Data query method and device

Abstract

The present invention provides a data query method and device, which can be applied to the fields of computer technology and artificial intelligence technology. The method includes: in response to a query statement submitted by a user to a database system, determining the statement features of the query statement, wherein the statement features include structural features, content features and time features, and the time features include the submission time of the query statement; inputting the statement features into a query prediction model to determine the query resources and statement execution time occupied by executing the query statement; based on the submission time, determining the first resource occupancy of the database system; based on the first resource, the first resource occupancy and the query resource, determining the response of the database system to the query statement; when the response situation indicates that the query statement can be responded to in real time, feeding back the query result to the user, wherein the query result is obtained by executing the query statement in the database system after the statement execution time.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and artificial intelligence technology, and in particular to a data query method and device. Background Art

[0002] A database is a system that stores, retrieves, and manages large amounts of data. The data stored in the database can usually be queried and operated using the Structured Query Language (SQL). As the amount of data stored in the database increases, the efficiency of querying the data in the database using SQL statements becomes lower.

[0003] In the process of implementing the present invention, it is found that there are at least the following problems in the prior art: since it is difficult to estimate the running time of SQL statements, it may cause the user to wait too long after submitting a query instruction, resulting in a poor user experience. In addition, when a large number of SQL statements are submitted at the same time, it may cause the database load to be too high, affecting the stability of the database. Summary of the invention

[0004] In view of the above problems, the present invention provides a data query method and device.

[0005] According to a first aspect of the present invention, a data query method is provided, comprising: in response to a query statement submitted by a user to a database system, determining statement features of the query statement, wherein the statement features include structural features, content features and time features, and the time features include the submission time of the query statement; inputting the statement features into a query prediction model to determine the query resources occupied by executing the query statement and the statement execution time; based on the submission time, determining the first resource occupancy of the database system; based on the first resource, the first resource occupancy and the query resource, determining the response of the database system to the query statement, wherein the database system includes a first resource and a second resource, the first resource representing a resource for responding to the query statement in real time, and the second resource representing a resource for responding to the query statement that cannot be responded to in real time; when the response situation represents that the query statement can be responded to in real time, feeding back the query result to the user, wherein the query result is obtained by executing the query statement in the database system after the statement execution time.

[0006] According to an embodiment of the present invention, based on the first resource occupancy and the query resource, the response of the database system to the query statement is determined, including: based on the first resource occupancy and the first resource, determining the first available resource of the database system; when the first available resource is greater than the query resource, the response situation indicates that the query statement can be responded to in real time; when the first available resource is less than the query resource, the response situation indicates that the query statement cannot be responded to in real time.

[0007] According to an embodiment of the present invention, in response to a query statement submitted by a user to a database system, statement features of the query statement are determined, including: determining structural features based on the statement structure of the query statement, wherein the structural features include the statement length, the number of clauses, and the number of connected tables of the query statement; determining content features based on the content of the query statement, wherein the content features include the number of data tables, the number of indexes, and the amount of data in the data tables included in the query statement; determining time features based on the time when the database system receives the query statement; and determining statement features based on structural features, content features, and time features.

[0008] According to an embodiment of the present invention, a query prediction model is pre-trained, and a training method includes: obtaining historical query records of a database system, wherein the historical query records include historical query statements and historical execution times and historical query resources corresponding to the historical query statements; determining historical statement features of the historical query statements; determining statement labels of the historical query statements based on the historical execution times and historical query resources corresponding to the historical query statements; inputting the historical statement features into an initial query prediction model to obtain a model output; adjusting model parameters of the initial query prediction model based on the model output and the statement label until the loss between the model output and the statement label is less than a preset loss threshold; and determining the initial query prediction model after adjusting the model parameters as the query prediction model.

[0009] According to an embodiment of the present invention, the data query method further includes: determining the periodic load condition of the database system based on historical query records; determining the optimal running time for the database system to execute query statements based on the periodic load condition; and feeding back the optimal running time to the user.

[0010] According to an embodiment of the present invention, the data query method also includes: when the query resources and statement execution time indicate that the resource allocation of the database system needs to be adjusted, determining an adjustment method based on the query resources and statement execution time; and dynamically adjusting the first resource and the second resource based on the adjustment method.

[0011] According to an embodiment of the present invention, the data query method also includes: when the response situation indicates that the query statement cannot be responded to in real time, adding the query statement to a query queue; based on the submission time and query resources, determining a recommended time when the query statement can be responded to in real time; and feeding back the recommended time to the user to obtain the user's query selection, wherein the query selection includes queuing and timed feedback.

[0012] According to an embodiment of the present invention, the data query method further includes: when the query selection is timed feedback, deleting the query statement from the query queue; executing the query statement at the recommended time, and feeding back the query result to the user.

[0013] According to an embodiment of the present invention, the data query method also includes: when the query is selected to wait in queue, obtaining the second available resource in real time, wherein the second available resource represents the available part of the second resource; when the query statement is at the head of the query queue and the second available resource is greater than the query resource, executing the query statement and feeding back the query result to the user; and deleting the query statement from the query queue.

[0014] A second aspect of the present invention provides a data query device, comprising:

[0015] A feature determination module, for determining the statement features of the query statement in response to the query statement submitted by the user to the database system, wherein the statement features include structural features, content features and time features, and the time features include the time when the query statement is submitted;

[0016] The model prediction module is used to input the statement features into the query prediction model to determine the query resources and statement execution time occupied by executing the query statement;

[0017] A resource determination module, used to determine a first resource occupancy of the database system based on the submission time;

[0018] A response status determination module, used to determine the response status of the database system to the query statement based on the first resource, the first resource occupancy and the query resource, wherein the database system includes the first resource and the second resource, the first resource represents the resource used to respond to the query statement in real time, and the second resource represents the resource used to respond to the query statement that cannot be responded to in real time;

[0019] The result feedback module is used to feed back the query result to the user when the response situation indicates that the query statement can be responded to in real time, wherein the query result is obtained by executing the query statement in the database system after the statement execution time.

[0020] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0021] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0022] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0023] According to an embodiment of the present invention, based on the query statement submitted by the user, the statement feature is determined, and the statement feature is input into the query prediction model, and the query resources and statement execution time required to execute the query statement are predicted. Since the output is predicted by the model, the query resources and statement execution time can be determined more accurately, thereby ensuring the accuracy of the prediction, thereby improving the accuracy of the process of determining the response situation. According to the first resource occupancy and query resources of the database system, the response of the database system to the query statement is determined, and when the response situation indicates that the query statement can be responded to in real time, the query result is fed back to the user to complete the data query. Since the response situation is determined based on the first resource occupancy and the query resources required to execute the query statement, it is possible to correctly judge whether the query statement can be responded to in real time, and when the query statement can be responded to in real time, the query result is fed back to the user in a timely manner, thereby improving the interactive experience of the user in the data query process. In addition, when a large number of query statements are submitted, the load capacity of the database system is determined based on the first available resource, which can avoid the problem of excessive database load caused by parallel query of too many query statements, and enhance the stability of the database system. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings.

[0025] Figure 1 An application scenario diagram of a data query method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0026] Figure 2 A flow chart of a data query method according to an embodiment of the present invention is shown.

[0027] Figure 3 A flowchart of executing query statements in a query queue of a data query method according to an embodiment of the present invention is shown.

[0028] Figure 4 A structural block diagram of a data query device according to an embodiment of the present invention is shown.

[0029] Figure 5 A block diagram of an electronic device suitable for implementing a data query method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0030] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0031] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0032] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0033] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0034] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, invention and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0035] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present invention provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0036] An embodiment of the present invention provides a data query method, comprising: in response to a query statement submitted by a user to a database system, determining statement features of the query statement, wherein the statement features include structural features, content features and time features, and the time features include the submission time of the query statement; inputting the statement features into a query prediction model to determine the query resources occupied by executing the query statement and the statement execution time; based on the submission time, determining the first resource occupancy of the database system; based on the first resource, the first resource occupancy and the query resource, determining the response of the database system to the query statement, wherein the database system includes a first resource and a second resource, the first resource representing a resource for responding to the query statement in real time, and the second resource representing a resource for responding to the query statement that cannot be responded to in real time; when the response situation represents that the query statement can be responded to in real time, feeding back the query result to the user, wherein the query result is obtained by executing the query statement in the database system after the statement execution time.

[0037] Figure 1 An application scenario diagram of a data query method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown.

[0038] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0039] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0040] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0041] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0042] It should be noted that the data query method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the data query device provided in the embodiment of the present invention can generally be set in the server 105. The data query method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the data query device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0043] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0044] The following will be based on Figure 1 The scene described by Figure 2~Figure 3 The data query method according to the embodiment of the present invention is described in detail.

[0045] Figure 2 A flow chart of a data query method according to an embodiment of the present invention is shown.

[0046] like Figure 2As shown, the data query method of this embodiment includes operations S210 to S250.

[0047] In operation S210, in response to a query statement submitted by a user to a database system, statement features of the query statement are determined, wherein the statement features include structural features, content features, and time features, and the time features include the time when the query statement is submitted.

[0048] In operation S220 , the statement features are input into a query prediction model to determine the query resources and statement execution time occupied by executing the query statement.

[0049] In operation S230 , a first resource occupancy of the database system is determined based on the submission time.

[0050] In operation S240, a response of the database system to the query statement is determined based on the first resource, the first resource occupancy, and the query resource.

[0051] In operation S250 , when the response situation indicates that the query statement can be responded to in real time, the query result is fed back to the user.

[0052] According to an embodiment of the present invention, when a user needs to perform a data query, he needs to submit a query statement to the database. In response to the query statement submitted by the user, the structural characteristics, content characteristics and time characteristics of the query statement are determined, thereby determining the statement characteristics of the query statement, wherein the query statement includes information such as the database name involved in the data query, one or more data table names, external connection conditions, query conditions, etc.

[0053] According to an embodiment of the present invention, statement features are input into a query prediction model, wherein the query prediction model can be used to predict the query resources and statement execution time required when executing the query statement based on the statement features of the query statement.

[0054] According to an embodiment of the present invention, the submission time of the query statement is determined based on the time feature of the query statement, and based on the submission time, the first resource occupancy of the database system at that time is determined.

[0055] According to an embodiment of the present invention, based on the first resource, the first resource occupancy amount and the query resource, the size relationship between the currently unoccupied resource amount in the first resource and the resource amount of the query resource is determined, and based on the size relationship, the response of the database system to the query statement is determined, wherein the database system includes a first resource and a second resource, the first resource represents a resource for responding to the query statement in real time, and the second resource represents a resource for responding to the query statement that cannot be responded to in real time.

[0056] According to an embodiment of the present invention, when the amount of unoccupied resources in the first resources is greater than the amount of query resources, the remaining first resources representing the database system can meet the resource requirements for executing query statements, and thus the query statements can be responded to in real time; and when the amount of unoccupied resources in the first resources is less than the amount of query resources, the remaining first resources representing the database system cannot meet the resource requirements for executing query statements, and thus the query statements cannot be responded to in real time.

[0057] According to an embodiment of the present invention, when the response situation indicates that the query statement can be responded to in real time, the query statement is executed in the database system, the query result is obtained after the statement is executed, and the query result is fed back to the user.

[0058] According to an embodiment of the present invention, based on the query statement submitted by the user, the statement feature is determined, and the statement feature is input into the query prediction model, and the query resources and statement execution time required to execute the query statement are predicted. Since the output is predicted by the model, the query resources and statement execution time can be determined more accurately, thereby ensuring the accuracy of the prediction, thereby improving the accuracy of the process of determining the response situation. According to the first resource occupancy and query resources of the database system, the response of the database system to the query statement is determined, and when the response situation indicates that the query statement can be responded to in real time, the query result is fed back to the user to complete the data query. Since the response situation is determined based on the first resource occupancy and the query resources required to execute the query statement, it is possible to correctly judge whether the query statement can be responded to in real time, and when the query statement can be responded to in real time, the query result is fed back to the user in a timely manner, thereby improving the interactive experience of the user in the data query process. In addition, when a large number of query statements are submitted, the load capacity of the database system is determined based on the first available resource, which can avoid the problem of excessive database load caused by parallel query of too many query statements, and enhance the stability of the database system.

[0059] According to an embodiment of the present invention, based on the first resource occupancy and the query resource, the response of the database system to the query statement is determined, including: based on the first resource occupancy and the first resource, determining the first available resource of the database system; when the first available resource is greater than the query resource, the response situation indicates that the query statement can be responded to in real time; when the first available resource is less than the query resource, the response situation indicates that the query statement cannot be responded to in real time.

[0060] According to an embodiment of the present invention, a first available resource of the database system is determined based on the first resource occupancy and the first resource, wherein the first available resource can be determined by using the difference between the resource amount of the first resource and the first resource occupancy.

[0061] According to an embodiment of the present invention, when the first available resource is greater than the query resource, the remaining first resource characterizing the database system can meet the resource demand for executing the query statement, and thus the response situation characterizing that the query statement can be responded to in real time; and when the first available resource is less than the query resource, the remaining first resource characterizing the database system cannot meet the resource demand for executing the query statement, and thus the response situation characterizing that the query statement cannot be responded to in real time.

[0062] According to an embodiment of the present invention, based on the first resource occupancy and the first resource, the first available resource is determined, and based on the size relationship between the first available resource and the query resource, it is determined whether the query statement can be responded to in real time, thereby ensuring the accuracy of the response.

[0063] According to an embodiment of the present invention, in response to a query statement submitted by a user to a database system, statement features of the query statement are determined, including: determining structural features based on the statement structure of the query statement, wherein the structural features include the statement length, the number of clauses, and the number of connected tables of the query statement; determining content features based on the content of the query statement, wherein the content features include the number of data tables, the number of indexes, and the amount of data in the data tables included in the query statement; determining time features based on the time when the database system receives the query statement; and determining statement features based on structural features, content features, and time features.

[0064] According to an embodiment of the present invention, the statement features of the query statement are determined based on the statement structure of the query statement, i.e., the statement length, the number of clauses, and the number of connected tables, wherein the statement length includes the number of words in the query statement. The number of clauses includes the clauses used in combination in the query statement to construct complex SQL queries and database operations, including SELECT clauses, FROM clauses, WHERE clauses, etc., wherein the clauses used to characterize sorting and grouping, i.e., ORDER BY clauses and GROUP BY clauses, can be encoded using one-hot encoding. The number of connected tables includes the number of tables involved in the outer join clause in the query statement.

[0065] According to an embodiment of the present invention, given the content of a query statement, content features are determined, wherein the content features include the number of data tables, the number of indexes and the amount of data in the data tables included in the query statement, wherein the number of data tables can be determined based on the number of table names introduced by the FROM clause of the query statement, the number of indexes can be determined based on information such as an index list in the data table corresponding to the table name introduced by the FROM clause, the amount of data in the data table can be determined based on the amount of data in the data table corresponding to the table name introduced by the FROM clause, and when the number of data tables is greater than 1, the amount of data in the data table can be obtained by summing the amounts of data in multiple data tables.

[0066] According to the embodiment of the present invention, since the delay between submission and receipt of the query statement is short and can be ignored, the time feature can be determined based on the time when the database system receives the query statement.

[0067] According to an embodiment of the present invention, the sentence features of the query sentence are determined based on the structural features, the content features and the time features.

[0068] According to an embodiment of the present invention, some features in the sentence features can be normalized so that the corresponding features have a distribution with a mean of 0 and a standard deviation of 1, thereby improving the prediction efficiency based on the sentence features and reducing the computational load caused by the prediction.

[0069] According to an embodiment of the present invention, the structural features, content features and time features of the query are determined based on the query structure, content and time of receiving the query, thereby determining the sentence features. Since multiple aspects of the query are considered respectively, the determined sentence features can more comprehensively reflect the characteristics of the query, and prediction based on the sentence features can obtain more accurate results.

[0070] According to an embodiment of the present invention, a query prediction model is pre-trained, and a training method includes: obtaining historical query records of a database system, wherein the historical query records include historical query statements and historical execution times and historical query resources corresponding to the historical query statements; determining historical statement features of the historical query statements; determining statement labels of the historical query statements based on the historical execution times and historical query resources corresponding to the historical query statements; inputting the historical statement features into an initial query prediction model to obtain a model output; adjusting model parameters of the initial query prediction model based on the model output and the statement label until the loss between the model output and the statement label is less than a preset loss threshold; and determining the initial query prediction model after adjusting the model parameters as the query prediction model.

[0071] According to an embodiment of the present invention, the query prediction model is pre-trained according to the following method to obtain historical query records of the database system, wherein the historical query records include historical query statements and historical execution time and historical query resources corresponding to the historical query statements.

[0072] According to an embodiment of the present invention, the historical sentence features of the historical query sentences are determined in the same manner as the sentence features of the query sentences.

[0073] According to an embodiment of the present invention, historical sentence features are input into an initial query prediction model to obtain a model output, wherein the model output includes the time and resources required for executing the historical query sentences predicted based on the historical sentence features, and the initial query prediction model may include linear regression, decision tree, random forest, etc., and based on the difference between the model output and the sentence label, the model parameters of the initial query prediction model are adjusted until the loss between the model output and the sentence label is less than a preset loss threshold, wherein the sentence label includes the historical execution time and historical query resources corresponding to the historical query sentences. The initial query prediction model after adjusting the model parameters is determined as the query prediction model.

[0074] According to an embodiment of the present invention, the historical query statements in the historical query records and the historical execution time and historical query resources corresponding to the historical query statements are used to adjust the parameters of the initial query prediction model, and a trained model is obtained and used as a query prediction model. The model is trained by using historical query statements and their corresponding resource occupancy and query time to ensure the accuracy of the prediction of the query prediction model, thereby improving the prediction accuracy of the recommended time for query statements that cannot be responded to in real time and improving the user experience.

[0075] According to an embodiment of the present invention, the data query method further includes: determining the periodic load condition of the database system based on historical query records; determining the optimal running time for the database system to execute query statements based on the periodic load condition; and feeding back the optimal running time to the user.

[0076] According to an embodiment of the present invention, based on multiple historical query statements in the historical query records and the historical execution time and historical query resources corresponding to the historical query statements, the load conditions of the database system at multiple historical moments are determined, and the periodic load conditions of the database system within a predetermined period are determined, where the predetermined period can be one day, one week, one month, etc.

[0077] According to an embodiment of the present invention, since the database system usually has a low period of system access within a predetermined period, for example, when the predetermined period is one day, the low period may be the early morning of each day, and therefore the low period is not suitable for being fed back to the user as the optimal operating time.

[0078] According to an embodiment of the present invention, based on the periodic load conditions, the time or period when the database system has the lowest load during the access peak period within a predetermined period is determined, the time or period is determined as the optimal running time for executing the query statement, and the optimal running time is fed back to the user.

[0079] According to an embodiment of the present invention, based on the historical operation status of the database system, the periodic load status of the database system is determined according to a predetermined period, and the time or a period with the smallest load during the access peak within the predetermined period is determined as the optimal operation time, and the optimal operation time is fed back to the user so that the user can choose to query data at the optimal operation time, thereby reducing the user's waiting time, improving query efficiency, and optimizing user experience.

[0080] According to an embodiment of the present invention, the data query method also includes: when the query resources and statement execution time indicate that the resource allocation of the database system needs to be adjusted, determining an adjustment method based on the query resources and statement execution time; and dynamically adjusting the first resource and the second resource based on the adjustment method.

[0081] According to an embodiment of the present invention, when the statement execution time is too long or the proportion of query resources in the first available resources is too large, the resource allocation of the characterizing database system needs to be adjusted. When the query resources and statement execution time characterize the resource allocation of the database system and need to be adjusted, the adjustment method is determined based on the query resources and statement execution time, wherein the idle ratio of the first resource can be preset, for example, the idle ratio is set to 20%, that is, the first resource occupancy of the first resource in the ideal state does not exceed 80%, and when the sum of the first resource occupancy and the query resource is greater than 80% of the first resource, based on the size of the query resource, the adjustment method is determined to adjust the first resource to 1.25 times the sum of the first resource occupancy and the query resource, so that the sum of the first resource occupancy and the query resource does not exceed 80% of the first resource. When the proportion of the first resource occupancy to the first resource is too low, the adjustment method is determined to reduce the first resource.

[0082] According to an embodiment of the present invention, the first resource and the second resource are dynamically adjusted based on the adjustment method, wherein, when the adjustment method indicates that the first resource needs to be expanded, part of the second resource is divided into the first resource, and when the adjustment method indicates that the first resource needs to be reduced, part of the first resource is divided into the second resource.

[0083] According to an embodiment of the present invention, when it is determined that the resource allocation of the database system needs to be adjusted based on the query resources and the statement execution time, the first resource and the second resource are dynamically adjusted according to the adjustment method, so as to improve resource utilization and adapt to changes in the database system.

[0084] According to an embodiment of the present invention, the data query method also includes: when the response situation indicates that the query statement cannot be responded to in real time, adding the query statement to a query queue; based on the submission time and query resources, determining a recommended time when the query statement can be responded to in real time; and feeding back the recommended time to the user to obtain the user's query selection, wherein the query selection includes queuing and timed feedback.

[0085] According to an embodiment of the present invention, when the response situation indicates that the query statement cannot be responded to in real time, the query statement is added to a query queue, wherein the query queue is used to store query statements that cannot be responded to in real time and need to be queued to be executed.

[0086] According to an embodiment of the present invention, based on the query resources, the waiting time for the first resource to respond to the query statement is determined, and based on the submission time and the waiting time, the recommended time is determined, and the recommended time is fed back to the user to obtain the user's query selection, wherein the query selection includes queuing and timed feedback, and the timed feedback indicates that the user can exit the database system, and the query result is fed back to the user at the recommended time. The queuing indicates that the user needs to wait in the query queue, and when the situation of the second resource and the query queue meets the execution condition of the query statement, the query statement is executed and the query result is fed back to the user.

[0087] According to an embodiment of the present invention, when the response situation indicates that a query statement cannot be responded to in real time, a recommended time when the query statement can be responded to is determined, and the recommended time is fed back to the user, and a corresponding query is performed based on the user's query selection. By providing the user with different choices, the user's experience of data query can be improved.

[0088] According to an embodiment of the present invention, the data query method further includes: when the query selection is timed feedback, deleting the query statement from the query queue; executing the query statement at the recommended time, and feeding back the query result to the user.

[0089] According to an embodiment of the present invention, when the query is selected as timed feedback, since the user needs to wait for the first resource to execute the query statement, the query statement is deleted from the query queue, and the query statement is executed at the recommended time, and the query result is fed back to the user, where the method of feeding back the query result may include email, pop-up window, etc.

[0090] According to an embodiment of the present invention, when the user's query selection is timed feedback, the query statement is deleted from the query queue to prevent the query statement with timed feedback from affecting the query efficiency of the query statements waiting in the queue. The query results are fed back to the user by means of email, pop-up windows, etc. at the recommended time, which can prevent the user from missing the query results and improve the user experience.

[0091] According to an embodiment of the present invention, the data query method also includes: when the query is selected to wait in queue, obtaining the second available resource in real time, wherein the second available resource represents the available part of the second resource; when the query statement is at the head of the query queue and the second available resource is greater than the query resource, executing the query statement and feeding back the query result to the user; and deleting the query statement from the query queue.

[0092] According to an embodiment of the present invention, when the query selection is queue waiting, the second available resource is calculated based on the second resource and the second resource occupancy.

[0093] According to an embodiment of the present invention, after other query statements before the current query statement are executed, the executed query statement will be deleted from the query queue, and the ranking of the current query statement in the query queue will be improved until the current query statement ranks first in the query queue, that is, at the head of the queue. The resource amount of the second available resource is judged, and when the resource amount of the second available resource is greater than the query resource, the query statement is executed to determine the query result.

[0094] According to an embodiment of the present invention, the query result is fed back to the user. After the query statement is executed, the query statement is deleted from the query queue so that subsequent query statements in the query queue can be executed.

[0095] Figure 3 A flowchart of executing query statements in a query queue of a data query method according to an embodiment of the present invention is shown.

[0096] like Figure 3 As shown, the database system includes a first resource 310, a second resource 320 and a query queue 330, wherein the first resource 310 includes a first resource occupancy 311 and a first available resource 312, the second resource 320 includes a second resource occupancy 321 and a second available resource 322, and the query queue 330 includes a first query statement 331, a second query statement 332, a third query statement 333, a fourth query statement 334, a fifth query statement 335 and a sixth query statement 336. The query resources of the multiple query statements in the query queue 330 are all greater than the first available resource 312, and therefore they are stored in the query queue 330, wherein the query of the user who submitted the first query statement 331 is selected as queued waiting, and in Figure 3 At the moment shown, the first query statement 331 is at the head of the query queue 330. When the query resources of the first query statement 331 are less than the second available resources 322, the second available resources 322 are used to execute the first query statement 331, obtain the query result and feed back the query result to the user.

[0097] According to an embodiment of the present invention, when the query selection priority is high, the user's query selection is to wait in queue, and the second resource can be used to execute the query statement, avoiding the problem of long response time that may occur when using the first resource to execute the statement, thereby improving the efficiency of data query.

[0098] Based on the above data query method, the present invention also provides a data query device. Figure 4 The device is described in detail.

[0099] Figure 4 A structural block diagram of a data query device according to an embodiment of the present invention is shown.

[0100] like Figure 4 As shown, the data query device 400 of this embodiment includes a feature determination module 410 , a model prediction module 420 , a resource determination module 430 , a response situation determination module 440 and a result feedback module 450 .

[0101] The feature determination module 410 is used to determine the statement features of the query statement in response to the query statement submitted by the user to the database system, wherein the statement features include structural features, content features and time features, and the time features include the time when the query statement is submitted. In one embodiment, the feature determination module 410 can be used to perform the operation S210 described above, which will not be repeated here.

[0102] The model prediction module 420 is used to input the statement features into the query prediction model to determine the query resources and statement execution time occupied by executing the query statement. In one embodiment, the model prediction module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0103] The resource determination module 430 is used to determine the first resource occupation amount of the database system based on the submission time. In one embodiment, the resource determination module 430 can be used to perform the operation S230 described above, which will not be described in detail here.

[0104] The response situation determination module 440 is used to determine the response of the database system to the query statement based on the first resource, the first resource occupancy and the query resource, wherein the database system includes the first resource and the second resource, the first resource represents a resource for responding to the query statement in real time, and the second resource represents a resource for responding to the query statement that cannot be responded to in real time. In one embodiment, the response situation determination module 440 can be used to perform the operation S240 described above, which will not be repeated here.

[0105] The result feedback module 450 is used to feed back the query result to the user when the response situation represents that the query statement can be responded to in real time, wherein the query result is obtained by executing the query statement in the database system after the statement execution time. In one embodiment, the result feedback module 450 can be used to perform the operation S250 described above, which will not be repeated here.

[0106] According to an embodiment of the present invention, the response situation determination module 440 includes an available resource determination submodule, a first situation determination submodule and a second situation determination submodule.

[0107] The available resource determination submodule is used to determine a first available resource of the database system based on the first resource occupancy and the first resource.

[0108] The first situation determination submodule is used to determine that, when the first available resource is greater than the query resource, the response situation represents that the query statement can be responded to in real time.

[0109] The second situation determination submodule is used for determining that, when the first available resource is less than the query resource, the response situation represents that the query statement cannot be responded to in real time.

[0110] According to an embodiment of the present invention, the feature determination module 410 includes a structure feature determination submodule, a content feature determination submodule, a time feature determination submodule and a feature determination submodule.

[0111] The structural feature determination submodule is used to determine the structural features based on the sentence structure of the query sentence, wherein the structural features include the sentence length, the number of clauses and the number of connection tables of the query sentence.

[0112] The content feature determination submodule is used to determine content features based on the content of the query statement, wherein the content features include the number of data tables, the number of indexes and the amount of data in the data tables included in the query statement.

[0113] The time feature determination submodule is used to determine the time feature based on the time when the database system receives the query statement.

[0114] The feature determination submodule is used to determine the sentence features based on the structural features, content features and time features.

[0115] According to an embodiment of the present invention, the query prediction model is pre-trained, and the data query device 400 further includes a history record acquisition module, a history feature determination module, a label determination module, a feature input module, a parameter adjustment module and a model determination module.

[0116] The historical record acquisition module is used to acquire the historical query records of the database system, wherein the historical query records include historical query statements and the historical execution time and historical query resources corresponding to the historical query statements.

[0117] The historical feature determination module is used to determine the historical statement features of the historical query statements.

[0118] The label determination module is used to determine the statement label of the historical query statement based on the historical execution time and the historical query resources corresponding to the historical query statement.

[0119] The feature input module is used to input the historical sentence features into the initial query prediction model to obtain the model output.

[0120] The parameter adjustment module is used to adjust the model parameters of the initial query prediction model based on the model output and the sentence label until the loss between the model output and the sentence label is less than a preset loss threshold.

[0121] The model determination module is used to determine the initial query prediction model after adjusting the model parameters as the query prediction model.

[0122] According to an embodiment of the present invention, the data query device 400 further includes a load determination module, a time determination module and a time feedback module.

[0123] The load determination module is used to determine the periodic load condition of the database system based on historical query records.

[0124] The time determination module is used to determine the optimal running time for the database system to execute query statements based on periodic load conditions.

[0125] The time feedback module is used to feed back the optimal running time to the user.

[0126] According to an embodiment of the present invention, the data query device 400 further includes an adjustment mode determination module and a resource adjustment module.

[0127] The adjustment mode determination module is used to determine the adjustment mode based on the query resources and the statement execution time when the query resources and the statement execution time indicate that the resource allocation of the database system needs to be adjusted.

[0128] The resource adjustment module is used to dynamically adjust the first resource and the second resource based on the adjustment method.

[0129] According to an embodiment of the present invention, the data query device 400 further includes a statement enqueue module, a time determination module and a selection acquisition module.

[0130] The statement enqueue module is used to add the query statement to the query queue when the response situation indicates that the query statement cannot be responded to in real time.

[0131] The time determination module is used to determine the recommended time that can respond to the query statement in real time based on the submission time and the query resource.

[0132] The selection acquisition module is used to feed back the recommendation time to the user and obtain the user's query selection, wherein the query selection includes queuing and timing feedback.

[0133] According to an embodiment of the present invention, the data query device 400 further includes a first statement dequeue module and a result push module.

[0134] The first statement dequeue module is used to delete the query statement from the query queue when the query selection is timed feedback.

[0135] The result push module is used to execute the query statement at the recommended time and feed back the query results to the user.

[0136] According to an embodiment of the present invention, the data query device 400 further includes a resource acquisition module, a statement execution module and a second statement dequeue module.

[0137] The resource acquisition module is used to acquire the second available resource in real time when the query selection is queue waiting, wherein the second available resource represents the available part of the second resource.

[0138] The statement execution module is used to execute the query statement and feed back the query result to the user when the query statement is at the head of the query queue and the second available resource is greater than the query resource.

[0139] The second statement dequeue module is used to delete the query statement from the query queue.

[0140] According to an embodiment of the present invention, any multiple modules of the feature determination module 410, the model prediction module 420, the resource determination module 430, the response determination module 440 and the result feedback module 450 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the feature determination module 410, the model prediction module 420, the resource determination module 430, the response determination module 440 and the result feedback module 450 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the feature determination module 410, the model prediction module 420, the resource determination module 430, the response situation determination module 440 and the result feedback module 450 can be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function can be performed.

[0141] Figure 5 A block diagram of an electronic device suitable for implementing a data query method according to an embodiment of the present invention is shown.

[0142] like Figure 5 As shown, the electronic device 500 according to an embodiment of the present invention includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 to a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include an onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0143] In RAM 503, various programs and data required for the operation of electronic device 500 are stored. Processor 501, ROM 502 and RAM 503 are connected to each other via bus 504. Processor 501 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 502 and / or RAM 503. It should be noted that the program can also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0144] According to an embodiment of the present invention, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the input / output (I / O) interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that the computer program read therefrom is installed into the storage portion 508 as needed.

[0145] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0146] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0147] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0148] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 501. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0149] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the removable medium 511. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0150] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0151] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0152] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0154] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A data query method, characterized in that: The method comprises: In response to a query statement submitted by a user to a database system, determining statement features of the query statement, wherein the statement features include structural features, content features, and time features, and the time features include the time when the query statement is submitted; Inputting the statement features into a query prediction model to determine the query resources and statement execution time occupied by executing the query statement; Based on the submission time, determining a first resource occupancy of the database system; Determining a response of the database system to the query statement based on a first resource, an amount of the first resource occupied, and the query resource, wherein the database system includes the first resource and a second resource, the first resource represents a resource for responding to the query statement in real time, and the second resource represents a resource for responding to the query statement that cannot be responded to in real time; When the response condition indicates that the query statement can be responded to in real time, feeding back a query result to the user, wherein the query result is obtained by executing the query statement in the database system after the statement execution time; In a case where the query resources and the statement execution time indicate that resource allocation of the database system needs to be adjusted, determining an adjustment method based on the query resources and the statement execution time; Based on the adjustment method, dynamically adjust the first resource and the second resource; The dynamically adjusting the first resource and the second resource based on the adjustment method includes: In a case where the adjustment mode indicates that the first resource needs to be expanded, allocating part of the second resource to the first resource; and When the adjustment mode indicates that the first resource needs to be reduced, part of the first resource is allocated to the second resource.

2. The method according to claim 1, characterized in that The determining, based on the first resource occupancy and the query resource, a response of the database system to the query statement includes: Determine a first available resource of the database system based on the first resource occupation and the first resource; In a case where the first available resource is greater than the query resource, the response condition indicates that the query statement can be responded to in real time; In the case where the first available resource is smaller than the query resource, the response condition indicates that the query statement cannot be responded to in real time.

3. The method according to claim 1, characterized in that The step of determining the statement features of the query statement in response to the query statement submitted by the user to the database system includes: Determining the structural features based on the sentence structure of the query sentence, wherein the structural features include the sentence length, the number of clauses, and the number of connection tables of the query sentence; Determine the content feature based on the content of the query statement, wherein the content feature includes the number of data tables, the number of indexes and the amount of data in the data table included in the query statement; Determining the time feature based on the time when the database system receives the query statement; The sentence feature is determined based on the structural feature, the content feature and the time feature.

4. The method according to claim 1, characterized in that The query prediction model is pre-trained, and the training method includes: Acquire historical query records of the database system, wherein the historical query records include historical query statements and historical execution time and historical query resources corresponding to the historical query statements; Determining historical statement features of the historical query statements; Determining a statement tag of the historical query statement based on a historical execution time and a historical query resource corresponding to the historical query statement; Inputting the historical sentence features into the initial query prediction model to obtain a model output; Adjusting a model parameter of the initial query prediction model based on the model output and the sentence label until a loss between the model output and the sentence label is less than a preset loss threshold; The initial query prediction model after adjusting the model parameters is determined as the query prediction model.

5. The method according to claim 4, characterized in that The data query method further includes: Based on the historical query records, determining the periodic load condition of the database system; Based on the periodic load condition, determining an optimal running time for the database system to execute the query statement; The optimal running time is fed back to the user.

6. The method according to any one of claims 1 to 5, characterized in that The data query method further includes: When the response condition indicates that the query statement cannot be responded to in real time, adding the query statement to a query queue; Determining a recommended time capable of responding to the query statement in real time based on the submission time and the query resource; The recommendation time is fed back to the user to obtain the user's query selection, wherein the query selection includes queuing and timing feedback.

7. The method according to claim 6, characterized in that The data query method further includes: In the case where the query selection is timed feedback, deleting the query statement from the query queue; The query statement is executed at the recommendation time, and the query result is fed back to the user.

8. The method according to claim 6, characterized in that The data query method further includes: In the case where the query is selected as queuing, obtaining a second available resource in real time, wherein the second available resource represents an available portion of the second resource; When the query statement is at the head of the query queue and the second available resource is greater than the query resource, executing the query statement and feeding back the query result to the user; The query statement is deleted from the query queue.

9. A data query device, characterized in that: The device comprises: A feature determination module, configured to determine, in response to a query statement submitted by a user to a database system, statement features of the query statement, wherein the statement features include structural features, content features, and time features, and the time features include the time when the query statement is submitted; A model prediction module, used to input the statement features into a query prediction model to determine the query resources and statement execution time occupied by executing the query statement; A resource determination module, configured to determine a first resource occupancy amount of the database system based on the submission time; a response status determination module, configured to determine the response status of the database system to the query statement based on a first resource, an amount of the first resource occupied, and the query resource, wherein the database system includes the first resource and a second resource, the first resource represents a resource for responding to the query statement in real time, and the second resource represents a resource for responding to the query statement that cannot be responded to in real time; A result feedback module is used to feed back the query result to the user when the response situation indicates that the query statement can be responded to in real time, wherein the query result is obtained by executing the query statement in the database system after the statement execution time; An adjustment mode determination module, configured to determine an adjustment mode based on query resources and the statement execution time when the query resources and the statement execution time indicate that resource allocation of the database system needs to be adjusted; A resource adjustment module, configured to dynamically adjust the first resource and the second resource based on the adjustment method; Wherein, the resource adjustment module includes: In a case where the adjustment mode indicates that the first resource needs to be expanded, allocating part of the second resource to the first resource; and When the adjustment mode indicates that the first resource needs to be reduced, part of the first resource is allocated to the second resource.

Images