Encoding methods for query requests, data query methods, and related equipment
By encoding the query plan through the Attention mechanism and generating the target code, the problem of low efficiency and insufficient accuracy of database parameter tuning in the existing technology is solved, and efficient adjustment of complex query plans is achieved.
Patent Information
- Application Number
- CN202411449630.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-10-16
AI Technical Summary
In existing technologies, database parameter tuning relies on human experience, which is inefficient and difficult to adjust in real time. Machine learning methods have limitations when encoding query requests, and cannot accurately reflect fine-grained information of the query execution process, especially in complex queries where it is difficult to model node relationships.
A query request encoding method based on the attention mechanism is adopted. By converting the query plan into a directed acyclic graph, multiple node vectors are generated, and the connection relationship between nodes is calculated using the self-attention mechanism to generate target encoding, thereby enriching the query information.
It improves query accuracy, especially in query plans with a large number of nodes and complex relationships, enhancing the accuracy and efficiency of database parameter adjustment.
Smart Images

Figure CN119441270B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a method for encoding a query request, a data query method, and related equipment. Background Technology
[0002] In the field of computer technology, databases are commonly used to store data and can be retrieved based on query requests. Databases contain adjustable system parameters. In related technologies, database administrators typically rely on their own experience to tune these parameters, requiring a significant amount of time to optimize them.
[0003] The inventors of this disclosure have discovered that machine learning methods can be used to automatically adjust the parameters of a database in related technologies, but this method has certain limitations when encoding query requests. Summary of the Invention
[0004] This disclosure proposes an encoding method for query requests, a data query method, and related equipment to solve or partially solve the above-mentioned problems.
[0005] In a first aspect, this disclosure provides a method for encoding a query request, comprising:
[0006] Receive query requests;
[0007] Obtain the query plan corresponding to the query request, wherein the query plan includes multiple nodes and the connection relationships between the multiple nodes;
[0008] Encode each of the multiple nodes to obtain multiple vectors;
[0009] Based on the connection relationships between the multiple nodes and the multiple vectors, the target encoding corresponding to the query request is determined.
[0010] A second aspect of this disclosure provides a data query method, including:
[0011] Receive a query request for target data, the query request being used to request the query for the target data in the target database;
[0012] Using the method described in the first aspect, the query request is encoded to obtain the target code corresponding to the query request;
[0013] Based on the target encoding, at least one target database parameter of the target database is determined;
[0014] Based on the at least one target database parameter, adjust at least one database parameter of the target database;
[0015] Based on the query request, the target data is queried in the target database after parameter adjustment.
[0016] A third aspect of this disclosure provides an encoding device for a query request, comprising:
[0017] The receiving module is configured to receive query requests;
[0018] The acquisition module is configured to: acquire the query plan corresponding to the query request, wherein the query plan includes multiple nodes and the connection relationship between the multiple nodes;
[0019] The encoding module is configured to encode the plurality of nodes respectively to obtain a plurality of vectors;
[0020] The determination module is configured to: determine the target encoding corresponding to the query request based on the connection relationships between the plurality of nodes and the plurality of vectors.
[0021] A fourth aspect of this disclosure provides a data query apparatus, comprising:
[0022] The receiving module is configured to receive a query request for target data, wherein the query request is used to request a query for the target data in the target database;
[0023] The encoding module is configured to: use the method described in the first aspect to encode the query request to obtain the target encoding corresponding to the query request;
[0024] The determining module is configured to: determine at least one target database parameter of the target database based on the target encoding;
[0025] The adjustment module is configured to: adjust at least one database parameter of the target database based on the at least one target database parameter;
[0026] The query module is configured to: query the target data in the target database after parameter adjustment based on the query request.
[0027] A fifth aspect of this disclosure provides a computer device including one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the programs including instructions for performing the method according to the first or second aspect.
[0028] A sixth aspect of this disclosure provides a non-volatile computer-readable storage medium comprising a computer program that, when executed by one or more processors, causes the processors to perform the method described in the first or second aspect.
[0029] A seventh aspect of this disclosure provides a computer program product including computer program instructions that, when executed on a computer, cause the computer to perform the method as described in the first or second aspect.
[0030] The query request encoding method, data query method, and related device provided in this disclosure generate multiple vectors based on multiple nodes of the query plan, and combine the connection relationship information between nodes in the multiple vectors, so that the target encoding can contain richer query information and improve query accuracy. Attached Figure Description
[0031] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0032] Figure 1 A schematic diagram of an exemplary system provided by an embodiment of this disclosure is shown.
[0033] Figure 2A A schematic diagram of an exemplary encoding method provided in an embodiment of this disclosure is shown.
[0034] Figure 2B A schematic flowchart illustrating the computation of self-attention encoding according to an embodiment of the present disclosure is shown.
[0035] Figure 3 A flowchart illustrating an exemplary method provided in an embodiment of this disclosure is shown.
[0036] Figure 4 Another flowchart illustrating an exemplary method provided by an embodiment of this disclosure is shown.
[0037] Figure 5 A schematic diagram of the hardware structure of an exemplary computer device provided in an embodiment of this disclosure is shown.
[0038] Figure 6 A schematic diagram of an exemplary apparatus provided by an embodiment of the present disclosure is shown.
[0039] Figure 7 A schematic diagram of another exemplary apparatus provided by an embodiment of this disclosure is shown. Detailed Implementation
[0040] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.
[0041] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0042] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.
[0043] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.
[0044] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0045] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.
[0046] Figure 1 A schematic diagram of an exemplary system 100 provided in an embodiment of this disclosure is shown.
[0047] like Figure 1As shown, system 100 may include terminal device 102, terminal device 104, server 106, and database server 108. A medium (e.g., a network) may be provided to provide a communication link between terminal device 102, terminal device 104, server 106, and database server 108. This network may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.
[0048] For example, various applications (APPs) can be installed on the terminal device 104, such as life service applications, collaborative office applications, video conferencing applications, reading applications, video applications, social applications, payment applications, web browsers, and instant messaging tools.
[0049] The terminal devices 102 and 104 here can be hardware or software. When terminal devices 102 and 104 are hardware, they can be various electronic devices with displays, including but not limited to smartphones, tablets, e-book readers, MP3 players, laptops, and desktop computers (PCs). When terminal devices 102 and 104 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module. No specific limitations are made here.
[0050] Server 106 can be a server providing various services, such as a backend server supporting various applications displayed on terminal devices 102 and 104. Database server 108 can also be a database server providing various services. It is understood that if server 106 can implement the relevant functions of database server 108, database server 108 may not need to be set up in system 100.
[0051] The server 106 and database server 108 here can be either hardware or software. When they are hardware, they can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When they are software, they can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module. No specific limitations are made here.
[0052] It should be understood that Figure 1 The number of terminal devices, users, servers, and database servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, users, servers, and database servers.
[0053] As an example scenario, one or more database servers 108 can constitute a database for storing data and retrieving and returning relevant data based on a query request. For instance, user 112 can request a service using any application on terminal device 104. This service needs to retrieve relevant data to return to terminal device 104. In this case, a query request can be generated based on user 112's request and sent to the database to retrieve the data. The database retrieves the relevant data based on the query request and returns it to terminal device 104.
[0054] As mentioned earlier, there are many adjustable system parameters in a database. In related technologies, these parameters are typically adjusted by database administrators based on their experience (for example, user 110 remotely adjusts the hardware parameters of any database server 108 via terminal device 102). Furthermore, this manual parameter tuning can be time-consuming. This method is inefficient and makes real-time parameter adjustment difficult, resulting in poor system performance.
[0055] As an example, machine learning methods can be used for parameter tuning. This approach reduces the time required for tuning while also improving the quality of the tuning.
[0056] For example, this type of parameter tuning method typically requires encoding the query request as input to the machine learning model, and then outputting the corresponding parameters for parameter tuning. Furthermore, the machine learning model also needs to be trained based on the encoding of historical query requests.
[0057] The inventors of this disclosure have discovered that the encoding of query requests significantly impacts the quality of parameter tuning. In particular, the effectiveness of query request encoding greatly influences the model's learning performance. If the query request encoding is inaccurate or ineffective, the model will struggle to accurately learn the relationship between queries, parameters, and database performance. This, in turn, affects the quality of the final recommended parameters. Therefore, query request encoding is a crucial step in the entire parameter tuning process and requires sufficient attention and optimization.
[0058] In related technologies, some text-based encoding methods for query requests have significant limitations. These methods merely encode the textual representation of the query request (e.g., SQL statement) into a feature vector as model input, failing to encompass information about the query execution process. This means they lack fine-grained, effective information. The query execution process involves many crucial factors, such as the execution order, data flow, and processing of intermediate results. Without this information, the encoding cannot fully reflect the actual query situation, potentially leading to inaccurate and ineffective subsequent parameter tuning.
[0059] In other related technologies, query operations, table names, cost information, and other details can be encoded along with the query request. However, this method of encoding query operations, table names, and cost information also has problems. When the query is very complex (e.g., containing multiple nested subqueries), these methods struggle to accurately represent the query plan information. In complex queries, the query plan typically consists of multiple nodes with intricate relationships between them. The encoding methods of these related technologies often fail to model such complex relationships.
[0060] In view of this, embodiments of the present disclosure provide an encoding method for query requests to solve or partially solve the above-mentioned problems.
[0061] Figure 2A A schematic diagram of an exemplary encoding method 200 provided in an embodiment of this disclosure is shown. This method 200 can be used to generate a target encoding based on a query request. This target encoding can be used to train a parameter generation model or as input to the parameter generation model to obtain relevant parameters for adjusting the database. Optionally, the system 200 can be implemented as... Figure 1 Server-side 106, or, implemented as Figure 1 The server 106 and the database server 108.
[0062] like Figure 2A As shown, a query request (Query) 202 can be received first. Optionally, the query request 202 can be, for example, a Structured Query Language (SQL) command used to request the execution of a corresponding query operation.
[0063] Optionally, the query request 202 can be real-time or historical. Real-time query requests can be used for real-time data retrieval, while historical query requests can be used to train a parameter generation model so that the parameter generation model can be used to generate recommended parameters to adjust the parameters of the database.
[0064] Next, as Figure 2AAs shown, the corresponding query plan 204 can be obtained based on query request 202.
[0065] Query plan 204 is the specific steps in a database system used to execute query request 202 (e.g., an SQL command). It details how to obtain the required data, including which indexes to use, how to join tables, and the order in which data is read.
[0066] In some embodiments, within a relational database management system (RDBMS), query plan 204 can be generated by the DBMS based on a submitted query request 202 (e.g., an SQL command), and is an optimized representation for efficiently accessing, retrieving, and processing data. Optionally, query plan 204 can be a product of a query optimizer, a component of the RDBMS used to build, evaluate, and select the most efficient query plan. Query plan 204 generated by the query optimizer can reduce the resource consumption and time required to execute a query, while improving query speed and accuracy.
[0067] As an alternative embodiment, the query plan can be abstracted into multiple plan nodes, and these nodes can also have an execution order according to the query steps described in the query plan. This execution order can be abstracted as the connection relationship between the nodes.
[0068] In some embodiments, based on the information contained in query plan 204, query plan 204 can be converted into a directed acyclic graph (DAG). Figure 2A (A tree-like structure) or tree diagram. For example... Figure 2A As shown, the directed acyclic graph (DAG) consists of vertices and edges, where each edge points from one vertex to another. Each vertex corresponds to a node, and the edges represent the connections between nodes. In this way, information about the query execution process can be extracted based on the DAG, and this information can be incorporated into the target encoding, allowing the target encoding to contain relevant information about the query execution process and improving query accuracy.
[0069] Optionally, an adjacency matrix 210 can be determined based on the directed acyclic graph. This adjacency matrix 210 indicates the connection relationships between the plurality of nodes. During subsequent encoding, this adjacency matrix can be used to mask the pre-encoding, thereby incorporating the connection relationships between nodes into the encoding and improving query accuracy.
[0070] Furthermore, after obtaining multiple nodes of query plan 204, these nodes can be encoded to obtain multiple vectors, such as vectors 206A to 206E. Figure 2A As shown.
[0071] Specifically, each node can be encoded as a vector. Optionally, the encoding can be based on multiple descriptive information contained in each node that is related to the query request 202. Optionally, the multiple descriptive information may include at least one of operators, joins, scan tables, predicates, and cardinality estimates. In this way, the vector encoded based on the descriptive information can contain more information about the query plan, thereby improving query accuracy.
[0072] In some embodiments, for each node, the plurality of description information of the node can be converted into a plurality of one-hot vectors. Taking the operators as a total of three types, A, B, and C, as an example, if the node's operator is A, then its corresponding one-hot vector can be [1,0,0]; if the node's operator is B, then its corresponding one-hot vector can be [0,1,0]; and if the node's operator is C, then its corresponding one-hot vector can be [0,0,1]. Using this method, each description information can be encoded into a one-hot vector.
[0073] Next, the multiple one-hot vectors corresponding to each node can be concatenated to obtain the vector corresponding to that node. In this way, the vector corresponding to each node is encoded.
[0074] In some embodiments, if it is necessary to calculate the self-attention score corresponding to each node subsequently, the plurality of one-hot vectors can be concatenated in a predetermined order to obtain the vector corresponding to the node; wherein the predetermined order corresponding to each node is the same. This ensures accuracy when calculating the self-attention score. It is understood that the predetermined order of concatenation can be arbitrary, as long as the concatenation order of each vector is the same.
[0075] Optionally, after obtaining the multiple vectors corresponding to the multiple nodes, the target code 212 corresponding to the query request 202 can be determined based on the connection relationship between the multiple nodes and the multiple vectors.
[0076] In this way, multiple vectors are generated based on multiple nodes of the query plan, and the connection relationship information between nodes is combined in multiple vectors, so that the target code 212 can contain richer query information, thereby improving query accuracy. In particular, this method is suitable for query plans with a large number of nodes and complex relationships between nodes, and the effect of improving accuracy is more significant.
[0077] Attention mechanisms play a crucial role in machine learning. They can be understood as a resource allocation mechanism that allows models to focus on important parts of information while ignoring less critical content. In practical applications, when models handle complex tasks, such as machine translation and text summarization in natural language processing, or image recognition and object detection in computer vision, the input data often contains a large amount of information. Attention mechanisms can dynamically calculate the importance weights of different parts based on the current task requirements.
[0078] To address the encoding scheme issues in related technologies to some extent, the query plan is encoded into a vector. Given a specified vector size, as much value information as possible is encoded, allowing for a trade-off between encoded information and encoding space.
[0079] Therefore, in some embodiments, such as Figure 2A As shown, the self-attention module 208 can be used to calculate self-attention scores and obtain self-attention codes. Based on the connection relationships between the multiple nodes, self-attention scores can be calculated for the multiple vectors, resulting in multiple self-attention codes corresponding to the multiple nodes. Then, based on these multiple self-attention codes, the target code corresponding to the query request is determined. The self-attention mechanism allows the model to automatically focus on different parts of the input based on attention scores and assign different weights according to their importance when processing data. Unlike traditional attention mechanisms, self-attention primarily focuses on the relationships within the input itself. Thus, through the self-attention mechanism, long-distance dependencies between multiple nodes can be captured, rather than being limited to local contextual information, which is very helpful for understanding complex language structures and semantic relationships. Furthermore, combined with the connection relationships, the self-attention scores of nodes with connection relationships can be introduced into the target code, enriching the encoding information as much as possible while ensuring the conciseness of the encoding. In addition, the self-attention mechanism can be computed in parallel, greatly improving computational efficiency. Furthermore, it can adaptively learn the relationships between different nodes without requiring the manual design of complex feature extractors. This makes the model more flexible and powerful, enabling it to better handle various complex tasks and data. In addition, self-attention provides a degree of interpretability; by observing the distribution of attention weights, one can understand the key regions the model focuses on when processing data.
[0080] In some embodiments, taking any node as the target node as an example, self-attention scores are calculated on the multiple vectors based on the connection relationships between the multiple nodes, resulting in multiple self-attention codes corresponding to the multiple nodes, such as... Figure 2B As shown, the following steps may be further included.
[0081] In step 222, the target node can be determined from the plurality of nodes, for example, the node corresponding to vector 206A.
[0082] In step 224, the self-attention scores between the target vector corresponding to the target node and the other vectors among the plurality of vectors excluding the target node can be calculated to obtain the first self-attention score corresponding to the target node.
[0083] In some embodiments, step 204, which calculates the self-attention score between the target vector corresponding to the target node and other vectors among the plurality of vectors excluding the target node, to obtain the first self-attention score corresponding to the target node, may further include the following steps:
[0084] The multiple vectors are converted into multiple queries, multiple keys, and multiple values. Then, based on the target query and target key of the target vector of the target node and the multiple other queries and multiple other keys of the other vectors, the first self-attention score corresponding to the target node is calculated.
[0085] Specifically, for an input sequence containing many vectors (e.g., vectors 206A to 206E), the self-attention mechanism can first use three different linear layers to project each vector into a new representation, denoted as Q, K, and V, which are query, key, and value, respectively.
[0086] Then, attention computation is defined as:
[0087]
[0088] The attention function involves calculating the correlation score between Q and K, and then multiplying it by the normalized value V. is a scaling factor designed to obtain more stable gradients during training. The output is the encoded representation of the input elements. Before normalization (e.g., by a softmax function), the matrix form in parentheses can be decomposed, where the unnormalized attention score between any two nodes i and j is given by:
[0089]
[0090] Here, H represents the input vector (e.g., vectors 206A to 206E), Q represents the query, and K represents the key, which is the token in the query.
[0091] Therefore, when encoding the target vector, the equation calculates the attention score between the target vector and other vectors.
[0092] In related technologies, these attention scores can be used to calculate the weight sum with the target value V corresponding to the target vector, serving as the encoded representation of the target vector. However, because there is no data flow between two unconnected nodes, and the attention mechanism itself lacks the ability to capture information about the internal structure of the DAG, in this embodiment, for a given pair of nodes i and j, attention scores are only calculated if an edge connecting these two nodes exists.
[0093] Therefore, in step 226, the second self-attention score corresponding to the target node can be determined based on the adjacency matrix 210 and the first self-attention score.
[0094] Here, adjacency matrix 210 is obtained through DAG analysis. In adjacency matrix 210, if node i and node j are connected by an edge, the item bij in adjacency matrix 210 is set to 1; otherwise, it is 0. This matrix is used to mask the attention score, i.e., the attention score is equal to Aij multiplied by bij. Therefore, the attention score of any unconnected node pair is set to 0, so that subsequent calculations based on these attention scores are unnecessary, saving computational resources.
[0095] In step 228, the target self-attention code corresponding to the target node can be determined based on the second self-attention score.
[0096] Here, since the second self-attention score is obtained by processing the first self-attention score through the adjacency matrix 210 mask, when performing weighted calculations on the target value corresponding to the target vector, the target self-attention code corresponding to the target node can be determined based on the second self-attention score and the target value of the target vector.
[0097] In this way, the self-attention encoding is obtained by using the above method for the vector corresponding to each node, thereby obtaining the target encoding 212 corresponding to the query request 202.
[0098] As can be seen from the above embodiments, the query request encoding method provided in this disclosure proposes a database query plan encoding scheme based on the Attention mechanism, which can encode information such as the operation type and table name cost of the query plan node, and is applicable to query plans with a large number of nodes and complex relationships between nodes. It can fully encode the information of the query execution process, and still has a good encoding effect for query plans with a large number of nodes and complex relationships between nodes.
[0099] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.
[0100] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0101] This disclosure also provides an encoding method for query features. Figure 3 A flowchart illustrating an exemplary method 300 provided in an embodiment of this disclosure is shown. This method 300 can be... Figure 1 It can be implemented by server 106, or by server 106 and database server 108 together, or by... Figure 1 The system is 100% implemented. For example... Figure 3 As shown, the method 300 may include the following steps.
[0102] In step 302, a query request is received (e.g., Figure 2A Query request 202);
[0103] In step 304, obtain the query plan corresponding to the query request (e.g., Figure 2A The query plan 204 includes multiple nodes and the connection relationships between the multiple nodes;
[0104] In step 306, the plurality of nodes are encoded respectively to obtain a plurality of vectors (e.g., Figure 2A Vectors 206A~206E);
[0105] In step 308, based on the connection relationships between the plurality of nodes and the plurality of vectors, the target encoding corresponding to the query request is determined (e.g., Figure 2A Target encoding 212).
[0106] The query request encoding method provided in this disclosure generates multiple vectors based on multiple nodes of the query plan, and combines the connection relationship information between nodes in the multiple vectors, so that the target encoding can contain richer query information and improve query accuracy.
[0107] In some embodiments, the node includes multiple descriptive information of the node associated with the query request;
[0108] Encode each of the multiple nodes to obtain multiple vectors, including: for each node, perform the following steps:
[0109] The multiple description information of the node is converted into multiple one-hot vectors;
[0110] The vector corresponding to the node is obtained by connecting the multiple one-hot vectors.
[0111] In this way, by encoding the descriptive information into one-hot vectors, each of the vectors (e.g., Figure 2A The lengths of the vectors (206A to 206E) are kept consistent to facilitate subsequent attention score calculations.
[0112] In some embodiments, concatenating the plurality of one-hot vectors to obtain the vector corresponding to the node includes:
[0113] The multiple one-hot vectors are connected in a predetermined order to obtain the vector corresponding to the node;
[0114] The predetermined order corresponding to each of the nodes is the same.
[0115] This ensures accuracy when calculating self-attention scores.
[0116] In some embodiments, the plurality of descriptive information includes at least one of operators, joins, scan tables, predicates, and cardinality estimates. Thus, the vector encoded based on this descriptive information can contain more information about the query plan, thereby improving query accuracy.
[0117] In some embodiments, determining the target encoding corresponding to the query request based on the connection relationships between the plurality of nodes and the plurality of vectors includes:
[0118] Based on the connection relationships between the multiple nodes, self-attention scores are calculated for the multiple vectors to obtain multiple self-attention codes corresponding to the multiple nodes;
[0119] The target code corresponding to the query request is determined based on the multiple self-attention codes.
[0120] In this way, multiple vectors are generated based on the multiple nodes of the query plan, and the connection relationship information between nodes is combined into these vectors. This allows the target encoding to contain richer query information, thereby improving query accuracy. In particular, this method is suitable for query plans with a large number of nodes and complex relationships between them, where the improvement in accuracy is more significant.
[0121] In some embodiments, obtaining the query plan corresponding to the query request further includes:
[0122] Convert the query plan into a directed acyclic graph (see reference). Figure 2A As shown, the directed acyclic graph includes the plurality of nodes;
[0123] Based on the directed acyclic graph, determine the adjacency matrix (e.g., Figure 2A The adjacency matrix 210 is used to indicate the connection relationship between the plurality of nodes.
[0124] In this way, information about the query execution process can be extracted from the directed acyclic graph and incorporated into the target encoding. This allows the target encoding to include relevant information about the query execution process, improving query accuracy. During subsequent encoding, the adjacency matrix can be used to mask the pre-encoding, further integrating the connection relationships between nodes into the encoding and improving query accuracy.
[0125] In some embodiments, self-attention scores are calculated on the plurality of vectors based on the connection relationships between the plurality of nodes to obtain plurality of self-attention codes corresponding to the plurality of nodes, including:
[0126] Identify the target node among the plurality of nodes;
[0127] Calculate the self-attention score between the target vector corresponding to the target node and the other vectors among the plurality of vectors excluding the target node, to obtain the first self-attention score corresponding to the target node;
[0128] Based on the adjacency matrix and the first self-attention score, determine the second self-attention score corresponding to the target node;
[0129] Based on the second self-attention score, the target self-attention code corresponding to the target node is determined.
[0130] In this way, the attention scores are masked using the adjacency matrix. The attention scores of any unconnected node pairs are set to 0, so that subsequent calculations based on these attention scores are not required, thus saving computational resources.
[0131] In some embodiments, calculating the self-attention score between the target vector corresponding to the target node and other vectors among the plurality of vectors excluding the target node, to obtain the first self-attention score corresponding to the target node includes:
[0132] Convert the multiple vectors into multiple queries, multiple keys, and multiple values;
[0133] The first self-attention score corresponding to the target node is calculated based on the target query and target key of the target vector of the target node, as well as multiple other queries and multiple other keys of the other vectors.
[0134] The self-attention mechanism allows the model to automatically focus on different parts of the input based on attention scores and assign different weights according to their importance when processing data. Unlike traditional attention mechanisms, self-attention primarily focuses on the relationships within the input itself. This allows the model to capture long-range dependencies between multiple nodes, rather than being limited to local contextual information, which is very helpful for understanding complex language structures and semantic relationships. Furthermore, by incorporating these connections, the self-attention scores of nodes with connections can be introduced into the target encoding, enriching the encoded information while maintaining conciseness. In addition, the self-attention mechanism can be computed in parallel, greatly improving computational efficiency. Simultaneously, it can adaptively learn the relationships between different nodes without the need for manually designing complex feature extractors, making the model more flexible and powerful, and better able to handle various complex tasks and data. Moreover, self-attention provides a degree of interpretability to the model; by observing the distribution of attention weights, one can understand the key regions the model focuses on when processing data.
[0135] In some embodiments, determining the target self-attention code corresponding to the target node based on the second self-attention score includes:
[0136] Based on the second self-attention score and the target value of the target vector, the target self-attention code corresponding to the target node is determined.
[0137] Since the second self-attention score is obtained by processing the first self-attention score through the adjacency matrix mask, when performing weighted calculations on the target value corresponding to the target vector, the target self-attention code corresponding to the target node can be determined based on the second self-attention score and the target value of the target vector, which reduces the amount of computation, saves resources, and improves computational efficiency.
[0138] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.
[0139] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0140] This disclosure also provides a data query method. Figure 4 A flowchart illustrating an exemplary method 400 provided in an embodiment of this disclosure is shown. Method 400 can be... Figure 1 It can be implemented by server 106, or by server 106 and database server 108 together, or by... Figure 1 The system is 100% implemented. For example... Figure 4 As shown, the method 400 may include the following steps.
[0141] In step 402, a query request for the target data is received (e.g., Figure 2A The query request 202 is used to request the query of the target data in the target database.
[0142] In step 404, using any embodiment or arrangement / combination of embodiments of method 300, the query request is encoded to obtain the target code corresponding to the query request (e.g., Figure 2A Target encoding 212).
[0143] In step 406, based on the target encoding, the target database is determined (e.g., Figure 1 At least one target database parameter of the database provided by the database server 108.
[0144] In step 408, based on the at least one target database parameter, at least one database parameter of the target database is adjusted.
[0145] In step 410, based on the query request, the target data is queried in the target database after parameter adjustment.
[0146] The data query method provided in this disclosure generates multiple vectors based on multiple nodes of the query plan, and combines the connection relationship information between nodes in the multiple vectors, so that the target encoding can contain richer query information and improve query accuracy.
[0147] In some embodiments, determining at least one target database parameter of the target database based on the target encoding includes: inputting the target encoding into a parameter generation model and outputting at least one target database parameter of the target database. Since the target encoding contains richer query information, the parameter generation model can output more accurate target database parameters, thereby improving query accuracy after parameter adjustment of the target database.
[0148] Optionally, the parameter generation model can also be pre-trained. The pre-training training data can be multiple codes obtained by encoding multiple historical query requests using any embodiment or arrangement / combination of the embodiments of method 300. These codes, along with the database parameters corresponding to the historical query requests, are used as training data. Then, the initial model is trained using this training data to obtain the parameter generation model. In this way, the model trained using multiple codes obtained by encoding multiple historical query requests using any embodiment or arrangement / combination of the embodiments of method 300 can make the output database parameters more closely match the query requests, improving query accuracy. It is understood that the parameter generation model can be any machine learning model, such as a neural network model, a deep neural network model, a reinforcement learning model, a deep reinforcement learning model, etc.
[0149] This disclosure also provides a computer device for implementing the above-described method 300 or method 400. Figure 5 A schematic diagram of the hardware structure of an exemplary computer device 500 provided in an embodiment of this disclosure is shown. The computer device 500 can be used to implement... Figure 1 Server 106 can also be used to implement Figure 1 Terminal devices 102 and 104. In some scenarios, this computer device 500 can also be used to implement... Figure 1 Database server 108.
[0150] like Figure 5 As shown, the computer device 500 may include: a processor 502, a memory 504, a network module 506, a peripheral interface 508, and a bus 510. The processor 502, memory 504, network module 506, and peripheral interface 508 are interconnected within the computer device 500 via the bus 510.
[0151] Processor 502 may be a central processing unit (CPU), image processor, neural network processor (NPU), microcontroller (MCU), programmable logic device, digital signal processor (DSP), application-specific integrated circuit (ASIC), or one or more integrated circuits. Processor 502 can be used to perform functions related to the techniques described in this disclosure. In some embodiments, processor 502 may also include multiple processors integrated as a single logic component. For example, such as... Figure 5 As shown, processor 502 may include multiple processors 502a, 502b and 502c.
[0152] Memory 504 can be configured to store data (e.g., instructions, computer code, etc.). Figure 5 As shown, the data stored in memory 504 may include program instructions (e.g., program instructions for implementing method 300 or method 400 of the embodiments of this disclosure) and data to be processed (e.g., the memory may store configuration files of other modules, etc.). Processor 502 may also access the program instructions and data stored in memory 504 and execute the program instructions to operate on the data to be processed. Memory 504 may include volatile storage devices or non-volatile storage devices. In some embodiments, memory 504 may include random access memory (RAM), read-only memory (ROM), optical disk, magnetic disk, hard disk, solid-state drive (SSD), flash memory, memory stick, etc.
[0153] Network interface 506 can be configured to provide communication with other external devices to computer device 500 via a network. This network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, Near Field Communication (NFC), etc.), a cellular network, the Internet, or a combination thereof. It is understood that the type of network is not limited to the specific examples described above.
[0154] The peripheral interface 508 can be configured to connect the computer device 500 to one or more peripheral devices to enable information input and output. For example, peripheral devices may include input devices such as keyboards, mice, touchpads, touch screens, microphones, and various sensors, as well as output devices such as displays, speakers, vibrators, and indicator lights.
[0155] Bus 510 can be configured to transfer information between various components of computer device 500 (e.g., processor 502, memory 504, network interface 506, and peripheral interface 508), such as internal buses (e.g., processor-memory bus), external buses (USB port, PCI-E bus), etc.
[0156] It should be noted that although the architecture of the computer device 500 described above only shows the processor 502, memory 504, network interface 506, peripheral interface 508, and bus 510, in specific implementations, the architecture of the computer device 500 may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the architecture of the computer device 500 described above may only include the components necessary for implementing the embodiments of this disclosure, and does not necessarily include all the components shown in the figures.
[0157] This disclosure also provides an encoding device for query requests. Figure 6 A schematic diagram of an exemplary apparatus 600 provided in an embodiment of this disclosure is shown. For example... Figure 6 As shown, the device 600 can be used to implement method 300 and may further include the following modules.
[0158] The receiving module 602 is configured to receive query requests;
[0159] The acquisition module 604 is configured to: acquire the query plan corresponding to the query request, wherein the query plan includes multiple nodes and the connection relationship between the multiple nodes;
[0160] Encoding module 606 is configured to encode the plurality of nodes respectively to obtain a plurality of vectors;
[0161] The determination module 608 is configured to: determine the target encoding corresponding to the query request based on the connection relationship between the plurality of nodes and the plurality of vectors.
[0162] In some embodiments, the node includes multiple descriptive information of the node associated with the query request;
[0163] Encoding module 606 is configured as follows:
[0164] For each of the nodes, perform the following steps:
[0165] The multiple description information of the node is converted into multiple one-hot vectors;
[0166] The vector corresponding to the node is obtained by connecting the multiple one-hot vectors.
[0167] In some embodiments, the encoding module 606 is configured to:
[0168] The multiple one-hot vectors are connected in a predetermined order to obtain the vector corresponding to the node;
[0169] The predetermined order corresponding to each of the nodes is the same.
[0170] In some embodiments, the plurality of descriptive information includes at least one of operators, joins, scan tables, predicates, and cardinality estimates.
[0171] In some embodiments, the determining module 608 is configured to:
[0172] Based on the connection relationships between the multiple nodes, self-attention scores are calculated for the multiple vectors to obtain multiple self-attention codes corresponding to the multiple nodes;
[0173] The target code corresponding to the query request is determined based on the multiple self-attention codes.
[0174] In some embodiments, the acquisition module 604 is configured to:
[0175] The query plan is converted into a directed acyclic graph, which includes the plurality of nodes;
[0176] Based on the directed acyclic graph, an adjacency matrix is determined, which is used to indicate the connection relationships between the plurality of nodes.
[0177] In some embodiments, the determining module 608 is configured to:
[0178] Identify the target node among the plurality of nodes;
[0179] Calculate the self-attention score between the target vector corresponding to the target node and the other vectors among the plurality of vectors excluding the target node, to obtain the first self-attention score corresponding to the target node;
[0180] Based on the adjacency matrix and the first self-attention score, determine the second self-attention score corresponding to the target node;
[0181] Based on the second self-attention score, the target self-attention code corresponding to the target node is determined.
[0182] In some embodiments, the determining module 608 is configured to:
[0183] Convert the multiple vectors into multiple queries, multiple keys, and multiple values;
[0184] The first self-attention score corresponding to the target node is calculated based on the target query and target key of the target vector of the target node, as well as multiple other queries and multiple other keys of the other vectors.
[0185] In some embodiments, the determining module 608 is configured to:
[0186] Based on the second self-attention score and the target value of the target vector, the target self-attention code corresponding to the target node is determined.
[0187] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.
[0188] The apparatus of the above embodiments is used to implement the corresponding method 300 in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0189] This disclosure also provides a data query device. Figure 7 A schematic diagram of an exemplary apparatus 700 provided in an embodiment of this disclosure is shown. For example... Figure 7 As shown, the device 700 can be used to implement method 400 and may further include the following modules.
[0190] The receiving module 702 is configured to receive a query request for target data, wherein the query request is used to request a query for the target data in the target database;
[0191] The encoding module 704 is configured to: encode the query request to obtain the target code corresponding to the query request by using any embodiment or arrangement and combination of the embodiments of the method 300;
[0192] The determining module 706 is configured to: determine at least one target database parameter of the target database based on the target encoding;
[0193] The adjustment module 708 is configured to: adjust at least one database parameter of the target database based on the at least one target database parameter;
[0194] The query module 710 is configured to query the target data in the target database after parameter adjustment based on the query request.
[0195] In some embodiments, the determining module 706 is configured to: generate a model from the target encoding input parameters and output at least one target database parameter to obtain the target database.
[0196] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.
[0197] The apparatus of the above embodiments is used to implement the corresponding method 400 in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0198] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method 300 or method 400 as described in any of the above embodiments.
[0199] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.
[0200] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the method 300 or method 400 as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0201] Based on the same inventive concept, corresponding to method 300 or method 400 in any of the above embodiments, this disclosure also provides a computer program product, which includes a computer program. In some embodiments, the computer program is executable by one or more processors to cause the processors to perform the method 300 or method 400. Corresponding to the execution entity for each step in each embodiment of method 300 or method 400, the processor executing the corresponding step may belong to the corresponding execution entity.
[0202] The computer program product of the above embodiments is used to cause a processor to execute the method 300 or method 400 as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0203] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.
[0204] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this disclosure, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this disclosure, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this disclosure will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuitry) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this disclosure may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0205] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.
[0206] This disclosure is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A method for encoding a query request, comprising: Receive a query request, the query request being used to request the query of the target data in the target database; Obtain the query plan corresponding to the query request. The query plan is generated based on the query request and includes steps for executing the query request. The query plan is converted into multiple nodes and the connection relationship between the multiple nodes. The nodes correspond to the query steps of the query plan, and the connection relationship is used to indicate the execution order of the query steps corresponding to the multiple nodes. Encode each of the multiple nodes to obtain multiple vectors; Based on the connection relationships between the multiple nodes and the multiple vectors, the target encoding corresponding to the query request is determined; The target code is used to determine the target database parameters, and the target database parameters are used to adjust the database parameters of the target database; The node includes multiple descriptive information about the node related to the query request; Encode each of the multiple nodes to obtain multiple vectors, including: for each node, perform the following steps: The multiple description information of the node is converted into multiple one-hot vectors; The vector corresponding to the node is obtained by connecting the multiple one-hot vectors. Specifically, determining the target encoding corresponding to the query request based on the connection relationships between the plurality of nodes and the plurality of vectors includes: Based on the connection relationships between the multiple nodes, self-attention scores are calculated for the multiple vectors to obtain multiple self-attention codes corresponding to the multiple nodes; Based on the multiple self-attention codes, the target code corresponding to the query request is determined; Obtaining the query plan corresponding to the query request further includes: The query plan is converted into a directed acyclic graph, which includes the plurality of nodes; Based on the directed acyclic graph, an adjacency matrix is determined, which is used to indicate the connection relationships between the plurality of nodes.
2. The method as described in claim 1, wherein, The vector corresponding to the node is obtained by concatenating the multiple one-hot vectors, including: The multiple one-hot vectors are connected in a predetermined order to obtain the vector corresponding to the node; The predetermined order corresponding to each of the nodes is the same.
3. The method as described in claim 1, wherein, The plurality of descriptive information includes at least one of operators, joins, scan tables, predicates, and cardinality estimates.
4. The method of claim 1, wherein, Based on the connection relationships between the multiple nodes, self-attention scores are calculated for the multiple vectors to obtain multiple self-attention codes corresponding to the multiple nodes, including: Identify the target node among the plurality of nodes; Calculate the self-attention score between the target vector corresponding to the target node and the other vectors among the plurality of vectors excluding the target node, to obtain the first self-attention score corresponding to the target node; Based on the adjacency matrix and the first self-attention score, determine the second self-attention score corresponding to the target node; Based on the second self-attention score, the target self-attention code corresponding to the target node is determined.
5. The method of claim 4, wherein, Calculate the self-attention scores between the target vector corresponding to the target node and the other vectors among the plurality of vectors excluding the target node, to obtain the first self-attention score corresponding to the target node, including: Convert the multiple vectors into multiple queries, multiple keys, and multiple values; The first self-attention score corresponding to the target node is calculated based on the target query and target key of the target vector of the target node, as well as multiple other queries and multiple other keys of the other vectors.
6. The method of claim 5, wherein, Based on the second self-attention score, the target self-attention code corresponding to the target node is determined, including: Based on the second self-attention score and the target value of the target vector, the target self-attention code corresponding to the target node is determined.
7. A data query method, comprising: Receive a query request for target data, the query request being used to request the query for the target data in the target database; The query request is encoded to obtain the target code corresponding to the query request by employing the method described in any one of claims 1 to 6; Based on the target encoding, at least one target database parameter of the target database is determined; Based on the at least one target database parameter, adjust at least one database parameter of the target database; Based on the query request, the target data is queried in the target database after parameter adjustment.
8. The method of claim 7, wherein, Based on the target encoding, at least one target database parameter of the target database is determined, including: The target encoding input parameters are used to generate a model, and at least one target database parameter is output to obtain the target database.
9. An encoding device for a query request, comprising: The receiving module is configured to receive a query request, wherein the query request is used to request the query of the target data in the target database; The acquisition module is configured to: acquire the query plan corresponding to the query request, the query plan being generated based on the query request and including steps for executing the query request, the query plan being converted into multiple nodes and connection relationships between the multiple nodes, the nodes corresponding to the query steps of the query plan, and the connection relationships being used to indicate the execution order of the query steps corresponding to the multiple nodes; The encoding module is configured to encode the plurality of nodes respectively to obtain a plurality of vectors; The determination module is configured to: determine the target encoding corresponding to the query request based on the connection relationship between the plurality of nodes and the plurality of vectors; The target code is used to determine the target database parameters, and the target database parameters are used to adjust the database parameters of the target database; The node includes multiple descriptive information about the node related to the query request; Encode each of the multiple nodes to obtain multiple vectors, including: for each node, perform the following steps: The multiple description information of the node is converted into multiple one-hot vectors; The vector corresponding to the node is obtained by connecting the multiple one-hot vectors. Specifically, determining the target encoding corresponding to the query request based on the connection relationships between the plurality of nodes and the plurality of vectors includes: Based on the connection relationships between the multiple nodes, self-attention scores are calculated for the multiple vectors to obtain multiple self-attention codes corresponding to the multiple nodes; Based on the multiple self-attention codes, the target code corresponding to the query request is determined; Obtaining the query plan corresponding to the query request further includes: The query plan is converted into a directed acyclic graph, which includes the plurality of nodes; Based on the directed acyclic graph, an adjacency matrix is determined, which is used to indicate the connection relationships between the plurality of nodes.
10. A data query device, comprising: The receiving module is configured to receive a query request for target data, wherein the query request is used to request a query for the target data in the target database; The encoding module is configured to: encode the query request to obtain the target encoding corresponding to the query request using the method described in any one of claims 1 to 6; The determining module is configured to: determine at least one target database parameter of the target database based on the target encoding; The adjustment module is configured to: adjust at least one database parameter of the target database based on the at least one target database parameter; The query module is configured to: query the target data in the target database after parameter adjustment based on the query request.
11. A computer device comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the programs comprising instructions for performing the method according to any one of claims 1 to 6 or the method according to claim 7 or 8.
12. A non-volatile computer-readable storage medium comprising a computer program, which, when executed by one or more processors, causes the processors to perform the method of any one of claims 1 to 6 or the method of claim 7 or 8.
13. A computer program product comprising computer program instructions that, when executed on a computer, cause the computer to perform the method as claimed in any one of claims 1 to 6 or the method as claimed in claim 7 or 8.
Citation Information
Patent Citations
Data query method and device, terminal equipment and storage medium
CN116775724A
Reading sequence extraction method and device, computer readable storage medium and system
CN118334688A
