Data query and model training method and device, equipment and storage medium

Abstract

The application provides a data query and model training method and device, equipment and a storage medium, which can be applied to various scenes such as databases. The method comprises the following steps: in response to a received query request, obtaining a resource load index of a database system at the current time; generating N candidate query plans of the query request; selecting a target query plan from the N candidate query plans based on the resource load index; and based on the target query plan, performing data query in the database to obtain a query result of the query request. That is, the application generates N candidate query plans of the query request, and then selects a target query plan that matches the load status of the database system at the current time from the N candidate query plans based on the resource load index of the database system at the current time, thereby avoiding the high latency problem caused by queuing and waiting, and significantly improving the system operation efficiency and stability, thereby improving the data query efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data query and model training method, apparatus, device and storage medium. Background Technology

[0002] In data query scenarios, when a user submits a database query request, the database system's query optimizer generates a query plan, which determines the data access path (such as index selection, table join order, etc.).

[0003] The current data query method involves parsing the query request submitted by an object, generating a query plan, and estimating the resources required to execute the query request based on this plan. If the available resources of the database system are greater than the resources required for the query request, the query request is initiated. However, if the available resources of the database system are less than the resources required for the query request, the query request is queued and placed to wait. This increases the data query waiting time and thus reduces the data query efficiency. Summary of the Invention

[0004] This application provides a data query and model training method, apparatus, device, and storage medium, which can improve the efficiency of data query.

[0005] Firstly, this application provides a data query method, including:

[0006] In response to a received query request, obtain the resource load indicators of the database system at the current moment;

[0007] Generate N candidate query plans for the query request, where N is a positive integer greater than 1;

[0008] Based on the resource load metric, a target query plan is selected from the N candidate query plans;

[0009] Based on the target query plan, a data query is performed in the database to obtain the query results of the query request.

[0010] Secondly, this application provides a model training method, including:

[0011] Obtain training samples, which include: a set of resource load indicators of the database system and a candidate query plan. The candidate query plan is any one of N candidate query plans for a historical query request. The resource load indicators are a set of P sets of resource load indicators of the database system, where N and P are both positive integers greater than 1.

[0012] Extract the feature information of the candidate query plan;

[0013] By using a query plan prediction model, the feature information of the candidate query plan and the resource load index are fused to obtain the estimated query duration of the candidate query plan.

[0014] Based on the estimated query duration of the candidate query plans, the model loss is determined, and the query plan prediction model is trained based on the model loss.

[0015] Thirdly, this application provides a data query device, comprising:

[0016] The acquisition unit is used to obtain the resource load indicators of the database system at the current moment in response to the received query request;

[0017] A generation unit is used to generate N candidate query plans for the query request, where N is a positive integer greater than 1;

[0018] The selection unit is used to select a target query plan from the N candidate query plans based on the resource load index.

[0019] The query unit is used to perform data queries in the database based on the target query plan to obtain the query results of the query request.

[0020] In some embodiments, the selection unit is specifically used to determine the query duration of each of the N candidate query plans based on the resource load index; and to select the target query plan from the N candidate query plans based on the query duration.

[0021] In some embodiments, the resource load metric includes at least one of the system-level bottleneck metric and the key time-consuming metric of the database system, wherein the key time-consuming metric refers to the time-consuming metric of each key step in the execution of the query plan by the database system.

[0022] In some embodiments, if the resource load indicators include system-level bottleneck indicators of the database system, the acquisition unit is specifically used to acquire the collected system-level bottleneck indicators of the database system at different times in the current time period; for each of the system-level bottleneck indicators, the average value of the indicator at different times in the current time period is determined as the value of the indicator at the current time.

[0023] In some embodiments, if the resource load indicators include key time-consuming indicators of the database system, the acquisition unit is specifically used to acquire the collected key time-consuming indicators of the database system when executing different query plans in the current time period; for each of the key time-consuming indicators, the average value of the indicator in the current time period is determined as the value of the indicator at the current moment.

[0024] In some embodiments, the selection unit is specifically used to extract feature information of the i-th candidate query plan from the N candidate query plans, where i is a positive integer from 1 to N; and to fuse the feature information of the i-th candidate query plan and the resource load index through a query plan prediction model to obtain the query duration of the i-th candidate query plan.

[0025] In some embodiments, the selection unit is specifically used to standardize and normalize each of the resource load indicators to obtain resource load coding information; and to fuse the feature information of the i-th candidate query plan and the resource load coding information through a query plan prediction model to obtain the query duration of the i-th candidate query plan.

[0026] In some embodiments, the query plan prediction model includes a cross-attention module and a selection unit, specifically used to perform fusion processing on the feature information of the i-th candidate query plan and the resource load encoding information through the cross-attention module to obtain the cross-attention result of the i-th candidate query plan; and to determine the query duration of the i-th candidate query plan based on the cross-attention result of the i-th candidate query plan.

[0027] In some embodiments, the selection unit is specifically used to multiply the feature information of the i-th candidate query plan with the key weight matrix and value weight matrix in the cross-attention module respectively to obtain the key matrix and value matrix corresponding to the i-th candidate query plan; multiply the resource load encoding information with the query weight matrix in the cross-attention module to obtain the query matrix corresponding to the i-th candidate query plan; and perform attention calculation on the key matrix, value matrix and query matrix corresponding to the i-th candidate query plan to obtain the cross-attention result of the i-th candidate query plan.

[0028] In some embodiments, the cross-attention module further includes a fully connected layer and a selection unit, specifically used to process the cross-attention result of the i-th candidate query plan through the fully connected layer to obtain the query duration of the i-th candidate query plan.

[0029] In some embodiments, the selection unit is specifically used to extract features from the i-th candidate query plan using a feature extraction model to obtain feature information of the i-th candidate query plan.

[0030] In some embodiments, the feature extraction model includes a tree convolutional network and a pooling layer. The selection unit is specifically used to capture the hierarchical relationship and semantic information in the tree structure of the i-th candidate query plan through the tree convolutional network to obtain the structural semantic information of the i-th candidate query plan; and to process the structural semantic information of the i-th candidate query plan through the pooling layer to obtain the feature information of the i-th candidate query plan.

[0031] In some embodiments, the generation unit is specifically used to generate N candidate query plans for the query request through an optimizer, wherein the optimizer is trained based on N different prompts, and the prompts are used to describe the requirements that the candidate query plans need to meet.

[0032] Thirdly, this application provides a model training apparatus, comprising:

[0033] The acquisition unit is used to acquire training samples, which include: a set of resource load indicators of the database system and a candidate query plan. The candidate query plan is any one of N candidate query plans for a historical query request. The resource load indicators are a set of P sets of resource load indicators of the database system, where N and P are both positive integers greater than 1.

[0034] Extraction unit, used to extract feature information of the candidate query plan;

[0035] The estimation unit is used to fuse the feature information of the candidate query plan and the resource load index through a query plan estimation model to obtain the estimated query duration of the candidate query plan.

[0036] The training unit is used to determine the model loss based on the estimated query duration of the candidate query plan, and to train the query plan prediction model based on the model loss.

[0037] In some embodiments, the training unit is specifically used to determine the actual query duration of the candidate query plan under the resource load metric; and to determine the model loss based on the estimated query duration and the actual query duration of the candidate query plan.

[0038] In some embodiments, the acquisition unit is further configured to generate the P group resource load index by adjusting the state and database configuration of the database system.

[0039] Fourthly, this application provides a computing device including a processor and a memory. The memory is used to store a computer program, and the processor is used to invoke and run the computer program stored in the memory to perform the methods described in the first or second aspect.

[0040] Fifthly, a chip is provided for implementing the methods of the first or second aspect described above. Specifically, the chip includes a processor for retrieving and running a computer program from a memory, causing a device equipped with the chip to perform the methods of the first or second aspect described above.

[0041] In a sixth aspect, a computer-readable storage medium is provided for storing a computer program that causes a computer to perform the methods described in the first or second aspect.

[0042] In a seventh aspect, a computer program product is provided, including computer program instructions that cause a computer to perform the methods described in the first or second aspect.

[0043] Eighthly, a computer program is provided that, when run on a computer, causes the computer to perform the methods described in the first or second aspect.

[0044] In summary, this application, in response to a received query request, obtains the resource load index of the database system at the current moment, where the query request is used to request data in the database. Next, N candidate query plans are generated for the query request, where N is a positive integer greater than 1. Then, based on the resource load index, a target query plan is selected from the N candidate query plans. Finally, based on the target query plan, a data query is performed in the database to obtain the query result of the query request. That is, this application embodiment generates N candidate query plans for the query request, and then, based on the resource load index of the database system at the current moment, selects a target query plan from these N candidate query plans that matches the current load status of the database system. In other words, when faced with a query request with insufficient resources, this application embodiment can select a suitable target query plan from the N candidate query plans to continue execution, avoiding high latency caused by queuing, thereby significantly improving system operating efficiency and stability, meeting the requirements for efficient and stable operation of the database system, and thus improving the efficiency of data querying. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of an implementation environment of an embodiment of this application;

[0047] Figure 2 A schematic flowchart illustrating a data query method provided in an embodiment of this application;

[0048] Figure 3 A schematic diagram of a system-level bottleneck indicator;

[0049] Figure 4 A diagram illustrating key time-consuming metrics;

[0050] Figure 5 A schematic diagram of resource load indicators;

[0051] Figure 6 A schematic diagram for generating N candidate query plans;

[0052] Figure 7 A schematic diagram for extracting feature information;

[0053] Figure 8 This is a schematic diagram of a network for a feature extraction model;

[0054] Figure 9 A diagram illustrating the prediction of query duration;

[0055] Figure 10 Another illustration for predicting query duration;

[0056] Figure 11 A network diagram for a query plan prediction model;

[0057] Figure 12 This is another network diagram for the query plan prediction model;

[0058] Figure 13 A network diagram illustrating a cross-attention module;

[0059] Figure 14 This is a schematic diagram of a data query method provided in an embodiment of this application;

[0060] Figure 15 A schematic diagram illustrating the framework of the data query method provided in the embodiments of this application;

[0061] Figure 16 This is a schematic diagram of a model training method provided in an embodiment of this application;

[0062] Figure 17 This is a schematic block diagram of a data query device provided in an embodiment of this application;

[0063] Figure 18 This is a schematic block diagram of a model training apparatus provided in an embodiment of this application;

[0064] Figure 19 This is a schematic block diagram of a computing device provided in an embodiment of this application. Detailed Implementation

[0065] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0066] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. In embodiments of the invention, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0067] The data query and model training method provided in this application can be applied to various fields such as databases, data querying, and intelligent retrieval. Based on the resource load index of the database system at the current moment, it selects a candidate query plan from N candidate query plans that matches the current load status of the database system as the target query plan. This improves the accuracy of the target query plan determination. Based on this accurately determined target query plan, data queries are performed, avoiding high latency issues caused by queuing, significantly improving system operating efficiency and stability, meeting the requirements for efficient and stable database system operation, and thus improving data query efficiency.

[0068] To facilitate understanding of the embodiments of this application, the relevant concepts involved in the embodiments of this application will first be introduced:

[0069] A database system (DBS) is a relatively ideal data processing system developed to meet the needs of data processing. It is also a software system that provides data for practical, operational storage, maintenance, and application systems; it is a collection of storage media, processing objects, and management systems. A database system typically consists of software, a database, and a data administrator. Its software mainly includes the operating system, various host languages, utilities, and the database management system. The database is managed uniformly by the database management system; data insertion, modification, and retrieval are all performed through the database management system. The data administrator is responsible for creating, monitoring, and maintaining the entire database, ensuring that the data can be effectively used by anyone with the appropriate authority. Database administrators are generally highly skilled and experienced personnel.

[0070] Structured Query Language (SQL) is a database query and programming language used to access, query, update, and manage relational database systems. SQL statements are a language for manipulating databases. For example, the SQL statement to create a database is: `CREATE DATABASE database-name`; the SQL statement to delete a database is: `drop database dbname`; the SQL statement to create a new table is: `createtable tabname(col1 type1[not null][primary key],col2 type2[not null],...)`; the SQL statement to delete a new table is: `drop table tabname`; and the SQL statement to add a column is: `ALTER TABLE tabname ADD COLUMN col type`.

[0071] Load awareness refers to the ability of a system or model to identify, understand, and respond to different load conditions. It can collect various load-related information, such as system resource usage and workload characteristics, and then make corresponding adjustments and decisions based on this information to achieve the goal of optimizing performance and improving efficiency.

[0072] An SQL query plan is a set of operational procedures and strategies for a database to execute SQL queries. It details how the database accesses table data, including which indexes to use, which table join method to employ (such as nested loop joins or hash joins), and how to perform data filtering and sorting, directly impacting query execution efficiency.

[0073] Cross-attention is a special attention mechanism in deep learning. It enables interaction and information fusion between different feature representations. By calculating the correlation between different features and assigning corresponding attention weights to each feature, it can more accurately capture the relationships between features and improve the model's performance when handling complex tasks.

[0074] Tree-based Convolutional Neural Networks (TCNNs) are a variant of convolutional neural networks specifically designed for processing tree-structured data. They can automatically extract hierarchical relationships and semantic information from tree structures, performing feature extraction and representation learning on the tree structure through convolutional operations, thereby enabling effective analysis and processing of tree-structured data.

[0075] A buffer is a memory area used for temporary data storage. Its function is to coordinate the differences in data transfer speeds between different devices or components, preventing data loss and processing delays. By caching data, redundant data reads and writes can be reduced, improving overall system performance.

[0076] Optimizer switches are settings options in databases or other systems that control the behavior of the optimizer. By turning different switches on or off, you can change the optimizer's strategies and preferences when generating execution plans, thereby affecting how the system processes queries and its performance.

[0077] The current data query method involves parsing the query request submitted by an object, generating a query plan, and estimating the resources required to execute the query request based on this plan. If the available resources of the database system are greater than the resources required for the query request, the query request is initiated. However, if the available resources of the database system are less than the resources required for the query request, the query request is queued and placed to wait. This increases the data query waiting time and thus reduces the data query efficiency.

[0078] To address this technical problem, this application provides a novel data query method. In response to a received query request, the method obtains the resource load index of the database system at the current moment, where the query request is used to request data from the database. Next, N candidate query plans are generated for the query request, where N is a positive integer greater than 1. Then, based on the resource load index, a target query plan is selected from the N candidate query plans. Finally, based on the target query plan, a data query is performed in the database to obtain the query result. In other words, this application generates N candidate query plans for the query request and then selects a target query plan that matches the current database system load status from these N candidate query plans based on the current database system resource load index. That is, when faced with a query request with insufficient resources, this application can select a suitable target query plan from the N candidate query plans to continue execution, avoiding high latency caused by queuing, thereby significantly improving system operating efficiency and stability, meeting the requirements for efficient and stable operation of the database system, and thus improving the efficiency of data querying.

[0079] The implementation environment of the embodiments of this application is described below.

[0080] Figure 1 This is a schematic diagram of an implementation environment of an embodiment of this application, such as... Figure 1 As shown, the implementation environment includes a terminal device 101 and a database system 102. The terminal device 101 and the database system 102 are connected via wired or wireless means.

[0081] In this embodiment, an object (e.g., a user) can query data from the database system 102 via a terminal device 101. For example, a data query client is installed on the terminal device 101, and the object can enter a query request on the client. Then, the terminal device 101 sends the query request to the database system 102 to perform the data query. The database system 102 returns the queried data to the terminal device 101 for display.

[0082] The database system 102 in this embodiment mainly includes a database user system, a database management system, and a database. A database (DB) is a collection of structured data, such as tables, indexes, and views. For example, a `user` table stores user information. A database management system (DBMS) is a software system that manages and operates the database, providing functions such as data storage, querying, transaction management, and security control. The database user system is responsible for managing user identities, permissions, and access control. For example, when an object initiates a query request, the database user system verifies the object's identity and checks its permissions (such as whether it is allowed to read a certain table). After successful verification, the DBMS executes the specific operations of the query request, such as parsing SQL, generating a query plan, and then reading data from the database.

[0083] Based on this, the data query method in this embodiment may involve an object inputting a query request on a terminal device 101, which then sends the query request to a database system 102. Upon receiving the query request, the database system 102 obtains the resource load indicators of the database system at the current moment, such as CPU utilization, disk I / O rate and occupancy, memory usage, context switching frequency, and network throughput. Next, the query request is parsed to generate multiple candidate query plans, for example, N candidate query plans. Then, based on the resource load indicators of the system at the current moment, a target query plan is selected from the N candidate query plans. For example, for the i-th candidate query plan among the N candidate query plans, its feature information is extracted, and then, through a query plan prediction model, the feature information and resource load indicators of the i-th candidate query plan are fused to obtain the query duration of the i-th candidate query plan. Finally, based on the target query plan, a data query is performed in the database to obtain the query result of the query request. Optionally, the database system 102 sends the query result of the query request to the terminal device 101, and the terminal device 101 displays the query result. That is, this embodiment generates N candidate query plans for the query request, and then, based on the resource load index of the database system at the current moment, selects a target query plan from these N candidate query plans that matches the current load status of the database system. In other words, when faced with a query with insufficient resources, this embodiment can select a suitable target query plan from the N candidate query plans to continue execution, avoiding high latency caused by queuing, thereby significantly improving system operating efficiency and stability, meeting the requirements for efficient and stable operation of the database system, and thus improving the efficiency of data query.

[0084] In some embodiments, the terminal device 101 includes, but is not limited to, desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices may include smartwatches, smart bracelets, and head-mounted devices. Terminal devices are often equipped with a display device, which may be a monitor, display screen, touchscreen, etc., and the touchscreen may be a touchscreen, touch panel, etc.

[0085] In some embodiments, the database system 102 includes servers, which may be one or more. When there are multiple servers, at least two servers provide different services, and / or at least two servers provide the same service, such as providing the same service in a load-balanced manner; this embodiment does not limit this. The servers may be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Servers may also be nodes in a blockchain.

[0086] It should be noted that the implementation environment of this application embodiment includes, but is not limited to, Figure 1 As shown.

[0087] The technical solutions of the embodiments of this application will be described in detail below through some examples. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0088] Figure 2 This is a flowchart illustrating a data query method provided in an embodiment of this application. The execution entity of this embodiment can be a device with data query functionality, such as a data query device. This data query device can be the one described above. Figure 1 This refers to a portion of the database system 102 shown, such as a device within a database management system. For ease of description, the following embodiments use a data query device as an example to illustrate the method of this application.

[0089] like Figure 2 As shown, the data query method in this application embodiment includes:

[0090] S101. In response to the received query request, obtain the resource load index of the database system at the current moment.

[0091] It should be noted that the data used in executing the embodiments of this application, as well as the process of obtaining such data, comply with the relevant laws and regulations.

[0092] In database query scenarios, the optimal execution plan for the same SQL statement may differ significantly under different resource loads on the database system (such as CPU utilization, disk I / O, lock contention, remote data access, etc.). Traditional optimizers lack the ability to dynamically model the correlation between real-time system load characteristics and query resource requirements, and cannot accurately perceive the current system bottlenecks and the types of resources required for SQL execution. This makes it difficult for optimization decisions to adaptively adjust with load changes, and they cannot achieve the best system performance under different dynamic load scenarios.

[0093] This application embodiment obtains the resource load index of the database system at the current moment, and then determines the optimal query plan based on the resource load index of the database system at the current moment. This enables the optimization decision to be adaptively adjusted with load changes, and can achieve the best performance of the system under different dynamic load scenarios.

[0094] In this embodiment, the object (e.g., a user) first connects to the database before querying data. An example command line is shown in Table 1:

[0095] Table 1

[0096] bash mysql -u username -p password -h host address -P port number

[0097] At the same time, specify the database to be operated on. For example, the command line is shown in Table 2:

[0098] Table 2

[0099] SQL USE database_name;

[0100] Next, write the query statement. Here are some examples of commonly used query syntax:

[0101] SELECT [column name]

[0102] FROM [table_name]

[0103] [WHERE condition]

[0104] [GROUP BY group column]

[0105] [HAVING grouping conditions]

[0106] [ORDER BY sorted list[ASC|DESC]]

[0107] [LIMIT returns the number of rows].

[0108] The query request in this application embodiment includes a query statement. This application embodiment does not limit the specific content of the query statement, but determines it according to actual needs.

[0109] For example, the query statement can be shown in Table 3:

[0110] Table 3

[0111]

[0112]

[0113] After the user completes the query statement on the client side of the terminal device, they input a send command. For example, they can press Enter to execute the query, or send the query statement via code. In response to the send command, the terminal device sends a query request to the database system, which includes the query statement.

[0114] Optionally, the query request may also include object information and the identifier of the target database. Upon receiving the query request, the database system parses it to obtain the object information and the identifier of the target database. Based on the object information, it then determines whether the object has permission to access the target database. If it is determined that the object has permission to access the target database, the query statement in the query request is executed.

[0115] In some embodiments, the terminal device may first send object information and the identifier information of the target database to the database system. The database system determines whether the object has permission to access the target database based on the object information. If it determines that the object has permission to access the target database, it sends a verification success response to the terminal device. Based on this response, the terminal device displays a query input interface, where the object can enter a query statement and a send command. In response to the send command, the terminal device sends a query request to the database system.

[0116] In this embodiment of the application, when the data query device of the database system receives the query request, it responds to the query request by obtaining the resource load index of the database system at the current moment.

[0117] The resource load index in this application embodiment is any index that can represent the load status of the database system at the current moment.

[0118] In some embodiments, resource load metrics include at least one of system-level bottleneck metrics and key time-consuming metrics of the database system. The key time-consuming metrics refer to the time consumption of each key step in the database system's execution of the query plan. In this case, the resource load metrics in this embodiment include the following three cases:

[0119] Case 1: In this embodiment of the application, the resource load indicators include system-level bottleneck indicators of the database system. That is, the data query device obtains the system-level bottleneck indicators of the database system at the current moment.

[0120] For example, such as Figure 3 As shown, system-level bottleneck metrics can include at least two of the following: system CPU utilization, disk I / O rate, disk occupancy, memory utilization, context switching frequency, network throughput, buffer size, etc.

[0121] Scenario 2: In this embodiment of the application, the resource load metrics include key time-consuming metrics of the database system. Key time-consuming metrics refer to the time consumption of each key step in the database system when executing a query plan. That is, the data query device obtains the time consumption of each key step in the database system when executing query plans over a recent period.

[0122] For example, such as Figure 4 As shown, key time-consuming metrics may include at least two of the following: network time, memory allocation time, disk I / O time, CPU time, lock time, and concurrency control-related time.

[0123] Case 3: In this embodiment of the application, the resource load indicators include system-level bottleneck indicators and key time-consuming indicators of the database system. That is, the data query device obtains the system-level bottleneck indicators and key time-consuming indicators of the database system at the current moment.

[0124] For example, such as Figure 5 As shown, the resource load metrics at this time include at least one of the following: system CPU utilization, disk I / O rate, disk occupancy, memory utilization, context switching frequency, network throughput, buffer size, etc., and at least one of the following: network latency, memory allocation latency, disk I / O latency, CPU latency, lock latency, and concurrency control related latency.

[0125] As described above, the resource load indicators in this application embodiment include multi-dimensional resource indicators, such as at least two of the following: system CPU utilization, disk I / O rate and occupancy, memory utilization, context switching frequency, network throughput, buffer size, etc.; or at least two of the following: network latency, memory allocation latency, disk I / O latency, CPU latency, lock and concurrency control related latency, etc.; or at least one of the following: system CPU utilization, disk I / O rate and occupancy, memory utilization, context switching frequency, network throughput, buffer size, etc., and at least one of the following: network latency, memory allocation latency, disk I / O latency, CPU latency, lock and concurrency control related latency, etc. Compared to existing technologies that mainly focus on storage resources in resource prediction without simultaneously considering multi-dimensional resources such as CPU, memory, disk, lock contention, and remote data access, resource management in complex database environments is not refined enough, leading to data query failures or low efficiency. In contrast, this application embodiment comprehensively assesses the system resource status by considering multi-dimensional resource indicators of the database system, providing accurate prediction basis for query plan selection, thereby improving the efficiency and reliability of data queries.

[0126] The following describes the specific process of obtaining the resource load indicators of the database system at the current moment.

[0127] In some embodiments, if the source load metric includes the system-level bottleneck metric of the database system, then obtaining the resource load metric of the database system at the current moment in S101 may include the following steps S101-A1 and S101-A2:

[0128] S101-A1: Obtain system-level bottleneck indicators of the collected database system at different times in the current time period;

[0129] S101-A2. For each indicator in the system-level bottleneck indicators, the average value of the indicator at different times in the current time period is determined as the value of the indicator at the current time.

[0130] In this implementation, the detection modules on the database system's operating system and related hardware continuously collect system-level bottleneck metrics such as CPU utilization, disk I / O rate and occupancy, memory utilization, context switching frequency, and network throughput. Specifically, upon receiving a query request, the system-level bottleneck metrics of the database system at different times within the current time period are obtained. For example, assuming the current time period is a 10-minute time window and the current time is t0, the system-level bottleneck metrics collected at different times within the time period (t0-10 minutes, t0) are obtained. The average value of the system-level bottleneck metrics at different times within the current time period is then determined as the system-level bottleneck metric of the database system at the current time.

[0131] For example, suppose the current time period is (t0-10 minutes, t0), which includes 4 data collection times, denoted as t0. 01 t 02 t 03 and t 04 Suppose that at t 01 The CPU utilization rate collected at that time was a1, disk I / O rate was b1, disk occupancy rate was c1, memory usage rate was d1, context switching frequency was e1, and network throughput rate was f1. At t 02 The CPU utilization rate collected at that time was a2, the disk I / O rate was b2, the disk occupancy rate was c2, the memory utilization rate was d2, the context switching frequency was e2, and the network throughput rate was f2. At t 03 The CPU utilization rate collected at that time was a3, the disk I / O rate was b3, the disk occupancy rate was c3, the memory utilization rate was d3, the context switching frequency was e3, and the network throughput rate was f3. At t 04 The CPU utilization rate was a4, the disk I / O rate was b4, the disk occupancy rate was c4, the memory usage rate was d4, the context switching frequency was e4, and the network throughput rate was f4.

[0132] For example, to facilitate caching of the system-level bottleneck metrics collected at different times, they are cached in tabular form, as shown in Table 4:

[0133] Table 4

[0134]

[0135] For each of the above system-level bottleneck metrics, the average value of that metric at different times within the current time period is determined as the value of that metric at the current time. For example, the CPU utilization value at the current time is (a1+a2+a3+a4) / 4. The disk I / O rate value at the current time is (b1+b2+b3+b4) / 4. The disk utilization value at the current time is (c1+c2+c3+c4) / 4. The memory utilization value at the current time is (d1+d2+d3+d4) / 4. The context switching frequency value at the current time is (e1+e2+e3+e4) / 4. The network throughput value at the current time is (f1+f2+f3+f4) / 4.

[0136] In some examples, for each system-level bottleneck metric, the weighted average of its values ​​at different times within the current time period is used to determine its value at the current time. The weights can be determined based on the time elapsed since the current time; for example, values ​​collected closer to the current time have a higher weight, while values ​​collected further back have a lower weight.

[0137] In some embodiments, if the source load metric includes key time-consuming metrics of the database system, then obtaining the current resource load metric of the database system in S101 may include the following steps S101-B1 and S101-B2:

[0138] S101-B1: Obtain key time-consuming metrics of the collected database system when executing different query plans in the current time period;

[0139] S101-B2. For each of the key time-consuming indicators, the average value of the indicator in the current time period is determined as the value of the indicator at the current moment.

[0140] In this implementation, the execution time of the database query plan is continuously collected, including specific values ​​for network time, memory allocation time, disk I / O time, CPU time, and time related to locks and concurrency control. In this embodiment, the above key time-consuming indicators are statistically analyzed according to a time window (e.g., within 10 minutes) to reflect the load situation under the current resource load. Specifically, upon receiving a query request, the key time-consuming indicators of the database system executing different query requests within the current time period are obtained. For example, assuming the current time period is a 10-minute time window and the current time is t0, the key time-consuming indicators of different query requests within the time period (t0-10 minutes, t0) are obtained. The average value of the key time-consuming indicators of different query requests within the current time period is then determined as the key time-consuming indicator of the database system at the current time.

[0141] For example, suppose the current time period is (t0 - 10 minutes, t0), and four query plans are executed during this period, denoted as Query Plan 1, Query Plan 2, Query Plan 3, and Query Plan 4. Assume the key performance indicators (KPIs) for executing Query Plan 1 are: network time g1, memory allocation time h1, disk I / O time k1, CPU time m1, and lock and concurrency control related time n1. For Query Plan 2, the KPIs are: network time g2, memory allocation time h2, disk I / O time k2, CPU time m2, and lock and concurrency control related time n2. For Query Plan 3, the KPIs are: network time g3, memory allocation time h3, disk I / O time k3, CPU time m3, and lock and concurrency control related time n3. The key consumption metrics for executing query plan 4 are: network time g4, memory allocation time h4, disk I / O time k4, CPU time m4, and lock and concurrency control related time n4.

[0142] For example, to facilitate caching of key consumption metrics for the different query plans mentioned above, they are cached in tabular form, as shown in Table 5:

[0143] Table 5

[0144]

[0145]

[0146] For each of the key consumption metrics mentioned above, the average value of that metric over the current time period is determined as its value at the current moment. For example, the network latency at the current moment is (g1+g2+g3+g4) / 4. The memory allocation latency at the current moment is (h1+h2+h3+h4) / 4. The disk I / O latency at the current moment is (k1+k2+k3+k4) / 4. The CPU latency at the current moment is (m1+m2+m3+m4) / 4. The lock and concurrency control related latency at the current moment is (n1+n2+n3+n4) / 4.

[0147] In some examples, for each key consumption metric, the weighted average of the different values ​​of that metric within the current time period is used to determine the value of that metric at the current moment. The weights can be determined based on the time elapsed since the current moment; for example, values ​​collected closer to the current moment have a higher weight, while values ​​collected further back have a lower weight.

[0148] The above embodiments determine the system-level bottleneck indicators and key time-consuming indicators of the database system at the current moment based on the system-level bottleneck indicators and key time-consuming indicators within the current time period.

[0149] In some embodiments, in response to a received query request, system-level bottleneck metrics and key time-consuming metrics at the current moment can also be collected.

[0150] In this embodiment of the application, in response to the received query request, after obtaining the resource load index of the database system at the current moment, the following step S102 is executed.

[0151] S102. Generate N candidate query plans for the query request.

[0152] Where N is a positive integer greater than 1.

[0153] In this embodiment of the application, in order to select an appropriate query plan for a query request under different resource load conditions and avoid high latency caused by queuing, this embodiment generates multiple candidate query plans for the query request. For example, N candidate query plans are generated for the query request.

[0154] This application does not limit the specific method of generating N candidate query plans for a production query request.

[0155] In some embodiments, N candidate query plans for the query request are selected from a plurality of preset query plans. For example, N query plans are randomly selected from a plurality of preset query plans as N candidate query plans for the query request.

[0156] In some embodiments, S102 above includes the following steps S102-A:

[0157] S102-A: The optimizer generates N candidate query plans for the query request.

[0158] The optimizer is trained based on N different hints, which describe the requirements that the candidate query plan needs to meet.

[0159] In this embodiment, the N candidate query plans for the query request are generated by an optimizer. To enable the optimizer to generate diverse query plans, during the training phase, a cueing intervention is used to assist the optimizer in generating a set of differentiated query plans.

[0160] Hint intervention refers to explicitly adding database-specific hints to SQL statements, thereby adjusting the execution plan without changing the SQL semantics. These hints are like explicit instructions given to the database optimizer, telling it how to choose indexes, determine the order of table joins, and so on.

[0161] For example, in MySQL databases, you can use the hint " / *+INDEX(table_nameindex_name)* / ", which is an index hint. When executing a complex query, if the database optimizer does not choose an appropriate index, resulting in poor query performance, adding this hint forces the optimizer to use the specified index, thereby changing the query plan.

[0162] For example, using the hint " / *+JOIN_PREFIX(t1,t2)* / " instructs the optimizer to perform join operations in the order the tables appear in the hint, rather than having the optimizer evaluate the join order itself. This approach allows for precise intervention in certain key aspects of the execution plan to obtain different execution paths, thereby resulting in diverse query plans.

[0163] This application embodiment does not limit the specific content of the prompt information; it can be set according to actual needs. After the above training, the optimizer of this application embodiment can generate N candidate query plans. Thus, as... Figure 6 As shown, after receiving a query request, the current data query device calls the optimizer and inputs the query request into the optimizer. In this way, the optimizer can generate N candidate query plans for the query request.

[0164] S103. Based on the resource load index, select the target query plan from N candidate query plans.

[0165] This application embodiment comprehensively and accurately depicts the resource contention status of the system at the current moment by acquiring the system's resource load indicators. For example, real-time collection of system-level bottleneck indicators ensures a grasp of the overall resource status of the system. Statistics on the time consumption of each key step in the database query plan execution provide a detailed reflection of the consumption of different resources during the query plan process within the current time window. Furthermore, based on the system's resource load indicators at the current moment, a candidate query plan that matches the resource load indicators is selected from N candidate query plans for the query request and used as the target query plan for that query request.

[0166] This application does not restrict the specific method of selecting the target query plan from N candidate query plans based on resource load indicators.

[0167] In some embodiments, the resource load index required for the execution of each of the N candidate query plans is determined. Then, based on the resource load index corresponding to each of the N candidate query plans and the resource load index of the data system at the current moment, a target query plan is selected from the N candidate query plans. One possible way to determine the resource load index required for the execution of each of the N candidate query plans is to predict the resource load index required by the candidate query plan using a neural network model.

[0168] In some embodiments, S103 above includes the following steps S103-A and S103-B:

[0169] S103-A. Based on resource load indicators, determine the query duration of each candidate query plan among N candidate query plans;

[0170] S103-B: Based on the query duration, select the target query plan from N candidate query plans.

[0171] In this implementation, for each of the N candidate query plans, the query duration of each candidate plan is determined under the current system resource load indicators. Then, based on the query duration of each candidate plan, the target query plan is selected from the N candidate query plans.

[0172] This application does not limit the specific method for determining the query duration of each candidate query plan among N candidate query plans based on resource load indicators.

[0173] In some embodiments, resource load metrics and N candidate query plans are input into a prediction model, which outputs the query duration of each of the N candidate query plans.

[0174] In some embodiments, S103-A includes the following steps S103-A1 and S103-A2:

[0175] S103-A1. For the i-th candidate query plan among N candidate query plans, extract the feature information of the i-th candidate query plan, where i is a positive integer from 1 to N;

[0176] S103-A2. By using the query plan prediction model, the feature information and resource load indicators of the i-th candidate query plan are fused to obtain the query duration of the i-th candidate query plan.

[0177] In this embodiment of the application, the specific method for determining the query duration of each of the N candidate query plans is the same. For ease of description, the method for determining the query duration of the i-th query plan is used as an example.

[0178] When determining the query duration of the i-th query plan, the feature information of the i-th candidate query plan is first extracted.

[0179] For example, such as Figure 7 As shown, the feature extraction model is used to extract features from the i-th candidate query plan to obtain the feature information of the i-th candidate query plan.

[0180] The embodiments of this application do not limit the specific network structure of the feature extraction model.

[0181] In some embodiments, such as Figure 8 As shown, the feature extraction model includes a tree convolutional network and pooling layers. Therefore, S103-A1 above includes the following steps S103-A11 and S103-A12:

[0182] S103-A11. By using a tree convolutional network, capture the hierarchical relationships and semantic information in the tree structure of the i-th candidate query plan to obtain the structural semantic information of the i-th candidate query plan.

[0183] S103-A12. The structural semantic information of the i-th candidate query plan is processed by the pooling layer to obtain the feature information of the i-th candidate query plan.

[0184] In this implementation, such as Figure 8 As shown, the i-th candidate query plan is input into the Tree Convolutional Network (TCNN). This TCNN effectively captures the hierarchical relationships and semantic information in the tree structure of the i-th candidate query plan, performing deep feature extraction to obtain the structural semantic information of the i-th candidate query plan. Then, a pooling layer is used to reduce the spatial size of the structural semantic information of the i-th candidate query plan while retaining important information, thus obtaining the feature information of the i-th candidate query plan.

[0185] Next, the query plan prediction model is used to fuse the feature information and resource load indicators of the i-th candidate query plan to obtain the query duration of the i-th candidate query plan.

[0186] For example, such as Figure 9 As shown, the feature information and resource load index of the i-th candidate query plan are input into the query plan prediction model for fusion processing to obtain the query duration of the i-th candidate query plan.

[0187] In some embodiments, before inputting the system's resource load index at the current moment into the query plan estimation model, the resource load index is first encoded. In this case, S103-A2 may include the following steps S103-A21 and S103-A22:

[0188] S103-A21. Standardize and normalize each indicator in the resource load index to obtain resource load coding information;

[0189] S103-A22. By using the query plan prediction model, the feature information and resource load coding information of the i-th candidate query plan are fused to obtain the query duration of the i-th candidate query plan.

[0190] In this implementation, before inputting the system's resource load indicators at the current moment into the query plan prediction model, the resource load indicators are first encoded. Specifically, each indicator in the resource load indicators is standardized and normalized to obtain resource load encoding information.

[0191] For example, for resource load metrics with large differences in numerical range, such as context switching frequency and disk I / O rate, a standardization method is used to convert the data into a standard normal distribution with a mean of 0 and a standard deviation of 1.

[0192] For example, for resource load metrics with clear boundaries, such as memory utilization (0%-100%) and CPU utilization, normalization methods can be used to map them to the [0,1] interval.

[0193] Next, each indicator in the resource load index is standardized and normalized, and then combined in sequence into a multi-dimensional state vector to obtain the resource load encoding information.

[0194] like Figure 10 As shown, the feature information and resource load encoding information of the i-th candidate query plan are input into the query plan prediction model for fusion processing to obtain the query duration of the i-th candidate query plan.

[0195] This application embodiment does not limit the specific method of fusing the feature information and resource load encoding information of the i-th candidate query plan through the query plan prediction model to obtain the query duration of the i-th candidate query plan.

[0196] In one possible implementation, the feature information and resource load encoding information of the i-th candidate query plan are first fused to obtain fused feature information. For example, the feature information and resource load encoding information of the i-th candidate query plan are concatenated to obtain fused feature information. Another example is that the feature information and resource load encoding information of the i-th candidate query plan are added together to obtain fused feature information. Next, the fused feature information is input into the query plan prediction model for processing to obtain the query duration of the i-th candidate query plan.

[0197] In one possible implementation, such as Figure 11 As shown, the query plan prediction model in this embodiment includes a cross-attention module. In this case, S103-A22 includes the following steps: S103-A22-a1 and S103-A22-a2.

[0198] S103-A22-a1: The feature information and resource load encoding information of the i-th candidate query plan are fused through the cross-attention module to obtain the cross-attention result of the i-th candidate query plan;

[0199] S103-A22-a2: Based on the cross-attention result of the i-th candidate query plan, determine the query duration of the i-th candidate query plan.

[0200] Traditional Tree Convolutional Networks (TCNNs) struggle to tightly correlate system resource load metrics with query plans when facing complex and variable system loads, failing to adequately adapt to the optimization needs of query plans under different resource load scenarios. This application innovates upon TCNN by introducing a cross-attention module, aiming to effectively fuse query plans and system resource load encoding information. This allows the model to accurately predict better query plans under various resource load scenarios.

[0201] In this embodiment, the feature information and resource load encoding information of the i-th candidate query plan are fused through a cross-attention module to obtain the cross-attention result of the i-th candidate query plan.

[0202] This application embodiment does not limit the specific method by which the feature information and resource load encoding information of the i-th candidate query plan are fused through the cross-attention module to obtain the cross-attention result of the i-th candidate query plan.

[0203] In some embodiments, S103-A22-a1 includes the following steps S103-A22-a11 to S103-A22-a13:

[0204] S103-A22-a11. Multiply the feature information of the i-th candidate query plan with the key weight matrix and value weight matrix in the cross-attention module respectively to obtain the key matrix and value matrix corresponding to the i-th candidate query plan;

[0205] S103-A22-a12: Multiply the resource load encoding information with the query weight matrix in the cross-attention module to obtain the query matrix corresponding to the i-th candidate query plan;

[0206] S103-A22-a13, perform attention calculation on the key matrix, value matrix and query matrix corresponding to the i-th candidate query plan to obtain the cross-attention result of the i-th candidate query plan.

[0207] like Figure 12 As shown, the cross-attention module in this embodiment includes three parameter matrices, namely the key weight matrix W. K Value weight matrix W V And query weight matrix W Q The feature information X of the i-th candidate query plan. plan With the key weight matrix W in the cross-attention module K Multiplying them together yields the bond matrix K = W. K ·X plan The key matrix K represents the abstract resource-related features of the i-th candidate query plan, covering the potential resource requirements (CPU, memory, disk, etc.) for executing various operations in the i-th candidate query plan. The feature information X of the i-th candidate query plan... plan With the value weight matrix W in the cross-attention module V Multiplying them together yields the value matrix V = W. V ·X plan The value matrix V contains specific information about the i-th candidate query plan, such as operator type and data access path. This step uses matrix transformation to abstract the feature information of the i-th candidate query plan hierarchically, preparing for subsequent interaction with resource load encoding information.

[0208] Next, the resource load encoding information X workload With the query weight matrix W in the cross-attention module Q Multiplying them together yields the query matrix Q = W. Q ·X workload The query matrix Q represents the current resource load status of the system, including indicators such as CPU utilization, disk I / O rate, and memory utilization, and is a quantitative expression of the real-time status of the system.

[0209] Finally, attention is calculated on the key matrix K, value matrix V, and query matrix Q corresponding to the i-th candidate query plan to obtain the cross-attention result of the i-th candidate query plan.

[0210] For example, the cross-attention result of the i-th candidate query plan is determined by the following formula (1):

[0211]

[0212] Where, d K This represents the length of the feature vector. Attention(Q,K,V) is the result of the cross-attention.

[0213] In this process, the attention weight matrix is ​​obtained by multiplying the query matrix Q and the key matrix K. This matrix reflects the degree of correlation between the current system resource load state (Q) and the abstract resource characteristics (K) of the i-th candidate query plan. Based on this, the model can assign different weights to different positions in the value matrix V according to the resource load. For example, when the system CPU load is high, the model will automatically reduce the weight of CPU-intensive operations in the i-th candidate query plan and prioritize the execution path with lower CPU resource requirements, thus achieving dynamic resource adaptation.

[0214] Based on the above steps, after obtaining the cross-attention result of the i-th candidate query plan, the query duration of the i-th candidate query plan is determined based on the cross-attention result of the i-th candidate query plan.

[0215] In one example, such as Figure 13 As shown, the cross-attention module in this embodiment further includes a fully connected layer. In this case, the cross-attention result of the i-th candidate query plan is processed through the fully connected layer to obtain the query duration of the i-th candidate query plan. Specifically, the cross-attention result Attention(Q,K,V) of the i-th candidate query plan is passed to the fully connected layer. Through further processing by the fully connected layer, the fused features are mapped to the final query duration. This step completes the transformation from feature fusion to specific query plan prediction, realizing query plan inference and prediction based on resource load awareness.

[0216] In this embodiment, the cross-attention mechanism enables the model to dynamically capture the complex relationship between system resource load and query plan. Under different resource load scenarios, it flexibly adjusts the focus on different parts of the query plan, significantly improving the model's accuracy and adaptability in query plan selection. Compared to traditional models, this embodiment effectively solves the problem of the disconnect between query plan selection and system resource load, providing more efficient and accurate decision support for database query optimization.

[0217] The above embodiment takes determining the query duration of the i-th candidate query plan among N candidate query plans as an example. Based on the above method, this application embodiment can determine the query duration of each candidate query plan among N candidate query plans. Then, S103-B is executed as described above, and based on the query duration of each candidate query plan among N candidate query plans, the target query plan is selected from the N candidate query plans.

[0218] For example, the candidate query plan with the shortest query time among these N candidate query plans is determined as the target query plan for this query request.

[0219] S104. Based on the target query plan, perform a data query in the database to obtain the query results of the query request.

[0220] In this embodiment of the application, based on the resource load index of the database system at the current moment, after determining the target query plan from the N candidate query plans of the query request, a data query is performed in the database based on the target query plan to obtain the query result of the query request.

[0221] This application's embodiments utilize load awareness to accurately match query plans, significantly reducing query execution latency and improving data acquisition efficiency. In resource management, it comprehensively considers multiple resource dimensions (such as CPU, disk, memory, etc.) to avoid resource waste and overconsumption, improving resource utilization and reducing hardware costs. Furthermore, it enhances the system's adaptability and stability to complex loads, reducing failures and performance fluctuations, and ensuring the efficient and stable operation of the database system.

[0222] This application provides a novel data query method. In response to a received query request, it obtains the resource load index of the database system at the current moment, where the query request is used to request data from the database. Next, N candidate query plans are generated for the query request, where N is a positive integer greater than 1. Then, based on the resource load index, a target query plan is selected from the N candidate query plans. Finally, based on the target query plan, a data query is performed in the database to obtain the query result. In other words, this application generates N candidate query plans for the query request and then selects a target query plan that matches the current load status of the database system from these N candidate query plans based on the current resource load index of the database system. That is, when faced with a query request with insufficient resources, this application can select a suitable target query plan from the N candidate query plans to continue execution, avoiding high latency caused by queuing, thereby significantly improving system operating efficiency and stability, meeting the requirements for efficient and stable operation of the database system, and thus improving the efficiency of data query.

[0223] The foregoing provides an overall overview of the data query process provided in the embodiments of this application. The following section will combine... Figure 14 The data query method provided in the embodiments of this application will be further described.

[0224] Figure 14 This is a schematic flowchart of a data query method provided in an embodiment of this application. Figure 15 This is a schematic diagram illustrating the framework of the data query method provided in the embodiments of this application.

[0225] like Figure 14 and Figure 15 As shown, the method in this application embodiment includes:

[0226] S201. In response to the received query request, obtain the resource load index of the database system at the current moment.

[0227] In some embodiments, resource load metrics include at least one of system-level bottleneck metrics and key time-consuming metrics of the database system, wherein key time-consuming metrics refer to the time-consuming metrics of each key step in the execution of the query plan by the database system.

[0228] In one example, if the resource load metrics include system-level bottleneck metrics of the database system, then the collected system-level bottleneck metrics of the database system at different times in the current time period are obtained; for each of the system-level bottleneck metrics, the average value of the metric at different times in the current time period is determined as the value of the metric at the current time.

[0229] In one example, if the resource load metrics include key time-consuming metrics of the database system, then the key time-consuming metrics of the database system when executing different query plans in the current time period are obtained; for each of the key time-consuming metrics, the average value of the metric in the current time period is determined as the value of the metric at the current moment.

[0230] The specific implementation process of S201 can be referred to the relevant description of S101 above, and will not be repeated here.

[0231] S202. Generate N candidate query plans for the query request.

[0232] Where N is a positive integer greater than 1.

[0233] For example, an optimizer generates N candidate query plans for a query request. This optimizer is trained based on N different hints, where the hints describe the requirements that the candidate query plans need to meet.

[0234] The specific implementation process of S202 can be referred to the relevant description of S102 above, and will not be repeated here.

[0235] S203. For the i-th candidate query plan among N candidate query plans, the i-th candidate query plan is extracted using a feature extraction model to obtain the feature information of the i-th candidate query plan.

[0236] Where i is a positive integer from 1 to N.

[0237] In some embodiments, such as Figure 15 As shown, the feature extraction model includes a tree convolutional network and a pooling layer. The tree convolutional network can capture the hierarchical relationship and semantic information in the tree structure of the i-th candidate query plan to obtain the structural semantic information of the i-th candidate query plan. The pooling layer processes the structural semantic information of the i-th candidate query plan to obtain the feature information of the i-th candidate query plan.

[0238] S204. Standardize and normalize each indicator in the resource load index to obtain resource load coding information.

[0239] It should be noted that there is no difference in the execution order between S204 and S202. For example, S204 can be executed after S202, or before S202, or simultaneously with S202. This application embodiment does not impose any restrictions on this, and the specific execution order is determined according to actual needs.

[0240] S205. The feature information and resource load encoding information of the i-th candidate query plan are fused through the cross-attention module to obtain the cross-attention result of the i-th candidate query plan.

[0241] Specifically, such as Figure 15 As shown, the resource load encoding information of the system at the current moment and the feature information of the i-th candidate query plan are input into the query plan prediction model for processing to obtain the query duration of the i-th candidate query plan.

[0242] like Figure 15 As shown, the query plan prediction model in this application embodiment includes a cross-attention module. This allows the feature information and resource load encoding information of the i-th candidate query plan to be fused through the cross-attention module to obtain the cross-attention result of the i-th candidate query plan.

[0243] In some embodiments, the feature information of the i-th candidate query plan is multiplied by the key weight matrix and the value weight matrix in the cross-attention module to obtain the key matrix and value matrix corresponding to the i-th candidate query plan; the resource load encoding information is multiplied by the query weight matrix in the cross-attention module to obtain the query matrix corresponding to the i-th candidate query plan; attention is calculated on the key matrix, value matrix and query matrix corresponding to the i-th candidate query plan to obtain the cross-attention result of the i-th candidate query plan.

[0244] S206. Based on the cross-attention result of the i-th candidate query plan, determine the query duration of the i-th candidate query plan.

[0245] For example, by using a fully connected layer, the cross-attention result of the i-th candidate query plan is processed to obtain the query duration of the i-th candidate query plan.

[0246] By using the same method as determining the query duration of the i-th candidate query plan, the query duration of each of the N candidate query plans can be determined.

[0247] S207. Based on the query duration, select the target query plan from N candidate query plans.

[0248] For example, the candidate query plan with the shortest query time among N candidate query plans is determined as the target query plan.

[0249] S208. Based on the target query plan, perform a data query in the database to obtain the query results of the query request.

[0250] The data query method provided in this application, in response to a received query request, obtains the resource load indicators of the database system at the current moment. It generates N candidate query plans for the query request. For the i-th candidate query plan among the N candidate query plans, a feature extraction model is used to extract features from the i-th candidate query plan to obtain its feature information. Each indicator in the resource load indicators is standardized and normalized to obtain resource load encoding information. Next, a cross-attention module is used to fuse the feature information and resource load encoding information of the i-th candidate query plan to obtain the cross-attention result of the i-th candidate query plan. Based on the cross-attention result of the i-th candidate query plan, the query duration of the i-th candidate query plan is determined. Finally, based on the query duration, a target query plan is selected from the N candidate query plans, and based on the target query plan, a data query is performed in the database to obtain the query result of the query request. This application embodiment, combined with a learning-based optimizer, can generate N candidate query plans in batches when faced with resource-insufficient queries. This allows for the selection of a suitable query plan based on system resource load metrics, avoiding high latency issues caused by queuing. Furthermore, this application embodiment effectively integrates resource load metrics and candidate query plans through a cross-attention mechanism. This enables the model to comprehensively assess system resource status by simultaneously considering multiple resource factors. Consequently, it can select appropriate target query plans under different load scenarios, avoiding resource waste, improving the accuracy of target query plan prediction, and significantly enhancing system operating efficiency and stability, meeting the requirements for efficient and stable database system operation.

[0251] The above section introduced the specific process of the data query method. The following section describes the training process of the model used in the data query method.

[0252] Figure 16 This is a schematic flowchart of a model training method provided in an embodiment of this application. The execution entity in this embodiment is a device with model training functionality, such as a model training apparatus. This model training apparatus can be a computing device, or a part of a computing device. The following description uses a computing device as the execution entity as an example.

[0253] like Figure 16 As shown, the model training method of this application embodiment includes the following steps:

[0254] S301. Obtain training samples.

[0255] The training samples include: the system's resource load index and a candidate query plan. The candidate query plan is any one of N candidate query plans for a historical query request. The resource load index is one of P sets of resource load indices of the database system, where N and P are both positive integers greater than 1.

[0256] The following section describes the specific process of creating the training sample set.

[0257] In this embodiment, multiple historical query requests can be obtained from historical query records. For each of these multiple historical query requests, N candidate query plans are generated. The specific generation process of the candidate query plans is described in the above embodiments and will not be repeated here.

[0258] For example, historical query requests and corresponding candidate query plans are shown in Table 6:

[0259] Table 6

[0260]

[0261] In terms of model training, in order to improve the predictive ability of the model under different resource loads, this application embodiment creates multiple resource load scenarios of the system, and then obtains a set of resource load indicators of the system under each of these multiple resource load scenarios, thereby obtaining multiple sets of resource load indicators, such as obtaining P sets of resource load indicators.

[0262] In some embodiments, the P-group resource load index of the system can be generated by adjusting the state of the database system and the configuration of the database.

[0263] For example, a set of resource load metrics may include at least one of the system-level bottleneck metrics and key time-consuming metrics of the database system.

[0264] System-level bottleneck metrics can include: CPU utilization, disk I / O rate, disk occupancy, memory utilization, context switching frequency, network throughput, buffer size, etc.

[0265] Key time-consuming metrics may include: network time, memory allocation time, disk I / O time, CPU time, and time related to locks and concurrency control.

[0266] In this embodiment, the P-group resource load index of the system is generated by adjusting the system buffer size, query request concurrency, and optimizing the optimizer.

[0267] The size of the buffer directly impacts the database's data caching capacity. Increasing the buffer size allows more data to be cached, reducing disk I / O operations and lowering disk load while increasing memory usage. Conversely, decreasing the buffer size increases disk I / O, putting greater pressure on disk load. Therefore, adjusting the buffer size can provide different resource load metrics for the system.

[0268] Increasing query request concurrency puts greater pressure on system resources such as CPU, memory, disk I / O, and locks. This not only keeps the CPU busy processing multiple query tasks but may also lead to increased lock contention, increased disk I / O, and a shift in resource bottlenecks. Conversely, decreasing concurrency reduces resource contention. Therefore, by adjusting query request concurrency, different sets of system resource load metrics can be obtained.

[0269] The optimizer switches can alter the bias of plan generation. Taking the table scan mode switch as an example, different switch settings may favor full table scans over index scans, resulting in significant differences in the load ratio on CPU and disk. Full table scans typically generate higher disk I / O load, while index scans rely more on CPU for index lookups and data location. Therefore, by adjusting the optimizer switches, different sets of resource load metrics for the system can be obtained.

[0270] For example, the resource load indicators of group P of the system generated in this embodiment are shown in Table 7:

[0271] Table 7

[0272]

[0273] It should be noted that Table 7 above is only an example of the resource load indicators of group P in the embodiments of this application. The indicators included in a group of resource load indicators in the embodiments of this application include, but are not limited to, the indicators shown in Table 7 above. For example, it may include more indicators than Table 7 above, or it may include fewer indicators than Table 7 above, depending on actual needs.

[0274] In this embodiment of the application, after generating the candidate query plan shown in Table 6 and the P group of resource load indicators shown in Table 7, one candidate query plan in Table 6 is combined with a group of resource load indicators in Table 7 to generate a training sample, and then multiple training samples can be obtained.

[0275] For example, the generated training sample set is shown in Table 8:

[0276] Table 8

[0277] 1 Candidate Query Plan 11 First set of resource load indicators 2 Candidate Query Plan 12 First set of resource load indicators ...... ...... ...... N Candidate Query Plan 1N First set of resource load indicators N+1 Candidate Query Plan 11 Second set of resource load indicators ...... ...... ......

[0278] In this embodiment of the application, after generating the training sample set, the training sample set is used to train the model.

[0279] In this embodiment, the specific process of training the model using each training sample in the training sample set is basically the same. For ease of description, we can use one training sample as an example. That is, the training sample obtained in this step is one training sample in the training sample set.

[0280] S302. Extract the feature information of candidate query plans.

[0281] In this embodiment of the application, when using the training sample to train the model, the feature information of the candidate query plans included in the training sample is first extracted.

[0282] The specific process of extracting the feature information of candidate query plans from the training sample can be referred to the description in the above embodiments, and will not be repeated here.

[0283] In some embodiments, when extracting feature information of the candidate query plan using a feature extraction model, the feature extraction model can be pre-trained and is not trained during the model training process of this application embodiment. Optionally, the feature extraction model may not have been pre-trained and may be trained end-to-end with the subsequent query plan prediction model during the model training process of this application embodiment.

[0284] S303. By using the query plan prediction model, the feature information and resource load indicators of the candidate query plans are fused and processed to obtain the estimated query duration of the candidate query plans.

[0285] In some embodiments, each indicator in the resource load index is standardized and normalized to obtain resource load coding information; the feature information and resource load coding information of the candidate query plan are fused through the query plan prediction model to obtain the query duration of the candidate query plan.

[0286] In some embodiments, the query plan prediction model includes a cross-attention module. In this case, the cross-attention module can be used to fuse the feature information and resource load encoding information of the candidate query plan to obtain the cross-attention result of the candidate query plan. Based on the cross-attention result of the candidate query plan, the query duration of the candidate query plan is determined.

[0287] For example, the feature information of the candidate query plan is multiplied by the key weight matrix and value weight matrix in the cross-attention module to obtain the key matrix and value matrix corresponding to the candidate query plan; the resource load encoding information is multiplied by the query weight matrix in the cross-attention module to obtain the query matrix corresponding to the candidate query plan; attention is calculated on the key matrix, value matrix, and query matrix corresponding to the candidate query plan to obtain the cross-attention result of the candidate query plan. Then, the cross-attention result of the candidate query plan is processed through a fully connected layer to obtain the query duration of the candidate query plan.

[0288] The specific implementation process of S303 can be referred to the relevant description of S103 above, and will not be repeated here.

[0289] S304. Based on the estimated query duration of the candidate query plan, determine the model loss, and train the query plan prediction model based on the model loss.

[0290] During model training, a batch of training samples is selected from the training sample set to train the model. Specifically, for each training sample in this batch, the query duration of each training sample is determined based on the steps described above. Then, based on the query duration of each training sample in this batch, the model loss is determined. Subsequently, based on this model loss, the parameters in the query plan prediction model are updated to obtain the query plan prediction model after the first round of training. Next, a new batch of training samples is selected from the training sample set to train the model. Specifically, for each training sample in the second batch, the query duration of each training sample is determined based on the steps described above. Then, based on the query duration of each training sample in the second batch, the model loss is determined. Subsequently, based on this model loss, the parameters in the query plan prediction model after the first round of training are updated to obtain the query plan prediction model after the second round of training. This process is repeated multiple times until the model training termination condition is met, such as the number of training rounds reaching a preset value, or the model loss reaching a preset loss value. At this point, model training stops, and the parameters in the model are fixed, resulting in the trained query plan prediction model.

[0291] In some embodiments, during model training, the feature extraction model and the query plan prediction model described above can be trained end-to-end together to obtain the trained feature extraction model and the trained query plan prediction model.

[0292] This application does not limit the specific method for determining model loss based on the estimated query duration of candidate query plans in the embodiments.

[0293] In some embodiments, candidate query plans are determined, and the actual query duration under resource load metrics is determined; based on the estimated query duration of the candidate query plans and the actual query duration, the model loss is determined.

[0294] For example, the resource load metrics of the database system are adjusted to match those in the training sample, and the candidate query plan is executed to obtain its actual query duration. Then, based on the estimated and actual query durations of the candidate query plan, the model loss is determined. For instance, the deviation between the estimated and actual query durations of the candidate query plan is determined, and the model loss is then determined based on this deviation.

[0295] The model training method provided in this application involves acquiring training samples, which include: a set of resource load indicators of the database system and a candidate query plan. The candidate query plan is any one of N candidate query plans for a historical query request, and the resource load indicators are one of P sets of resource load indicators of the database system, where N and P are both positive integers greater than 1. Feature information of the candidate query plans is extracted. The feature information and resource load indicators of the candidate query plans are fused using a query plan prediction model to obtain the estimated query duration of the candidate query plans. Based on the estimated query duration of the candidate query plans, the model loss is determined, and the query plan prediction model is trained based on the model loss. Thus, the trained query plan prediction model can accurately predict the query duration of candidate query plans based on the system's resource load indicators, thereby improving the efficiency of data querying in the database.

[0296] The above text combined Figures 2 to 16 The method embodiments of this application are described in detail below, in conjunction with... Figure 17 and Figure 18 The following describes in detail the device embodiments of this application.

[0297] Figure 17 This is a schematic block diagram of a data query device provided in an embodiment of this application.

[0298] like Figure 17 As shown, the data query device 10 includes:

[0299] The acquisition unit 11 is used to acquire the resource load index of the database system at the current moment in response to the received query request;

[0300] The generation unit 12 is used to generate N candidate query plans for the query request, where N is a positive integer greater than 1;

[0301] Selection unit 13 is used to select a target query plan from the N candidate query plans based on the resource load index;

[0302] The query unit 14 is used to perform data query in the database based on the target query plan to obtain the query result of the query request.

[0303] In some embodiments, the selection unit 13 is specifically used to determine the query duration of each of the N candidate query plans based on the resource load index; and to select the target query plan from the N candidate query plans based on the query duration.

[0304] In some embodiments, the resource load metric includes at least one of the system-level bottleneck metric and the key time-consuming metric of the database system, wherein the key time-consuming metric refers to the time-consuming metric of each key step in the execution of the query plan by the database system.

[0305] In some embodiments, if the resource load indicators include system-level bottleneck indicators of the database system, the acquisition unit 11 is specifically used to acquire the collected system-level bottleneck indicators of the database system at different times in the current time period; for each of the system-level bottleneck indicators, the average value of the indicator at different times in the current time period is determined as the value of the indicator at the current time.

[0306] In some embodiments, if the resource load indicators include key time-consuming indicators of the database system, the acquisition unit 11 is specifically used to acquire the collected key time-consuming indicators of the database system when executing different query plans in the current time period; for each of the key time-consuming indicators, the average value of the indicator in the current time period is determined as the value of the indicator at the current moment.

[0307] In some embodiments, the selection unit 13 is specifically used to extract the feature information of the i-th candidate query plan among the N candidate query plans, where i is a positive integer from 1 to N; and to fuse the feature information of the i-th candidate query plan and the resource load index through a query plan prediction model to obtain the query duration of the i-th candidate query plan.

[0308] In some embodiments, the selection unit 13 is specifically used to standardize and normalize each of the resource load indicators to obtain resource load coding information; and to fuse the feature information of the i-th candidate query plan and the resource load coding information through a query plan prediction model to obtain the query duration of the i-th candidate query plan.

[0309] In some embodiments, the query plan prediction model includes a cross-attention module and a selection unit 13, which is specifically used to perform fusion processing on the feature information of the i-th candidate query plan and the resource load encoding information through the cross-attention module to obtain the cross-attention result of the i-th candidate query plan; and to determine the query duration of the i-th candidate query plan based on the cross-attention result of the i-th candidate query plan.

[0310] In some embodiments, the selection unit 13 is specifically used to multiply the feature information of the i-th candidate query plan with the key weight matrix and value weight matrix in the cross-attention module respectively to obtain the key matrix and value matrix corresponding to the i-th candidate query plan; multiply the resource load encoding information with the query weight matrix in the cross-attention module to obtain the query matrix corresponding to the i-th candidate query plan; and perform attention calculation on the key matrix, value matrix and query matrix corresponding to the i-th candidate query plan to obtain the cross-attention result of the i-th candidate query plan.

[0311] In some embodiments, the cross-attention module further includes a fully connected layer and a selection unit 13, which is specifically used to process the cross-attention result of the i-th candidate query plan through the fully connected layer to obtain the query duration of the i-th candidate query plan.

[0312] In some embodiments, the selection unit 13 is specifically used to extract features from the i-th candidate query plan using a feature extraction model to obtain feature information of the i-th candidate query plan.

[0313] In some embodiments, the feature extraction model includes a tree convolutional network and a pooling layer. The selection unit 13 is specifically used to capture the hierarchical relationship and semantic information in the tree structure of the i-th candidate query plan through the tree convolutional network to obtain the structural semantic information of the i-th candidate query plan; and to process the structural semantic information of the i-th candidate query plan through the pooling layer to obtain the feature information of the i-th candidate query plan.

[0314] In some embodiments, the generation unit 12 is specifically used to generate N candidate query plans for the query request through an optimizer, wherein the optimizer is trained based on N different prompts, and the prompts are used to describe the requirements that the candidate query plans need to meet.

[0315] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 17The apparatus shown can execute the above-described data query method embodiments, and the aforementioned and other operations and / or functions of each module in the apparatus are for implementing the above-described method embodiments, which will not be described in detail here for the sake of brevity.

[0316] Figure 18 This is a schematic block diagram of a model training device provided in an embodiment of this application.

[0317] like Figure 18 As shown, the model training device 20 includes:

[0318] The acquisition unit 21 is used to acquire training samples, which include: a set of resource load indicators of the database system and a candidate query plan. The candidate query plan is any one of N candidate query plans for a historical query request. The resource load indicators are a set of P sets of resource load indicators of the database system, where N and P are both positive integers greater than 1.

[0319] Extraction unit 22 is used to extract feature information of the candidate query plan;

[0320] The estimation unit 23 is used to fuse the feature information of the candidate query plan and the resource load index through the query plan estimation model to obtain the estimated query duration of the candidate query plan.

[0321] Training unit 24 is used to determine the model loss based on the estimated query duration of the candidate query plan, and to train the query plan prediction model based on the model loss.

[0322] In some embodiments, the training unit 24 is specifically used to determine the actual query duration of the candidate query plan under the resource load index; and to determine the model loss based on the estimated query duration and the actual query duration of the candidate query plan.

[0323] In some embodiments, the acquisition unit 21 is further configured to generate the P group resource load index by adjusting the state and database configuration of the database system.

[0324] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 18 The apparatus shown can execute the embodiments of the above-described model training method, and the foregoing and other operations and / or functions of each module in the apparatus are for implementing the above-described method embodiments, which will not be described in detail here for the sake of brevity.

[0325] The apparatus of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0326] Figure 19 This is a schematic block diagram of a computing device provided in an embodiment of this application, which is used to execute the above-described data query method or model training method.

[0327] like Figure 19 As shown, the computing device 40 may include:

[0328] The system includes a memory 41 and a processor 42. The memory 41 stores a computer program 43 and transfers the program code 43 to the processor 42. In other words, the processor 42 can retrieve and run the computer program 43 from the memory 41 to implement the methods described in the embodiments of this application.

[0329] For example, the processor 42 can be used to execute the steps in the method 200 described above according to the instructions in the computer program 43.

[0330] In some embodiments of this application, the processor 42 may include, but is not limited to:

[0331] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASIKs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0332] In some embodiments of this application, the memory 41 includes, but is not limited to:

[0333] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synkhronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synkh Link DRAM (SLDRAM), and Direct Memory Bus RAM (DR RAM).

[0334] In some embodiments of this application, the computer program 43 may be divided into one or more modules, which are stored in the memory 41 and executed by the processor 42 to complete the page recording method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 43 in the computing device 900.

[0335] like Figure 19 As shown, the computing device 40 may further include:

[0336] Transceiver 34, which can be connected to processor 42 or memory 41.

[0337] The processor 42 can control the transceiver 34 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 34 may include a transmitter and a receiver. The transceiver 34 may further include antennas, and the number of antennas may be one or more.

[0338] It should be understood that the various components in the computing device 40 are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0339] According to one aspect of this application, a computer storage medium is provided that stores a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0340] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in the above-described method embodiments.

[0341] In other words, when implemented using software, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disk (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0342] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0343] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0344] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0345] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data query method, characterized in that, include: In response to a received query request, obtain the resource load indicators of the database system at the current moment; Generate N candidate query plans for the query request, where N is a positive integer greater than 1; Based on the resource load metric, a target query plan is selected from the N candidate query plans; Based on the target query plan, a data query is performed in the database to obtain the query results of the query request.

2. The method according to claim 1, characterized in that, The step of selecting a target query plan from the N candidate query plans based on the resource load metric includes: Based on the resource load metric, determine the query duration of each of the N candidate query plans; Based on the query duration, the target query plan is selected from the N candidate query plans.

3. The method according to claim 1 or 2, characterized in that, The resource load indicators include at least one of the system-level bottleneck indicators and key time-consuming indicators of the database system. The key time-consuming indicators refer to the time consumption indicators of each key step when the database system executes the query plan.

4. The method according to claim 3, characterized in that, If the resource load metric includes the system-level bottleneck metric of the database system, obtaining the resource load metric of the database system at the current moment includes: Obtain the system-level bottleneck indicators of the collected database system at different times in the current time period; For each of the system-level bottleneck indicators, the average value of the indicator at different times in the current time period is determined as the value of the indicator at the current time.

5. The method according to claim 3, characterized in that, If the resource load metrics include key time-consuming metrics of the database system, obtaining the resource load metrics of the database system at the current moment includes: Obtain key time-consuming metrics of the collected database system when executing different query plans in the current time period; For each of the key time-consuming indicators, the average value of the indicator during the current time period is determined as the value of the indicator at the current moment.

6. The method according to claim 2, characterized in that, The step of determining the query duration of each of the N candidate query plans based on the resource load metric includes: For the i-th candidate query plan among the N candidate query plans, extract the feature information of the i-th candidate query plan, where i is a positive integer from 1 to N; The query duration of the i-th candidate query plan is obtained by fusing the feature information of the i-th candidate query plan and the resource load index through the query plan prediction model.

7. The method according to claim 6, characterized in that, The query duration of the i-th candidate query plan is obtained by fusing the feature information and resource load index of the i-th candidate query plan using a query plan prediction model, including: Each of the resource load indicators is standardized and normalized to obtain resource load coding information; The query duration of the i-th candidate query plan is obtained by fusing the feature information and resource load encoding information of the i-th candidate query plan using the query plan prediction model.

8. The method according to claim 7, characterized in that, The query plan prediction model includes a cross-attention module. The process of fusing the feature information of the i-th candidate query plan and the resource load encoding information using the query plan prediction model to obtain the query duration of the i-th candidate query plan includes: The cross-attention module fuses the feature information of the i-th candidate query plan and the resource load encoding information to obtain the cross-attention result of the i-th candidate query plan. Based on the cross-attention result of the i-th candidate query plan, the query duration of the i-th candidate query plan is determined.

9. The method according to claim 8, characterized in that, The step of performing cross-attention processing on the feature information and resource load encoding information of the i-th candidate query plan through the cross-attention module to obtain the cross-attention result of the i-th candidate query plan includes: The feature information of the i-th candidate query plan is multiplied by the key weight matrix and value weight matrix in the cross-attention module to obtain the key matrix and value matrix corresponding to the i-th candidate query plan; Multiply the resource load encoding information by the query weight matrix in the cross-attention module to obtain the query matrix corresponding to the i-th candidate query plan; Attention is calculated on the key matrix, value matrix, and query matrix corresponding to the i-th candidate query plan to obtain the cross-attention result of the i-th candidate query plan.

10. The method according to claim 8, characterized in that, The cross-attention module further includes a fully connected layer. The step of determining the query duration of the i-th candidate query plan based on the cross-attention result of the i-th candidate query plan includes: The cross-attention result of the i-th candidate query plan is processed through the fully connected layer to obtain the query duration of the i-th candidate query plan.

11. The method according to claim 7, characterized in that, The extraction of feature information of the i-th candidate query plan includes: The i-th candidate query plan is extracted using a feature extraction model to obtain its feature information.

12. The method according to claim 11, characterized in that, The feature extraction model includes a tree convolutional network and pooling layers. The feature extraction model is used to extract features from the i-th candidate query plan to obtain its feature information, including: The hierarchical relationships and semantic information in the tree structure of the i-th candidate query plan are captured through the tree convolutional network to obtain the structural semantic information of the i-th candidate query plan. The structural semantic information of the i-th candidate query plan is processed by the pooling layer to obtain the feature information of the i-th candidate query plan.

13. The method according to claim 1, characterized in that, The N candidate query plans that generate the query request include: The optimizer generates N candidate query plans for the query request. The optimizer is trained based on N different hints, which describe the requirements that the candidate query plans need to meet.

14. A model training method, characterized in that, include: Obtain training samples, which include: a set of resource load indicators of the database system and a candidate query plan. The candidate query plan is any one of N candidate query plans for a historical query request. The resource load indicators are a set of P sets of resource load indicators of the database system, where N and P are both positive integers greater than 1. Extract the feature information of the candidate query plan; By using a query plan prediction model, the feature information of the candidate query plan and the resource load index are fused to obtain the estimated query duration of the candidate query plan. Based on the estimated query duration of the candidate query plans, the model loss is determined, and the query plan prediction model is trained based on the model loss.

15. The method according to claim 14, characterized in that, The determination of model loss based on the estimated query duration of the candidate query plan includes: Determine the actual query duration of the candidate query plan under the resource load metric; The model loss is determined based on the estimated query duration and the actual query duration of the candidate query plan.

16. The method according to claim 14, characterized in that, The method further includes: The resource load index of group P is generated by adjusting the status and configuration of the database system.

17. A data query device, characterized in that, include: The acquisition unit is used to obtain the resource load indicators of the database system at the current moment in response to the received query request; A generation unit is used to generate N candidate query plans for the query request, where N is a positive integer greater than 1; The selection unit is used to select a target query plan from the N candidate query plans based on the resource load index. The query unit is used to perform data queries in the database based on the target query plan to obtain the query results of the query request.

18. A model training device, characterized in that, include: The acquisition unit is used to acquire training samples, which include: a set of resource load indicators of the database system and a candidate query plan. The candidate query plan is any one of N candidate query plans for a historical query request. The resource load indicators are a set of P sets of resource load indicators of the database system, where N and P are both positive integers greater than 1. Extraction unit, used to extract feature information of the candidate query plan; The estimation unit is used to fuse the feature information of the candidate query plan and the resource load index through a query plan estimation model to obtain the estimated query duration of the candidate query plan. The training unit is used to determine the model loss based on the estimated query duration of the candidate query plan, and to train the query plan prediction model based on the model loss.

19. A computing device, comprising a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the method as described in any one of claims 1 to 13 or 14 to 16.

20. A computer-readable storage medium, characterized in that, Used to store computer programs; The computer program causes the computer to perform the method as described in any one of claims 1 to 13 or 14 to 16.

Images