Database query method, device and storage medium based on large language model

Abstract

Disclosed are a database query method, device, and storage medium based on a large language model. The method includes: obtaining a query set and a materialized view set; training a query load trend prediction model based on the query set and a constructed event knowledge base; training a Bayesian network based on the materialized view set; receiving a target query, retrieving related events, and inputting the query load trend prediction model to obtain a first materialized view statement, which is converted into a first variable set and then input into a Bayesian network and configured with evidence to obtain a second variable set; executing a second materialized view statement converted from the second variable set; and rewriting subsequent queries that match the second materialized view statement into queries that read data from a materialized view table and executing them. The present disclosure introduces event factors to predict query load changes, breaking through the limitations of historical queries, using a large language model and a Bayesian network to generate high-quality materialized views, and improving database query efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of information retrieval technology, and in particular to a database query method, device, and storage medium based on a large language model. Background Art

[0002] Materialized views are an important technology for optimizing database query speed, especially for complex analytical queries. Database query workloads often contain numerous repeated subqueries, leading to unnecessary recalculation and wasting valuable computing resources. To address this issue, materialized view technology, based on the principle of trading space for time, precomputes and stores intermediate results for some queries, allowing future queries to directly utilize these precomputed results, effectively avoiding recalculation.

[0003] However, existing materialized view management methods primarily rely on mining subqueries from historical queries and generating materialized views based on these subqueries. This approach has significant historical limitations and lags, making it difficult to optimize newly emerging queries. This slows down user query analysis and reduces the database experience. To address this issue, researchers have proposed using n-gram models to model query loads and learn how query load characteristics shift over time. However, this approach has the disadvantage that the predicted query load characteristics do not include the query statements, query conditions, and other information required for materialized view generation, making them unsuitable for materialized view generation.

[0004] Subsequently, researchers proposed methods for predicting future queries to a certain extent, such as the QueryBot5000 method, which uses a logical combination of historical queries to predict the arrival rate of a certain type of query in the future. However, the queries predicted by this method are still extracted from historical queries and do not break through the limitations of the historical query set. As a result, the generated materialized views cannot be applied to optimize new queries that do not appear in historical queries. In addition, the design objectives of existing query load prediction methods do not consider the need to generate materialized views. As a result, the materialized views generated based on the query load predicted by existing methods perform poorly.

[0005] Therefore, a new technical solution is needed to solve the above problems. Summary of the Invention

[0006] The purpose of the present disclosure is to provide a database query method based on a large language model, which predicts the changing trend of query load by introducing event factors, and adopts a query prediction method based on a large language model and a materialized view generation method based on a Bayesian network, so as to break through the limitations of historical queries and predict and optimize future queries in a targeted manner.

[0007] According to one embodiment of the present disclosure, a database query method based on a large language model is proposed, comprising:

[0008] Obtaining a query set and a materialized view set, wherein the query set includes multiple queries and corresponding timestamps, and the materialized view set includes multiple materialized view statements and corresponding features and hit rates, wherein the materialized view statements are generated based on query conditions embedded in the queries in the query set;

[0009] A query load trend prediction model based on a large language model is trained based on the query set and the constructed event knowledge base. The query load trend prediction model is used to predict the change trend of queries under the influence of events and the materialized view statements that match the changed queries. The event knowledge base stores multiple events and corresponding time features.

[0010] Building and training a Bayesian network based on the materialized view set, wherein the Bayesian network is used to sample a variable set converted from the materialized view statements, features, and hit rates according to the configured evidence, and output a new variable set obtained by the sampling;

[0011] Receive a target query, retrieve events that are close in time to the target query and semantically related to the target query from an event knowledge base, input the target query and the retrieved events into a trained query load trend prediction model, predict a first materialized view statement, input a first variable set converted based on the first materialized view statement into a trained Bayesian network, configure evidence to set desired features and hit rates, and obtain a second variable set output by the Bayesian network;

[0012] Execute the second materialized view statement transformed by the second variable set in the database, generate the materialized view table and store it;

[0013] The subsequent queries that match the second materialized view statement are rewritten into queries that read data from the materialized view table and executed.

[0014] In some implementations, obtaining a materialized view set includes:

[0015] For each query in the query set, enumerate a subset of query conditions embedded in the query to form a subquery;

[0016] Convert the subquery into a materialized view statement;

[0017] The hit rate of the transformed materialized view statement for the queries in the query set is calculated, and the characteristics of the materialized view statement are determined.

[0018] In some embodiments, the method further comprises constructing the event knowledge base according to the following steps:

[0019] enumerate at least one category of events based on temporal characteristics;

[0020] Obtain relevant information of events of various categories under various time characteristics through search;

[0021] Use a large language model to extract event description text from search results;

[0022] Inputting the event description text and the domain features of the query set into a large language model, and cleaning and removing the event description text irrelevant to the domain features of the query set;

[0023] Use the text semantic encoding model to convert the cleaned event description text into the event feature vector;

[0024] An event knowledge base is constructed, wherein the event knowledge base stores event description text, event feature vectors and corresponding time features.

[0025] In some embodiments, training a query load trend prediction model based on a large language model based on the query set and the constructed event knowledge base includes:

[0026] Clustering queries in a query set;

[0027] Sample query pairs q in the same cluster i and q j , where t i <t j , t i and t j For query q i and query q j timestamp;

[0028] Retrieve and query q from the event knowledge base j Events g that are close in time and semantically related are obtained from the materialized view set and are related to the query q. j Matching materialized view statement v j , get the materialized view training set;

[0029] The query q i and event g as input, query q j As a label for the intermediate process, the materialized view statement v j As the final output label, the query load trend prediction model is trained step by step.

[0030] In some implementations, retrieving events that are close in time to the query and semantically related includes:

[0031] Use a large language model to convert the query to be retrieved into a natural language description, and use a text semantic encoding model to further convert the natural language description into a feature vector of the query to be retrieved;

[0032] The semantic similarity is calculated based on the Euclidean distance between the feature vector of the query to be retrieved and the feature vector of the event in the event knowledge base, and the temporal similarity is calculated based on the timestamp of the query to be retrieved and the temporal features of the event;

[0033] Calculating a comprehensive similarity score between the event and the query to be searched by combining the semantic similarity and the temporal similarity;

[0034] The event with the highest score is selected as the event that is close in time to the query to be retrieved and semantically related.

[0035] In some embodiments, the semantic similarity Dis(x g , x q ):

[0036]

[0037] Among them, x g is the feature vector of event g, x q is the feature vector of the query q to be retrieved, and d represents the dimension of the feature vector;

[0038] The time similarity Sim(t k , t j ):

[0039]

[0040] Among them, t q is the q timestamp of the query to be retrieved, t g is the time characteristic of event g, σ is the preset time window parameter;

[0041] The comprehensive similarity score between event g and query q is calculated based on the following formula:

[0042]

[0043] In some embodiments, building and training a Bayesian network includes:

[0044] Selecting materialized view statements with a hit rate exceeding a preset threshold from the materialized view set to train a Bayesian network;

[0045] Convert each selected materialized view statement and its corresponding features and hit rates into a variable set;

[0046] The transformed variable set is used as training samples to construct and train a Bayesian network to learn the network connection structure and conditional probability distribution.

[0047] In some implementations, the materialized view statement, characteristics, and hit rate are converted into a variable set in the following manner:

[0048] The field name, operator, and value triples corresponding to the equality-type query condition are used as query condition variables. The IN-type query condition is split into multiple equality-type query conditions. The triples corresponding to each equality-type query condition are also used as query condition variables. Each query condition variable is assigned a value. If the materialized view statement contains the query condition corresponding to the query condition variable, the value is assigned 1. If the materialized view statement does not contain the query condition corresponding to the query condition variable, the value is assigned 0.

[0049] Map the hit rate to the real number domain as the hit rate variable;

[0050] Map features to corresponding predefined value sets as feature variables.

[0051] In some implementations, configuring evidence includes:

[0052] Soft evidence and hard evidence are configured for the Bayesian network, where the soft evidence includes a hit rate and the hard evidence includes part or all of the features.

[0053] According to one embodiment of the present disclosure, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer instructions executable on the processor, and the processor is used to implement any of the above methods when executing the computer instructions.

[0054] According to one embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in any one of the above items is implemented.

[0055] The technical solution proposed in this disclosure has at least the following beneficial effects:

[0056] First, we introduce an event knowledge base to predict changes in query load. By retrieving events that are close in time to queries and semantically related, we can effectively predict query change trends caused by these events. This breaks through the limitations of historical query sets, allows us to predict and optimize new queries in advance, and improves the accuracy of query load prediction.

[0057] Second, this solution uses a large language model to understand query intent and events. This enables the model to accurately predict query trends based on events, effectively processing event information in natural language, and improving the ability to predict query load trends.

[0058] 3. This solution designs a generative materialized view model based on a Bayesian network. By learning variable representation and probabilistic dependencies, it can batch generate high-quality materialized views based on predicted query trends. This not only ensures the legitimacy of the generated materialized view statements, but also improves the hit rate of the materialized views, significantly enhancing the optimization effect of database queries.

[0059] Other features and advantages of the technical solution proposed in this disclosure are described in detail below. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0061] Figure 1 A flowchart of a database query method based on a large language model according to an embodiment of the present disclosure is shown.

[0062] Figure 2 A flow chart of generating a materialized view based on event-aware prediction according to an exemplary embodiment of the present disclosure is shown.

[0063] Figure 3 A schematic diagram of query load changes according to an exemplary embodiment of the present disclosure is shown.

[0064] Figure 4 A schematic diagram of a materialized view prediction framework according to an exemplary embodiment of the present disclosure is shown.

[0065] Figure 5 A schematic diagram of variable representation of a materialized view statement according to an exemplary embodiment of the present disclosure is shown.

[0066] Figure 6 A schematic diagram of a directed acyclic graph of a Bayesian network obtained according to an exemplary embodiment of the present disclosure is shown.

[0067] Figure 7 A flow chart of a materialized view generation process according to an exemplary embodiment of the present disclosure is shown.

[0068] Figure 8 It is a schematic structural diagram of an electronic device according to at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0069] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0070] The disclosed embodiments may be applied to a computer system / server that is operable with numerous other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with the computer system / server include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above, among others.

[0071] Computer systems / servers may be described in the general context of computer system-executable instructions, such as program modules, executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and the like, that perform specific tasks or implement specific abstract data types. Computer systems / servers may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communications network. In a distributed cloud computing environment, program modules may be located on local or remote computer system storage media, including storage devices.

[0072] The core concept of the database query solution based on a large language model proposed in this disclosure lies in: introducing event factors into the database query optimization process, using the large language model to understand the impact of events on query patterns, and breaking through the limitations of historical queries; and using Bayesian networks to probabilistically model the characteristics of materialized views, enabling batch generation of high-quality materialized views. The two complement each other: the former predicts changes in query trends, while the latter ensures the diversity and quality of the materialized views generated based on the predicted results. This solution combines semantic understanding with probabilistic reasoning to form a complete query optimization solution.

[0073] Figure 1 A flowchart of a database query method based on a large language model according to an embodiment of the present disclosure is shown. As shown in the figure, the method includes steps 1 to 6.

[0074] Step 1, obtain a query set and a materialized view set. The query set includes multiple queries and corresponding timestamps. The materialized view set includes multiple materialized view statements and corresponding features and hit rates. The materialized view statements are generated based on the query conditions embedded in the queries in the query set.

[0075] A query set can be input into the database. The query set can include multiple queries submitted by users, and each query has a corresponding timestamp, which is the time when the user submits the query. For example, the query is "SELECT COUNT(*)FROM user_search WHERE user_search.keyword IN('rose')", and the timestamp is February 14th. This query set is used for subsequent training of the query load trend prediction model.

[0076] In some embodiments, the materialized view set can be obtained through the following steps: for each query in the query set, enumerate subsets of the query conditions embedded in the query to form subqueries; convert the subqueries into materialized view statements; calculate the hit rate of the converted materialized view statements for the queries in the query set, and determine the features of the materialized view statements.

[0077] For example, for the query "SELECT COUNT(*)FROM users WHERE users.crowd_type IN('2')", the query conditions in its WHERE clause can be enumerated to form subqueries, and these subqueries are converted into materialized view statements.

[0078] The hit rate of the converted materialized view statements for the queries in the query set can be calculated according to the following method. For example, sort the queries in the query set by timestamp. In the time interval [t1, t |Q| , simulate the execution process of the query load, and calculate the quotient of the number of query hits of the materialized view statement converted from the subquery of query q i in the time interval divided by the number of query loads in this time interval (i.e., |Q|-i) as the hit rate corresponding to this materialized view statement.

[0079] The characteristics of the converted materialized view statement can be determined in the following ways. For example, the structure of the materialized view statement can be analyzed to determine its query category. For example, a materialized view statement containing aggregate functions such as COUNT and SUM can be determined as a statistical category, a materialized view statement containing a GROUP BY clause can be determined as a grouping category, and a materialized view statement containing only a WHERE clause can be determined as a filtering category. The time characteristics can be determined based on the timestamp of the original query corresponding to the materialized view statement. For example, the timestamp of the original query can be mapped to predefined time categories, such as holidays (New Year's Day, Spring Festival, Qingming Festival, May Day, Dragon Boat Festival, Mid-Autumn Festival, National Day), weekdays, weekends, etc., or mapped to specific time periods of the day, such as the morning rush hour (6:00-9:00) and working hours (9:00-17:00). The tables and fields involved in the materialized view statement can be analyzed to determine its business type. For example, fields involving user attributes can be determined as user profile types, and fields involving transaction-related fields can be determined as transaction statistics types.

[0080] Step 2: Based on the query set and the constructed event knowledge base, a query load trend prediction model based on a large language model is trained. The query load trend prediction model is used to predict the changing trend of queries under the influence of events and the materialized view statements matching the changed queries. The event knowledge base stores multiple events and corresponding time features.

[0081] This query load trend prediction model solves the following prediction task: predicting query trends when an event occurs. For example, queries in the "gift search analysis" category often shift in focus as different holidays approach. For example, a query like "SELECT COUNT(*) FROM user_search WHERE user_search.keyword IN('roses')" might shift to queries focused on products like "Christmas hats" during the "Christmas" event. This model not only predicts query trends but also generates materialized view statements that align with the query trend β. For example, it outputs the materialized view statement "CREATE MATERIALIZEDVIEW MV AS SELECT COUNT(*) FROM users, search WHERE age IN('2', '3') AND keyword IN('Christmas hats')."

[0082] Events that affect query load trends (such as "Christmas is coming" in the example above) are stored in an event knowledge base. Each entry in the event knowledge base includes an event and a corresponding time feature, which is used to represent the time when the event occurred.

[0083] In some implementations, the event knowledge base may be constructed by the following steps:

[0084] enumerate at least one category of events based on temporal characteristics;

[0085] Obtain relevant information of events of various categories under various time characteristics through search;

[0086] Use a large language model to extract event description text from search results;

[0087] Inputting the event description text and the domain features of the query set into a large language model, and cleaning and removing the event description text irrelevant to the domain features of the query set;

[0088] Use the text semantic encoding model to convert the cleaned event description text into the event feature vector;

[0089] An event knowledge base is constructed, wherein the event knowledge base stores event description text, event feature vectors and corresponding time features.

[0090] You can select the category of interest as needed, such as seasons, solar terms, festivals, and / or activities. For each category, the events of each category are enumerated based on the time feature (e.g., all dates in a year), such as "January 1st", "Seasons on January 1st", "Solar terms on January 1st", "Festivals on January 1st", and / or "Activities on January 1st".

[0091] These categories of events can then be searched using, for example, an internet search engine to retrieve relevant webpage content. These webpages can contain relevant information for each category of date, such as definitions, descriptions, corresponding national holidays, holiday customs, activities, major historical events, and / or famous people.

[0092] Next, a large language model is used to extract event description text from the search results. The extracted event description text is fed into the large language model along with the domain features of the query set to clean and remove event description text irrelevant to the domain. Domain features are summarized and extracted from the query set, representing the data scope of the query or the business scope of the user submitting the query. For example, a domain feature such as "gift search analysis" can be extracted from queries analyzing product keywords and user behavior. By cleaning the data for the "gift search analysis" domain feature, event description text related to holiday shopping, gift promotions, and other events can be retained while removing event description text irrelevant to the domain.

[0093] A text semantic encoding model can be used to convert the cleaned event description text into a feature vector. The event description text, feature vector, and time features are stored in a database and indexed to build an event knowledge base. Each entry can be represented as (t, g), where t is the time feature and g includes the event description text and the event feature vector.

[0094] In some embodiments, training a query load trend prediction model based on a large language model based on the query set and the constructed event knowledge base includes:

[0095] Clustering queries in a query set;

[0096] Sample query pairs q in the same cluster i and q j , where t i <t j , t i and t j For query q i and query q j timestamp;

[0097] Retrieve and query q from the event knowledge base j Events g that are close in time and semantically related are obtained from the materialized view set and are related to the query q. j Matching materialized view statement v j , get the materialized view training set;

[0098] The query q i and event g as input, query q j As a label for the intermediate process, the materialized view statement v j As the final output label, the query load trend prediction model is trained step by step.

[0099] The query load trend prediction model based on a large language model includes a vector embedding neural network, an attention mechanism transformation matrix, an attention mechanism weighted average matrix, and an output neural network. The model accepts text input and outputs text.

[0100] It will be understood by those skilled in the art that the query q i The query statement itself is used as the input of the query load trend prediction model, and the query q i The transformed natural language description is used as the input of the query load trend prediction model. This application does not limit this. Those skilled in the art can convert the query q into i The specific form of is used as the input of the query load trend prediction model. The event description text of event g can be used as the input of the query load trend prediction model.

[0101] Get the same query q from the materialized view collection j When matching materialized view statements, you can retrieve j Materialized view statements, such as enumerating materialized views and selecting j Optimize the query execution time of the largest materialized view statement as v j.

[0102] In some embodiments, the following time-weighted event retrieval method can be used to retrieve and query q j Events that are close in time and semantically related:

[0103] Use a large language model to retrieve the query q j Convert it into a natural language description, and use the text semantic encoding model to further convert the natural language description into the query to be retrieved q j The eigenvector of

[0104] Based on the query to be retrieved q j The semantic similarity is calculated based on the Euclidean distance between the feature vector of the event and the feature vector of the event in the event knowledge base, and the semantic similarity is calculated based on the query to be retrieved q j Calculate the temporal similarity between the timestamp and the temporal features of the event;

[0105] Combine the semantic similarity and the temporal similarity to calculate the event and the query to be retrieved q j The comprehensive similarity score between

[0106] The event with the highest score is selected as the event that is close in time to the query to be retrieved and semantically related.

[0107] In some embodiments, the semantic similarity Dis(x g , x q ):

[0108]

[0109] Among them, x g is the feature vector of event g, x q is the feature vector of the query q to be retrieved, and d represents the dimension of the feature vector.

[0110] In some embodiments, the time similarity Aim(t k , t j ):

[0111]

[0112] Among them, t q is the q timestamp of the query to be retrieved, t g is the time characteristic of event g, and σ is the preset time window parameter, which can be 7 days, for example.

[0113] The comprehensive similarity score between event g and query q can be calculated based on the following formula:

[0114]

[0115] In a specific example, we can first calculate the semantic similarity Dis(x g , x q ) Preliminarily screen out multiple events that are semantically related to the query to be retrieved from the event knowledge base. Then calculate the comprehensive similarity score Score(t k , t j ,g,q), select the event with the highest score as the event that is close in time to the query to be retrieved and semantically related.

[0116] During the training phase, the single event with the highest score can be selected as the event that is close in time and semantically relevant to the query to be retrieved. When using the trained model for inference, multiple (for example, three) events with the highest scores can be selected as events that are close in time and semantically relevant to the query to be retrieved, further improving the coverage of query predictions.

[0117] The large language model used in the process of building the event knowledge base and the large language model used in the process of converting the query into the feature vector are preferably the same large language model. Similarly, the text semantic encoding model used when generating the feature vector of the event and the text semantic encoding model used when generating the feature vector of the query are preferably the same text semantic encoding model. This helps to ensure the consistency of the semantic space, ensure that the event and the query are mapped to the same semantic space, facilitate the measurement of similarity based on their feature vectors, avoid differences in the semantic representations and vectors generated by different models, and thus enhance the robustness of the system.

[0118] After obtaining the training samples, the following prompt words can be designed to convert the training samples into natural language form and input into the query load trend prediction model based on the large language model. The large language model used here can be different from the large language model used in the process of building the event knowledge base and obtaining the query feature vector.

[0119] A prompt word can contain three parts: instruction, context, and answer.

[0120] The instruction can include the materialized view prediction task objectives and prediction step instructions.

[0121] The context may include a materialized view profile, an event g, and a query q. i content.

[0122] The answer may include a description of the reasoning steps, intermediate results (predicted q j ) and the final answer (the final predicted target materialized view v j ).

[0123] The model is fed with the prompt text and the error is calculated against the expected output. The model parameters are optimized using a gradient descent algorithm. The model's effectiveness is evaluated by testing how well the generated materialized view matches queries after the corresponding timestamp in the query payload.

[0124] The prompt word template given according to an example is as follows: (the content in “<>” is filled in according to the training sample):

[0125] Instruction: Based on the provided query q i and future events g to predict the future materialized view. Follow the steps below to perform step-by-step reasoning: Step 1 - Understand user intent: q i Interpret it as a natural language description and infer the user's background and query intention. Infer the query category of qi. Step 2 - Integrate event factor knowledge: i Combined with the relevant event knowledge g, reason about how g affects the query trend. Step 3 - Predicting the future materialized view: Based on the integrated event g and user intent, predict q i Query after future changes q j . Recommend query q that can be optimized after the change j Materialized view v j Step 4 - Evaluate and Output Materialized View: Evaluate the materialized view v j Query availability and value in the load after the change. Output the materialized view statement after the "answer" delimiter.

[0126] Context: Introduction to the materialized view concept: <Introduction to Materialized View Technology Principles>. Related event factor knowledge <event g>. Old query q i : <old query in the old and new query pair>.

[0127] Model answer: q i The explanation is i The query category is <the “user category” data in the query feature>. The event related to the query is <event g>. Under the influence of the event, q i Changes to <q in the query pair j >. In order to optimize the query after the change, it is recommended to generate a materialized view <Materialized View v j >. The estimated hit rates for these materialized views are <v j The hit rate p>. Answer: <v j >.

[0128] Step 3: construct and train a Bayesian network based on the materialized view set. The Bayesian network is used to sample the variable set converted from the materialized view statements, features, and hit rates according to the configured evidence, and output a new variable set obtained by sampling. ​

[0129] A Bayesian network is a probabilistic graphical model structured as a directed acyclic graph. Nodes represent variables, and edges represent dependencies between variables. Each node has a corresponding conditional probability distribution table, describing the probability distribution of how the node's value depends on the value of its parent node. Bayesian networks simplify the representation of joint probability distributions based on the assumption of conditional independence between variables. They can also predict the values ​​of other nodes through configuration evidence (known node value information) and probabilistic reasoning. Hard evidence ensures that nodes take certain values, while soft evidence ensures that node values ​​conform to a specific probability distribution.

[0130] In some embodiments, constructing and training a Bayesian network may include: selecting materialized view statements with a hit rate exceeding a preset threshold from the materialized view set to train the Bayesian network; converting each selected materialized view statement and corresponding features and hit rate into a variable set; and using the converted variable set as training samples to construct and train the Bayesian network to learn the network connection structure and conditional probability distribution.

[0131] According to this embodiment, materialized view statements with a hit rate exceeding a preset threshold are first selected from the materialized view set as Bayesian network training data. Each selected materialized view statement and its corresponding features and hit rate are then converted into variable representations. In some embodiments, the conversion can be performed as follows:

[0132] Query condition conversion: Convert the equality type query condition into a (field name, operator, value) triple as the query condition variable. Split the IN type query condition into multiple equality query conditions and perform the same conversion. For example, the query conditions "age=1" and "price>10" are represented as var age,＝,1 ∈{0,1} and var price,>,10 ∈{0,1}, the query condition "ageIN(2,3)" will be converted to "age=2 OR age=3" and represented as var age,＝,2 ∈{0,1} and var age,＝,3 ∈{0,1}. When the materialized view statement contains the query condition corresponding to the query condition variable, the query condition variable takes the value of 1, otherwise it takes the value of 0;

[0133] Hit rate conversion: Map the hit rate to the real number domain, for example, expressed as var p ∈R;

[0134] Feature conversion: Mapping features to a predefined set of values, for example, querying a category variable can be represented as var cate ∈[1,|categories|], for example, time information can be mapped to a predefined time set, represented as var time_seg∈{"0~5","6~11","12~17","18~23"}, for example, the query template can be mapped to the template number set, represented by var tmpl ∈[1,|templates|].

[0135] For example, for Figure 5 In the materialized view statement 1 "CREATE...age IN(1,2,3)AND crowd_id IN(343,596)", its hit rate is 0.4, the query category code is 2, the time period is '6~11', and the query template number is 2, which can be expressed as a variable set {var p =0.4,var cate =2,var time_seg =′6-11′,var tmpl =2,var age,＝,1 =

[0136] 1,var age,＝,2 =1,var age,＝,3 =1,var age,＝,4 =0,var crowd_id,＝,343 =

[0137] 1,var crowd_id,＝,596 =1}.

[0138] Step 4: Receive the target query, retrieve events that are close in time to the target query and semantically related from the event knowledge base, input the target query and the retrieved events into the trained query load trend prediction model, predict a first materialized view statement, and input the first variable set converted based on the first materialized view statement into the trained Bayesian network. Configure evidence to set the desired features and hit rate to obtain a second variable set output by the Bayesian network.

[0139] In one example, the target query can first be converted from an SQL statement into a feature vector. For example, the query condition can be converted into a more semantically rich form, such as converting "age=2" to "age=10-19 years old." A large language model is then used to convert the converted query into a natural language description. Finally, a text semantic encoding model is used to convert the natural language description into a query feature vector. The event knowledge base stores event text descriptions, event feature vectors, and temporal features. A time-weighted event retrieval method can be used, as described above, to calculate a comprehensive similarity score between the target query and the events. The three events with the highest scores are selected as events that are close in time and semantically relevant to the target query. For example, for queries in the "gift search analysis" category, the three events with the highest comprehensive similarity scores retrieved near December 25th are: Christmas events, year-end sales, and winter flu alerts. These events indicate that the query may be interested in holiday items such as "Christmas hats," discounted items, and warm and health-related items.

[0140] Pairing the target query with each of the three retrieved events and inputting them into the query load trend prediction model yields three corresponding first materialized view statements. These first materialized view statements can each include keywords related to the three different events, further improving the query prediction coverage.

[0141] A first variable set can be generated based on the first materialized view statement, and soft evidence and hard evidence can be configured for the Bayesian network. In some implementations, a higher hit rate can be set using soft evidence, which helps improve the hit rate of the resulting materialized view. Hard evidence can be used to enumerate desired feature combinations, such as all or some combinations of some or all of time, query templates, and query categories, which helps increase the richness of the generated second variable set and the diversity of the resulting materialized view.

[0142] Under the same input and evidence configuration, multiple sampling can be performed, each time generating a second variable set. For example, if the input variable set and evidence configuration remain unchanged, sampling can be performed 1000 times to generate 1000 new variable sets, corresponding to the expansion of 1000 materialized view statements.

[0143] Step 5: Execute the second materialized view statement transformed by the second variable set in the database to generate and store the materialized view table.

[0144] The process of converting the second variable set into the second materialized view statement can be done by using the reverse process of converting the materialized view statement into the variable set described above, which will not be described in detail here. The second materialized view statement is executed in the database to generate a materialized view table and store it in the database. For example, executing the statement "CREATE MATERIALIZEDVIEW MV ASSELECT COUNT(*)FROM users,search WHERE age IN('2','3')and keyword IN('Christmas hat')" generates a materialized view table named MV, which pre-calculates and stores statistical results that meet the age and keyword conditions. By storing these pre-calculated results in the database, future queries can directly use these results to avoid repeated calculations.

[0145] Step 6: Rewrite the subsequent queries that match the second materialized view statement into queries that read data from the materialized view table and execute the queries.

[0146] Those skilled in the art may select an appropriate strategy to determine whether a subsequent query matches the second materialized view statement. For example, in some examples, when the query conditions of a subsequent query include the query conditions of the second materialized view statement, it can be considered that the query can be optimized by the materialized view. In this case, the subsequent query can be rewritten into a query that reads pre-calculated results from the matching materialized view table, and the rewritten query can be executed, and the query results can be directly returned to the user, thereby avoiding repeated calculations and significantly improving query efficiency.

[0147] Figure 1 The illustrated embodiment has at least the following beneficial effects: an event knowledge base is introduced to predict changes in query load, and by retrieving events that are close to the query time and semantically related, the query change trend under the influence of the event can be effectively predicted, and the limitations of the historical query set can be broken through, and new queries can be predicted and optimized in advance, thereby improving the accuracy of query load prediction; a large language model is used to understand query intent and events, so that the model can accurately predict the query change trend based on events, effectively process event information in natural language form, and improve the ability to predict the query load change trend; a generative materialized view model based on a Bayesian network is designed, and through the learning of variable representation and probabilistic dependencies, high-quality materialized views can be generated in batches based on predicted query trends, which not only ensures the legitimacy of the generated materialized view statements, but also improves the hit rate of the materialized view, significantly improving the optimization effect of database queries.

[0148] Figure 2 The flowchart of generating a materialized view based on event-aware prediction according to an exemplary embodiment of the present disclosure is shown, which mainly includes three parts:

[0149] Input: Time series query load;

[0150] Core processing modules (in the middle dashed box): document parsing, event knowledge base, vector retrieval, query model prediction (the example event is "New Year's Day on January 1st"), and large language model;

[0151] Output: Generated materialized view.

[0152] This process is guided by events to achieve prediction generation from query load to materialized views.

[0153] Figure 3 A schematic diagram of query load changes according to an exemplary embodiment of the present disclosure is shown.

[0154] Figure 4 A schematic diagram of the overall framework of materialized view prediction according to an exemplary embodiment of the present disclosure is shown, which includes three parts.

[0155] The top section mainly shows the event knowledge base construction and time series prediction. The left side shows the event knowledge base construction process. The middle section shows query load trend mining, which is the time-weighted event retrieval method mentioned above. The right side shows the use of a large language model for prediction.

[0156] The middle part mainly includes data processing and feature representation, the left side shows query clustering, the middle is materialized view mining, that is, materialized view matching, and the right side is materialized view generation based on Bayesian network.

[0157] The bottom part shows the inference flow, which illustrates the complete process from query input to the final generation of materialized views.

[0158] Figure 4 The medium blue portion represents the processing related to the query load trend prediction model based on the large language model, and the orange portion represents the processing related to the Bayesian network.

[0159] Figure 5 A variable diagram of a materialized view statement according to an exemplary embodiment of the present disclosure is shown, including two materialized view statements and their variable representations. Materialized view statement 1 has been described in detail above, and materialized view statement 2 is similar thereto.

[0160] Figure 6 A schematic diagram of a directed acyclic graph of a Bayesian network obtained according to an exemplary embodiment of the present disclosure is shown.

[0161] Figure 7A block diagram of the materialized view generation process according to an exemplary embodiment of the present disclosure is shown. As shown in the figure, the event knowledge base and the event retrieval process are shown in the upper part; the large language model processes the historical query load in the middle part to generate the materialized view seeds; the processing process of the Bayesian network is shown in the lower part. Starting from the old materialized view, the query template and variables 1, 2, and 3 are sampled to generate a new materialized view. Figure 7 The overall evolution process from query load i - 1 to query load i is shown.

[0162] A specific query process according to the above embodiment is given below.

[0163] (a) Input a query set {q1, q2, q3, q4, q5, q6, q7} containing timestamps into the database. The query set is as Figure 3 shown, Figure 3 where q1 to q7 are arranged from top to bottom in the figure. The timestamp of query q6 "SELECT COUNT(*) FROM user_search WHERE user_search.keyword IN ('rose')" is February 14th.

[0164] (b) Inductively extract domain features from the query set. For example, extract the "gift search analysis" domain feature from {q6, q7}. Use the text semantic encoding model to encode the query into a feature vector, and retrieve relevant events in the event knowledge base based on the time-weighted event retrieval method. For example, retrieve the Christmas event on December 25th.

[0165] (c) Extract subqueries from the queries in the query set. For example, the subquery "SELECT COUNT(*) FROM users WHERE users.crowd_type IN ('2')" that appears 3 times in {q1, q2, q3}. Convert these subqueries into materialized view statements, calculate the hit rate, and determine the features to form a materialized view set.

[0166] (d) Use the trained query load trend prediction model to predict the change trend of the query load. For example, for the query q6 of the "gift search analysis" type, predict that its focus will shift to "Christmas hats" near Christmas. Accordingly, generate the materialized view statement "CREATE MATERIALIZED VIEW MV AS SELECT COUNT(*) FROM users, search WHERE age IN ('2', '3') and keyword IN ('Christmas hats')".

[0167] (e) Convert the materialized view statement into a variable set. For example, for a materialized view statement containing "age IN(1,2,3)" and "crowd_id IN(343,596)", its hit rate is 0.4, the category code is 2, and the time period is '6-11', it can be represented as a variable set

[0168]

[0169] (f) Input the variable set into the trained Figure 6 The Bayesian network shown in the figure configures evidence and samples to generate a new set of variables, which are then converted into materialized view statements. These materialized view statements are consistent with the predicted query load change trend.

[0170] (g) Execute the materialized view statement in the database to generate the materialized view table. For example, if the subsequent query is "SELECT COUNT(*) FROM users, search WHERE age IN('2','3') and keyword IN('Christmas hat')", it can be rewritten as "SELECT * FROM MV" to read the results directly from the materialized view table, achieving efficient query.

[0171] Figure 8 An electronic device provided for at least one embodiment of the present disclosure includes a memory and a processor, wherein the memory is used to store computer instructions that can be executed on the processor, and the processor is used to implement the database query method based on a large language model described in any embodiment or implementation of the present disclosure when executing the computer instructions.

[0172] At least one embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the database query method based on a large language model described in any embodiment or implementation of the present disclosure.

[0173] Those skilled in the art will appreciate that one or more embodiments of this specification may be provided as a method, system, or computer program product. Thus, one or more embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the data processing device embodiment is generally similar to the method embodiment, so its description is relatively simple. For relevant portions, refer to the description of the method embodiment.

[0175] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0176] Embodiments of the subject matter and functional operations described in this specification may be implemented in the following: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier to be executed by a data processing device or to control the operation of the data processing device. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information and transmit it to a suitable receiver device for execution by the data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0177] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform the corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0178] Computers suitable for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or the computer will be operably coupled to such mass storage devices to receive data from them or to transmit data to them, or both. However, a computer does not necessarily have such devices. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0179] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0180] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of specific embodiments of specific inventions. Certain features described in multiple embodiments within this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claimed as such, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of the sub-combination.

[0181] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0182] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential sequence to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0183] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included in the scope of protection of one or more embodiments of this specification.

Claims

1. A database query method based on a large language model, characterized in that: include: Obtaining a query set and a materialized view set, wherein the query set includes multiple queries and corresponding timestamps, and the materialized view set includes multiple materialized view statements and corresponding features and hit rates, wherein the materialized view statements are generated based on query conditions embedded in the queries in the query set; A query load trend prediction model based on a large language model is trained based on the query set and the constructed event knowledge base. The query load trend prediction model is used to predict the change trend of queries under the influence of events and the materialized view statements that match the changed queries. The event knowledge base stores multiple events and corresponding time features. Building and training a Bayesian network based on the materialized view set, wherein the Bayesian network is used to sample a variable set converted from the materialized view statements, features, and hit rates according to the configured evidence, and output a new variable set obtained by the sampling; Receive a target query, retrieve events that are close in time to the target query and semantically related to the target query from an event knowledge base, input the target query and the retrieved events into a trained query load trend prediction model, predict a first materialized view statement, input a first variable set converted based on the first materialized view statement into a trained Bayesian network, configure evidence to set desired features and hit rates, and obtain a second variable set output by the Bayesian network; Execute the second materialized view statement transformed by the second variable set in the database, generate the materialized view table and store it; The subsequent queries that match the second materialized view statement are rewritten into queries that read data from the materialized view table and executed.

2. The method according to claim 1, characterized in that Obtaining a materialized view collection includes: For each query in the query set, enumerate a subset of query conditions embedded in the query to form a subquery; Convert the subquery into a materialized view statement; The hit rate of the transformed materialized view statement for the queries in the query set is calculated, and the characteristics of the materialized view statement are determined.

3. The method according to claim 1, characterized in that The method further comprises constructing the event knowledge base according to the following steps: enumerate at least one category of events based on temporal characteristics; Obtain relevant information of events of various categories under various time characteristics through search; Use a large language model to extract event description text from search results; Inputting the event description text and the domain features of the query set into a large language model, and cleaning and removing the event description text irrelevant to the domain features of the query set; Use the text semantic encoding model to convert the cleaned event description text into the event feature vector; An event knowledge base is constructed, wherein the event knowledge base stores event description text, event feature vectors and corresponding time features.

4. The method according to claim 1, wherein Training a query load trend prediction model based on a large language model based on the query set and the constructed event knowledge base includes: Clustering queries in a query set; Sample query pairs q in the same cluster i and q j , where t i <t j , t i and t j For query q i and query q j timestamp; Retrieve and query q from the event knowledge base j Events g that are close in time and semantically related are obtained from the materialized view set and are related to the query q. j Matching materialized view statement v j , get the materialized view training set; The query q i and event g as input, query q j As a label for the intermediate process, the materialized view statement v j As the final output label, the query load trend prediction model is trained step by step.

5. The method according to claim 1, wherein Retrieve events that are close in time to the query and semantically related, including: Use a large language model to convert the query to be retrieved into a natural language description, and use a text semantic encoding model to further convert the natural language description into a feature vector of the query to be retrieved; The semantic similarity is calculated based on the Euclidean distance between the feature vector of the query to be retrieved and the feature vector of the event in the event knowledge base, and the temporal similarity is calculated based on the timestamp of the query to be retrieved and the temporal features of the event; Calculating a comprehensive similarity score between the event and the query to be searched by combining the semantic similarity and the temporal similarity; The event with the highest score is selected as the event that is close in time to the query to be retrieved and semantically related.

6. The method according to claim 5, characterized in that: The semantic similarity Dis(x g , x q ): Among them, x g is the feature vector of event g, x q is the feature vector of the query q to be retrieved, and d represents the dimension of the feature vector; The time similarity Sim(t k , t j ): Among them, t q is the q timestamp of the query to be retrieved, t g is the time characteristic of event g, σ is the preset time window parameter; The comprehensive similarity score between event g and query q is calculated based on the following formula:

7. The method according to claim 1, characterized in that Building and training a Bayesian network involves: Selecting materialized view statements with a hit rate exceeding a preset threshold from the materialized view set to train a Bayesian network; Convert each selected materialized view statement and its corresponding features and hit rates into a variable set; The transformed variable set is used as training samples to construct and train a Bayesian network to learn the network connection structure and conditional probability distribution.

8. The method according to claim 1, characterized in that Materialized view statements, characteristics, and hit rates are converted to variable sets in the following way: The field name, operator, and value triples corresponding to the equality-type query condition are used as query condition variables. The IN-type query condition is split into multiple equality-type query conditions. The triples corresponding to each equality-type query condition are also used as query condition variables. Each query condition variable is assigned a value. If the materialized view statement contains the query condition corresponding to the query condition variable, the value is assigned 1. If the materialized view statement does not contain the query condition corresponding to the query condition variable, the value is assigned 0. Map the hit rate to the real number domain as the hit rate variable; Map features to corresponding predefined value sets as feature variables.

9. The method according to claim 1, characterized in that Configuration evidence includes: Soft evidence and hard evidence are configured for the Bayesian network, where the soft evidence includes a hit rate and the hard evidence includes part or all of the features.

10. An electronic device, characterized in that: The device includes a memory and a processor, wherein the memory is used to store computer instructions that can be executed on the processor, and the processor is used to implement the method according to any one of claims 1 to 9 when executing the computer instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images