Database query method and device based on large language model and storage medium

Abstract

The invention discloses a database query method and device based on a large language model and a storage medium. The method comprises the steps of obtaining a query set and a materialized view set; training a query load trend prediction model based on the query set and the constructed event knowledge base; training a Bayesian network based on the materialized view set; receiving a target query, retrieving a related event, inputting a query load trend prediction model to obtain a first materialized view statement, converting the first materialized view statement into a first variable set, inputting the first variable set into a Bayesian network, and configuring evidence to obtain a second variable set; executing a second materialized view statement converted from the second set of variables; and rewriting a subsequent query matched with the second materialized view statement into a query for reading data from the materialized view table and executing the query. According to the method, event factors are introduced to predict the query load change, the limitation of historical query is broken through, the high-quality materialized view is generated by using the large language model and the Bayesian network, and the database query efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of information retrieval technology, and in particular to a database query method, device and storage medium based on a large language model. Background Art

[0002] Materialized views are an important technology for optimizing query speed in databases, especially for complex analytical queries. Database query loads often contain a large number of repeated subqueries, which leads to unnecessary repeated calculations and wastes valuable computing resources. To solve this problem, materialized view technology pre-calculates and stores the intermediate results of some queries based on the principle of using space for time, so that future queries can directly use these pre-calculated results, thereby effectively avoiding repeated calculations.

[0003] However, existing materialized view management methods mainly rely on mining subqueries from historical queries and generating materialized views based on these subqueries. This method has strong historical limitations and lags, and it is difficult to optimize newly emerging queries. As a result, the user's query analysis speed slows down and the database usage experience deteriorates. To solve this problem, researchers proposed using the n-gram model to model the query load and learn the offset of the query load characteristics over time. However, its disadvantage is that the predicted query load characteristics do not contain the query statements, query conditions and other information required for materialized view generation, and therefore cannot be applied to the generation of materialized views.

[0004] Subsequently, researchers proposed to predict future queries to a certain extent, such as the QueryBot5000 method, which uses a logical combination method on historical queries to predict the arrival rate of a certain type of query in the future. However, the queries predicted by this method are still extracted from historical queries, without breaking through the limitations of the historical query set, resulting in the inability to apply the generated materialized views to optimize new queries that did not appear in historical queries. In addition, the design goals of existing query load prediction methods do not take into account the need to generate materialized views, and the materialized views generated based on the query load predicted by existing methods perform poorly.

[0005] Therefore, a new technical solution is needed to solve the above problems. Summary of the invention

[0006] The purpose of the present disclosure is to provide a database query method based on a large language model, which predicts the changing trend of the query load by introducing event factors, and adopts a query prediction method based on a large language model and a materialized view generation method based on a Bayesian network, so as to break through the historical query limitations and predict and optimize future queries in a targeted manner.

[0007] According to an embodiment of the present disclosure, a database query method based on a large language model is proposed, comprising:

[0008] Acquire a query set and a materialized view set, wherein the query set includes multiple queries and corresponding timestamps, and the materialized view set includes multiple materialized view statements and corresponding features and hit rates, wherein the materialized view statements are generated based on query conditions embedded in the queries in the query set;

[0009] Based on the query set and the constructed event knowledge base, a query load trend prediction model based on a large language model is trained, wherein the query load trend prediction model is used to predict the change trend of queries under the influence of events and the materialized view statements matching the changed queries, wherein the event knowledge base stores multiple events and corresponding time features;

[0010] Building and training a Bayesian network based on the materialized view set, the Bayesian network is used to sample a variable set converted from materialized view statements, features, and hit rates according to configured evidence, and output a new variable set obtained by sampling;

[0011] Receive a target query, retrieve events that are close in time to the target query and semantically related from an event knowledge base, input the target query and the retrieved events into a trained query load trend prediction model, predict and obtain a first materialized view statement, input a first variable set converted based on the first materialized view statement into a trained Bayesian network, configure evidence to set expected features and hit rates, and obtain a second variable set output by the Bayesian network;

[0012] Execute the second materialized view statement transformed by the second variable set in the database, generate the materialized view table and store it;

[0013] A query in a subsequent query that matches the second materialized view statement is rewritten into a query that reads data from the materialized view table and executed.

[0014] In some implementations, obtaining a materialized view set includes:

[0015] For each query in the query set, enumerate a subset of query conditions embedded in the query to form a subquery;

[0016] Convert the subquery into a materialized view statement;

[0017] The hit rate of the transformed materialized view statement for the query in the query set is calculated, and the characteristics of the materialized view statement are determined.

[0018] In some implementations, the method further comprises constructing the event knowledge base according to the following steps:

[0019] enumerating at least one category of events based on a temporal feature;

[0020] Obtain relevant information of events of various categories under various time characteristics through search;

[0021] Use a large language model to extract event description text from search results;

[0022] Inputting the event description text and the domain features of the query set into a large language model, and cleaning and removing the event description text irrelevant to the domain features of the query set;

[0023] Use the text semantic encoding model to convert the cleaned event description text into the feature vector of the event;

[0024] An event knowledge base is constructed, wherein the event knowledge base stores event description text, event feature vectors, and corresponding time features.

[0025] In some implementations, training a query load trend prediction model based on a large language model based on the query set and the constructed event knowledge base includes:

[0026] Clustering queries in a query set;

[0027] Sample query pairs q in the same cluster i and q j , where t i <t j , t i and t j The query q i and query q j timestamp;

[0028] Retrieve and query q from the event knowledge base j Events g that are close in time and semantically related are obtained from the materialized view set and are related to the query q j Matching materialized view statement v j , get the materialized view training set;

[0029] The query q i and event g as input, and query q j As a label for the intermediate process, the materialized view statement v j As the final output label, the query load trend prediction model is trained step by step.

[0030] In some implementations, retrieving events that are close in time to the query and semantically related includes:

[0031] Use a large language model to convert the query to be retrieved into a natural language description, and use a text semantic encoding model to further convert the natural language description into a feature vector of the query to be retrieved;

[0032] The semantic similarity is calculated based on the Euclidean distance between the feature vector of the query to be retrieved and the feature vector of the event in the event knowledge base, and the temporal similarity is calculated based on the timestamp of the query to be retrieved and the temporal feature of the event;

[0033] Calculating a comprehensive similarity score between the event and the query to be searched by combining the semantic similarity and the temporal similarity;

[0034] The event with the highest score is selected as the event that is close in time to the query to be retrieved and semantically related.

[0035] In some embodiments, the semantic similarity Dis(x g , x q ):

[0036]

[0037] Among them, x g is the feature vector of event g, x q is the feature vector of the query q to be retrieved, and d represents the dimension of the feature vector;

[0038] The temporal similarity Sim(t k , t j ):

[0039]

[0040] Among them, t q is the q timestamp of the query to be retrieved, t g is the time characteristic of event g, σ is the preset time window parameter;

[0041] The comprehensive similarity score between event g and query q is calculated based on the following formula:

[0042]

[0043] In some embodiments, constructing and training a Bayesian network includes:

[0044] Selecting materialized view statements whose hit rates exceed a preset threshold from the materialized view set to train a Bayesian network;

[0045] Convert each selected materialized view statement and the corresponding features and hit rates into a variable set;

[0046] The transformed variable set is used as training samples to construct and train a Bayesian network to learn the network connection structure and conditional probability distribution.

[0047] In some implementations, the materialized view statement, features, and hit rates are converted into a variable set in the following manner:

[0048] The field name, operator, and value triples corresponding to the equality type query condition are used as query condition variables, and the IN type query condition is split into multiple equality type query conditions. The triples corresponding to each equality type query condition after splitting are also used as query condition variables, and each query condition variable is assigned a value. When the materialized view statement contains the query condition corresponding to the query condition variable, the value is assigned 1, and when the materialized view statement does not contain the query condition corresponding to the query condition variable, the value is assigned 0;

[0049] Map the hit rate to the real number domain as a hit rate variable;

[0050] Map features to corresponding predefined value sets as feature variables.

[0051] In some implementations, configuring evidence includes:

[0052] Soft evidence and hard evidence are configured for the Bayesian network, wherein the soft evidence includes a hit rate, and the hard evidence includes part or all of the features.

[0053] According to one embodiment of the present disclosure, an electronic device is provided, the device comprising a memory and a processor, the memory being used to store computer instructions executable on the processor, the processor being used to implement any of the above methods when executing the computer instructions.

[0054] According to an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, any of the above methods is implemented.

[0055] The technical solution proposed in this disclosure has at least the following beneficial effects:

[0056] First, we introduce an event knowledge base to predict changes in query load. By retrieving events that are close to the query time and semantically related, we can effectively predict the query change trend caused by the event, break through the limitations of the historical query set, predict and optimize new queries in advance, and improve the accuracy of query load prediction.

[0057] Second, this solution uses a large language model to understand query intent and events, so that the model can accurately predict the change trend of queries by referring to events, effectively process event information in natural language form, and improve the ability to predict the change trend of query load;

[0058] 3. This solution designs a generative materialized view model based on Bayesian networks. Through the learning of variable representation and probabilistic dependencies, it can batch generate high-quality materialized views based on predicted query trends. It not only ensures the legitimacy of the generated materialized view statements, but also improves the hit rate of materialized views, significantly improving the optimization effect of database queries.

[0059] Other features and advantages of the technical solution proposed in the present disclosure are described in detail below. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0061] Figure 1 A flowchart of a database query method based on a large language model according to an embodiment of the present disclosure is shown.

[0062] Figure 2 A flowchart of generating a materialized view based on event-aware prediction according to an exemplary embodiment of the present disclosure is shown.

[0063] Figure 3 A schematic diagram of query load change according to an exemplary embodiment of the present disclosure is shown.

[0064] Figure 4 A schematic diagram of a materialized view prediction framework according to an exemplary embodiment of the present disclosure is shown.

[0065] Figure 5 A schematic diagram of variable representation of a materialized view statement according to an exemplary embodiment of the present disclosure is shown.

[0066] Figure 6 A schematic diagram of a directed acyclic graph of a Bayesian network obtained according to an exemplary embodiment of the present disclosure is shown.

[0067] Figure 7 A flow chart of a materialized view generation process according to an exemplary embodiment of the present disclosure is shown.

[0068] Figure 8 It is a schematic diagram of the structure of an electronic device shown in at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0069] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0070] The disclosed embodiments may be applied to a computer system / server that may operate with numerous other general or special computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with a computer system / server include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above systems, and the like.

[0071] Computer systems / servers may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. In general, program modules may include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located on local or remote computing system storage media including storage devices.

[0072] The core concept of the database query solution based on a large language model proposed in this disclosure is to introduce event factors into the database query optimization process, understand the impact of events on query patterns through a large language model, and break through the limitations of historical queries; use Bayesian networks to probabilistically model the characteristics of materialized views to achieve batch generation of high-quality materialized views. The two complement each other, the former predicts changes in query trends, and the latter ensures the diversity and quality of materialized views generated based on the prediction results. This solution combines semantic understanding with probabilistic reasoning to form a complete query optimization solution.

[0073] Figure 1 A flowchart of a database query method based on a large language model according to an embodiment of the present disclosure is shown. As shown in the figure, the method includes steps 1 to 6.

[0074] Step 1, obtain a query set and a materialized view set. The query set includes multiple queries and corresponding timestamps. The materialized view set includes multiple materialized view statements and corresponding features and hit rates. The materialized view statements are generated based on the query conditions embedded in the queries in the query set.

[0075] A query set can be input into the database. The query set may include multiple queries submitted by users, and each query has a corresponding timestamp, which is the time when the user submits the query. For example, the query is "SELECT COUNT(*)FROM user_search WHERE user_search.keyword IN('rose')", and the timestamp is February 14th. This query set is used for subsequent training of the query load trend prediction model.

[0076] In some embodiments, the materialized view set can be obtained through the following steps: for each query in the query set, enumerate subsets of the query conditions embedded in the query to form subqueries; convert the subqueries into materialized view statements; calculate the hit rate of the converted materialized view statements for the queries in the query set, and determine the features of the materialized view statements.

[0077] For example, for the query "SELECT COUNT(*)FROM users WHERE users.crowd_type IN('2')", the query conditions in its WHERE clause can be enumerated to form subqueries, and these subqueries can be converted into materialized view statements.

[0078] The hit rate of the converted materialized view statements for the queries in the query set can be calculated in the following way. For example, sort the queries in the query set by timestamp. In the time interval [t 1 ,t |Q| of the query set, simulate the execution process of the query load, and calculate the quotient of the number of query hits of the materialized view statement converted from the subquery of query q i in the time interval divided by the number of query loads in this time interval (i.e., |Q|-i) as the hit rate corresponding to this materialized view statement.

[0079] The characteristics of the converted materialized view statement can be determined in the following ways. For example, the structure of the materialized view statement can be analyzed to determine its query category. For example, a materialized view statement containing aggregate functions such as COUNT and SUM can be determined as a statistical category, a materialized view statement containing a GROUP BY clause can be determined as a grouping category, and a materialized view statement containing only a WHERE clause can be determined as a filtering category, etc.; the time feature can be determined based on the timestamp of the original query corresponding to the materialized view statement, for example, the timestamp of the original query is mapped to a predefined time category, such as holidays (New Year's Day, Spring Festival, Qingming Festival, May Day, Dragon Boat Festival, Mid-Autumn Festival, National Day), weekdays, weekends, etc., or mapped to a specific time period of a day, such as the morning rush hour (6:00-9:00), working hours (9:00-17:00), etc.; the tables and fields involved in the materialized view statement can be analyzed to determine its business type. For example, fields related to user attributes can be determined as user portrait types, and fields related to transactions can be determined as transaction statistics types, etc.

[0080] Step 2: Based on the query set and the constructed event knowledge base, a query load trend prediction model based on a large language model is trained. The query load trend prediction model is used to predict the change trend of queries under the influence of events and the materialized view statements matching the changed queries. The event knowledge base stores multiple events and corresponding time features.

[0081] The query load trend prediction model is used to solve the following prediction tasks: how to predict the changing trend of queries when an event occurs. For example, for queries in the category of "gift search analysis", the focus of the query will change as different holidays approach. For example, a query like "SELECT COUNT(*)FROM user_search WHERE user_search.keyword IN('rose')" may become a query focusing on products such as "Christmas hats" when the event "Christmas is coming" occurs. The model not only predicts the changing trend of queries, but also generates materialized view statements that conform to the query change trend β, such as the output materialized view statement "CREATE MATERIALIZEDVIEW MV AS SELECT COUNT(*)FROM users,search WHERE age IN('2','3')and keyword IN('Christmas hat')".

[0082] Events that affect the query load change trend (such as "Christmas is coming" in the above example) are stored in the event knowledge base. Each entry in the event knowledge base includes an event and a corresponding time feature, and the time feature is used to characterize the time when the event occurs.

[0083] In some implementations, the event knowledge base may be constructed by the following steps:

[0084] enumerating at least one category of events based on a temporal feature;

[0085] Obtain relevant information of events of various categories under various time characteristics through search;

[0086] Use a large language model to extract event description text from search results;

[0087] Inputting the event description text and the domain features of the query set into a large language model, and cleaning and removing the event description text irrelevant to the domain features of the query set;

[0088] Use the text semantic encoding model to convert the cleaned event description text into the feature vector of the event;

[0089] An event knowledge base is constructed, wherein the event knowledge base stores event description text, event feature vectors, and corresponding time features.

[0090] You can select the category of interest as needed, such as seasons, solar terms, festivals and / or activities, etc. For each category, the events of each category are enumerated based on time characteristics (such as all dates in a year), such as "January 1st", "January 1st season", "January 1st solar term", "January 1st festival" and / or "January 1st activity", etc.

[0091] These categories of events can then be searched using, for example, an Internet search engine to obtain relevant web page content. These web pages can contain relevant information for each category of each date, such as definitions, descriptions, corresponding national holidays, holiday customs, activities, major events in history and / or famous people, etc.

[0092] Then use the large language model to extract event description text from the search results. The extracted event description text is input into the large language model together with the domain features of the query set, and the event description text that is not related to the field is cleaned and removed. The domain features are summarized and extracted from the query set, which characterize the data range of the query or the business scope of the user who submitted the query. For example, the domain features such as "gift search analysis" can be extracted from the query that analyzes product keywords and user behavior. For the domain features of "gift search analysis", through data cleaning, event description texts related to holiday shopping, gift promotions, etc. can be retained, and event description texts that are not related to the field can be removed.

[0093] The cleaned event description text can be converted into a feature vector using a text semantic encoding model. The event description text, feature vector and time feature are stored in a database and indexed to build an event knowledge base. Each entry can be represented as (t, g), where t is the time feature and g includes the event description text and the event feature vector.

[0094] In some implementations, training a query load trend prediction model based on a large language model based on the query set and the constructed event knowledge base includes:

[0095] Clustering queries in a query set;

[0096] Sample query pairs q in the same cluster i and q j , where t i <t j , t i and t j The query q i and query q j timestamp;

[0097] Retrieve and query q from the event knowledge base j Events g that are close in time and semantically related are obtained from the materialized view set and are related to the query q j Matching materialized view statement v j , get the materialized view training set;

[0098] The query q i and event g as input, and query q j As a label for the intermediate process, the materialized view statement v j As the final output label, the query load trend prediction model is trained step by step.

[0099] The query load trend prediction model based on a large language model includes a vector embedding neural network, an attention mechanism transformation matrix, an attention mechanism weighted average matrix and an output neural network. The model accepts text input and outputs text.

[0100] It will be understood by those skilled in the art that the query q i The query statement itself is used as the input of the query load trend prediction model. i The transformed natural language description is used as the input of the query load trend prediction model. This application does not limit this. Those skilled in the art can convert the query q into i The specific form of can be used as the input of the query load trend prediction model. The event description text of event g can be used as the input of the query load trend prediction model.

[0101] Get the same query q from the materialized view collection j When a matching materialized view statement is found, it can be used to optimize the query q j Materialized view statements, such as enumerating materialized views and selecting j Optimize the query execution time of the largest materialized view statement as v j.

[0102] In some embodiments, the following time-weighted event retrieval method can be used to retrieve and query q j Events that are close in time and semantically related:

[0103] Use a large language model to retrieve the query q j Convert it into a natural language description, and use the text semantic encoding model to further convert the natural language description into the query to be searched q j The eigenvector of

[0104] Based on the query to be retrieved q j The semantic similarity is calculated by the Euclidean distance between the feature vector of the event and the feature vector of the event in the event knowledge base, and based on the query to be retrieved q j Calculate the temporal similarity between the timestamp and the temporal features of the event;

[0105] Combine the semantic similarity and the temporal similarity to calculate the event and the query to be retrieved q j The comprehensive similarity score between

[0106] The event with the highest score is selected as the event that is close in time to the query to be retrieved and semantically related.

[0107] In some implementations, the semantic similarity Dis(x) between the query to be retrieved q and the event g can be calculated according to the following formula: g , x q ):

[0108]

[0109] Among them, x g is the feature vector of event g, x q is the feature vector of the query q to be retrieved, and d represents the dimension of the feature vector.

[0110] In some implementations, the temporal similarity Aim(t k , t j ):

[0111]

[0112] Among them, t q is the q timestamp of the query to be retrieved, t g is the time characteristic of event g, and σ is the preset time window parameter, for example, it can be 7 days.

[0113] The comprehensive similarity score between event g and query q can be calculated based on the following formula:

[0114]

[0115] In a specific example, the semantic similarity Dis(x g , x q ) Preliminarily select multiple events that are semantically related to the query to be retrieved from the event knowledge base. Then calculate the comprehensive similarity score Score(t k , t j ,g,q), select the event with the highest score as the event that is close in time to the query to be retrieved and semantically related.

[0116] During the training phase, the event with the highest score can be selected as the event that is close in time to the query to be retrieved and semantically related. When using the trained model for reasoning, multiple (for example, three) events with the highest scores can be selected as the events that are close in time to the query to be retrieved and semantically related to further improve the coverage of query prediction.

[0117] The large language model used in the process of building the event knowledge base and the large language model used in the process of converting the query into the feature vector are preferably the same large language model. Similarly, the text semantic encoding model used when generating the feature vector of the event and the text semantic encoding model used when generating the feature vector of the query are preferably the same-text semantic encoding model. This is conducive to ensuring the consistency of the semantic space, ensuring that the event and the query are mapped to the same semantic space, facilitating the measurement of similarity based on their feature vectors, avoiding differences in the semantic representations and vectors generated by different models, thereby enhancing the robustness of the system.

[0118] After obtaining the training samples, the following prompt words can be designed to convert the training samples into natural language form and input into the query load trend prediction model based on the large language model. The large language model used here can be different from the large language model used in the process of building the event knowledge base and obtaining the query feature vector.

[0119] A prompt word can contain three parts: instructions, context, and answer.

[0120] The instruction may include the materialized view prediction task objectives and prediction step instructions.

[0121] The context may include a materialized view profile, an event g, and a query q. i content.

[0122] The answer may include a description of the reasoning steps, intermediate results (predicted q j ) and the final answer (the final predicted target materialized view v j ).

[0123] The prompt word text is used as input to the model and the error is calculated with the expected output. The model parameters are optimized by the gradient descent algorithm. The model effect is evaluated by testing the matching degree between the generated materialized view and the query after the corresponding timestamp in the query load.

[0124] The prompt word template given according to an example is as follows: (the content in "<>" is filled in according to the training sample):

[0125] Instructions: Based on the provided query q i and future events g to predict the future materialized view. Follow the steps below to perform step-by-step reasoning: Step 1 - Understand user intent: i Interpret it as a natural language description and infer the user's background and query intent. Infer the query category of qi. Step 2 - Integrate event factor knowledge: i Combined with the relevant event knowledge g, reasoning how g affects the query trend. Step 3 - Predicting the future materialized view: Based on the integrated event g and user intent, predict q i Query after future changes q j . Recommendations that can optimize the changed query q j The materialized view v j Step 4 - Evaluate and Output Materialized Views: Evaluate the materialized view v j Availability and value in query load after changes. Output materialized view statements after the "answer" delimiter.

[0126] Context information: Introduction to the concept of materialized views: <Introduction to the technical principles of materialized views>. Related event factor knowledge <event g>. Old query q i : <old query in the old and new query pair>.

[0127] Model answer: q i The explanation is i The query category is <the “user category” data in the query feature>. The event related to the query is <event g>. Under the influence of the event, q i Changes to <q in the query pair j >. In order to optimize the query after the change, it is recommended to generate a materialized view <Materialized View v j >. The estimated hit rates for these materialized views are <v j The hit rate p>. Answer: <v j >.

[0128] Step 3: construct and train a Bayesian network based on the materialized view set. The Bayesian network is used to sample a variable set converted from materialized view statements, features, and hit rates according to configured evidence, and output a new variable set obtained by sampling. ​

[0129] The Bayesian network is a probabilistic graph model, whose structure is a directed acyclic graph. The nodes in the graph represent variables, and the edges represent the dependencies between variables. Each node has a corresponding conditional probability distribution table, which describes the probability distribution of the node value depending on the value of its parent node. The Bayesian network simplifies the representation of the joint probability distribution based on the conditional independence assumption between variables, and can predict the values ​​of other nodes by configuring evidence (known node value information) and probabilistic reasoning. Among them, the node takes a certain value under hard evidence, and the soft evidence makes the node value conform to a specific probability distribution.

[0130] In some embodiments, constructing and training a Bayesian network may include: selecting materialized view statements whose hit rates exceed a preset threshold from the materialized view set to train the Bayesian network; converting each selected materialized view statement and the corresponding features and hit rates into a variable set; and using the converted variable set as a training sample to construct and train a Bayesian network to learn the network connection structure and conditional probability distribution.

[0131] According to this embodiment, first, materialized view statements whose hit rates exceed a preset threshold are selected from the materialized view set as Bayesian network training data. Then, each selected materialized view statement and the corresponding features and hit rates are converted into variable representations. In some embodiments, the conversion can be performed in the following manner:

[0132] Query condition conversion: Convert the equality type query condition into a (field name, operator, value) triple as the query condition variable, and split the IN type query condition into multiple equality query conditions and perform the same conversion. For example, the query conditions "age=1" and "price>10" are represented as var age,＝,1 ∈{0,1} and var price,>,10 ∈{0,1}, the query condition "ageIN(2,3)" will be converted to "age=2 or age=3" and represented as var age,＝,2 ∈{0,1} and var age,＝,3 ∈{0,1}. When the materialized view statement contains the query condition corresponding to the query condition variable, the query condition variable takes the value of 1, otherwise it takes the value of 0;

[0133] Hit rate conversion: Map the hit rate to the real number domain, for example, represented by var p ∈R;

[0134] Feature conversion: Mapping features to a predefined set of values. For example, querying a category variable can be represented as var cate ∈[1,|categories|], for example, time information can be mapped to a predefined time set, represented by var time_seg∈{"0~5","6~11","12~17","18~23"}, for example, the query template can be mapped to the template number set, represented by var tmpl ∈[1,|templates|].

[0135] For example, for Figure 5 In the materialized view statement 1 "CREATE...age IN(1,2,3)and crowd_id IN(343,596)", its hit rate is 0.4, the query category code is 2, the time period is '6~11', and the query template number is 2, which can be expressed as a variable set {var p =0.4,var cate =2,var time_seg =′6-11′,var tmpl =2,var age,＝,1 =

[0136] 1,var age,＝,2 =1,var age,＝,3 =1,var age,＝,4 =0,var crowd_id,＝,343 =

[0137] 1,var crowd_id,＝,596 =1}.

[0138] Step 4: Receive a target query, retrieve events that are close in time to the target query and semantically related from an event knowledge base, input the target query and the retrieved events into a trained query load trend prediction model, predict a first materialized view statement, and input a first variable set converted based on the first materialized view statement into a trained Bayesian network, configure evidence to set desired features and hit rates, and obtain a second variable set output by the Bayesian network.

[0139] In one example, the target query can be first converted from an SQL statement into a feature vector, for example, the query condition is converted into a more semantically rich form, such as converting "age = 2" into "age = 10 to 19 years old", and then the converted query is converted into a natural language description using a large language model, and then the natural language description is converted into a feature vector of the query using a text semantic encoding model. The event knowledge base stores event text descriptions, feature vectors of events, and time features. The time-weighted event retrieval method can be used with reference to the above description to calculate the comprehensive similarity score between the target query and the event, and the three events with the highest scores are selected as events that are close in time to the target query and semantically related. For example, for queries in the "gift search analysis" category, when approaching December 25, the three events with the highest comprehensive similarity scores retrieved are: Christmas activities, year-end promotions, and winter flu high incidence warnings. These events indicate that the query may focus on holiday products such as "Christmas hats", discounted products, and warm and health products.

[0140] The target query is paired with each of the three retrieved events and input into the query load trend prediction model, and three first materialized view statements can be predicted accordingly. These three first materialized view statements can respectively include keywords related to the three different events, which is conducive to further improving the coverage of query prediction.

[0141] The first variable set can be converted based on the first materialized view statement, and soft evidence and hard evidence can be configured for the Bayesian network. In some embodiments, a higher hit rate can be set through soft evidence, which is conducive to improving the hit rate of the materialized view finally generated; the desired feature combination can be enumerated through hard evidence, such as all or part of all combinations of time, query template, query category, etc., which is conducive to improving the richness of the generated second variable set and the diversity of the materialized view finally generated.

[0142] Under the same input and evidence configuration, multiple sampling can be performed, and each sampling can obtain a second variable set. For example, if the input variable set and evidence configuration remain unchanged, sampling can be performed 1000 times to generate 1000 new variable sets, corresponding to the expansion of 1000 materialized view statements.

[0143] Step 5: Execute the second materialized view statement transformed by the second variable set in the database, generate a materialized view table, and store it.

[0144] The process of converting the second variable set into the second materialized view statement can be the reverse process of the process of converting the materialized view statement into the variable set mentioned above, which will not be repeated here. The second materialized view statement is executed in the database to generate a materialized view table and store it in the database. For example, execute the statement "CREATE MATERIALIZEDVIEW MV ASSELECT COUNT(*)FROM users,search WHERE age IN('2','3')and keyword IN('Christmas hat')" to generate a materialized view table named MV, which pre-calculates and stores the statistical results that meet the age and keyword conditions. By storing these pre-calculated results in the database, future queries can directly use these results to avoid repeated calculations.

[0145] Step 6: Rewrite the query that matches the second materialized view statement in the subsequent query into a query that reads data from the materialized view table and execute it.

[0146] Those skilled in the art may select appropriate strategies to determine whether a subsequent query matches the second materialized view statement. For example, in some examples, when the query conditions of the subsequent query include the query conditions of the second materialized view statement, it can be considered that the query can be optimized by the materialized view. The subsequent query can be rewritten into a query that reads pre-calculated results from the matching materialized view table, and the rewritten query is executed, and the query results are directly returned to the user, thereby avoiding repeated calculations and significantly improving query efficiency.

[0147] Figure 1 The illustrated embodiment has at least the following beneficial effects: an event knowledge base is introduced to predict changes in query load, and by retrieving events that are close to the query time and semantically related, the query change trend under the influence of the event can be effectively predicted, and the limitation of the historical query set can be broken through, and new queries can be predicted and optimized in advance, thereby improving the accuracy of query load prediction; a large language model is used to understand query intent and events, so that the model can accurately predict the query change trend based on events, effectively process event information in the form of natural language, and improve the prediction ability of the query load change trend; a generative materialized view model based on a Bayesian network is designed, and through the learning of variable representation and probabilistic dependencies, high-quality materialized views can be generated in batches based on predicted query trends, which not only ensures the legitimacy of the generated materialized view statements, but also improves the hit rate of the materialized view, and significantly improves the optimization effect of database queries.

[0148] Figure 2 A flowchart of generating a materialized view based on event-aware prediction according to an exemplary embodiment of the present disclosure is shown, which mainly includes three parts:

[0149] Input: Time series query load;

[0150] Core processing modules (in the middle dotted box): document parsing, event knowledge base, vector retrieval, query model prediction (the example event is "New Year's Day on January 1"), and large language model;

[0151] Output: Generated materialized view.

[0152] This process is guided by events to achieve prediction generation from query load to materialized views.

[0153] Figure 3 A schematic diagram of query load change according to an exemplary embodiment of the present disclosure is shown.

[0154] Figure 4 A schematic diagram of the overall framework of materialized view prediction according to an exemplary embodiment of the present disclosure is shown, which includes three parts.

[0155] The top part mainly shows the event knowledge base construction and time series prediction. The left side shows the construction process of the event knowledge base. The middle part is the query load trend mining, which is the time-weighted event retrieval method mentioned above. The right side shows the prediction using a large language model.

[0156] The middle part mainly includes data processing and feature representation, the left side shows query clustering, the middle is materialized view mining, that is, materialized view matching, and the right side is materialized view generation based on Bayesian network.

[0157] The bottom part shows the inference flow, which illustrates the complete process from input query to the final generation of materialized view.

[0158] Figure 4 The medium blue part represents the query load trend prediction model-related processing based on the large language model, and the orange part represents the Bayesian network-related processing.

[0159] Figure 5 A variable diagram of a materialized view statement according to an exemplary embodiment of the present disclosure is shown, including two materialized view statements and their variable representations. Materialized view statement 1 has been described in detail above, and materialized view statement 2 is similar thereto.

[0160] Figure 6 A schematic diagram of a directed acyclic graph of a Bayesian network obtained according to an exemplary embodiment of the present disclosure is shown.

[0161] Figure 7Shows a block diagram of the materialized view generation process according to an exemplary embodiment of the present disclosure. As shown in the figure, the upper part shows the event knowledge base and the event retrieval process; the middle part shows the processing of the large language model on the historical query load to generate the materialized view seeds; the lower part shows the processing process of the Bayesian network, starting from the old materialized view, sampling the query template and variables 1, 2, and 3 to generate a new materialized view. Figure 7 Overall, it shows the evolution process from query load i - 1 to query load i.

[0162] The following gives a specific query process according to the above embodiment.

[0163] (a) Input a query set {q 1 , q 2 , q 3 , q 4 , q 5 , q 6 , q 7} containing timestamps into the database. The query set is as Figure 3 shown, Figure 3 from top to bottom in which are q 1 to q 7 . The timestamp of query q 6 “SELECT COUNT(*)FROM user_search WHERE user_search.keyword IN('rose')” is February 14th.

[0164] (b) Inductively extract domain features from the query set. For example, extract the domain feature of “gift search analysis” from {q 6 , q 7}. Use the text semantic encoding model to encode the query into a feature vector, and retrieve relevant events in the event knowledge base based on the time - weighted event retrieval method. For example, retrieve the Christmas event on December 25th.

[0165] (c) Extract sub - queries from the queries in the query set. For example, the sub - query “SELECT COUNT(*)FROM users WHERE users.crowd_type IN('2')” that appears 3 times in {q 1 , q 2 , q 3}. Convert these sub - queries into materialized view statements, calculate the hit rate and determine the features to form a materialized view set.

[0166] (d) Use the trained query load trend prediction model to predict the query load change trend. For example, for the query q6 of the "gift search analysis" type, it is predicted that the focus will shift to "Christmas hats" as Christmas approaches. Based on this, the materialized view statement "CREATE MATERIALIZEDVIEW MV AS SELECT COUNT(*)FROM users,searchWHERE age IN('2','3')and keyword IN('Christmas hat')" is generated.

[0167] (e) Convert the materialized view statement into a variable set. For example, for the materialized view statement containing "age IN(1,2,3)" and "crowd_id IN(343,596)", its hit rate is 0.4, the category code is 2, and the time period is '6~11', which can be represented as a variable set

[0168]

[0169] (f) Input the variable set into the trained Figure 6 The Bayesian network shown in the figure configures evidence and samples to generate a new set of variables, which are converted into materialized view statements. These materialized view statements are consistent with the predicted query load change trend.

[0170] (g) Execute the materialized view statement in the database to generate the materialized view table. For example, when the subsequent query is "SELECT COUNT (*) FROM users, search WHERE age IN ('2', '3') and keyword IN ('Christmas hat')", it can be rewritten as "SELECT * FROM MV" to directly read the results from the materialized view table to achieve efficient query.

[0171] Figure 8 An electronic device provided for at least one embodiment of the present disclosure includes a memory and a processor, wherein the memory is used to store computer instructions executable on the processor, and the processor is used to implement the database query method based on a large language model described in any embodiment or implementation of the present disclosure when executing the computer instructions.

[0172] At least one embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the database query method based on a large language model described in any embodiment or implementation of the present disclosure.

[0173] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the data processing device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0175] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0176] The embodiments of the subject matter and functional operations described in this specification may be implemented in the following: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules in computer program instructions encoded on a tangible non-temporary program carrier to be executed by a data processing device or to control the operation of the data processing device. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information and transmit it to a suitable receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0177] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits), and the apparatus can also be implemented as special purpose logic circuits.

[0178] Computers suitable for executing computer programs include, for example, general and / or special microprocessors, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, the computer will also include one or more large-capacity storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or the computer will be operably coupled to this large-capacity storage device to receive data from it or to transmit data to it, or both. However, the computer does not necessarily have such a device. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0179] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0180] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of the specific embodiments of specific inventions. Certain features described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claim protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0181] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or requiring that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0182] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0183] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. A database query method based on a large language model, characterized in that: include: Acquire a query set and a materialized view set, wherein the query set includes multiple queries and corresponding timestamps, and the materialized view set includes multiple materialized view statements and corresponding features and hit rates, wherein the materialized view statements are generated based on query conditions embedded in the queries in the query set; Based on the query set and the constructed event knowledge base, a query load trend prediction model based on a large language model is trained, wherein the query load trend prediction model is used to predict the change trend of queries under the influence of events and the materialized view statements matching the changed queries, wherein the event knowledge base stores multiple events and corresponding time features; Building and training a Bayesian network based on the materialized view set, the Bayesian network is used to sample a variable set converted from materialized view statements, features, and hit rates according to configured evidence, and output a new variable set obtained by sampling; Receive a target query, retrieve events that are close in time to the target query and semantically related from an event knowledge base, input the target query and the retrieved events into a trained query load trend prediction model, predict and obtain a first materialized view statement, input a first variable set converted based on the first materialized view statement into a trained Bayesian network, configure evidence to set expected features and hit rates, and obtain a second variable set output by the Bayesian network; Execute the second materialized view statement transformed by the second variable set in the database, generate the materialized view table and store it; A query in a subsequent query that matches the second materialized view statement is rewritten into a query that reads data from the materialized view table and executed.

2. The method according to claim 1, characterized in that: Getting a materialized view collection includes: For each query in the query set, enumerate a subset of query conditions embedded in the query to form a subquery; Convert the subquery into a materialized view statement; The hit rate of the transformed materialized view statement for the query in the query set is calculated, and the characteristics of the materialized view statement are determined.

3. The method according to claim 1, characterized in that The method further comprises constructing the event knowledge base according to the following steps: enumerating at least one category of events based on a temporal feature; Obtain relevant information of events of various categories under various time characteristics through search; Use a large language model to extract event description text from search results; Inputting the event description text and the domain features of the query set into a large language model, and cleaning and removing the event description text irrelevant to the domain features of the query set; Use the text semantic encoding model to convert the cleaned event description text into the feature vector of the event; An event knowledge base is constructed, wherein the event knowledge base stores event description text, event feature vectors, and corresponding time features.

4. The method according to claim 1, characterized in that: Training a query load trend prediction model based on a large language model based on the query set and the constructed event knowledge base includes: Clustering queries in a query set; Sample query pairs q in the same cluster i and q j , where t i <t j , t i and t j The query q i and query q j timestamp; Retrieve and query q from the event knowledge base j Events g that are close in time and semantically related are obtained from the materialized view set and are related to the query q j Matching materialized view statement v j , get the materialized view training set; The query q i and event g as input, and query q j As a label for the intermediate process, the materialized view statement v j As the final output label, the query load trend prediction model is trained step by step.

5. The method according to claim 1, characterized in that: Retrieve events that are close in time to the query and semantically related, including: Use a large language model to convert the query to be retrieved into a natural language description, and use a text semantic encoding model to further convert the natural language description into a feature vector of the query to be retrieved; The semantic similarity is calculated based on the Euclidean distance between the feature vector of the query to be retrieved and the feature vector of the event in the event knowledge base, and the temporal similarity is calculated based on the timestamp of the query to be retrieved and the temporal feature of the event; Calculating a comprehensive similarity score between the event and the query to be searched by combining the semantic similarity and the temporal similarity; The event with the highest score is selected as the event that is close in time to the query to be retrieved and semantically related.

6. The method according to claim 5, characterized in that: The semantic similarity Dis(x) between the query q and the event g is calculated according to the following formula: g , x q ): Among them, x g is the feature vector of event g, x q is the feature vector of the query q to be retrieved, and d represents the dimension of the feature vector; The temporal similarity Sim(t k , t j ): Among them, t q is the q timestamp of the query to be retrieved, t g is the time characteristic of event g, σ is the preset time window parameter; The comprehensive similarity score between event g and query q is calculated based on the following formula:

7. The method according to claim 1, characterized in that Building and training a Bayesian network involves: Selecting materialized view statements whose hit rates exceed a preset threshold from the materialized view set to train a Bayesian network; Convert each selected materialized view statement and the corresponding features and hit rates into a variable set; The transformed variable set is used as training samples to construct and train a Bayesian network to learn the network connection structure and conditional probability distribution.

8. The method according to claim 1, characterized in that: Materialized view statements, characteristics, and hit rates are converted to variable sets in the following way: The field name, operator, and value triples corresponding to the equality type query condition are used as query condition variables, and the IN type query condition is split into multiple equality type query conditions. The triples corresponding to each equality type query condition after splitting are also used as query condition variables, and each query condition variable is assigned a value. When the materialized view statement contains the query condition corresponding to the query condition variable, the value is assigned 1, and when the materialized view statement does not contain the query condition corresponding to the query condition variable, the value is assigned 0; Map the hit rate to the real number domain as a hit rate variable; Map features to corresponding predefined value sets as feature variables.

9. The method according to claim 1, characterized in that: Configuration evidence includes: Soft evidence and hard evidence are configured for the Bayesian network, wherein the soft evidence includes a hit rate, and the hard evidence includes part or all of the features.

10. An electronic device, characterized in that: The device comprises a memory and a processor, wherein the memory is used to store computer instructions executable on the processor, and the processor is used to implement the method according to any one of claims 1 to 9 when executing the computer instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images