Query method, system and equipment, program product and storage medium

Abstract

The invention discloses a query method, system and device, a program product and a storage medium. Obtaining a query statement; if the data tables indicated by the query statement comprise a plurality of data tables with a connection relationship, determining the data tables in an invariable state and the data tables in a variable state in the plurality of data tables; for the data table in the invariable state, determining the query quantity by using a corresponding Bayesian network model, for the data table in the variable state, extracting sample data in the connected data table, and determining conditional probability distribution information corresponding to the sample data; based on the query number corresponding to the data table in the invariable state and the conditional probability distribution information corresponding to the data table in the variable state, a query plan is determined, and the query plan is used for indicating the execution process of query.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a query method, system, device, program product and storage medium. Background Art

[0002] In modern banking, with the widespread application of information technology, banks have accumulated a large amount of data, including but not limited to customer information, transaction records, financial statements, etc. The large-scale data accumulation has brought tremendous pressure to the bank's information processing.

[0003] When users perform query operations, the database needs to quickly retrieve relevant information from massive amounts of data. Currently, cardinality estimation methods are used for querying. Cardinality estimation methods rely on some simplified statistical assumptions to simplify the problem and provide fast estimation results. The most common ones are the independence assumption and the uniform distribution assumption. The independence assumption assumes that the data distribution on different columns is independent of each other, which means that the change of one attribute will not affect the probability distribution of another attribute. The uniform distribution assumption assumes that the data is uniformly distributed within a certain range, that is, the probability of each value appearing is the same. However, in complex application scenarios such as banking, the possibility of data distribution changing before and after the data table is connected is often much higher than the possibility of remaining unchanged. This is mainly due to the complex associations between data and the variability of business logic. Therefore, traditional cardinality estimation methods perform well when processing data sets that meet their assumptions, but when the actual data does not meet these assumptions, the estimation results may have large deviations. Since traditional cardinality estimation methods cannot accurately reflect the actual situation of data distribution, the execution plan generated by the database query optimizer may not be optimal, which in turn affects query efficiency. Summary of the invention

[0004] Based on the above problems, the present application provides a query method, system, device, program product and storage medium.

[0005] The embodiments of the present application disclose the following technical solutions:

[0006] A second aspect of the embodiment of the present application provides a query method, including:

[0007] Get the query statement;

[0008] If the data table indicated by the query statement includes multiple data tables that are connected, determine the data tables in an unchanged state and the data tables in a changing state among the multiple data tables; for the data tables in an unchanged state, use the corresponding Bayesian network model to determine the number of queries; for the data tables in a changing state, extract sample data from the connected data tables, and determine the conditional probability distribution information corresponding to the sample data;

[0009] Based on the number of queries corresponding to the data table in an unchanged state and the conditional probability distribution information corresponding to the data table in a changing state, a query plan is determined, where the query plan is used to indicate the execution process of the query.

[0010] In a possible implementation, the Bayesian network model is determined by:

[0011] Get multiple data tables stored in the database;

[0012] According to the dependency relationship between variables in each data table, a directed acyclic graph corresponding to each data table is constructed;

[0013] For each node of the directed acyclic graph, construct a corresponding conditional probability table;

[0014] A Bayesian network model corresponding to each data table is obtained based on the training set and the directed acyclic graph training, and the Bayesian network model is used to obtain probability distribution through probability reasoning.

[0015] In a possible implementation, the method further includes:

[0016] For each data table, add an index to the primary key of the data table.

[0017] In a possible implementation manner, determining the data tables in the multiple data tables that are in an unchanged state and the data tables that are in a changed state includes:

[0018] For each data table, determine the interval between the current time point and the last update time point;

[0019] If the interval exceeds the preset interval threshold, the data table is divided into a data table in a constant state; if the interval does not exceed the preset interval threshold, the data table is divided into a data table in a changing state.

[0020] In a possible implementation, the method further includes:

[0021] If the data tables indicated by the query statement include only a single data table, the number of queries corresponding to the single data table is determined using the corresponding Bayesian network model.

[0022] In a possible implementation, the method further includes:

[0023] Get the actual execution result after the query plan is executed;

[0024] The actual execution result is compared with the expected result, and the corresponding Bayesian network model is adjusted based on the comparison result.

[0025] A second aspect of the embodiment of the present application provides a query system, including:

[0026] An acquisition unit, used for acquiring a query statement;

[0027] A first determining unit is configured to, if the data table indicated by the query statement includes multiple data tables that are connected, determine a data table in an unchanged state and a data table in a changing state among the multiple data tables; for a data table in an unchanged state, determine the number of queries using a corresponding Bayesian network model; for a data table in a changing state, extract sample data from the connected data table and determine conditional probability distribution information corresponding to the sample data;

[0028] The second determination unit is used to determine a query plan based on the number of queries corresponding to the data table in an unchanged state and the conditional probability distribution information corresponding to the data table in a changing state, wherein the query plan is used to indicate the execution process of the query.

[0029] A third aspect of an embodiment of the present application provides an electronic device, characterized in that it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the query method described in the first aspect above.

[0030] A fourth aspect of an embodiment of the present application provides a computer program product, characterized in that when the computer program product is run on a computer, the computer executes the query method as described in the first aspect above.

[0031] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, characterized in that instructions are stored in the computer-readable storage medium, and when the instructions are executed on a terminal device, the terminal device executes the query method described in the first aspect above.

[0032] Compared with the prior art, this application has the following beneficial effects:

[0033] Obtain a query statement, parse the query statement to determine the data tables involved and the connection relationship between them. For multiple data tables involved in the query that have a connection relationship, determine which data tables are in an unchanged state and which data tables are in a changing state. For the data tables in an unchanged state, use the trained Bayesian network model to determine the number of queries. The Bayesian network can capture the complex dependencies within the data table and across tables, so as to estimate the query cardinality more accurately. For the data tables in a changing state, a sampling method is adopted to extract sample data from the connected data tables, and conditional probability distribution information is calculated based on these samples. This method allows the impact of the connection operation to be directly observed, thereby obtaining a more accurate cardinality estimate. Combining the above two types of cardinality estimation results, namely the number of queries for the unchanged data table and the conditional probability distribution information for the changing data table, determine the query execution plan. Not only the interaction between different data tables is considered, but also the query performance and resource utilization are comprehensively considered to ensure that the query operation can be completed in the shortest time and with high accuracy. Therefore, it is no longer completely dependent on overly simplified assumptions such as independence and uniform distribution, but a suitable estimation method is flexibly selected according to the actual data situation. By identifying the status of the data table and adopting different strategies in a targeted manner, the cardinality estimate is made closer to the actual value, reducing the deviation caused by incorrect assumptions. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 A flowchart of a query method provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of a second query method flow chart provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the database structure of a table is provided for the embodiment of the present application;

[0038] Figure 4 A structural diagram of a query system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the professional terms involved in the embodiments of the present application will be explained below.

[0041] Artificial Intelligence: Artificial Intelligence is a field of science and engineering that aims to study and develop computer systems that can simulate, extend and expand human intelligence. These systems are able to perform a range of complex cognitive tasks such as perception, learning, reasoning, planning, and understanding natural language. The design and implementation of artificial intelligence systems involve multiple disciplines, including machine learning, computer vision, natural language processing, knowledge representation, expert systems, etc. At present, artificial intelligence has been applied to various fields, including healthcare, finance, transportation, manufacturing, etc., providing new solutions to improve production efficiency, improve the quality of life, and solve the challenges facing mankind.

[0042] Machine Learning: Machine learning is a branch of artificial intelligence that enables computer systems to automatically improve and optimize their behavior by allowing them to learn patterns and regularities from data. This learning process involves analyzing and processing large amounts of data to identify features and relationships in the data and perform tasks such as prediction, classification, clustering, or decision-making based on them. Machine learning algorithms can be divided into supervised learning, unsupervised learning, and reinforcement learning based on different learning methods, each of which has different application scenarios and algorithm models. In supervised learning, the algorithm uses labeled training data to learn to predict the labels or attributes of new data; in unsupervised learning, the algorithm automatically discovers patterns and structures from unlabeled data; and in reinforcement learning, the algorithm learns the optimal decision-making strategy through interaction with the environment.

[0043] Bayesian network: A Bayesian network is a probabilistic graphical model used to represent dependencies between random variables and use Bayes' theorem for inference. It consists of a directed acyclic graph and a set of conditional probability distributions, where nodes represent random variables, edges represent dependencies between variables, and conditional probability distributions represent the probability distribution of each node given its parent node. Bayesian networks are widely used in fields such as machine learning, artificial intelligence, and decision analysis to model and infer complex probabilistic relationships to support decision making and prediction.

[0044] Sampling: Sampling is a statistical method that selects some individuals or observations from a population to represent the population for statistical inference or analysis. The sampling process aims to obtain a representative sample from the population so that the characteristics, attributes or behavioral patterns of the population can be inferred through the study of the sample. The choice of sampling method depends on factors such as the purpose of the research, the characteristics of the population and resource limitations. Common sampling methods include random sampling, systematic sampling, stratified sampling, etc.

[0045] Data distribution: Data distribution refers to the statistical overview that describes the distribution of each value or observation in the data set within the overall range. It covers statistical characteristics such as the central tendency of the data (such as mean, median), the degree of dispersion of the data (such as standard deviation, quartiles), and the shape of the data (such as normal distribution, skewed distribution). Understanding data distribution is important for analyzing the characteristics of data sets, predicting future trends, and making decisions.

[0046] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.

[0047] As mentioned above, when users perform query operations, the database needs to quickly retrieve relevant information from massive amounts of data. The cardinality estimation method currently used usually relies on some simplified statistical assumptions to simplify the problem and provide fast estimation results, such as the independence assumption and the uniform distribution assumption. However, in complex application scenarios such as banking, due to the complex associations between data and the variability of business logic, these assumptions often cannot accurately reflect the actual situation, resulting in large deviations in the estimation results.

[0048] In order to solve the above problems, the embodiment of the present application receives the query request submitted by the user, and parses the data tables involved in the query and the connection relationship between them. Identify the data tables in the unchanged state and the changed state: Next, for the multiple data tables involved in the query with connection relationship, determine which data tables have relatively stable data distribution after the connection operation (i.e., in the unchanged state), and which data tables may have changed significantly due to the connection (i.e., in the changed state). For those data tables that are determined to be in the unchanged state, the pre-trained Bayesian network model is used to estimate the number of queries on these tables. The Bayesian network can capture the conditional dependency between the attributes in the data table, thereby providing a more accurate cardinality estimation result than the simple statistical assumption. In this way, the accuracy of the cardinality estimation can be effectively improved, especially when facing a data set with complex correlations. For those data tables whose data distribution changes due to the connection operation, the present application adopts another strategy-extracting sample data from the connected data table, and estimating its conditional probability distribution based on these sample data. This method allows the direct observation of the changes in the actual data distribution, rather than relying on unrealistic assumptions. By adopting sampling technology, the computational overhead can be reduced to a certain extent, while still being able to obtain relatively accurate probability distribution information. Finally, based on the results of the above two parts - the estimated number of queries for the unchanged state data table and the conditional probability distribution information for the changed state data table, this application combines them to develop an optimized query execution plan. This plan not only takes into account the complex interactions between data tables, but also takes into account the balance between efficiency and accuracy, thereby ensuring a faster response speed while also improving the quality of query results.

[0049] Through the above process, this application effectively solves the problems faced by traditional cardinality estimation methods when dealing with conditions that do not meet their basic assumptions. It not only utilizes the powerful modeling capabilities of the Bayesian network to process the static part, but also flexibly responds to the dynamically changing part through direct sampling, achieving better adaptation to changes in data distribution during the query process. This method, which combines model prediction with actual observation, can significantly improve the accuracy of cardinality estimation, thereby improving overall query performance and enhancing user experience. In addition, such a design also makes the entire solution more flexible and scalable, and can better support new business scenarios that may emerge in the future.

[0050] It should be noted that the query method, system, device and medium provided in the present application can be applied to the field of computer technology. The above is only an example and does not limit the application field of the query method, system, device and medium provided in the present application. In addition, the embodiments of the present application may not limit the execution subject of the query. For example, the query method of the embodiment of the present application can be applied to data processing devices such as terminal devices or servers. Among them, the terminal device can be an electronic device such as a computer, a personal digital assistant (PDA). The server can be a stand-alone server, a cloud server, or a cluster server composed of multiple servers.

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] The following is an example of an example to illustrate a query method provided by the present application. Figure 1 ,Should Figure 1 A flowchart of a query method provided in an embodiment of the present application, the method comprising:

[0053] S101: Obtain a query statement.

[0054] First, the SQL query statement submitted by the user is received and parsed, and all the data tables involved and the connection relationships between these tables are identified, ensuring that subsequent processing can be accurately based on the actual needs of the user.

[0055] S102. If the data table indicated by the query statement includes multiple data tables that are connected, determine the data tables in an unchanged state and the data tables in a changing state among the multiple data tables; for the data tables in an unchanged state, use the corresponding Bayesian network model to determine the number of queries, and for the data tables in a changing state, extract sample data from the connected data tables, and determine the conditional probability distribution information corresponding to the sample data.

[0056] For each data table involved in the query, we determine whether it will change significantly after the join operation based on its degree of association with other tables and business logic analysis. Through this classification, we can apply different strategies to deal with the two types of data tables in a targeted manner, thereby estimating the cardinality more accurately. A data table in an unchanged state usually means that its data distribution is relatively stable, while a data table in a changing state may have its data distribution changed due to the join operation.

[0057] Tables that remain unchanged are usually those that are not affected by other tables during queries. For example, in a banking system, the basic information of customers (such as name, address) generally does not change frequently, so their data distribution can be considered relatively stable.

[0058] For those data tables that are judged to be in an unchanged state, the pre-trained Bayesian network model is used to estimate their cardinality. Bayesian networks can capture the conditional dependencies between attributes and therefore provide more accurate results than simple statistical assumptions. Using Bayesian network models can effectively improve the accuracy of cardinality estimation, especially when facing data sets with complex correlations. This method avoids the bias caused by traditional independence and uniform distribution assumptions, making cardinality estimation closer to reality.

[0059] Tables whose data distribution may change: These tables usually contain data that changes over time or other factors, such as transaction records, account balances, etc. Since the data in these tables is updated dynamically, their data distribution may change with the input of new data. In this case, it is no longer appropriate to directly apply the existing Bayesian network model for estimation, because the model may not accurately reflect the current data situation. Therefore, additional techniques are needed to handle such changes.

[0060] For data tables that are in a changing state, samples are drawn from the result set after the connection, and their conditional probability distribution is calculated based on these samples. This method allows direct observation of changes in the actual data distribution. The sampling technique can directly reflect the actual state of the data instead of relying on unrealistic assumptions. This helps to improve the accuracy of the estimate, especially when the data distribution has indeed changed significantly.

[0061] Select an appropriate sampling method based on the characteristics of the data table and query requirements. Common sampling methods include simple random sampling, stratified sampling, systematic sampling, and cluster sampling. Each method has its applicable scenarios and advantages and disadvantages. Determine the sampling ratio, that is, the proportion of sample data to the original data. The selection of the sampling ratio requires a trade-off between accuracy and computational cost. Based on the selected sampling method and ratio, extract sample data from the original data table. This usually involves randomly selecting records in the table or filtering them according to certain rules. Perform the same join operation on the extracted sample data as the original query. This can be done by building a temporary table in memory or using the join function provided by the database management system. Ensure that the extracted sample data can maintain data integrity during the join operation, that is, the records in the sample can be correctly matched to the corresponding join fields.

[0062] Clean and organize the concatenated sample data as necessary to ensure the accuracy and consistency of the data. Calculate the statistics of the target variable on the concatenated sample data, such as mean, variance, frequency distribution, etc. These statistics will be used to estimate the corresponding values ​​in the original data. Based on the statistics in the sample data, use the principles of probability theory and statistics to estimate the conditional probability distribution of the target variable. Bayesian theorem, maximum likelihood estimation or other statistical methods can be applied.

[0063] Use data that is not involved in sampling (such as a validation set) to verify the accuracy of the estimated conditional probability distribution. If the verification results show that the estimated conditional probability distribution is significantly different from the actual distribution, you can reselect the sampling method, adjust the sampling ratio, or modify the details of the connection operation, and perform iterative optimization.

[0064] S103: Determine a query plan based on the number of queries corresponding to the data table in an unchanged state and conditional probability distribution information corresponding to the data table in a changing state.

[0065] The query plan is used to indicate the execution process of the query. The query optimizer determines the best physical execution plan based on the above estimation results and other factors (such as hardware resource conditions, concurrent access requirements, etc.). This may include selecting appropriate indexes, deciding whether to use parallel processing, adjusting caching strategies, etc. The ultimate goal is to ensure that the query can return correct results in the shortest time while minimizing the impact on overall performance. By comprehensively considering multiple factors, the query optimizer can develop the optimal operation path, thereby improving the user experience and the overall operation efficiency of the system.

[0066] In one possible implementation, the method for determining the Bayesian network model includes: obtaining multiple data tables stored in a database; constructing a directed acyclic graph corresponding to each data table for the dependency relationship between variables in each data table; constructing a corresponding conditional probability table for each node of the directed acyclic graph; and obtaining a Bayesian network model corresponding to each data table based on a training set and the directed acyclic graph training, wherein the Bayesian network model is used to obtain a probability distribution through probability reasoning.

[0067] Extract all relevant business data tables from the bank's database. This includes but is not limited to customer information, account details, transaction records, etc. In order to ensure the effectiveness of subsequent analysis, it is usually necessary to clean and preprocess the data, such as removing duplicate entries, filling missing values, and correcting errors. This step is the basis of the entire process and provides a clean and structured data source for subsequent modeling and training.

[0068] For each independent data table, a corresponding Bayesian network model is constructed. The Bayesian network model is a probabilistic graph model that represents the conditional dependencies between variables through a directed acyclic graph (DAG). In an embodiment of the present application, each node represents a field or attribute in a data table, and the edge represents the conditional dependencies between these attributes. By learning the sample distribution in the data table, the conditional probability distribution of each node can be estimated. For example, if there is a data table about customer loan information, there may be nodes such as "income", "age", "credit score", and the probability of default under certain conditions can be determined through historical data2. After training, such a model can be used to predict the behavior pattern of new data points, which is particularly useful for estimating the number of single-table queries.

[0069] In order to improve query efficiency, especially when it comes to large amounts of data, it is very important to create indexes for the primary keys in the data table. An index is similar to the table of contents of a book, which allows the database to quickly locate a specific record without having to scan the entire table.

[0070] In one possible implementation, see Figure 2 The process shown, Figure 2 A flow chart of a second query method provided in an embodiment of the present application, wherein if the data table indicated by the query statement includes only a single data table, the corresponding Bayesian network model is used to determine the number of queries corresponding to the single data table.

[0071] In one possible implementation, determining the data tables in the multiple data tables that are in an unchanged state and the data tables in a changing state includes: for each data table, determining the interval between the current time point and the last update time point; if the interval exceeds a preset interval threshold, classifying the data table as a data table in an unchanged state; if the interval does not exceed the preset interval threshold, classifying the data table as a data table in a changing state.

[0072] For each data table involved in the query statement, the system determines the interval between its current time point and the last update time point. If the interval exceeds the preset interval threshold, the data table is classified as a data table in an unchanged state; conversely, if the interval does not exceed the preset threshold, it is classified as a data table in a changing state. For data tables in an unchanged state, the Bayesian network model is used to quickly estimate the number of queries, reducing unnecessary computing overhead. For data tables in a changing state, conditional probability distribution information is obtained through sampling technology, avoiding full table scans and improving query efficiency.

[0073] In a possible implementation, an actual execution result after the query plan is executed is obtained; the actual execution result is compared with the expected result, and the corresponding Bayesian network model is adjusted based on the comparison result.

[0074] When the query plan is executed, the database system will record the actual execution statistics, including but not limited to the actual number of rows returned (A-Rows), the actual execution time of each operation step (A-Time), the number of logical reads (Buffers), etc. 1. This information can be obtained from the preset view, which stores the actual execution plan of the SQL statement. By collecting this runtime data, the actual situation of query execution can be obtained, which helps to evaluate the effectiveness of the decisions made by the query optimizer.

[0075] Compare the actual execution results obtained from the preset view with the results predicted by the Bayesian network model. Focus on indicators with large differences, such as the gap between the estimated number of rows (E-Rows) and the actual number of rows (A-Rows). This comparison can help identify areas where the model estimation deviation is large, thus providing a basis for subsequent model adjustments. For example, if it is found that the estimated number of rows for a certain join operation is much lower than the actual number of rows, it means that the model may underestimate the amount of data in the table or the selectivity under certain conditions.

[0076] According to the comparison results, adjust the parameters in the Bayesian network, such as the conditional probability distribution parameters. If there is a large error in the estimation of a certain part, the accuracy of the model can be improved by increasing the number of training samples, improving feature engineering, or adjusting the learning algorithm. If the existing Bayesian network structure is not sufficient to capture the complex dependencies between data, you need to consider adding new nodes or edges, or even rebuilding the entire network structure.

[0077] Thus, a closed-loop optimization mechanism is formed by obtaining the actual execution results and comparing them with the expected results, and then adjusting the corresponding Bayesian network model based on the comparison results. This method can not only improve the efficiency of the current query, but also promote the intelligence level of the entire database system, enabling it to maintain efficient and stable performance in a constantly changing data environment.

[0078] The following is an example to illustrate the embodiment of the present application.

[0079] In the scenario of an e-commerce website, you are faced with a database structure that contains three tables: customers (C), purchase records (P), and stores (S). Figure 3 , Figure 3A schematic diagram of the database structure of the table is provided for the embodiment of the present application. The Customer table includes NATIONALITY: the nationality of the customer. HAIR: the hair color of the customer. SALARY: the salary of the customer. The Purchase table includes NAME: the name of the product. DAY OF WEEK: the day of the week when the purchase occurs. The Store table includes NAME: the name of the store. OWNER: the owner of the store. CITY: the city where the store is located. SIZE: the size of the store. COMMODITY: the types of goods sold in the store. The arrows in the figure represent the dependencies between different entities and their attributes: the NATIONALITY of the Customer affects HAIR and SALARY. The NAME of the Purchase is affected by NATIONALITY. The NAME of the Store is directly associated with OWNER, CITY, SIZE, and COMMODITY.

[0080] These three tables are associated through a common ID field, forming a complex multi-table join query. The customer table (C) and the store table (S) are the master tables, and the purchase table (P) is the detail table. Query Q2 needs to obtain data from these three tables, and the conditions include that the customer's nationality is China (CHINESE), the hair color is black (BLACK), and the name of the purchased product is iPhone (IPHONE).

[0081] Build a connected Bayesian network where nodes represent random variables and edges represent dependencies between variables. Nodes may include attributes such as C.NATIONALITY, C.HAIR, S.NAME, etc. By analyzing the data in these tables, the probabilistic dependencies between the attributes can be determined and the corresponding Bayesian network can be constructed.

[0082] According to the Bayesian network, the following decomposition formula of joint distribution is obtained: Pcps(BLACK, CHINESE, IPHONE) = Pc(BLACK|CHINESE)*Pcp(CHINESE)*Psp(IPHONE|CHINESE)

[0083] Pc(BLACK|CHINESE) represents the probability of a black-haired customer given Chinese nationality; Pcp(CHINESE) represents the purchase probability of a Chinese nationality customer; sp(IPHONE|CHINESE) represents the probability of purchasing an iPhone given known Chinese nationality.

[0084] The attribute value dependency preservation assumption assumes that the dependency of the attributes of the main table remains unchanged before and after the connection. This means that after the customer table is connected to the purchase table, the data distribution of the customer table remains unchanged. Therefore, the conditional probability Pcp(BLACK|CHINESE) required in the inference can be replaced by Pc(BLACK|CHINESE). This is the main source of error in the algorithm, because the distribution of attribute values ​​will change in most cases after the connection.

[0085] Therefore, Pcp(BLACK|CHINESE) and Pc(BLACK|CHINESE) are not equal in most cases, and the two values ​​may even be far apart. In reasoning, the more conditional probabilities are based on this assumption to obtain their values, the greater the error of the final estimate. Therefore, this application proposes to use sampling to simulate the value of Pcp(BLACK|CHINESE). The estimate obtained by reasonable sampling is more accurate than the one obtained by the connection invariant assumption.

[0086] Sampling methods include but are not limited to stratified sampling, Bernoulli sampling, universe sampling, random walk sampling, and newly designed sampling methods. Stratified sampling ensures that each subgroup has a representative sample; Bernoulli sampling independently decides whether to select each record with a certain probability; universe sampling selects samples at fixed intervals in an ordered data set; random walk sampling is suitable for graph data or data with complex link structures; and the newly designed sampling method can be customized according to specific application scenarios.

[0087] The above are some specific implementations of the query method provided in the embodiment of the present application. Based on this, the present application also provides a corresponding query system. The system provided in the embodiment of the present application will be introduced from the perspective of functional modularization. Figure 4 A structural diagram of a query system provided in an embodiment of the present application.

[0088] The system comprises:

[0089] An acquisition unit 110, used to acquire a query statement;

[0090] A first determining unit 111 is configured to, if the data table indicated by the query statement includes multiple data tables that are connected, determine a data table in an unchanged state and a data table in a changing state among the multiple data tables; for a data table in an unchanged state, determine the number of queries using a corresponding Bayesian network model; for a data table in a changing state, extract sample data from the connected data table and determine conditional probability distribution information corresponding to the sample data;

[0091] The second determining unit 112 is used to determine a query plan based on the number of queries corresponding to the data table in an unchanged state and the conditional probability distribution information corresponding to the data table in a changing state, wherein the query plan is used to indicate the execution process of the query.

[0092] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the test method solutions provided in the embodiments of the present application.

[0093] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the test method described in any embodiment of the present application.

[0094] The computer storage medium stores codes, and when the codes are executed, a device executing the codes implements the test method described in any embodiment of the present application.

[0095] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0096] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0097] It should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0098] It should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0099] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0100] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0101] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A query method, characterized in that: include: Get the query statement; If the data table indicated by the query statement includes multiple data tables that are connected, determine the data tables in an unchanged state and the data tables in a changing state among the multiple data tables; for the data tables in an unchanged state, use the corresponding Bayesian network model to determine the number of queries; for the data tables in a changing state, extract sample data from the connected data tables, and determine the conditional probability distribution information corresponding to the sample data; Based on the number of queries corresponding to the data table in an unchanged state and the conditional probability distribution information corresponding to the data table in a changing state, a query plan is determined, where the query plan is used to indicate the execution process of the query.

2. The method according to claim 1, characterized in that The Bayesian network model is determined in the following manner: Get multiple data tables stored in the database; According to the dependency relationship between variables in each data table, a directed acyclic graph corresponding to each data table is constructed; For each node of the directed acyclic graph, construct a corresponding conditional probability table; A Bayesian network model corresponding to each data table is obtained based on the training set and the directed acyclic graph training, and the Bayesian network model is used to obtain probability distribution through probability reasoning.

3. The method according to claim 2, characterized in that Also includes: For each data table, add an index to the primary key of the data table.

4. The method according to claim 1, characterized in that: The determining of a data table in a constant state and a data table in a changed state among the multiple data tables comprises: For each data table, determine the interval between the current time point and the last update time point; If the interval exceeds the preset interval threshold, the data table is divided into a data table in a constant state; if the interval does not exceed the preset interval threshold, the data table is divided into a data table in a changing state.

5. The method according to claim 1, characterized in that Also includes: If the data tables indicated by the query statement include only a single data table, the number of queries corresponding to the single data table is determined using the corresponding Bayesian network model.

6. The method according to claim 1, characterized in that Also includes: Get the actual execution result after the query plan is executed; The actual execution result is compared with the expected result, and the corresponding Bayesian network model is adjusted based on the comparison result.

7. A query system, characterized in that: include: An acquisition unit, used for acquiring a query statement; A first determining unit is configured to, if the data table indicated by the query statement includes multiple data tables that are connected, determine a data table in an unchanged state and a data table in a changing state among the multiple data tables; for a data table in an unchanged state, determine the number of queries using a corresponding Bayesian network model; for a data table in a changing state, extract sample data from the connected data table and determine conditional probability distribution information corresponding to the sample data; The second determination unit is used to determine a query plan based on the number of queries corresponding to the data table in an unchanged state and the conditional probability distribution information corresponding to the data table in a changing state, wherein the query plan is used to indicate the execution process of the query.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the query method according to any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that When the computer program product runs on a computer, the computer executes the query method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on the terminal device, the terminal device executes the query method according to any one of claims 1 to 6.

Images