A method and device for data query

Abstract

The embodiment of the present application provides a method and device for data query, the method comprising: presetting multiple query plans; obtaining real-time query conditions; wherein the real-time query conditions include query conditions for structured data and query conditions for unstructured data; for the real-time query conditions, determining the target selection rate of the pre-constructed center point in each query plan; according to the target selection rate, determining the query cost corresponding to each query plan; using the query cost, determining the target query plan from the multiple query plans, and executing the query for the real-time query condition according to the target query plan. Through the embodiment of the present application, the optimization of the fusion query is realized, and multiple query plans are generated while ensuring the output accuracy, and a query plan with good performance is selected from them, which can not only ensure the accuracy of the fusion query, but also reduce the cost of the fusion query and ensure the query response time.

Description

Technical Field

[0001] The present application relates to the field of search technology, and in particular to a method and device for data query. Background Art

[0002] With the development of science and technology, data queries usually include structured queries and unstructured queries. Structured queries are more common, and unstructured queries are vector queries, which convert images, texts, sounds, etc. into vectors and store and query them in the form of vectors.

[0003] In some cases, structured queries are included in unstructured queries, which are called fusion queries. Since fusion queries include structured queries and unstructured queries, there are problems such as low query accuracy and poor query performance in the fusion query process, and it is difficult to guarantee reasonable accuracy output and query response time, making it difficult to meet the needs. Summary of the invention

[0004] In view of the above problems, a method and device for data query is proposed to overcome the above problems or at least partially solve the above problems, including:

[0005] A method for data query, the method comprising:

[0006] Preset multiple query plans;

[0007] Acquire real-time query conditions; wherein the real-time query conditions include query conditions for structured data and query conditions for unstructured data;

[0008] Determine, for the real-time query condition, a target selection rate of the pre-built center point in each query plan;

[0009] Determine the query cost corresponding to each query plan according to the target selection rate;

[0010] The query cost is used to determine a target query plan from the multiple query plans, and a query for the real-time query condition is executed according to the target query plan.

[0011] Optionally, determining a target selection rate of a pre-built center point in each query plan for the real-time query condition includes:

[0012] Determine the number of returned results corresponding to the real-time query condition;

[0013] Determining a first selection rate and a second selection rate for the pre-constructed center point according to the number of returned results;

[0014] The first selection rate and the second selection rate are combined to obtain a target selection rate.

[0015] Optionally, determining a first selection rate of pre-constructed central points according to the number of returned results includes:

[0016] Determine a first query range corresponding to the number of returned results;

[0017] Obtaining a preset precision parameter, and determining a second query range corresponding to the precision parameter in the first query range;

[0018] Determine the number of records corresponding to the second query range;

[0019] The ratio of the number of records to the total number of records is determined to obtain a first selection rate of the pre-constructed center point.

[0020] Optionally, determining a second selection rate of pre-constructed central points according to the number of returned results includes:

[0021] Determine the number of structured query results; wherein the number of structured query results is the number of results obtained by executing the query condition for the structured data;

[0022] Determine whether the number of returned results is greater than a preset number;

[0023] If the number of returned results is greater than the preset number of results, the second selection rate of the pre-constructed center point is determined by using the number of structured query results and the number of returned results;

[0024] If the number of returned results is less than or equal to the preset number of results, the number of structured query results is used to determine a second selection rate for the pre-constructed center point.

[0025] Optionally, combining the first selection rate and the second selection rate to obtain a target selection rate includes:

[0026] Among the first selection rate and the second selection rate, the maximum selection rate is selected as the target selection rate.

[0027] Optionally, determining the query cost corresponding to each query plan according to the target selection rate includes:

[0028] Using the target selection rate, determine the number of selection results;

[0029] The number of selected results is used to determine a query cost corresponding to the query plan.

[0030] Optionally, the adopting the query cost to determine a target query plan from the multiple query plans includes:

[0031] From the multiple query plans, a query plan corresponding to a minimum query cost is determined as a target query plan.

[0032] Optionally, the presetting multiple query plans includes:

[0033] Preset one or more physical query operators;

[0034] The one or more physical query operators are used to generate multiple query plans.

[0035] A data query device, comprising:

[0036] A query plan preset module is used to preset multiple query plans;

[0037] A real-time query condition acquisition module, used to acquire real-time query conditions; wherein the real-time query conditions include query conditions for structured data and query conditions for unstructured data;

[0038] A target selection rate determination module, used to determine the target selection rate of the pre-built center point in each query plan according to the real-time query condition;

[0039] A query cost determination module, used to determine the query cost corresponding to each query plan according to the target selection rate;

[0040] The target query plan execution module is used to determine a target query plan from the multiple query plans using the query cost, and execute a query for the real-time query condition according to the target query plan.

[0041] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the data query method described above when executed by the processor.

[0042] A computer-readable storage medium stores a computer program, which implements the steps of the data query method described above when executed by a processor.

[0043] The embodiments of the present application have the following advantages:

[0044] In an embodiment of the present application, multiple query plans are pre-set to obtain real-time query conditions, which include query conditions for structured data and query conditions for unstructured data. For the real-time query conditions, the target selection rate of the pre-built center point in each query plan is determined, and the query cost corresponding to each query plan is determined based on the target selection rate. The query cost is used to determine the target query plan from multiple query plans, and according to the target query plan, the query for the real-time query condition is executed, thereby optimizing the fusion query. While ensuring the output accuracy, multiple query plans are generated, and a query plan with good performance is selected from them, which can not only ensure the accuracy of the fusion query, but also reduce the cost of the fusion query, thereby ensuring the query response time. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0046] Figure 1 This is a flowchart of a method for data query provided by an embodiment of the present application;

[0047] Figure 2a is a schematic diagram of a physical query operator provided by an embodiment of the present application;

[0048] Figure 2b is a schematic diagram of another physical query operator provided by an embodiment of the present application;

[0049] Figure 2c is a schematic diagram of another physical query operator provided by an embodiment of the present application;

[0050] Figure 2d is a schematic diagram of another physical query operator provided by an embodiment of the present application;

[0051] Figure 3 is a flowchart of another method for data query provided by an embodiment of the present application;

[0052] Figure 4a is a relationship diagram between target selection rate and accuracy provided by an embodiment of the present application;

[0053] Figure 4b This is a relationship diagram between the number of returned results and the target selection rate provided by an embodiment of the present application;

[0054] Figure 5 is a schematic diagram of a fusion query provided by an embodiment of the present application;

[0055] Figure 6 It is a structural diagram of a data query device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0057] In the data query scenario, there are structured queries and unstructured queries. Unstructured queries are vector queries, which convert image, text, sound and other data into vectors for storage. When performing unstructured queries, the queried images, text, sound and other data can be converted into vectors at the same time, and the distance is calculated with the currently stored vectors, and the vectors with close distances are output. Including structured queries in unstructured queries is called fusion queries.

[0058] In practical applications, structured data can be stored in relational databases, and unstructured data can be stored in the system. When performing a fusion query, the structured information of the relational database is queried and the unstructured information of the vector is retrieved at the same time, and then the intersection of the two queried data sets is calculated to complete the fusion query. However, this fusion query method may bring the following problems:

[0059] 1. Fusion query is based on two systems, and the communication overhead is relatively large;

[0060] 2. If the result set retrieved by the ANN (Approximate Nearest Neighbor) of the unstructured query returns too few results, the intersection with the result set filtered by the structured condition will obtain very few results, and the recall value will be very low;

[0061] 3. If there are too many results from structured filtering, the query performance will be very slow when performing ANN retrieval.

[0062] However, the result required by the user is:

[0063] 1. The accuracy of fusion query is guaranteed to be above a certain value, such as above 95%;

[0064] 2. The performance of fusion query is better, which can ensure the query response time and meet the requirements.

[0065] In an embodiment of the present application, a fusion query optimizer is set up on the basis of an analytical database system to generate richer query plans while ensuring accuracy, from which query plans with better performance, that is, query plans with low query cost, are selected to execute related queries.

[0066] The following is a detailed description of query Q1:

[0067] Q1=SELECT*FROMAWHEREA.c1=p1 ORDER BY DISTANCE(A.f1,v1)LIMITk

[0068] Table A contains two columns, one is structured data c1, and the other is unstructured data f1. Q1 is used for fusion query. In table A, when the query condition A.c1=p1 (p1 is structured data) for structured data is met, the distance between A.f1 and v1 (v1 is unstructured data) vectors is calculated, that is, the query condition for unstructured data is met, and the top k results with the closest distance are returned.

[0069] Reference Figure 1 , shows a flowchart of a method for data query provided by an embodiment of the present application, which may specifically include the following steps:

[0070] Step 101, presetting multiple query plans;

[0071] For fusion queries, multiple query plans can be set in advance to ensure the richness of the query plans, and then a query plan with better performance can be selected from them while ensuring accuracy.

[0072] In an embodiment of the present application, step 101 may include the following sub-steps:

[0073] Sub-step 11, presetting one or more physical query operators;

[0074] Since the optimization methods of CBO (Cost-Based Optimization) and RBO (Rule-Based Optimization) include various relational algebra optimization operators and various physical optimization algorithms, physical query operation operators (i.e., operators) for unstructured queries can be pre-set to generate relevant query plans.

[0075] For example, the following physical query operators can be included:

[0076] filter A.f1<->q (A):

[0077] Executing a structured query will generate a result set. When traversing the unstructured index, it will check whether the traversal result is in the result set of the previous structured query. If it is, it will be calculated. A.f1<->q is used to represent the distance between the query vector and the internal vector of the database.

[0078] IdxScan A.f1<->q (A,k):

[0079] Traverse the current vector q and return k results similar to q. The results are not sorted.

[0080] Sort_Full_View A.f1<->q (A,k):

[0081] According to the previously generated result set, relevant long features are extracted from it, the distance between the long features and the query q is sorted, and the top k results are found.

[0082] Sort_PQ A.f1<->q (A,k):

[0083] According to the previously generated result set, relevant PQ (Product Quantization) codes are extracted from it, the PQ distances are sorted, and the top k results are found.

[0084] Merge:

[0085] When the current system uses column storage, after the independent query of each column is completed, the results of all columns must be intersected to obtain relevant results. The current Merge operator can ensure that after the Merge is completed, the Merge results are still sorted according to the unstructured sorting.

[0086] IdxScan A.c1＝p1 (A):

[0087] The internally implemented structured index query creates an inverted index for each implicit structured column to facilitate query when the current system is a system with less writes and more reads.

[0088] Sub-step 12, using the one or more physical query operators to generate multiple query plans.

[0089] After obtaining the physical query operators, the physical query operators may be combined to generate multiple query plans for fusion queries.

[0090] The following is an exemplary description based on the above physical query operator:

[0091] like Figure 2a , by the physical query operator Sort_Full_View A.f1<->q(A,k)→SIdxScan A.c1＝p1 (A) Constructing query plan 1: First obtain the results of the structured query, and sort the structured query results by long features.

[0092] like Figure 2b , by the physical query operator Sort_Full_View A.f1<->q (A,k)→Sort_PQ A.f1<->q (A,αk)→SIdxScan A.c1＝p1 (A) Construct query plan 2: first obtain the results of the structured query, sort the structured query results by PQ code, select the results of αk, then sort the results of αk by long features, and return the final k results.

[0093] like Figure 2c , by the physical query operator Sort_Full_View A.f1<->q (A,k)→Sort_PQ A.f1<->q (A,αk)→filter A.f1<->q (A) → SIdxScan A.c1＝p1 (A) Construct query plan 3: first obtain the result of the structured query, pass the result of the structured query into the vector index, filter it, then sort the filtered results by PQ code and long features, and return the final k results.

[0094] like Figure 2d , from the physical query operator Merge→Sort_Full_View A.f1<->q (A,q,k)→Sort_PQ A.f1<->q (A,αk)→IdxScan A.f1<->q (A) and Merge→IdxScan A.c1＝p1 (A) Construct query plan 4: obtain the result of the structured query, obtain the result of the unstructured query, and then find the intersection of the two results.

[0095] Step 102, obtaining real-time query conditions; wherein the real-time query conditions include query conditions for structured data and query conditions for unstructured data;

[0096] When performing a fusion query, user input can be received, and then real-time query conditions for the fusion query can be obtained based on the user input. The user input can be directly used as a query condition for structured data, and the user input can be converted into a vector as a query condition for unstructured data.

[0097] Step 103, determining the target selection rate of the pre-built center point in each query plan according to the real-time query condition;

[0098] For each query plan, a target selection rate of the pre-constructed center point under the real-time query condition may be determined, that is, the probability of selecting the center point in all data records.

[0099] It should be noted that the center point can be obtained by clustering the original data records. One piece of data can have multiple center points. The selection ratio of the center points affects the accuracy and efficiency of the query.

[0100] For example, in the IVF-PQ (Inverted File-Product Quantization) algorithm, PQ is used as a code that divides a 128-dimensional vector into four parts and clusters each part of the data. For example, if the k-means clustering algorithm is used, 256 cluster centers are generated for each part. Then, the original vector can be re-encoded using the cluster center number. Only 4 bytes are needed for an original vector to record the vector of the cluster center.

[0101] To achieve good results after PQ coding compression, but the query volume is still large, you can add a rough filter in advance, cluster the vectors into 1024 cluster centers to form an inverted index, and calculate the residual between each original vector and its center, and then perform PQ quantization on this residual data set. Using PQ to process the residual is simpler than using the variance of the original data.

[0102] When performing a query, you can first find multiple center points close to the query vector, then calculate the top vector after PQ quantization at the center point, and return the top-K results.

[0103] Step 104, determining the query cost corresponding to each query plan according to the target selection rate;

[0104] After determining the target selection rate, for each query plan, the query cost corresponding to the query plan can be determined in combination with its corresponding target selection rate. The query cost may be the time required to execute the current fusion query.

[0105] In an embodiment of the present application, step 104 may include the following sub-steps:

[0106] Sub-step 21, using the target selection rate to determine the number of selection results;

[0107] In a specific implementation, the total number of records, that is, the number of all data records, can be obtained, and then the target selection rate is multiplied by the total number of records to obtain the number of selection results.

[0108] Sub-step 22, using the number of selected results, to determine the query cost corresponding to the query plan.

[0109] After determining the number of selected results, the time required for each selected result can be determined, and then the time required for selecting the results corresponding to the number of selected results can be determined, and then the query cost corresponding to the query plan can be obtained. Of course, the query cost is not only related to the target selection rate used to characterize the selection, but also related to other operations in the fusion query.

[0110] The following is a detailed description based on the above query plan:

[0111] Query plan 1 first calculates all the result sets R (n1*t0) that meet the structural conditions. Then, query plan 1 brute force scans and directly calculates the distance between v1 and all vectors in the result set R, sorts them according to the distance, and finds the top k results (n1*t1).

[0112] The query cost of query plan 1 is cost1 = (n1*t0) + (n1*t1)

[0113] Query plan 2 first calculates all the result sets R (n1*t0) that meet the structural conditions. Next, the PQ distance of each vector in the result set R is calculated, and the results are sorted according to the distance to find the first αk results (n1*t2). Then, the first αk results are rearranged by long features to obtain the first k results.

[0114] The query cost of query plan 2 is cost2 = (n1*t0) + (n1*t1) + (n2*t1)

[0115] Query plan 3 first calculates all the result sets R(n1*t0) that meet the structured conditions. Then, query plan 3 can be filtered using the IVF-PQ algorithm. The IVF-PQ algorithm will select (cr*n3) results, where CR is the proportion of the selected center points (i.e., the target selection rate). When the system traverses each result of cr*n3, it will check whether the result is in R. If so, it will be retained. Then, the PQ code is calculated for the filtered result n4 to find the first αk results. Then, the first αk results are rearranged by long features to obtain the first k results.

[0116] The query cost of query plan 3 is cost13 = n1*t0+(cr*n3)*t3+n4*t2+t1*n2

[0117] Query plan 4, first calculate all the result sets R(n1*t0) that meet the structured conditions. In order to ensure accuracy, it is necessary to obtain βk results in the unstructured query. According to the βk results, adjust the range of CR. At the same time, use the IVF-PQ algorithm to calculate the first (cr*n3) results, and then calculate the PQ code to find the first αk results. Then, the first αk results are rearranged by long features to obtain the first βk results.

[0118] The query cost of query plan 4 is cost4 = n1*t0+(cr*n3)*t3+αβk*t2+t1*βk

[0119] Table 1 below is an explanation of the above symbols:

[0120]

[0121] Table 1

[0122] Step 105 , using the query cost, determining a target query plan from the multiple query plans, and executing a query for the real-time query condition according to the target query plan.

[0123] After determining the query cost, a target query plan is determined from multiple query plans, and then a query for the real-time query condition can be executed according to the target query plan to achieve fusion query.

[0124] In an embodiment of the present application, step 105 may include the following sub-steps:

[0125] From the multiple query plans, a query plan corresponding to a minimum query cost is determined as a target query plan.

[0126] In a specific implementation, a query plan corresponding to the minimum query cost can be determined from multiple query plans as a target query plan, thereby ensuring the performance of the fusion query.

[0127] In an embodiment of the present application, multiple query plans are pre-set to obtain real-time query conditions, which include query conditions for structured data and query conditions for unstructured data. For the real-time query conditions, a target selection rate corresponding to each query plan is determined, and based on the target selection rate, a query cost corresponding to each query plan is determined. The query cost is used to determine a target query plan from multiple query plans, and according to the target query plan, a query is executed for the real-time query conditions, thereby optimizing the fusion query. While ensuring the output accuracy, multiple query plans are generated, and a query plan with better performance is selected from them, which can not only ensure the accuracy of the fusion query, but also reduce the cost of the fusion query, thereby ensuring the query response time.

[0128] Reference Figure 3 , shows a flowchart of another data query method provided by an embodiment of the present application, which may specifically include the following steps:

[0129] Step 301, presetting multiple query plans;

[0130] Step 302, obtaining real-time query conditions; wherein the real-time query conditions include query conditions for structured data and query conditions for unstructured data;

[0131] Step 303, determining the number of returned results corresponding to the real-time query condition;

[0132] In a specific implementation, the number of returned results included in the real-time query condition may be determined, that is, the number of query results required to be returned by the query. For example, in Q1 above, the number of returned results is k.

[0133] Step 304, determining a first selection rate and a second selection rate for the pre-constructed center point according to the number of returned results;

[0134] In practical applications, the accuracy of fusion query is guaranteed by two parts: the accuracy of unstructured query and the accuracy of fusion query. The accuracy of unstructured query can be represented by the first selection rate, and the accuracy of fusion query can be represented by the second selection rate.

[0135] The following is a detailed description:

[0136] In an embodiment of the present application, step 304 may include the following sub-steps:

[0137] Determine a first query range corresponding to the number of returned results; obtain a preset precision parameter, and determine a second query range corresponding to the precision parameter in the first query range; determine the number of records corresponding to the second query range; determine the proportion of the number of records to the total number of records, and obtain a first selection rate for the pre-constructed center point.

[0138] For the accuracy of unstructured queries, the main factor is the scope of index queries. The accuracy of unstructured queries can be improved by expanding the scope of queries. The size of the query range is determined by CR (the proportion of selected center points). As shown in 4a, as more and more center blocks are selected (CR), the accuracy of unstructured queries will continue to improve.

[0139] In addition, the precision and query range are also related to the selection of top-k (the number of results returned), such as Figure 4b ,As the value of top-k gradually increases, CR will also gradually increase.

[0140] Applied in the embodiment of the present application, since the system can record the selected query ranges corresponding to different k internally, the system can adaptively select different query ranges based on different top-k values ​​obtained in real time. Then, after determining the number K of returned results corresponding to the real-time query conditions, the first query range corresponding to the number of returned results can be determined.

[0141] In practical applications, the accuracy parameter can be set in advance, such as 95%. After determining the first query range corresponding to the number k of returned results, the second query range corresponding to the accuracy parameter can be determined in the first query range to ensure that subsequent operations are performed under the condition of satisfying the accuracy parameter.

[0142] After determining the second query range corresponding to the precision parameter, you can search under the second query range to determine the number of records found, and you can get the total number of records, that is, the number of all data records, and then you can determine the proportion of the number of records to the total number of records, and get the first selection rate CR0, as shown below

[0143] CR0=search_range(k) / l

[0144] Among them, search_range(k) is the number of records corresponding to the second query range. For example, according to different k, while ensuring the accuracy within the query range of 95%, the number of points searched, I is the total number of records, and CR0 is the quotient of these two numbers.

[0145] In an embodiment of the present application, step 304 may further include the following sub-steps:

[0146] Determine the number of structured query results; wherein the number of structured query results is the number of results obtained by executing the query conditions for the structured data; determine whether the number of returned results is greater than a preset number; if the number of returned results is greater than the preset number of results, use the number of structured query results and the number of returned results to determine a second selection rate; if the number of returned results is less than or equal to the preset number of results, use the number of structured query results to determine a second selection rate for a pre-constructed center point.

[0147] For the accuracy of fusion query, such as Figure 5 , E1 is the result set of structured query, S1 and S2 are the result sets of unstructured query. For S2, because the range of topk selected is relatively small, it will cause inaccurate precision to find the first K (such as K = 6) records in E1. If the query range is too large, such as S1 has covered the entire E1, unnecessary calculations will increase, resulting in poor performance of S1 query. In this case, a reasonable query range needs to be given to ensure both precision and speed.

[0148] Assuming that the structured column c1 and the unstructured column f1 are two unrelated columns, the following formula can be obtained through statistics to calculate the selection rate of the center point, that is, the second selection rate CR1.

[0149]

[0150] Among them, k is the first k records to be found in the query, and m refers to the records obtained by the structured query.

[0151] Applied in the embodiment of the present application, the number m of structured query results can be determined, and it can be determined whether the number k of returned results is greater than a preset number (such as the preset number is 5).

[0152] If the number k of returned results is greater than the preset number of results (such as the preset number is 5), the second selection rate can be determined by using the number of structured query results and the number of returned results, such as calculating the quotient of 3K divided by M as the second selection rate.

[0153] If the number of returned results is less than k or equal to the preset number of results (such as the preset number is 5), the second selection rate can be determined by using the number of structured query results, such as calculating the quotient of 15 divided by m as the second selection rate.

[0154] Step 305, combining the first selection rate and the second selection rate to obtain a target selection rate;

[0155] After obtaining the first selection rate and the second selection rate, one of them may be selected as the target selection rate corresponding to the query plan.

[0156] In an embodiment of the present application, step 305 may further include the following sub-steps:

[0157] Among the first selection rate and the second selection rate, the maximum selection rate is selected as the target selection rate.

[0158] In practical applications, the maximum selection rate can be selected as the target selection rate CR corresponding to the query plan, as shown in the following formula:

[0159] CR=max(CR0,CR1)

[0160] Step 306, determining the query cost corresponding to each query plan according to the target selection rate;

[0161] Step 307 : using the query cost, determining a target query plan from the multiple query plans, and executing a query for the real-time query condition according to the target query plan.

[0162] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0163] Reference Figure 6 , shows a schematic diagram of the structure of a data query device provided by an embodiment of the present application, which may specifically include the following modules:

[0164] A query plan presetting module 601 is used to preset multiple query plans;

[0165] A real-time query condition acquisition module 602 is used to acquire real-time query conditions; wherein the real-time query conditions include query conditions for structured data and query conditions for unstructured data;

[0166] A target selection rate determination module 603 is used to determine the target selection rate of the pre-built center point in each query plan according to the real-time query condition;

[0167] A query cost determination module 604, configured to determine a query cost corresponding to each query plan according to the target selection rate;

[0168] The target query plan execution module 605 is used to determine a target query plan from the multiple query plans using the query cost, and execute a query for the real-time query condition according to the target query plan.

[0169] In one embodiment of the present application, the target selection rate determination module 603 includes:

[0170] A submodule for determining the number of returned results, used to determine the number of returned results corresponding to the real-time query condition;

[0171] The first and second selection rate determination submodules are used to determine the first selection rate and the second selection rate of the pre-constructed center point according to the number of returned results;

[0172] A submodule for determining a target selection rate is combined to combine the first selection rate and the second selection rate to obtain a target selection rate.

[0173] In one embodiment of the present application, the first and second selection rate determination submodules include:

[0174] A first query range determining unit, configured to determine a first query range corresponding to the number of returned results;

[0175] A second query range determining unit, configured to obtain a preset precision parameter and determine a second query range corresponding to the precision parameter in the first query range;

[0176] A record quantity determination unit, used to determine the number of records corresponding to the second query range;

[0177] The first selection rate unit is used to determine the ratio of the number of records to the total number of records, and obtain a first selection rate for the pre-constructed center point.

[0178] In one embodiment of the present application, the first and second selection rate determination submodules include:

[0179] A structured query result number determination unit, used to determine the number of structured query results; wherein the number of structured query results is the number of results obtained by executing the query condition for the structured data;

[0180] A preset number judgment unit is used to judge whether the number of returned results is greater than the preset number; if the number of returned results is greater than the preset number of results, call the structured query and returned result number determination unit; if the number of returned results is less than or equal to the preset number of results, call the structured query result number determination unit.

[0181] A structured query and returned result quantity determination unit, configured to determine a second selection rate for a pre-constructed central point by using the structured query result quantity and the returned result quantity;

[0182] The structured query result quantity determining unit is used to determine a second selection rate of the pre-constructed central point by using the structured query result quantity.

[0183] In one embodiment of the present application, the combined determination target selection rate submodule includes:

[0184] The maximum selection rate determination unit is used to select the maximum selection rate as the target selection rate between the first selection rate and the second selection rate.

[0185] In an embodiment of the present application, the query cost determination module 604 includes:

[0186] A submodule for determining the number of selected results, used to determine the number of selected results using the target selection rate;

[0187] The submodule for determining the cost based on the number of selected results is used to determine the query cost corresponding to the query plan using the number of selected results.

[0188] In an embodiment of the present application, the target query plan execution module 605 includes:

[0189] The minimum query cost determination submodule is used to determine a query plan corresponding to the minimum query cost from the multiple query plans as a target query plan.

[0190] In an embodiment of the present application, the query plan presetting module 601 includes:

[0191] A physical query operator presetting submodule, used to presetting one or more physical query operators;

[0192] The physical query operator plan generation submodule is used to generate multiple query plans using the one or more physical query operators.

[0193] An embodiment of the present application also provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the steps of the above data query method are implemented.

[0194] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above data query method are implemented.

[0195] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0196] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0197] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0198] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0199] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0200] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0201] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0202] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0203] The above is a detailed introduction to a method and device for data query. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for data query, the method comprising: Preset multiple query plans; Acquire real-time query conditions; wherein the real-time query conditions include query conditions for structured data and query conditions for unstructured data; According to the real-time query condition, determine the target selection rate of the pre-constructed center point in each query plan; wherein the center point is obtained by clustering the original data records; the target selection rate is the probability of selecting the center point in all the data records; Determine the query cost corresponding to each query plan according to the target selection rate; The query cost is used to determine a target query plan from the multiple query plans, and a query for the real-time query condition is executed according to the target query plan.

2. According to the method of claim 1, determining the target selection rate of the pre-built center point in each query plan for the real-time query condition comprises: Determine the number of returned results corresponding to the real-time query condition; Determining a first selection rate and a second selection rate for the pre-constructed center point according to the number of returned results; The first selection rate and the second selection rate are combined to obtain a target selection rate.

3. The method according to claim 2, wherein determining a first selection rate of pre-constructed center points according to the number of returned results comprises: Determine a first query range corresponding to the number of returned results; Obtaining a preset precision parameter, and determining a second query range corresponding to the precision parameter in the first query range; Determine the number of records corresponding to the second query range; The ratio of the number of records to the total number of records is determined to obtain a first selection rate of the pre-constructed center point.

4. The method according to claim 2 or 3, wherein determining the second selection rate of the pre-constructed center point according to the number of returned results comprises: Determine the number of structured query results; wherein the number of structured query results is the number of results obtained by executing the query condition for the structured data; Determine whether the number of returned results is greater than a preset number of results; If the number of returned results is greater than the preset number of results, the second selection rate of the pre-constructed center point is determined by using the number of structured query results and the number of returned results; If the number of returned results is less than or equal to the preset number of results, the number of structured query results is used to determine a second selection rate for the pre-constructed center point.

5. The method according to claim 2, wherein combining the first selection rate and the second selection rate to obtain a target selection rate comprises: Among the first selection rate and the second selection rate, the maximum selection rate is selected as the target selection rate.

6. The method according to claim 1, wherein determining the query cost corresponding to each query plan according to the target selection rate comprises: Using the target selection rate, determine the number of selection results; The number of selected results is used to determine a query cost corresponding to the query plan.

7. The method according to claim 1, wherein the step of using the query cost to determine a target query plan from the plurality of query plans comprises: From the multiple query plans, a query plan corresponding to a minimum query cost is determined as a target query plan.

8. The method according to claim 1, wherein the presetting of multiple query plans comprises: Preset one or more physical query operators; The one or more physical query operators are used to generate multiple query plans.

9. A data query device, comprising: A query plan preset module is used to preset multiple query plans; A real-time query condition acquisition module, used to acquire real-time query conditions; wherein the real-time query conditions include query conditions for structured data and query conditions for unstructured data; A target selection rate determination module is used to determine the target selection rate of the pre-constructed center point in each query plan according to the real-time query condition; wherein the center point is obtained by clustering the original data records; and the target selection rate is the probability of selecting the center point in all the data records; A query cost determination module, used to determine the query cost corresponding to each query plan according to the target selection rate; The target query plan execution module is used to determine a target query plan from the multiple query plans using the query cost, and execute a query for the real-time query condition according to the target query plan.

10. An electronic device comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the data query method according to any one of claims 1 to 8 when executed by the processor.

11. A computer-readable storage medium storing a computer program, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data query method according to any one of claims 1 to 8 are implemented.

Images