Data frequency inference method and device based on SQL query price

Abstract

The invention discloses a method and device for deducing data frequency based on SQL query prices, and the method comprises the steps: constructing queries Q1 and Q2 for data v, obtaining prices p1 and p2, and directly purchasing and returning the frequency of the data v if any price is 0; otherwise, when the pricing algorithm is based on the usage amount, constructing a group of LIMIT queries Qk and querying prices pk corresponding to queries under different k values, and positioning the first element k meeting the condition that pk is equal to p1 as the frequency; and when the pricing algorithm is based on the information amount, constructing a group of HAVING queries Qi and inquiring the price pi of the corresponding query, determining the first element i of which pi is 0, constructing a group of new HAVING queries Qj, inquiring the price pj of the corresponding query, calculating the mean value of all the elements j meeting the condition that pj is equal to p2, and rounding down to obtain the frequency. The method can deduce the occurrence frequency of the original data from the SQL query price, the precision loss is less than or equal to 1, and the method is efficient.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data processing, and particularly relates to a data frequency inference method and device based on SQL query price. BACKGROUND

[0002] With the development of Internet of Things devices, 5G communication and large models, the application value of data is continuously highlighted. Under this background, the demand for big data interaction, integration and circulation continues to grow, which promotes the rise of big data transactions. In the face of diversified data analysis and transaction needs, how to build an efficient data transaction mode has become the core challenge for platform landing. The query-based transaction mode has broad application prospects, which can enable budget-constrained users to obtain the required data through accurate queries and avoid the high cost problem of full-scale procurement. As a mainstream query language, SQL (Structured Query Language) query pricing has attracted widespread attention from the academic community.

[0003] Existing SQL query pricing methods can be divided into two categories, including pricing based on data usage and pricing based on query information volume, aiming to provide reasonable and effective prices for different types of queries. The former method defines the price of each query as the minimum total price of the tuples or views that can generate the query results, while in the latter method, the database is regarded as a variable with multiple possible values, and the query price depends on the amount of information it reveals about the real database instance.

[0004] However, the high correlation between query price and query result poses a significant threat to query pricing, that is, disclosing the query price may expose sensitive information about the underlying data set. Clever data consumers can infer sensitive data by analyzing the query price without actually purchasing any query. Therefore, it is necessary to further explore the information inference and leakage problem in the data query market. Existing methods do not consider the information leakage risk in data query transactions, and effective information inference algorithms are urgently needed to provide effective reference for pricing strategy design. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a data frequency inference method and device based on SQL query price, to realize zero economic cost inference of data information, and to provide security basis for pricing strategy design.

[0006] According to a first aspect of the embodiments of the present application, a data frequency inference method based on SQL query price is provided, comprising: According to the input data v Constructing SQL queries Q 1 and Q 2, inquiring the query prices corresponding to the two queries p 1 and p2. If the queried price p 1 or p If 2 is zero, directly purchase the corresponding query. Q 1 or Q 2. Set and return data based on the query results. v The frequency of occurrence; otherwise, a new set of query inference data is reconstructed based on the underlying pricing algorithm. v The frequency; When the underlying pricing algorithm is based on data usage, then construct a set of LIMIT queries. Q k And ask different elements k The corresponding price for the given value. p k Find it based on binary search k =0, ..., N The first one to satisfy p k equal p 1 element k , data v The frequency of occurrence of the element k is set to the value of the element k and the result is returned. When the underlying pricing algorithm is based on the amount of information in the query, then construct a set of HAVING queries. Q i And ask for the corresponding price. p i Find it based on binary search i =0, ..., N The first one to satisfy p i Elements that are 0 i This element is the key threshold for inferring data frequency; According to the element i Construct a new set of HAVING queries Q j And ask for the corresponding price. p j Calculate all that satisfy p j equal p 2 elements j The mean of the data is then rounded down to the nearest integer. v The frequency of occurrence is calculated and the results are returned.

[0007] Optionally, the step of adjusting the input data v Constructing SQL queries Q 1 and Q 2, including: The constructed SQL query format is as follows Q1: SELECT * FROM Table WHERE Attribute = v and Q 2: SELECT count(*) FROM Table WHERE Attribute = v Where Table is the data v The relation table it belongs to, Attribute is data v The attribute it belongs to.

[0008] Optionally, construct a set of LIMIT queries. Q k And ask different elements k The corresponding price for the given value. p k Find it based on binary search k =0, ..., N The first one to satisfy p k equal p 1 element k ,include: Set the left and right endpoints of the binary search as follows: left = 0, right = N ,in N For data v The size of the table it belongs to, i.e., its maximum possible frequency of occurrence; like left Less than or equal to right The following process is repeated in a loop to find elements that meet the conditions: k Set as Inquiry Q k :SELECT * FROM Table WHERE Attribute = v LIMIT k price p k ,like p k Greater than or equal to p 1. Update the value of the right endpoint. right = k Otherwise, update the value of the left endpoint. left = k +1; when left Greater than right After the loop ends, set the data frequency to... left It takes a value and returns the result.

[0009] Optionally, construct a set of HAVING queries. Q i And ask for the corresponding price. p i Find it based on binary search i =0, ..., N The first one to satisfy p i Elements that are 0 i ,include: Set the left and right endpoints of the binary search as follows: left = 0, right = N ,in N For data v The size of the table it belongs to, i.e., its maximum possible frequency of occurrence; like left Less than or equal to right The following process is repeated in a loop to find elements that meet the conditions: i Set as Inquiry Q i :SELECT count(*) FROM Table WHERE Attribute = v HAVING count(*) i price p i ,like p i If the value is greater than or equal to 0, update the right endpoint value. right = k Otherwise, update the value of the left endpoint. left = k +1; when left Greater than right When the loop ends, i Set as left The value of .

[0010] Optionally, it also includes: According to the element i Construct a new set of HAVING queries Q j : SELECT count(*) FROM TableWHERE Attribute = v HAVING count(*) = j ; in And ask for the corresponding query. Q j pricep j Calculate all that satisfy p j equal p 2 elements j The mean of the data is then rounded down to the nearest integer. v The frequency of occurrence is calculated and the results are returned.

[0011] According to a second aspect of the embodiments of this application, a data frequency inference apparatus based on SQL query prices is provided, comprising: The data inference module is used to infer data based on the input data. v Constructing SQL queries Q 1 and Q 2. Query the prices corresponding to the two queries. p 1 and p 2. If the queried price p 1 or p If 2 is zero, directly purchase the query, set and return data based on the query results. v The frequency of occurrence; otherwise, a new set of queries is reconstructed based on the underlying pricing algorithm for inference. v The frequency; The first frequency inference module is used to construct a set of LIMIT queries when the underlying pricing algorithm is based on data usage. Q k And ask different k The corresponding price for the given value. p k Find it based on binary search k =0, ..., N The first one to satisfy p k equal p 1 element k , data v The frequency of occurrence is set to the value of the element and the result is returned. The threshold lookup module is used to construct a set of HAVING queries when the underlying pricing algorithm is based on the amount of query information. Q i And ask for the corresponding price. p i Find it based on binary search i =0, ..., N The first one to satisfy p i Elements that are 0 i This element is the key threshold for inferring data frequency; The second frequency inference module, based on the threshold element i Construct a new set of HAVING queriesQ j And ask for the corresponding price. p j Calculate all that satisfy p j equal p 2 elements j The mean of the data is then rounded down to the nearest integer. v The frequency of occurrence is calculated and the results are returned.

[0012] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect.

[0013] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0014] Beneficial effects: Based on the above technical solutions, this invention designs targeted query construction and inference methods according to the type of the underlying pricing algorithm (based on data usage or based on query information volume). For the special case where the query price is directly zero, the query is purchased directly, and the data frequency is returned based on the query results. In the data usage-based pricing scenario, a set of LIMIT queries is constructed. Q k Using binary search, we can quickly locate the price that matches the original query price. k Value, data v The frequency of occurrence is directly mapped to the corresponding k This avoids full data scanning and reduces computational complexity. In pricing scenarios based on the amount of query information, a two-stage HAVING query strategy is adopted. First, a binary search HAVING query is performed. Q i Price positioning key threshold i Then, through a second round of HAVING queries Q j Calculate the price that is equal to the original query price. jThe mean of the values ​​is calculated, and the final frequency estimate is obtained by rounding down. This method reduces the number of price queries while maintaining accuracy through hierarchical querying and binary search, allowing data buyers to infer the frequency of target data with only a few queries without purchasing the data. The method is computationally efficient and the accuracy loss does not exceed 1. Attached Figure Description

[0015] Figure 1 A flowchart illustrating a data frequency inference method based on SQL query prices, provided as an embodiment of the present invention.

[0016] Figure 2 An example diagram of a movie rating database provided in an embodiment of the present invention.

[0017] Figure 3 An example diagram of a movie box office database provided in an embodiment of the present invention.

[0018] Figure 4 A possible database example diagram provided for embodiments of the present invention.

[0019] Figure 5 This is an example diagram of a LIMIT query price provided in an embodiment of the present invention.

[0020] Figure 6 HAVING query provided for embodiments of the present invention Q i Price example image.

[0021] Figure 7 HAVING query provided for embodiments of the present invention Q j Price example image.

[0022] Figure 8 The data frequency inference method provided in this embodiment of the invention demonstrates its inference accuracy and runtime on the Walmart, Employment, TPC-H, and SSB datasets.

[0023] Figure 9 This is a block diagram of a data frequency inference device based on SQL query price provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the specific content of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention. Contents not described in detail in the embodiments of the present invention belong to the prior art known to those skilled in the art.

[0025] This invention provides a data frequency inference method based on SQL query prices, applicable to scenarios where data information is inferred from query prices. In this data market, each data consumer queries data according to their individual needs, and data buyers price data based on existing pricing algorithms. A savvy data consumer can accurately estimate the frequency of data occurrences by continuously querying different query prices. For example, in... Figure 2 In the movie rating dataset, a data analyst might be interested in the number of American movies. Instead of directly purchasing the query, they would infer the number of American movies based on the query price. Similarly, in the movie box office dataset in Figure 3, a data analyst might be interested in the number of American movie box office figures. Again, they would not directly purchase the query, but would infer the amount of American box office data based on the query price.

[0026] The following section provides a detailed explanation of each step in the process of creating the movie rating dataset mentioned above.

[0027] Figure 1 A flowchart illustrating a data frequency inference method based on SQL query prices provided in this embodiment of the invention is shown below. Figure 1 The method may include the following steps: S1: Based on the input data v Constructing SQL queries Q 1 and Q 2. Query the prices corresponding to the two queries. p 1 and p 2. If the queried price p 1 or p If 2 is zero, directly purchase the corresponding query. Q 1 or Q 2. Set and return data based on the query results. v The frequency of occurrence; otherwise, a new set of query inference data is reconstructed based on the underlying pricing algorithm. v The frequency; Specifically, the data analyst was interested in the number of US movies, so he constructed two queries on the Movie table (which contains movie information) to determine if the movies originated from the United States (region = 'USA'). Q1: SELECT * FROM MovieWHERE region = 'USA' AND Q 2: SELECT count(*) FROM Movie WHERE region = 'USA', to query the prices of data from data trading systems / sellers. p 1 and p 2. Under a pricing algorithm based on data usage, p 1= p 2 = 2, because there are 2 data entries that satisfy the query conditions; however, in pricing algorithms based on the amount of query information, assuming the following is used... Figure 4 Possible databases (each database) D i Only showed with Figure 2 Pricing is based on different data (from three possible databases). D 1. D 2 and D 4. Compared with the original database Q The query results for option 1 are different. Q The price of 1 is 3, and due to the possible database... D 1. Q The result of 2 is 1 (not 2), query Q The price of 2 (i.e.) p 2) is 1. Therefore, it can be seen that under both pricing algorithms, this query... Q The prices are all non-zero; this is to allow data consumers to accurately know the frequency of the target data simply by querying the price, avoiding the high cost and privacy risk of purchasing all results at once; at the same time, it greatly improves efficiency in the context of massive data by using strategies such as binary search.

[0028] S2: When the underlying pricing algorithm is based on data usage, construct a set of LIMIT queries. Q k And ask different elements k The corresponding price for the given value. p k Find it based on binary search k =0, ..., N The first one to satisfy p k equal p 1 element k , data v The frequency of occurrence of the element k is set to the value of the element k and the result is returned. Specifically, when a data analyst interested in the number of American films chooses to use a pricing algorithm based on data usage, the method of this invention constructs a set of queries. Qk :SELECT * FROM Movie WHERE region = 'USA'LIMIT k ,in k =0, ..., 4; Q k price p k and k Relationship such as Figure 5 As shown, when k When the price is less than or equal to 2, the query price will vary. k Monotonically increasing, when k When the value is greater than 2, the query price remains unchanged and is always 2. Therefore, the first condition satisfied by binary search is... p k equal p 1 element k The value is 2, from which the data can be deduced. v The frequency of occurrence of (i.e., American movies) is 2, and this result is returned. The method of this invention accurately determines the data frequency by observing the inflection point of the LIMIT query price without purchasing the entire set, which saves query costs and effectively reduces the risk of sensitive information exposure; at the same time, by using binary search, the number of price queries is reduced to logarithmic level, which significantly improves the inference efficiency in the context of massive data.

[0029] S3: When the underlying pricing algorithm is based on the amount of information in the query, construct a set of HAVING queries. Q i And ask for the corresponding price. p i Find it based on binary search i =0, ..., N The first one to satisfy p i Elements that are 0 i This element is the key threshold for inferring data frequency; Specifically, when a data analyst interested in the number of American films chooses to use a pricing algorithm based on the amount of query information, the method of this invention constructs a set of queries. Q i :SELECT count(*) FROM Movie WHERE region ='USA' HAVING count(*) i ,in i =0, ..., 4; Q i price p i and i Relationship such as Figure 6As shown, when i When the price is less than or equal to 2, the query price is always 1 because D 1. Always outputs results that differ from the original dataset, when k When the value is greater than 2, the query price drops to 0. Therefore, the element located based on binary search... i The value is 3; this allows data analysts to prepare for the next step of determining the true frequency based solely on the "breakpoints" in price changes, without having to purchase the complete query results, thus significantly saving costs and avoiding the exposure of sensitive information. S4: According to the element i Construct a new set of HAVING queries Q j And ask for the corresponding price. p j Calculate all that satisfy p j equal p 2 elements j The mean of the data is then rounded down to the nearest integer. v The frequency of occurrence is calculated and the results are returned.

[0030] Specifically, the method of the present invention uses the threshold obtained in S3. i Construct a set of queries Q j :SELECT count(*)FROM Movie WHERE region = 'USA' HAVING count(*) = j ,in j = 0, ..., 4; Q j price p j and j Relationship such as Figure 7 As shown, when j When the value is 0, since the query results for all possible databases and the original database are empty, the query price is 0. j When the price is less than or equal to 2, the query price is always 1 because D 1. Always outputs results that differ from the original dataset, when j When the value is greater than 2, the query price drops to 0. Therefore, all conditions that satisfy this condition will be met. p j equal p 2 elements j The values ​​are 1 and 2, and their mean is rounded down to 1. This value is then used as the frequency of occurrence of American movies and returned. At this point, the difference between the inferred data frequency and the true frequency (i.e., the precision loss) is 1. This is done by applying the values ​​of 1 and 2 to all movies that meet the condition... p j equalp 2 of j The average value is rounded down, which utilizes the information of the original query price to finely adjust the frequency estimate while ensuring that the error does not exceed 1. Without the need to purchase the complete results, the query cost and privacy exposure risk are further reduced, while the method maintains its efficiency through simple statistical calculations.

[0031] Example The transaction processing method of the present invention was implemented on an Ubuntu 18.04 system on a server with 192GB of memory and an Intel core of 2.80 GHz. The performance of the embodiment of the present invention in data frequency inference on the Walmart, Employment, TPC-H, and SSB datasets was tested.

[0032] The performance of the data frequency inference method proposed in this invention was tested and analyzed through simulation experiments, and the results are as follows: Figure 8 As shown, the present invention has a precision loss (Mean Absolute Error, MAE) of 0 under the UBP pricing algorithm based on data usage, and a precision loss of less than 1 under the IBP pricing algorithm based on query information. Furthermore, the present invention has a short running time and high computational efficiency.

[0033] Corresponding to the aforementioned embodiments of the data frequency inference method based on SQL query prices, this application also provides embodiments of a data frequency inference apparatus based on SQL query prices.

[0034] Figure 9 This is a block diagram illustrating a data frequency inference device based on SQL query prices, according to an exemplary embodiment. (Refer to...) Figure 9 The device includes: The data inference module is used to infer data based on the input data. v Constructing SQL queries Q 1 and Q 2. Query the prices corresponding to the two queries. p 1 and p 2. If the queried price p 1 or p If 2 is zero, directly purchase the query, set and return data based on the query results. v The frequency of occurrence; otherwise, a new set of queries is reconstructed based on the underlying pricing algorithm for inference. v The frequency; The first frequency inference module is used to construct a set of LIMIT queries when the underlying pricing algorithm is based on data usage. Q k And ask different k The corresponding price for the given value. pk Find it based on binary search k =0, ..., N The first one to satisfy p k equal p 1 element k , data v The frequency of occurrence is set to the value of the element and the result is returned. The threshold lookup module is used to construct a set of HAVING queries when the underlying pricing algorithm is based on the amount of query information. Q i And ask for the corresponding price. p i Find it based on binary search i =0, ..., N The first one to satisfy p i Elements that are 0 i This element is the key threshold for inferring data frequency; The second frequency inference module, based on the threshold element i Construct a new set of HAVING queries Q j And ask for the corresponding price. p j Calculate all that satisfy p j equal p 2 elements j The mean of the data is then rounded down to the nearest integer. v The frequency of occurrence is calculated and the results are returned.

[0035] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0036] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0037] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the data frequency inference method based on SQL query prices as described above.

[0038] Accordingly, this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the data frequency inference method based on SQL query prices as described above.

[0039] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for inferring data frequency based on SQL query prices, characterized in that, include: Based on the input data v Constructing SQL queries Q 1 and Q 2. Query the prices corresponding to the two queries. p 1 and p 2. If the queried price p 1 or p If 2 is zero, directly purchase the corresponding query. Q 1 or Q 2. Set and return data based on the query results. v The frequency of occurrence; otherwise, a new set of query inference data is reconstructed based on the underlying pricing algorithm. v The frequency; When the underlying pricing algorithm is based on data usage, then construct a set of LIMIT queries. Q k And ask different elements k The corresponding price for the given value. p k Find it based on binary search k =0, ..., N The first one to satisfy p k equal p 1 element k , data v The frequency of occurrence of the element k is set to the value of the element k and the result is returned. When the underlying pricing algorithm is based on the amount of information in the query, then construct a set of HAVING queries. Q i And ask for the corresponding price. p i Find it based on binary search i =0, ..., N The first one to satisfy p i Elements that are 0 i This element is the key threshold for inferring data frequency; According to the element i Construct a new set of HAVING queries Q j And ask for the corresponding price. p j Calculate all that satisfy p j equal p Elements of 2 j The mean of the data is then rounded down to the nearest integer. v The frequency of occurrence is calculated and the results are returned.

2. The method according to claim 1, characterized in that, The data based on the input v Constructing SQL queries Q 1 and Q 2, including: The constructed SQL query format is as follows Q 1: SELECT * FROM Table WHERE Attribute = v and Q 2: SELECTcount(*) FROM Table WHERE Attribute = v Where Table is the data v The relation table it belongs to, Attribute is data v The attribute it belongs to.

3. The method according to claim 1, characterized in that, Construct a set of LIMIT queries Q k And ask different elements k The corresponding price for the given value. p k Find it based on binary search k =0, ..., N The first one to satisfy p k equal p 1 element k ,include: Set the left and right endpoints of the binary search as follows: left = 0, right = N ,in N For data v The size of the table it belongs to, i.e., its maximum possible frequency of occurrence; like left Less than or equal to right The following process is repeated in a loop to find elements that meet the conditions: k Set as Inquiry Q k :SELECT * FROM Table WHERE Attribute = v LIMIT k price p k ,like p k Greater than or equal to p 1. Update the value of the right endpoint. right = k Otherwise, update the value of the left endpoint. left = k +1; when left Greater than right After the loop ends, set the data frequency to... left It takes a value and returns the result.

4. The method according to claim 1, characterized in that, Construct a set of HAVING queries Q i And ask for the corresponding price. p i Find it based on binary search i =0, ..., N The first one to satisfy p i Elements that are 0 i ,include: Set the left and right endpoints of the binary search as follows: left = 0, right = N ,in N For data v The size of the table it belongs to, i.e., its maximum possible frequency of occurrence; like left Less than or equal to right The following process is repeated in a loop to find elements that meet the conditions: i Set as Inquiry Q i :SELECT count(*) FROM Table WHERE Attribute = v HAVING count(*) i price p i ,like p i If the value is greater than or equal to 0, update the right endpoint value to 0. right = k Otherwise, update the value of the left endpoint. left = k +1; when left Greater than right When the loop ends, i Set as left The value of .

5. The method according to claim 1, characterized in that, Also includes: According to the element i Construct a new set of HAVING queries Q j : SELECT count(*) FROM Table WHEREAttribute = v HAVING count(*) = j ; in And ask for the corresponding query. Q j price p j Calculate all that satisfy p j equal p Elements of 2 j The mean of the data is then rounded down to the nearest integer. v The frequency of occurrence is calculated and the results are returned.

6. A data frequency inference device based on SQL query prices, characterized in that, include: The data inference module is used to infer data based on the input data. v Constructing SQL queries Q 1 and Q 2. Query the prices corresponding to the two queries. p 1 and p 2. If the queried price p 1 or p If 2 is zero, directly purchase the query, set and return data based on the query results. v The frequency of occurrence; otherwise, a new set of queries is reconstructed based on the underlying pricing algorithm for inference. v The frequency; The first frequency inference module is used to construct a set of LIMIT queries when the underlying pricing algorithm is based on data usage. Q k And ask different k The corresponding price for the given value. p k Find it based on binary search k =0, ..., N The first one to satisfy p k equal p 1 element k , data v The frequency of occurrence is set to the value of the element and the result is returned. The threshold lookup module is used to construct a set of HAVING queries when the underlying pricing algorithm is based on the amount of query information. Q i And ask for the corresponding price. p i Find it based on binary search i =0, ..., N The first one to satisfy p i Elements that are 0 i This element is the key threshold for inferring data frequency; The second frequency inference module, based on the threshold element i Construct a new set of HAVING queries Q j And ask for the corresponding price. p j Calculate all that satisfy p j equal p Elements of 2 j The mean of the data is then rounded down to the nearest integer. v The frequency of occurrence is calculated and the results are returned.

7. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

Images