Data query method, query engine, device, medium and program product

Abstract

The embodiment of the invention provides a data query method, a query engine, equipment, a medium and a program product, and the method comprises the steps that a query request is responded to determine to-be-queried data corresponding to the query request, and the to-be-queried data can be stored in a data organization mode of a file system and / or a database system. And then, according to the data organization mode of the to-be-queried data, determining statistical information of the to-be-queried data, and further determining respective execution costs of different query strategies according to the statistical information. And determining a target query strategy of the to-be-queried data according to the execution cost. And finally, realizing response to the query request according to the target query strategy. Therefore, in the query method, the statistical information used for determining the query strategy can be determined by adopting a mode corresponding to the data organization form. In other words, the method provides a universal statistical information acquisition process, and data with different data organization modes can be inquired by using the process.

Description

Technical Field

[0001] The present invention relates to the field of data query technology, and in particular to a data query method, query engine, device, medium and program product. Background Art

[0002] Big data is a collection of data characterized by large capacity, diverse types, fast access speed, and high application value. In practice, various storage systems have emerged for storing big data, such as data lakes, data warehouses, and integrated lake-warehouse systems. Furthermore, the data stored in these systems can come from different sources, meaning that data can be stored in the storage system in different data organization formats. Common data sources include file systems, database systems, and so on. In practice, queries on the data lake can be performed using statistical information from the data lake.

[0003] Based on the above description, how to query data in different data organization forms in the storage system becomes an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a data query method, query engine, device, medium, and program product for querying data in different organizational forms in a storage system.

[0005] In a first aspect, an embodiment of the present invention provides a data query method, comprising:

[0006] In response to a query request, determining data to be queried corresponding to the query request, the data to be queried including data stored in a data organization manner of a file system and / or a database system;

[0007] Determining statistical information of the data to be queried according to a data organization mode of the data to be queried;

[0008] Determine the execution costs of different query strategies based on the statistical information;

[0009] The query request is responded to according to a target query strategy determined according to the execution cost.

[0010] In a second aspect, an embodiment of the present invention provides a data query method, including:

[0011] In response to a query request, determining to-be-queried data corresponding to the query request, the to-be-queried data comprising files in a file directory stored in a first logical storage unit, the first logical storage unit being a logical storage unit at a first level in the file system;

[0012] Extracting the data to be queried to obtain sample data, wherein the sample data is stored in a second logical storage unit at a second level in the file system, where the second level is at least one level lower than the first level, and the first logical storage unit includes the second logical storage unit;

[0013] Determining the statistical information of the data to be queried based on the statistical information of the sample data;

[0014] Determining the execution costs of different query strategies based on the statistical information of the data to be queried;

[0015] The query request is responded to according to a target query strategy determined according to the execution cost.

[0016] In a third aspect, an embodiment of the present invention provides a data query method, including:

[0017] In response to a query request, determining data to be queried corresponding to the query request, the data to be queried including data stored in a data organization manner of a database system;

[0018] Receiving statistical information of the data to be queried fed back by the database system;

[0019] Determine the execution costs of different query strategies based on the statistical information;

[0020] The query request is responded to according to a target query strategy determined according to the execution cost.

[0021] In a fourth aspect, an embodiment of the present invention provides a query engine, comprising: a processing component and a query optimization component;

[0022] The processing component is configured to, in response to a query request, determine data to be queried corresponding to the query request, the data to be queried including data stored in a data organization manner of a file system and / or a database system; determine statistical information of the data to be queried based on the data organization manner of the data to be queried; and respond to the query request using a target query strategy;

[0023] The query optimization component is used to determine the execution costs of different query strategies according to the statistical information; and determine the target query strategy according to the execution costs.

[0024] In a fifth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory configured to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the data query method described in any one of aspects 1 to 4. The electronic device may also include a communication interface configured to communicate with other devices or communication systems.

[0025] In the sixth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a computing system of an electronic device, the computing system can at least implement the data query method described in any one of the first to fourth aspects.

[0026] In a seventh aspect, an embodiment of the present invention provides a computer program product. The computer program product includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the processor is enabled to implement the data query method as described in any one of the first to fourth aspects.

[0027] In the data query method provided by an embodiment of the present invention, in response to a query request, a query device can determine the corresponding query data. This query data may include data stored in a file system and / or database system. The query device then determines statistical information about the query data based on its data organization. Furthermore, based on this statistical information, it determines the execution costs of different query strategies. Finally, based on the execution costs, it determines a target query strategy for the query data. Finally, the query request is responded to according to this target query strategy.

[0028] As can be seen, in the above query method, the statistical information used to determine the query strategy can be determined in a manner corresponding to the data organization format. In other words, the above method provides a universal statistical information acquisition process that can be used to collect statistical information on data with different data organization formats in the storage system, thereby ultimately realizing data query. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 A flowchart of a data query method provided by an embodiment of the present invention;

[0031] Figure 2 A flowchart of a method for determining statistical information provided by an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of a statistical information determination process provided by an embodiment of the present invention;

[0033] Figure 4 A flowchart of another method for determining statistical information provided by an embodiment of the present invention;

[0034] Figure 5 A schematic diagram of a stratified sampling process provided by an embodiment of the present invention;

[0035] Figure 6 A flowchart of another data query method provided by an embodiment of the present invention;

[0036] Figure 7 A flowchart of another data query method provided by an embodiment of the present invention;

[0037] Figure 8 A structural diagram of a query engine provided by an embodiment of the present invention;

[0038] Figure 9 A schematic structural diagram of a data query device provided by an embodiment of the present invention;

[0039] Figure 10 A schematic structural diagram of another data query device provided by an embodiment of the present invention;

[0040] Figure 11 A structural diagram of another data query device provided by an embodiment of the present invention

[0041] Figure 12 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0043] The terms used in the embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The singular forms "a," "the," and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. "A plurality" generally includes at least two, but does not exclude the inclusion of at least one.

[0044] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0045] As used herein, the words “if” and “if” may be interpreted as “at the time of” or “when” or “in response to determining” or “in response to identifying,” depending on the context. Similarly, the phrases “if it is determined” or “if (stated condition or event) is identified” may be interpreted as “when it is determined” or “in response to determining” or “when identifying (stated condition or event)” or “in response to identifying (stated condition or event),” depending on the context.

[0046] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0047] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or system. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or system comprising the element.

[0048] Before describing the methods and engines provided by the following embodiments of the present invention, the following concepts may be explained:

[0049] Data lake: A storage system for large-scale, diverse data. Data on the lake can belong to different data sources, meaning that data from different data sources can be stored on the lake without changing their data organization. For example, a data source could be a file system or a database system. Data from a file system can be stored on the lake using the file system's data organization, and data from a database system can be stored on the lake using the database system's data organization.

[0050] However, it should be noted that data from different data sources is actually stored in the data lake through mapping. This means that data in a remote file system can be mapped to the data lake, forming a local file system within the data lake. Database systems operate similarly: data in a remote database system can be mapped to the data lake, forming a local database system within the data lake. Data in local or remote file systems can be stored using the Object Storage Service (OSS).

[0051] In addition, whether it is a remote or local file system, the file system includes multiple levels of logical storage units, and data can be stored in these different levels of logical storage units. The multi-level logical storage units of the file system may include: file directory - folder under the file directory - file - file block - file page, and the hierarchy of the above logical storage units is descending.

[0052] Regardless of whether it is a remote or local database system, the multi-level logical storage unit of the database system may include: table - data partition - data block - data page - record, and the levels of the above logical storage units are successively lowered.

[0053] Statistical information of lake data: This statistical result is obtained by taking the data in any logical storage unit at any level as a whole and performing statistics on it in units of fields. This statistical result is the statistical information of the logical storage unit. Common statistical information may include at least one of the following: number of rows, data volume, the maximum value of the data in a field, the null value ratio (Null Ratio) of a field, the number of different values ​​(NDV) in a field, which is also the field cardinality of the field, etc.

[0054] Query engine: A component used to query data in the data lake. More specifically, the query engine can collect statistics on the data in the data lake and complete queries based on the statistical information.

[0055] The following describes some embodiments of the present invention in detail with reference to the accompanying drawings. The following embodiments and features thereof may be combined with one another unless they conflict with each other. Furthermore, the sequence of steps in the following method embodiments is provided for illustrative purposes only and is not intended to be a strict limitation.

[0056] Figure 1A flowchart of a data query method provided by an embodiment of the present invention. The method provided by an embodiment of the present invention can be executed by a query device. Optionally, the query device may include a software device or hardware device with query capabilities, and the software device may specifically support different data architectures, such as databases, data warehouses, lake warehouses, etc. Figure 1 As shown, the method may include the following steps:

[0057] S101 , in response to a query request, determining data to be queried corresponding to the query request, where the data to be queried includes data stored in a data organization manner of a file system and / or a database system.

[0058] S102: Determine statistical information of the data to be queried according to the data organization mode of the data to be queried.

[0059] A user can generate a data query request. In response to this request, the query device can determine the data to be queried that corresponds to the query request. The query device can then further determine a statistical method based on the data organization method of the data to be queried, thereby generating statistical information about the data to be queried. This statistical information is then used to determine a query strategy corresponding to the query request. Finally, the query device can execute this query strategy to respond to the query request.

[0060] Optionally, the data to be queried involved in the query request may be stored in a storage system that may store data from different sources. The sources of the data to be queried may also be the same or different. Specifically, in one case, all of the data to be queried may be stored using the data organization of a file system. In another case, all of the data to be queried may be stored using the data organization of a database system. In yet another case, a portion of the data to be queried may be stored using the data organization of a file system, while another portion may be stored using the data organization of a database system.

[0061] Optionally, when the data to be queried is stored in a data organization manner of a file system, the query device may determine the statistical information of the data to be queried by sampling the data to be queried. Specific processes may refer to the description in the following related embodiments.

[0062] Optionally, when the data to be queried is stored in the data organization form of a database system, the query device can determine the statistical information of the data to be queried by interacting with the remote database system. The specific process can also be found in the description of the following related embodiments.

[0063] Optionally, the statistical information of the data to be queried may include at least one of the above-mentioned total number of rows, data volume, the maximum value in a column, the null value rate of a column, the field cardinality of a column, and the like.

[0064] S103: Determine the execution costs of different query strategies based on the statistical information.

[0065] S104: Respond to the query request according to the target query strategy determined based on the execution cost. Based on the statistical information of the query data obtained in the above steps, the query device can determine the execution costs of different query strategies based on the statistical information and determine the target query strategy based on the execution costs. Ultimately, the query device can respond to the query request by executing the query strategy.

[0066] Optionally, the query strategy may include query strategies for table connection order and data propagation mode. The table connection order may be determined by the join operator, and the data propagation mode may be determined by the exchange operator. The query device may use different operators to determine query strategies including different table connection orders and data propagation modes, that is, different query strategies. Then, the execution cost of the operator may be estimated based on the statistical information of the data to be queried, thereby obtaining the execution cost of each of the different query strategies. Next, the query device may determine the target query strategy based on the execution cost, and respond to the query request according to the target query strategy. The target query strategy may have the minimum execution cost.

[0067] The query device responds to a query request by joining the multiple tables involved in the query request according to the table join order specified in the target query strategy to obtain candidate data. Then, using the query conditions specified in the query request, the device retrieves the target data that meets the query conditions from the candidate data. Finally, the device uses the data propagation method specified in the target query strategy to deliver the target data as the query response to the user, effectively completing the query request response.

[0068] In this embodiment, in response to a query request, the query device can determine the query data corresponding to the query request. This query data can be stored in a data organization manner of a file system and / or database system. Then, based on the data organization manner of the query data, statistical information of the query data can be determined. The execution costs of different query strategies can be further determined based on this statistical information. A target query strategy for the query data can then be determined based on the execution costs. Finally, the query request is responded to according to this target query strategy.

[0069] As can be seen, in the above query method, the statistical information used to determine the query strategy can be determined in a manner corresponding to the data organization format. In other words, the above method provides a universal statistical information acquisition process that can be used to determine statistical information for data with different data organization formats, thereby ultimately achieving data query.

[0070] Optionally, the storage system storing the data to be queried in the above and following embodiments of the present invention may specifically be a data lake, that is, the data to be queried may be stored in the data lake in the data organization manner of a file system and / or a database system.

[0071] In this data lake scenario, Figure 1 In the illustrated embodiment, step S102 refers to the data to be queried, when all the data to be queried is stored in the data lake using the file system's data organization, that is, it is stored in the data lake's local file system. When all the data to be queried is stored in the data lake using the database system's data organization, that is, it is stored in the data lake's local database system. Some of the data to be queried is stored in the data lake using the file system's data organization, while the rest is stored in the data lake using the database system's data organization.

[0072] Optionally, when the data to be queried is stored in a local file system on the lake using a file system data organization, the query device can directly access data within logical storage units at different levels within the local file system. Specifically, the query device can access data in the local file system at various granularities, such as file directories, folders, and files. The query device can then determine statistical information about the data by sampling it. For more information, see the description of the relevant embodiments below.

[0073] Optionally, when the data to be queried is stored in a local database system on the lake in the form of a database system's data organization, the query device can access data stored in the highest-level logical storage unit in the local database system. Specifically, the query device can access data in the local database system on a table-by-table basis. The query device can then determine statistical information about the data to be queried by interacting with the remote database system. The specific process can also be found in the description of the related embodiments below.

[0074] It can be seen that when the data to be queried is specifically lake data, the method provided in the above and following embodiments of the present invention is actually a general statistical information acquisition process for lake data. This process can be used to determine statistical information for data with different data organization methods on the data lake, thereby ultimately realizing the query of data on the data lake.

[0075] Alternatively, the process of determining statistical information in the above-described and below-described embodiments of the present invention may be specifically performed by a processing component included in a query engine in a query device, where the input of the component is the query request and the output is the statistical information. The process of determining a query strategy may be specifically performed by a query optimization component included in the query engine, where the input of the component is the statistical information and the output is the query strategy.

[0076] In practice, a query request may include a query condition. Optionally, the query condition may describe the storage location of the data involved in the query request, or may describe the filtering condition of the data, i.e., the filtering predicate. The data to be queried determined by the query device using the query condition may include two parts: the data involved in the query request and the data that meets the filtering condition. For the sake of clarity in the subsequent description, the data involved in the query request may be referred to as original data, and the data in the original data that meets the filtering condition may be referred to as filtered data. The original data in the data to be queried may be stored in a logical storage unit, and this logical storage unit may be a logical storage unit of any level provided by the data organization method corresponding to the original data. Optionally, it is usually a logical storage unit of the highest level.

[0077] In a data lake scenario, the logical storage unit storing raw data can be the data lake's local file system or the highest-level logical storage unit in a local database system. Specifically, raw data can be all files in at least one folder within a directory in the local file system, while filtered data can be at least one row within all those files. Raw data can also be a table in a local database system, while filtered data can be at least one record within that table.

[0078] Optionally, the raw data in the data to be queried can be stored in the same or different logical storage units. Furthermore, the connection relationship between the different logical storage units storing the raw data can be included in the query conditions. The query device can then retrieve the raw data based on the connection relationship in the query request. The above connection relationship can also be understood as a table connection relationship. Optionally, the different logical storage units involved in the connection relationship can belong to the same or different data organization methods.

[0079] Furthermore, for the two parts of data included in the query data, the statistical information of the query data may specifically include statistical information of the original data and the filtered data. For clarity of subsequent descriptions, the statistical information of the two parts of data may be referred to as first statistical information and second statistical information, respectively. Optionally, the manner in which the first statistical information and the second statistical information are determined is related to the data organization method.

[0080] The following example illustrates the relationship between the data to be queried, the query conditions, and the statistical information:

[0081] Assume that data is stored in a data lake in the form of a database system. The data lake can store data with the fields "user name" and "height": [User 1, height 150cm], [User 2, height 170cm], [User 3, height 160cm], [User 4, height 150cm], [User 5, height 180cm], [User 6, height 175cm], [User 7, height 180cm], [User 8, height 185cm], and [User 9, height 160cm].

[0082] The above data can be stored in two tables respectively. For example, Table 1 may contain the following data: [User 1, height 150cm], [User 2, height 170cm], [User 3, height 160cm], [User 4, height 150cm], [User 5, height 180cm], [User 6, height 150cm]. Table 2 may contain the following data: [User 7, height 180cm], [User 8, height 185cm], [User 9, height 160cm].

[0083] Suppose the query request is to find users in Table 2 and Table 1 who are taller than 160cm. The original data involved in the query is the entire data in Table 1 and Table 2, and the filtering condition is "users taller than 160cm." The data to be queried is [User 2, height 170cm], [User 3, height 160cm], [User 5, height 180cm] in Table 1, as well as the entire data in Table 2.

[0084] The query device can perform statistics on the data in the "Height" field. The obtained first statistical information may include: the maximum value in Table 1 is 180 cm, the minimum value is 150 cm, the total number of rows is 6, and the field cardinality is 4. The maximum value in Table 2 is 185 cm, the minimum value is 160 cm, the total number of rows is 3, and the field cardinality is 3. The second statistical information may include: the maximum value is 185 cm, the minimum value is 160 cm, the total number of rows is 6, and the field cardinality is 4.

[0085] As can be seen from the above description, the statistical information of the data to be queried is the statistical information of both the original data and the filtered data. The specific method for determining the statistical information is related to the organization of the data. Optionally, the query device can obtain metadata of the data to be queried locally and use this metadata to determine the data organization of the data to be queried. Optionally, this metadata can specifically include the Data Definition Language (DDL) corresponding to the data to be queried.

[0086] The following will further describe in detail the respective processes of determining the first statistical information and the second statistical information when the original data and the filtered data are in different organizational forms.

[0087] When the original data is stored in the local file system of the data lake in the form of file system data organization, the query device can determine the first statistical information by sampling. For the specific sampling process, please refer to Figure 2 The embodiment shown. Figure 2 This is a flow chart of a method for determining statistical information provided by an embodiment of the present invention. This method can also be executed by a query device. Figure 2 As shown, the method may include the following steps:

[0088] S201 , determining the total amount of original data, where the original data includes files in a file directory stored in a first logical storage unit, and the first logical storage unit is a logical storage unit at a first level in a file system.

[0089] The query device can first determine the total amount of raw data. Since the raw data is stored in the data lake in the file system's data organization format, the raw data may include multiple files under a file directory stored in a first logical storage unit. The first logical storage unit is a logical storage unit at the first level of the local file system, and the first logical storage unit may store a file directory.

[0090] Regarding the method for obtaining the total amount of data, one optional method is that the query device can directly obtain the total amount of raw data using the interface provided by the remote file system. Another optional method is that the query device can traverse the raw data in the local file system to obtain the total amount of raw data.

[0091] To improve the efficiency of obtaining the total amount of data, another alternative method is to extract sample files from the original data according to the sampling ratio. The total amount of original data can then be determined based on the data volume of the sample files and the sampling ratio. Here, total data volume = sample file data volume / sampling ratio. In practice, this method is often suitable for scenarios with large amounts of original data.

[0092] To distinguish from the following steps, the sampling ratio used when obtaining the total amount of data can be referred to as the second ratio, and the sample file sampled from the original data using the second ratio can be referred to as the second sample file. Furthermore, both the second ratio and the first ratio in step S202 can be pre-set, and there is no strict relationship between the two.

[0093] S202: Extract a first sample file from the original data according to a first ratio, and store the sample file in a first logical storage unit.

[0094] S203: Determine first statistical information of the original data according to the statistical information of the first sample file, the data volume of the first sample file, and the total data volume.

[0095] Next, the query device can sample the original data at a first ratio to extract a first sample file. This refers to the sampling granularity of files in step S202. First statistical information about the original data is then determined based on the data volume of the first sample file, the statistical information about the first sample file, and the total amount of original data. The first statistical information about the original data is calculated as follows: (total amount of original data / total amount of first sample file)*statistical information about the first sample file.

[0096] The method for obtaining the data volume of the first sample file may optionally be the same as the method for obtaining the total data volume of the original data. For details, please refer to the relevant description in step S201 and will not be repeated here.

[0097] Regarding the method for obtaining statistical information about the first sample file, one optional method is for the query device to traverse and read the first sample file to obtain the statistical information about the first sample file. If the first sample file is stored in a format with metadata in the local file system, another optional method is for the query device to directly read the statistical information contained in the metadata. Optionally, the storage format of the first sample file can be columnar storage, such as OptimizedRow Columnar (ORC) or Parquet.

[0098] Optionally, in this embodiment, the statistical information of the first sample file and the first statistical information of the original data may be the same item, such as the number of rows or the null value ratio.

[0099] This embodiment provides a sampling and statistical method suitable for data lake scenarios. Specifically, a query device first samples raw data represented by multiple files at a preset ratio, using the file as the granularity, and then amplifies the statistical information of the first sample text according to the ratio to estimate the first statistical information of the raw data. Using this method, sampling can reduce the amount of data read by the query device when determining the first statistical information, thereby increasing the speed of determining the first statistical information, improving the speed of determining the query strategy, and ultimately improving the response speed of query requests.

[0100] in addition, Figure 2 The illustrated embodiment does not limit the number of first logical storage units storing original data. Optionally, the number of the first logical storage unit may be at least one.

[0101] When the number of the first logical storage unit is one, the original data is a plurality of files under a file directory, and the file directory can be stored in the first logical storage unit. Figure 2 In the manner of the illustrated embodiment, a set of statistical information corresponding to the files in the file directory is determined, and the statistical information is the first statistical information of the original data.

[0102] When there are multiple first logical storage units, the original data is multiple files under multiple file directories, and the multiple file directories can be stored in corresponding first logical storage units. Figure 2 In the manner of the illustrated embodiment, a set of statistical information corresponding to the files in each file directory is determined respectively, and the multiple sets of statistical information can be collectively referred to as the first statistical information of the original data.

[0103] When the original data is stored in the local database system of the data lake in the form of data organization of the database system, the query device cannot determine the statistical information of the data in the local database system. In this case, the query device can interact with the remote database system to obtain the first statistical information of the original data fed back by the remote database system. The first statistical information can be obtained by the query optimization component in the remote database and fed back to the query device.

[0104] Regarding the method for obtaining the first statistical information, one optional method is that after the data is stored locally, the remote database system can collect statistical information about the data and save the statistical information as metadata. The query device then directly uses the metadata access interface provided by the remote database to read the first statistical information of the original data. Optionally, the remote database providing the interface can be a relational database, such as MySQL, a database provided by a relational database service (RDS), etc.

[0105] Alternatively, the query device can send a statistical information query request to a remote database system. After the remote database system responds to the request, it can provide the query device with the first statistical information of the original data. Alternatively, the remote database system that can respond to the statistical information query request can be a database system that supports the Structured Query Language (SQL) protocol, such as MaxCompute. The optimized query component in the remote database system can respond to the statistical information query request.

[0106] In addition, the interaction method provided in this embodiment can also be combined with Figure 3 understand.

[0107] It needs to be explained that, as Figure 1 As can be seen from the description in the illustrated embodiment, statistical information can be input into the query optimization component included in the query engine in the query device. Therefore, the interaction between the query device and the remote database system in this embodiment is actually the interaction between the optimized query components included in each of the two.

[0108] In this embodiment, without intruding into the remote database system architecture, the query device can obtain the first statistical information of the original data through the interaction between the query optimization components of the query device and the remote database system.

[0109] When the filtered data is stored in the local database system of the data lake in the data organization form of the database system, the query device can also interact with the remote database system to obtain the second statistical information of the filtered data fed back by the remote database system.

[0110] In addition, the interaction method provided in this embodiment can also be combined with Figure 3 understand.

[0111] In this embodiment, without intruding into the remote database system architecture, the query device can obtain the second statistical information of the filtered data through the interaction between the query optimization components of the query device and the remote database system.

[0112] When the filtered data is stored in the local file system of the data lake in the form of file system data organization, the query device can also determine the second statistical information through sampling. The second statistical information is the same item as the first statistical information, such as the row or null value rate. For the specific sampling process, please refer to Figure 4 The embodiment shown. And compared to Figure 2 The sampling in the illustrated embodiment is based on the file granularity, while the sampling granularity in this embodiment is smaller and richer.

[0113] but Figure 4 This is a flow chart of another method for determining statistical information provided by an embodiment of the present invention. This method can also be executed by a query device. Figure 4 As shown, the method may include the following steps:

[0114] S301, extracting sample data from original data, the sample data being stored in a second logical storage unit at a second level in the file system, the second logical storage unit being included in the first logical storage unit, the second level including at least one level lower than the first level.

[0115] The query device can extract sample data from the original data. Figure 2The sample files sampled in the illustrated embodiment are different. The sample data extracted in this embodiment has a smaller granularity. The sample data can be specifically stored in a second logical storage unit at the second level in the file system. The second logical storage unit is included in the first logical storage unit, and the second level can be at least one level lower than the first level.

[0116] For example, the second level in the file system may correspond to a granularity smaller than a file, such as a file block and / or a file page. Then step S202 actually samples the original data in units of files to sample a first proportion of files from the original data containing multiple files. Step S301 actually samples in units of file blocks and / or file pages to obtain sample data.

[0117] Regarding the specific sampling method for sample data, optionally, assuming that the second level corresponds to file pages, and a file page includes at least one row, the query device may first randomly select at least one file from the raw data containing multiple files, then select at least one file block from the at least one file, and then further randomly select a certain number of file pages from the at least one file block. The selected file pages are the sample data extracted from the raw data.

[0118] The number of sample data extracted can be optionally pre-set or determined based on the first statistical information of the original data. The process of determining the first statistical information can be found in the description of the above-mentioned related embodiments and will not be repeated here.

[0119] Regarding the process of determining the amount of sample data based on the first statistical information, specifically, the query device may first determine reference statistical information based on the third ratio and the first statistical information of the original data. Then, based on the reference statistical information and the statistical information of the data stored in each of the plurality of second logical storage units, a third logical storage unit is determined from the plurality of second logical storage units, and the data in the third logical storage unit is ultimately determined as the sample data. The statistical information of the data in the third logical storage unit satisfies the reference statistical information.

[0120] For example, assuming that the statistical information is the number of rows, and the second logical storage unit stores file pages. The query device can then determine the third logical storage unit based on the preset number of rows (i.e., the reference statistical information) and the number of rows of file pages stored in different second logical storage units (i.e., the statistical information of the data stored in the second logical storage unit). The number of rows of file pages stored in the third logical storage unit is greater than or equal to the preset number of rows. Since the sample data is determined in units of file pages, the number of rows of the file page serving as the sample data does not necessarily equal the preset number of rows, and may also be greater than the preset number of rows.

[0121] In the above process of determining the number of sample data, the statistical information of the data stored in the second logical storage unit is used. Therefore, the statistical information of the data stored in any second logical storage unit can be optionally estimated in the following way: based on the data volume of the data stored in any logical storage unit, the total data volume of the original data and the first statistical information of the original data, the statistical information of the data stored in any logical storage unit is determined.

[0122] Statistical information of data stored in any logical storage unit=(first statistical information*data volume of data stored in any logical storage unit) / total data volume of original data.

[0123] Based on the description of step S301 above, it can be seen that compared with Figure 2 In the embodiment shown, the original data is sampled in a more hierarchical manner. The above multi-level sampling process can also be combined with Figure 5 Understand. Among them, Figure 5 The process of dividing the file pages into file rows at a finer granularity is not shown.

[0124] S302: Determine second statistical information of the filtered data based on the statistical information of the original data and the sample data, and the statistical information of the data in the sample data that meets the filtering condition.

[0125] Afterwards, the query device can further determine the data in the sample data that meets the query conditions and the statistical information of the data. Ultimately, the second statistical information of the filtered data = the statistical information of the data in the sample data that meets the filter conditions * (the first statistical information of the original data / the statistical information of the sample data).

[0126] The statistical information of different data mentioned in this embodiment may be the same item as the first statistical information, such as the number of rows or the null value ratio.

[0127] and Figure 2 Similar to the embodiment shown, this embodiment also provides a sampling and statistical method suitable for data lake scenarios, that is, the query device first performs multi-level and multi-granular sampling on the original data represented by multiple files to obtain sample data. Then, the statistical information of the data that meets the query conditions in the sample data is analyzed to estimate the second statistical information of the filtered data. Similarly, when using the above method, the amount of data read by the query device in the process of determining the second statistical information can be reduced by sampling, thereby improving the determination speed of the second statistical information, improving the determination speed of the query strategy, and ultimately improving the response speed of the query request.

[0128] As can be seen from the above embodiments, the data to be queried can include data from one or more sources. For data to be queried originating from a local file system, the query device can determine data statistics using different levels of sampling. For data to be queried originating from a local database system, the query device can determine data statistics by interacting with a query optimization component in a remote database system.

[0129] The following will take a specific query request as an example to illustrate the specific process of implementing data query using the methods provided by the above embodiments of the present invention.

[0130] Suppose a query request is: SELECT TABLE_A.col_a FROM TABLE_A,TABLE_B WHERETABLE_A.col_a=TABLE_B.col_a and TABLE_A.col_b=1.

[0131] In the query request, "TABLE_A.col_a=TABLE_B.col_a" represents the table join relationship described by the query condition, indicating that the query involves Table A and Table B. "TABLE_A.col_b=1" represents the filter condition described by the query condition. The query request intends to retrieve the data in Table A that meets the filter condition "TABLE_A.col_b=1" as well as all the data in Table B.

[0132] Assume that Table A is stored in the data lake using the file system's data organization format, and Table B is stored in the data lake using the database system's data organization format. For the query request above, the data in Tables A and B can be collectively referred to as the raw data mentioned in the above embodiments. The data in Table A that meets the filter condition "TABLE_A.col_b=1" is the filtered data mentioned in the above embodiments.

[0133] The statistical information required to respond to the query request may include the number of rows in table A, which is the first statistical information in the above embodiments, the number of rows in table B, which is the first statistical information in the above embodiments, and the number of rows of data in table A that meet the filtering conditions, which is the second statistical information in the above embodiments.

[0134] Based on the above premise, the query device can be Figure 2 The sampling method shown in the figure determines the number of rows in table A, which can be obtained by Figure 4 The stratified sampling method shown determines the number of rows in Table A that meet the filter conditions. The query optimization components can also interact to determine the number of rows in Table B. Ultimately, the query request can determine a query strategy based on the statistical information obtained above and respond to the query request according to this query strategy.

[0135] In addition, for the contents not described in detail in this embodiment and the technical effects that can be achieved, please refer to the relevant descriptions in the above embodiments and will not be repeated here.

[0136] Figure 6 This is a flowchart of another data query method provided by an embodiment of the present invention. When the data to be queried is stored in the data lake in the data organization mode of the file system, the following method can be used to implement the response to the query request. Figure 6 As shown, the method may include the following steps:

[0137] S401: In response to a query request, determine to-be-queried data corresponding to the query request, where the to-be-queried data includes files in a file directory stored in a first logical storage unit, where the first logical storage unit is a logical storage unit at a first level in a file system.

[0138] S402: Extract the query data to obtain sample data. The sample data is stored in a second logical storage unit at a second level in the file system. The second level is at least one level lower than the first level. The first logical storage unit includes the second logical storage unit.

[0139] S403: Determine the statistical information of the data to be queried based on the statistical information of the sample data.

[0140] S404, determining the execution costs of different query strategies based on statistical information of the data to be queried;

[0141] S405: Respond to the query request according to the target query strategy determined according to the execution cost.

[0142] The query device determines the data to be queried corresponding to the query request in response to the query request. The data to be queried may include files in a file directory stored in a first logical storage unit, and the first logical storage unit belongs to a logical storage unit at the first level in the file system. Then, the query device can extract the data to be queried to obtain sample data. The sample data is stored in a second logical storage unit at the second level in the file system, and the second level is at least one level of the first level, and the first logical storage unit includes the second logical storage unit.

[0143] The sample data extraction process and the method for determining the statistical information of the sample data can be found in Figure 4 Related description in the illustrated embodiment.

[0144] Finally, the query device can determine the execution cost of different query strategies based on the statistical information of the data to be queried, and determine the target query strategy based on the execution cost, and finally execute this target query strategy to respond to the query request. Figure 1 The relevant descriptions in the illustrated embodiment will not be repeated here. Optionally, the data to be queried mentioned in this embodiment can also be stored in different storage systems, such as in a data lake.

[0145] In this embodiment, the query device can determine statistical information about the data to be queried, stored in the file system's data organization format, through stratified sampling at different granularities. Sampling can reduce the amount of data read by the query device during the statistical information determination process, thereby increasing the speed of determining the second statistical information, increasing the speed of determining the query strategy, and ultimately increasing the speed of query request responses.

[0146] In addition, for the contents not described in detail in this embodiment and the technical effects that can be achieved, please refer to the relevant descriptions in the above embodiments and will not be repeated here.

[0147] Figure 7 A flowchart of another data query method provided by an embodiment of the present invention. When the data to be queried is stored in the local database system of the data lake in the data organization mode of the database system, the query device can use the following method to respond to the query request. Figure 7 As shown, the method may include the following steps:

[0148] S501 : In response to a query request, determine the to-be-queried data corresponding to the query request, where the to-be-queried data includes data stored in a data organization manner of a database system.

[0149] S502: Receive statistical information of the data to be queried fed back by the database system.

[0150] S503: Determine the execution costs of different query strategies based on the statistical information.

[0151] S504: Respond to the query request according to the target query strategy determined according to the execution cost.

[0152] In response to a query request, the query device determines the corresponding query data. The query data includes data stored in a database system's data organization format. Optionally, the query data may include data in a local database system within the data lake. The query device then receives statistical information about the query data from the remote database system and uses this statistical information to respond to the query request.

[0153] In this embodiment, without intruding on the remote database system architecture, the query device can obtain statistical information about the data to be queried through interaction between itself and the query optimization components of the remote database system. Furthermore, for details not described in detail in this embodiment and the technical effects achieved, please refer to the relevant descriptions of the above embodiments and will not be repeated here.

[0154] The above has described the response process of the query device to the query request from the perspective of method. Figure 1 As can be seen from the description of the illustrated embodiments, the above methods can more accurately be executed by a query engine deployed on a query device. The query device can be a device independent of the data lake. The following also describes the above data query process from the perspective of the query engine. Figure 8 A structural diagram of a query engine provided by an embodiment of the present invention. Figure 8 As shown, the query engine may include: a processing component and a query optimization component.

[0155] The processing component can determine the data to be queried associated with the query request in response to the query request. The data to be queried includes data stored in a data organization manner of a file system and / or a database system. Then, the processing component can determine statistical information of the data to be queried based on the data organization manner of the data to be queried. The statistical information can be sent to the query optimization component. Then, the query optimization component can determine the execution cost of each of the different query strategies based on the above statistical information, and determine the target query strategy based on the execution cost. Ultimately, the processing component can achieve a response to the query request by executing this target query strategy.

[0156] In addition, for the contents not described in detail in this embodiment, reference can be made to the relevant descriptions in the above embodiments, and no further details will be given here.

[0157] In this embodiment, the processing component in the query engine responds to the query request to determine the data to be queried corresponding to the query request. The data to be queried can be stored in the data organization method of the file system and / or database system. Then, the processing component determines the statistical information of the data to be queried based on the data organization method of the data to be queried. The query optimization component in the query engine can further determine the execution cost of each of the different query strategies based on this statistical information, and determine the target query strategy for the data to be queried based on the execution cost. Finally, the processing component can implement the response to the query request according to this target query strategy.

[0158] As can be seen from the above process, the statistical information used by the processing components to determine the query strategy can be determined in a manner that corresponds to the data organization format. In other words, the above method provides a universal statistical information acquisition process that the query engine can use to determine statistical information for data with different data organization formats, ultimately enabling data query.

[0159] Alternatively, the data to be queried mentioned in this embodiment can be stored in any storage system, such as a data lake. The query engine's operation process is thus a general statistical information acquisition process for lake data.

[0160] Optionally, according to the descriptions in the above embodiments, the data to be queried corresponding to the query request may include the original data and the filtered data in the original data that meets the query conditions. The processing component in the query engine can determine the statistical information by sampling or interactive means according to the data organization methods of the original data and the filtered data. The specific process of statistical information can be found in the above Figures 2 to 5 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0161] The data query device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these data query devices can be constructed using commercially available hardware components and configured according to the steps taught in this solution.

[0162] Figure 9 A schematic diagram of a data query device according to an embodiment of the present invention is shown in FIG. Figure 9 As shown, the device may include:

[0163] The first data determination module 11 is configured to determine, in response to a query request, data to be queried corresponding to the query request, where the data to be queried includes data stored in a data organization manner of a file system and / or a database system.

[0164] The first information determination module 12 is configured to determine statistical information of the data to be queried according to a data organization mode of the data to be queried.

[0165] The first response module 13 is configured to determine the execution costs of different query strategies according to the statistical information, and respond to the query request according to the target query strategy determined according to the execution costs.

[0166] Optionally, the data to be queried is stored in a data lake.

[0167] The first data determination module 11 is configured to obtain original data according to the query condition carried in the query request, wherein the original data is stored in a logical storage unit provided by a data organization method corresponding to the original data;

[0168] Determining filtered data that meets the query condition in the original data;

[0169] The data to be queried including the original data and the filtered data is determined.

[0170] Optionally, the first information determining module 12 is configured to determine first statistical information of the original data according to a data organization mode of the original data;

[0171] Second statistical information of the filtered data is determined according to a data organization mode of the filtered data.

[0172] Optionally, the first information determining module 12 is configured to determine the total amount of the original data, where the original data includes files in a file directory stored in a first logical storage unit, and the first logical storage unit belongs to a logical storage unit at a first level in the file system;

[0173] extracting a first sample file from the original data according to a first ratio, and storing the sample file in the first logical storage unit;

[0174] First statistical information of the original data is determined according to the statistical information of the first sample file, the data volume of the first sample file, and the total data volume.

[0175] Optionally, the first information determining module 12 is configured to extract a second sample file from the original data according to a second ratio;

[0176] The total amount of the original data is determined according to the data amount of the second sample file and the second ratio.

[0177] Optionally, the first information determination module 12 is configured to extract sample data from the original data, the sample data being stored in a second logical storage unit at a second level in the file system, the second logical storage unit being included in the first logical storage unit, the second level including at least one level lower than the first level;

[0178] Second statistical information of the filtered data is determined based on the statistical information of each of the original data and the sample data, and the statistical information of the data in the sample data that meets the query condition.

[0179] Optionally, the first information determining module 12 is configured to determine reference statistical information according to the third ratio and the first statistical information of the original data;

[0180] determining a third logical storage unit in the second logical storage unit according to the reference statistical information and statistical information of the data stored in each of the second logical storage units;

[0181] The data in the third logical storage unit is determined as the sample data.

[0182] Optionally, the first information determination module 12 is used to determine the statistical information of the data stored in any logical storage unit in the second logical storage unit based on the data volume of the data stored in the any logical storage unit, the total data volume of the original data, and the first statistical information of the original data.

[0183] Optionally, the original data includes a data table stored in the database system.

[0184] The first information determination module 12 is configured to receive first statistical information of the original data and second statistical information of the filtered data fed back by the database system.

[0185] Optionally, the query condition includes a connection relationship between different logical storage units, and the different logical storage units belong to at least one data organization form.

[0186] The first data determination module 11 is configured to determine the data stored in the different logical storage units corresponding to the connection relationship as the original data.

[0187] Optionally, the apparatus further comprises: an organization mode determination module 14, configured to obtain metadata of the data to be queried;

[0188] A data organization mode of the data to be queried is determined according to the metadata.

[0189] Figure 9 The device shown can perform Figures 1 to 5 For the method of the embodiment shown in FIG. 1 , reference may be made to the description of the part not described in detail in the embodiment. Figures 1 to 5 The implementation process and technical effects of this technical solution can be found in Figures 1 to 5 The description in the illustrated embodiment will not be repeated here.

[0190] Figure 10 A structural diagram of another data query device provided by an embodiment of the present invention, such as Figure 10 As shown, the device may include:

[0191] The second data determination module 21 is used to determine the data to be queried corresponding to the query request in response to the query request, and the data to be queried includes files in the file directory stored in the first logical storage unit, and the first logical storage unit belongs to the logical storage unit at the first level in the file system.

[0192] The sampling module 22 is used to extract the data to be queried to obtain sample data, and the sample data is stored in a second logical storage unit at the second level in the file system, where the second level is at least one level lower than the first level, and the first logical storage unit includes the second logical storage unit.

[0193] The second information determination module 23 is configured to determine the statistical information of the data to be queried based on the statistical information of the sample data.

[0194] The second response module 24 is configured to determine execution costs of different query requests according to the statistical information of the data to be queried, and respond to the query requests according to the target query strategy determined according to the execution costs.

[0195] Figure 10 The device shown can perform Figure 6 For the method of the embodiment shown in FIG. 1 , reference may be made to the description of the part not described in detail in the embodiment. Figure 6 The implementation process and technical effects of this technical solution can be found in Figure 6 The description in the illustrated embodiment will not be repeated here.

[0196] Figure 11 A structural diagram of another data query device provided by an embodiment of the present invention is shown in FIG. Figure 11 As shown, the device may include:

[0197] The third data determination module 31 is configured to determine, in response to a query request, data to be queried corresponding to the query request, where the data to be queried includes data stored in a data organization manner of a database system.

[0198] The receiving module 32 is configured to receive the statistical information of the data to be queried fed back by the database system.

[0199] The third response module 33 is configured to determine the execution costs of different query strategies based on the statistical information, and respond to the query request according to the target query strategy determined based on the execution costs.

[0200] Figure 11 The device shown can perform Figure 7 For the method of the embodiment shown in FIG. 1 , reference may be made to the description of the part not described in detail in the embodiment. Figure 7The implementation process and technical effects of this technical solution can be found in Figure 7 The description in the illustrated embodiment will not be repeated here.

[0201] In a possible design, the data query method provided in the above embodiments can be applied in an electronic device, such as Figure 12 As shown, the electronic device may include: a processor 41 and a memory 42. The memory 42 is used to store data that supports the electronic device to execute the above Figures 1 to 7 The program of the data query method provided in the illustrated embodiment, the processor 41 is configured to execute the program stored in the memory 42 .

[0202] The program includes one or more computer instructions, wherein the one or more computer instructions can be executed by the first processor 41 to achieve the above Figures 1 to 7 All or part of the steps in the illustrated embodiments.

[0203] The structure of the electronic device may further include a communication interface 43 for the electronic device to communicate with other devices or communication systems.

[0204] In addition, an embodiment of the present invention provides a computer storage medium for storing computer software instructions used by the above electronic device, which includes instructions for executing the above Figures 1 to 7 The procedures involved in the data query method shown.

[0205] In addition, an embodiment of the present invention provides a computer program product. The computer program product includes a computer program or instructions. When the computer program or instructions are executed by a processor, the processor is enabled to implement the above Figures 1 to 7 The steps or functions of the data query method shown.

[0206] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data query method, characterized in that: include: In response to a query request, determining data to be queried corresponding to the query request, the data to be queried including data stored in a data organization manner of a file system and / or a database system; Determining statistical information of the data to be queried according to a data organization mode of the data to be queried; Determine the execution costs of different query strategies based on the statistical information; The query request is responded to according to a target query strategy determined according to the execution cost.

2. The method according to claim 1, characterized in that The data to be queried is stored in a data lake; and determining the data to be queried corresponding to the query request includes: Acquire original data according to the query condition carried in the query request, wherein the original data is stored in a logical storage unit provided by a data organization method corresponding to the original data; Determining filtered data that meets the query condition in the original data; The data to be queried including the original data and the filtered data is determined.

3. The method according to claim 2, characterized in that The determining of statistical information of the data to be queried according to the data organization mode of the data to be queried includes: determining first statistical information of the original data according to a data organization mode of the original data; Second statistical information of the filtered data is determined according to a data organization mode of the filtered data.

4. The method according to claim 3, characterized in that The determining the first statistical information of the original data includes: Determining a total amount of the original data, the original data including files in a file directory stored in a first logical storage unit, the first logical storage unit being a logical storage unit at a first level in the file system; extracting a first sample file from the original data according to a first ratio, and storing the sample file in the first logical storage unit; First statistical information of the original data is determined according to the statistical information of the first sample file, the data volume of the first sample file, and the total data volume.

5. The method according to claim 4, characterized in that The determining the total amount of the original data includes: extracting a second sample file from the original data according to a second ratio; The total amount of the original data is determined according to the data amount of the second sample file and the second ratio.

6. The method according to claim 4, characterized in that Determining second statistical information of the filtered data includes: Extracting sample data from the original data, wherein the sample data is stored in a second logical storage unit at a second level in the file system, the second logical storage unit being included in the first logical storage unit, the second level including at least one level lower than the first level; Second statistical information of the filtered data is determined based on the statistical information of each of the original data and the sample data, and the statistical information of the data in the sample data that meets the query condition.

7. The method according to claim 6, characterized in that The extracting sample data from the original data includes: determining reference statistical information according to the third ratio and the first statistical information of the original data; determining a third logical storage unit in the second logical storage unit according to the reference statistical information and statistical information of the data stored in each of the second logical storage units; The data in the third logical storage unit is determined as the sample data.

8. The solution according to claim 7, characterized in that The method further comprises: For the data stored in any logical storage unit in the second logical storage unit, the statistical information of the data stored in any logical storage unit is determined based on the data volume of the data stored in the any logical storage unit, the total data volume of the original data, and the first statistical information of the original data.

9. The method according to claim 3, characterized in that The original data includes a data table stored in the database system; The determining the first statistical information of the original data includes: Receive first statistical information of the original data and second statistical information of the filtered data fed back by the database system.

10. The method according to claim 2, characterized in that The query condition includes a connection relationship between different logical storage units, and the different logical storage units belong to at least one data organization form; The acquiring of original data stored in the logical storage unit according to the query condition carried in the query request includes: The data stored in each of the different logical storage units corresponding to the connection relationship is determined as the original data.

11. The method according to claim 1, wherein The method further comprises: Obtaining metadata of the data to be queried; A data organization mode of the data to be queried is determined according to the metadata.

12. A data query method, characterized in that: include: In response to a query request, determining to-be-queried data corresponding to the query request, the to-be-queried data comprising files in a file directory stored in a first logical storage unit, the first logical storage unit being a logical storage unit at a first level in the file system; Extracting the data to be queried to obtain sample data, wherein the sample data is stored in a second logical storage unit at a second level in the file system, where the second level is at least one level lower than the first level, and the first logical storage unit includes the second logical storage unit; Determining the statistical information of the data to be queried based on the statistical information of the sample data; Determining the execution costs of different query strategies based on the statistical information of the data to be queried; The query request is responded to according to a target query strategy determined according to the execution cost.

13. A data query method, characterized in that: include: In response to a query request, determining data to be queried corresponding to the query request, the data to be queried including data stored in a data organization manner of a database system; Receiving statistical information of the data to be queried fed back by the database system; Determine the execution costs of different query strategies based on the statistical information; The query request is responded to according to a target query strategy determined according to the execution cost.

14. A query engine, characterized in that: include: processing components and query optimization components; The processing component is configured to determine, in response to a query request, data to be queried corresponding to the query request, wherein the data to be queried includes data stored in a data organization manner of a file system and / or a database system; Determining statistical information of the data to be queried according to a data organization mode of the data to be queried; Responding to the query request using a target query strategy; The query optimization component is used to determine the execution costs of different query strategies according to the statistical information; The target query strategy is determined according to the execution cost.

15. An electronic device, characterized in that: include: A memory and a computing system; wherein the memory stores executable code, and when the executable code is executed by the computing system, the computing system executes the data query method according to any one of claims 1 to 13.

16. A non-transitory machine-readable storage medium, characterized in that The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a computing system of an electronic device, the computing system is caused to execute the data query method according to any one of claims 1 to 13.

17. A computer program product, characterized in that The method comprises a computer program or instructions, which, when executed by a processor, enables the processor to implement the steps of the data query method according to any one of claims 1 to 13.

Images