Data query method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a data query method, which can be applied to the technical field of big data. The method comprises: in response to a first query request, obtaining a to-be-queried field carried in the first query request; determining a target field information table corresponding to the to-be-queried field from a data source; determining a first target query engine associated with the to-be-queried field based on the target field information table; constructing a first query statement based on the first target query engine; and executing the first query statement to obtain first target data associated with the to-be-queried field. In addition, the present disclosure also provides a data query device, equipment and storage medium.

Description

Technical Field

[0001] This disclosure relates to the field of big data technology, and more specifically, to a data query method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of data technology, in order to meet the market's demands for data retrieval and analysis, an increasing number of query engines are being applied to enterprise business systems. For the same query request, different query engines may exhibit completely different query efficiency, resource consumption, and query results. Therefore, when performing data queries, it is inevitable to choose a query engine.

[0003] In the process of realizing the present invention, the inventors discovered that the related technologies have at least the following problems: when performing data queries in the related technologies, the selection of query engines relies on the personal experience of technicians, resulting in high labor costs and low query efficiency. Summary of the Invention

[0004] In view of the above, this disclosure provides a data query method, a data query device, an electronic device, a readable storage medium, and a computer program product.

[0005] One aspect of this disclosure provides a data query method, comprising: in response to a first query request, obtaining a field to be queried carried in the first query request; determining a target field information table corresponding to the field to be queried from a data source; determining a first target query engine associated with the field to be queried based on the target field information table; constructing a first query statement based on the first target query engine; and executing the first query statement to obtain first target data associated with the field to be queried.

[0006] According to embodiments of this disclosure, the first query request further carries query time period information, and the data source includes a dictionary information table and multiple field information tables; wherein, determining the target field information table corresponding to the field to be queried from the data source includes: determining the target query type of the first query request based on the query time period and the response time information of the first query request; determining the base field associated with the field to be queried based on the dictionary information table; and determining the target field information table from the multiple field information tables based on the target query type and the base field.

[0007] According to an embodiment of this disclosure, determining the first target query engine associated with the field to be queried based on the target field information table includes: determining at least one first query engine associated with the benchmark field based on the target field information table; and determining the first target query engine from at least one of the first query engines.

[0008] According to an embodiment of this disclosure, the above-mentioned field information table is configured with query engine attribute values; wherein, the above-mentioned determination of at least one first query engine associated with the above-mentioned benchmark field based on the above-mentioned target field information table includes: parsing the query engine attribute values ​​of the above-mentioned target field information table to determine at least one of the above-mentioned first query engines.

[0009] According to embodiments of this disclosure, the method further includes: determining the target query dimension of the benchmark field; determining at least one second query engine related to the target query dimension; and determining at least one third query engine based on at least one first query engine and at least one second query engine; wherein determining the first target query engine from at least one first query engine includes determining the first target query engine from at least one third query engine.

[0010] According to an embodiment of this disclosure, the first query engine is configured with a priority; wherein, determining the first target query engine from at least one of the first query engines includes: sorting the at least one of the first query engines in descending order of priority to obtain a query engine sequence; and determining the first query engine at the beginning of the query engine sequence as the first target query engine.

[0011] According to embodiments of this disclosure, the method further includes: determining a preset resource consumption of the first target query engine when the first target query engine is a preset query engine; parsing a query plan based on the first query statement to obtain a predicted resource consumption; determining a second target query engine from the query engine sequence when the predicted resource consumption is greater than the preset resource consumption; and constructing a second query statement for the field to be queried based on the second target query engine.

[0012] According to embodiments of this disclosure, the method further includes: serializing the first target data to obtain serialized data; and storing the query field and the serialized data in a cache, using the query field as the cache primary key and the serialized data as the cache value.

[0013] According to embodiments of this disclosure, the method further includes: in response to a second query request, generating a target cache primary key based on a target query field carried in the second query request; querying the cache using the target cache primary key to obtain a query result; and if the query result indicates that a target cache value corresponding to the target cache primary key exists in the cache, deserializing the target cache value to obtain second target data associated with the target query field.

[0014] Another aspect of this disclosure provides a data query apparatus, comprising: a first acquisition module, a first determination module, a second determination module, a first construction module, and a first processing module. The first acquisition module is configured to acquire a field to be queried carried in the first query request in response to the first query request; the first determination module is configured to determine a target field information table corresponding to the field to be queried from a data source; the second determination module is configured to determine a first target query engine associated with the field to be queried based on the target field information table; the first construction module is configured to construct a first query statement based on the first target query engine; and the first processing module is configured to execute the first query statement to obtain first target data associated with the field to be queried.

[0015] Another aspect of this disclosure provides an electronic device, including: one or more processors; and a memory for storing one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors cause the one or more processors to perform the method as described above.

[0016] Another aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the method described above.

[0017] Another aspect of this disclosure provides a computer program product including computer-executable instructions that, when executed, implement the method described above.

[0018] According to embodiments of this disclosure, the query engines supported by each field can be pre-configured and stored in the data source. When performing a data query, a first target query engine can be determined based on the field to be queried and the field information table in the data source. A first query statement is then constructed based on this first target query engine, and subsequently, the first query statement can be executed to retrieve the first target data. Through the above technical means, the technical problems of relying on the personal experience of technical personnel to select the query engine during data querying, resulting in high labor costs and low query efficiency, can be overcome at least partially. This effectively reduces the learning cost of data querying and improves query efficiency. Attached Figure Description

[0019] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0020] Figure 1 An exemplary system architecture for applying data query methods and apparatus according to embodiments of this disclosure is illustrated.

[0021] Figure 2 A flowchart illustrating a data query method according to an embodiment of the present disclosure is shown schematically.

[0022] Figure 3 A flowchart illustrating a first target query engine determination method according to another embodiment of the present disclosure is shown.

[0023] Figure 4 A flowchart illustrating a first target query engine determination method according to yet another embodiment of the present disclosure is shown.

[0024] Figure 5 A flowchart illustrating a data query method according to another embodiment of this disclosure is shown.

[0025] Figure 6 A block diagram of a data query apparatus according to an embodiment of the present disclosure is shown schematically.

[0026] Figure 7 A block diagram of an electronic device suitable for implementing a data query method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0027] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0030] When using expressions such as "at least one of A, B, and C," the expression should generally be interpreted in accordance with the meaning commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). Similarly, when using expressions such as "at least one of A, B, or C," the expression should generally be interpreted in accordance with the meaning commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, or C" should include, but is not limited to, systems having A alone, having B alone, having C alone, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0031] With the development of data technology, and to meet market demands for data retrieval and analysis, an increasing variety of query engines have been launched and are widely used. Commonly used query engines in this field can include traditional relational databases such as Oracle, DB2, and MySQL, as well as engines designed for big data processing such as HIVE, Impala, and Druid. While various query engines share some similarities, they also have their own characteristics and suitable scenarios. Therefore, to adapt to different business needs, enterprises typically deploy multiple query systems to accommodate different query engines. However, for the same query request, query efficiency, resource consumption, and query results can vary significantly across different query engines. Therefore, when performing data queries, the selection of a query engine is unavoidable.

[0032] The methods used in related technologies for selecting query engines generally include steps such as data source filtering, query engine filtering, and query code compilation. However, these steps largely rely on the personal experience of business personnel. Furthermore, since the same query can be executed by different query engines, and the execution efficiency and resource consumption of different query engines are not the same, manual selection may not be able to select the optimal query engine, thus affecting query efficiency.

[0033] In view of the above, embodiments of this disclosure provide a data query method, apparatus, electronic device, readable storage medium, and computer program product. The method includes: in response to a first query request, obtaining a field to be queried carried in the first query request; determining a target field information table corresponding to the field to be queried from a data source; determining a first target query engine associated with the field to be queried based on the target field information table; constructing a first query statement based on the first target query engine; and executing the first query statement to obtain first target data associated with the field to be queried.

[0034] It should be noted that, unless it is explicitly stated that there is a sequential order of execution between different operations, or that there is a sequential order of execution between different operations in terms of technical implementation, the execution order between multiple operations may not be significant, and multiple operations may be executed simultaneously.

[0035] Figure 1 This illustration schematically depicts an exemplary system architecture to which data query methods and apparatus can be applied according to embodiments of this disclosure. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0036] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105.

[0037] Terminal devices 101, 102, and 103 can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers. Users can perform data queries through terminal devices 101, 102, and 103, and the results of the data queries can be displayed to the user on the display screen.

[0038] Network 104 is a medium used to provide a communication link between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired and / or wireless communication links.

[0039] Server 105 can be a server that provides various services, such as a backend management server that supports query requests initiated by users through terminal devices 101, 102, and 103. The backend management server can process the received query requests and return the data query structure to the terminal devices.

[0040] It should be noted that the data query method provided in this embodiment can generally be executed by server 105. Correspondingly, the data query device provided in this embodiment can generally be located in server 105. The data query method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the data query device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Alternatively, the data query method provided in this embodiment can also be executed by terminal devices 101, 102, or 103, or by other terminal devices different from terminal devices 101, 102, or 103. Correspondingly, the data query device provided in this embodiment can also be located in terminal devices 101, 102, or 103, or in other terminal devices different from terminal devices 101, 102, or 103.

[0041] For example, terminal device 101 can generate a query request in response to user input. Then, terminal device 101 can locally execute the data query method provided in this embodiment to process the query request and obtain the target data. Alternatively, terminal device 101 can send the query request to other terminal devices, servers, or server clusters, and other terminal devices, servers, or server clusters that receive the query request can execute the data query method provided in this embodiment.

[0042] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0043] Figure 2 A flowchart illustrating a data query method according to an embodiment of the present disclosure is shown schematically.

[0044] like Figure 2 As shown, the method includes operations S201 to S205.

[0045] In operation S201, in response to the first query request, the field to be queried carried in the first query request is obtained.

[0046] In operation S202, the target field information table corresponding to the field to be queried is determined from the data source.

[0047] In operation S203, based on the target field information table, the first target query engine associated with the field to be queried is determined.

[0048] In operation S204, a first query statement is constructed based on the first target query engine.

[0049] In operation S205, the first query statement is executed to obtain the first target data associated with the field to be queried.

[0050] According to embodiments of this disclosure, the first query request may be generated based on a user's selection operation on components such as checkboxes and dropdown lists on the display interface. The first query request may include the field to be queried, the data time range to be queried, and configured filtering conditions. The configured filtering conditions may include filter fields, filter relationships, and filter standard values, etc., and are not limited thereto.

[0051] According to embodiments of this disclosure, the number of fields to be queried carried in the first query request can be one or more.

[0052] According to embodiments of this disclosure, the data source may include multiple field information tables that are one-to-one related to multiple fields stored in the query engine. These field information tables may include information such as field ID, field name, and supported query engines. Each query engine can generate a field information table corresponding to a field when writing to it, or modify the content of an existing field information table based on the query engine used when writing to the field.

[0053] According to embodiments of this disclosure, determining the target field information table corresponding to the field to be queried from the data source may refer to: matching the field to be queried with attributes such as field ID and field name in multiple field information tables, and determining the field information table that has completed the matching as the target field information table.

[0054] According to embodiments of this disclosure, each field may have multiple corresponding query engines that support that field. Accordingly, the first target query engine may be a query engine that supports the field to be queried.

[0055] According to embodiments of this disclosure, constructing a first query statement based on a first target query engine can be achieved by extracting relevant fields from the fields to be queried and then using the relevant fields to replace the query statement template; or it can be achieved by any query statement generation tool, such as SqlMetal, etc., without limitation.

[0056] According to embodiments of this disclosure, the query engines supported by each field can be pre-configured and stored in the data source. When performing a data query, a first target query engine can be determined based on the field to be queried and the field information table in the data source. A first query statement is then constructed based on this first target query engine, and subsequently, the first query statement can be executed to retrieve the first target data. Through the above technical means, the technical problems of relying on the personal experience of technical personnel to select the query engine during data querying, resulting in high labor costs and low query efficiency, can be overcome at least partially. This effectively reduces the learning cost of data querying and improves query efficiency.

[0057] The following is for reference. Figures 3-5 In conjunction with specific embodiments, Figure 2 The method shown will be further explained.

[0058] According to embodiments of this disclosure, the data source may include a dictionary information table and multiple field information tables.

[0059] According to embodiments of this disclosure, the first query request may carry query time period information.

[0060] According to embodiments of this disclosure, operation S202 may include the following operations:

[0061] Based on the query period and the response time of the first query request, the target query type of the first query request is determined; based on the dictionary information table, the base field associated with the field to be queried is determined; and based on the target query type and the base field, the target field information table is determined from multiple field information tables.

[0062] According to embodiments of this disclosure, the query type of the current query can include real-time query, offline query, historical query, etc. The classification of the query type can be determined based on the time difference between the query period and the response time. For example, if the start time of the query period is no more than 3 days from the current time, the query type can be determined to be a real-time query; if the start time of the query period is more than 3 days but less than 3 months from the current time, the query type can be determined to be an offline query; if the start time of the query period is more than 3 months from the current time, the query type can be determined to be a historical query. The classification of the query type can also be determined based on some preset conditions. For example, if the time represented by the response time information is no later than 6:00 AM, the query type of the current query can be determined to be a real-time query, etc.

[0063] According to embodiments of this disclosure, the same field can have different field names in different query engines. A dictionary information table can record the field names of the same field in all query engines, and this dictionary information table can be used to convert the field to be queried into a unified baseline field. The baseline field can refer to the field name of the field to be queried in a baseline query engine, which can refer to any query engine, without limitation herein.

[0064] According to embodiments of this disclosure, the field information table may include information such as field ID, base field name, query type attribute value, and query engine attribute value, as shown in Table 1.

[0065] Table 1

[0066] Column names type Remark ID int Field ID Name varchar Baseline field name Query_Type int Query type attribute value Query_Engine int Query engine attribute values

[0067] According to embodiments of this disclosure, determining the target field information table from multiple field information tables based on the target query type and the base field can be achieved by matching the target query type with the query type attribute value in each field information table, matching the base field with the base field name in each field information table, and determining the matched field information table as the target field information table when both are matched.

[0068] According to embodiments of this disclosure, operation S203 may include the following operations:

[0069] Based on the target field information table, at least one first query engine associated with the baseline field is determined; and a first target query engine is determined from the at least one first query engine.

[0070] According to embodiments of this disclosure, determining at least one first query engine associated with a baseline field may include the following operations:

[0071] Parse the query engine attribute value of the target field information table to determine at least one first query engine.

[0072] According to embodiments of this disclosure, the query engine attribute value can be represented as the sum of the values ​​represented by all query engines supported by the base field. For example, the values ​​represented by the various query engines are 1: Impala, 2: Hive, 4: Kylin, 8: Druid, and 16: mysql. Parsing the query engine attribute value can be done by decomposing the query engine attribute value into a sum of powers of 2. Taking the query engine attribute value of the base field as 21 as an example, this query engine attribute value can be parsed as 21 = 1 × 2. 0 +0×2 1 +1×2 2 +0×2 3 +1×2 4Based on the parsing results, it can be determined that the query engines supported by this baseline field are Impala, Kylin, and mysql, that is, the first query engine includes Impala, Kylin, and mysql.

[0073] According to embodiments of this disclosure, the first query engine can be configured with a priority. This priority can be pre-configured by the user; for example, for the three query engines Impala, Kylin, and mysql, the preset priority order from high to low could be mysql, Kylin, and Impala. Alternatively, the priority can be automatically adjusted based on the resource usage of each query engine; for example, a higher priority can be configured for query engines with more remaining resources. This is not limited to any particular engine.

[0074] According to embodiments of this disclosure, determining a first target query engine from at least one first query engine may include the following operations:

[0075] At least one first query engine is sorted in descending order of priority to obtain a query engine sequence; and the first query engine at the top of the query engine sequence is determined as the first target query engine.

[0076] Figure 3 A flowchart illustrating a first target query engine determination method according to another embodiment of the present disclosure is shown.

[0077] like Figure 3 As shown, the method includes operations S301 to S306.

[0078] In operation S301, the target query type of the first query request is determined based on the query time period and the response time information of the first query request.

[0079] In operation S302, at least one first query engine corresponding to the target query type is determined.

[0080] In operation S303, determine the target query dimension for the baseline field.

[0081] In operation S304, at least one second query engine is identified that is relevant to the target query dimension.

[0082] In operation S305, at least one third query engine is determined based on at least one first query engine and at least one second query engine.

[0083] In operation S306, a first target query engine is determined from at least one third query engine.

[0084] According to embodiments of this disclosure, the target query dimension of a first query request can be determined based on the field name of a baseline field. For example, when the field name of the baseline field contains keywords such as "frequency," the target query dimension of the first query request can be determined to be the frequency query dimension; as another example, when the field name of the baseline field contains keywords specific to a particular scenario such as "traffic," the target query dimension of the first query request can be determined to be the traffic query dimension.

[0085] According to embodiments of this disclosure, each query dimension may have a supported query engine. For example, the query engine supporting the frequency query dimension may be Hive, and the query engine supporting the traffic query dimension may be Impala, etc.

[0086] According to embodiments of this disclosure, determining at least one third query engine based on at least one first query engine and at least one second query engine can be understood as: identifying the same query engine among the at least one first query engine and at least one second query engine as the third query engine. For example, each query engine can be represented as a power of 2, such as 1: Impala, 2: Hive, 4: Kylin, 8: Druid, and 16: mysql; the query engines supported by each query type and query dimension can be represented as the sum of their corresponding powers. For example, the first query engine supported by the target query type can be represented as (10101)2, meaning the first query engine includes Impala, Kylin, and mysql; the second query engine supported by the target query dimension can be represented as (01101)2, meaning the second query engine includes Hive, Kylin, and mysql; then the third query engine can be represented as the bitwise AND result of (10101)2 and (01101)2, that is, the third query engine can be represented as (00101)2, meaning the third query engine includes Kylin and mysql.

[0087] In some embodiments, after determining the first target query engine, which is a special query engine such as Impala, the query statement constructed based on the first query requirement can be analyzed, and the analysis results can be used to determine whether the first target query engine meets the requirements. If it does not meet the requirements, the first query engine can be re-determined after excluding the query engine.

[0088] Figure 4 A flowchart illustrating a first target query engine determination method according to yet another embodiment of the present disclosure is shown.

[0089] like Figure 4 As shown, the method includes operations S401 to S406.

[0090] In operation S401, a first query statement is constructed based on the first target query engine.

[0091] In operation S402, determine whether the first target query requirement is a preset query engine. If it is determined that the first target query requirement is a preset query engine, execute operation S403; if it is determined that the first target query requirement is not a preset query engine, execute operation S406.

[0092] In operation S403, the query plan of the first query statement is parsed to obtain the predicted resource consumption.

[0093] In operation S404, determine whether the predicted resource consumption is greater than the preset resource consumption. If it is determined that the predicted resource consumption is greater than the preset resource consumption, execute operation S405; if it is determined that the predicted resource consumption is less than or equal to the preset resource consumption, execute operation S406.

[0094] In operation S405, the first target query engine is re-determined from the query engine sequence. After completing operation S405, the process returns to execute operation S401.

[0095] In operation S406, return to the first target query engine.

[0096] According to embodiments of this disclosure, the preset query engine may be a memory-limited query engine, such as Impala.

[0097] According to embodiments of this disclosure, re-determining the first target query engine from the query engine sequence may involve taking the second query engine in the query engine sequence as the first target query engine, and then deleting the first query engine from the query engine sequence.

[0098] Figure 5 A flowchart illustrating a data query method according to another embodiment of this disclosure is shown.

[0099] like Figure 5 As shown, the method includes operations S501 to S507.

[0100] In operation S501, a query request is received.

[0101] In operation S502, determine if the same query exists in the cache. If it is determined that the same query does not exist in the cache, execute operation S503; if it is determined that the same query exists in the cache, execute operation S506.

[0102] When operating S503, adapt the query engine according to the query request.

[0103] When operating S504, a query statement is constructed based on the query engine.

[0104] In operation S505, execute the query statement.

[0105] Operation S506 returns the query results.

[0106] In operation S507, the cache is updated based on the query results.

[0107] According to embodiments of this disclosure, the data in the cache can be stored in the form of key-value pairs, wherein the cache primary key in the key-value pair can be generated as needed to query the field, and the cache value in the key-value pair can be generated based on the data returned by querying the field.

[0108] According to embodiments of this disclosure, the query request may carry a target query field, and correspondingly, the query result may contain target data corresponding to the target query field.

[0109] According to embodiments of this disclosure, operation S507 may include the following operations:

[0110] The target data is serialized to obtain serialized data; and the target query field is used as the cache primary key, and the serialized data is used as the cache value to store the target query field and the serialized data into the cache.

[0111] According to embodiments of this disclosure, serialization processing can reduce the storage space occupied by the target data, thereby enabling the cache to store more key-value pairs.

[0112] According to embodiments of this disclosure, operation S502 may include the following operations:

[0113] First, the target query field carried in the query request can be converted into the target cache primary key; then, the cache can be queried using the target cache primary key, that is, the target cache primary key is compared with the cache primary key of each key-value pair in the cache in turn to obtain the query result; then, if the query result indicates that there is a cache primary key in the cache that is the same as the target cache primary key, it can be determined that there is the same query in the cache.

[0114] According to embodiments of this disclosure, if the query result indicates that a cache primary key with the same name as the target cache primary key exists in the cache, it can be determined that a target cache value corresponding to the target cache primary key exists in the cache. Then, the target cache value can be deserialized, and the target data obtained is the query result.

[0115] Figure 6 A block diagram of a data query apparatus according to an embodiment of the present disclosure is shown schematically.

[0116] like Figure 6As shown, the data query device 600 includes a first acquisition module 610, a first determination module 620, a second determination module 630, a first construction module 640, and a first processing module 650.

[0117] The first acquisition module 610 is used to respond to the first query request and acquire the field to be queried carried in the first query request.

[0118] The first determination module 620 is used to determine the target field information table corresponding to the field to be queried from the data source.

[0119] The second determining module 630 is used to determine the first target query engine associated with the field to be queried based on the target field information table.

[0120] The first construction module 640 is used to construct the first query statement based on the first target query engine.

[0121] The first processing module 650 is used to execute the first query statement to obtain the first target data associated with the field to be queried.

[0122] According to embodiments of this disclosure, the query engines supported by each field can be pre-configured and stored in the data source. When performing a data query, a first target query engine can be determined based on the field to be queried and the field information table in the data source. A first query statement is then constructed based on this first target query engine, and subsequently, the first query statement can be executed to retrieve the first target data. Through the above technical means, the technical problems of relying on the personal experience of technical personnel to select the query engine during data querying, resulting in high labor costs and low query efficiency, can be overcome at least partially. This effectively reduces the learning cost of data querying and improves query efficiency.

[0123] According to embodiments of this disclosure, the first query request also carries query time period information, and the data source includes a dictionary information table and multiple field information tables.

[0124] According to embodiments of this disclosure, the first determining module 620 includes a first determining unit, a second determining unit, and a third determining unit.

[0125] The first determining unit is used to determine the target query type of the first query request based on the query time period and the response time information of the first query request.

[0126] The second determining unit is used to determine the base field associated with the field to be queried based on the dictionary information table.

[0127] The third determining unit is used to determine the target field information table from multiple field information tables based on the target query type and the benchmark field.

[0128] According to embodiments of this disclosure, the second determining module 630 includes a fourth determining unit and a fifth determining unit.

[0129] The fourth determining unit is used to determine at least one first query engine associated with the benchmark field based on the target field information table.

[0130] The fifth determining unit is used to determine the first target query engine from at least one first query engine.

[0131] According to embodiments of this disclosure, the field information table is configured with query engine attribute values.

[0132] According to embodiments of this disclosure, the fourth determining unit includes a first determining subunit.

[0133] The first determining subunit is used to parse the query engine attribute value of the target field information table to determine at least one first query engine.

[0134] According to embodiments of this disclosure, the second determining module 630 further includes a sixth determining unit, a seventh determining unit, and an eighth determining unit.

[0135] The sixth determining unit is used to determine the target query dimension of the baseline field.

[0136] The seventh determining unit is used to determine at least one second query engine related to the target query dimension.

[0137] The eighth determining unit is used to determine at least one third query engine based on at least one first query engine and at least one second query engine.

[0138] According to embodiments of this disclosure, the fifth determining unit includes a second determining subunit.

[0139] The second determining subunit is used to determine the first target query engine from at least one third query engine.

[0140] According to embodiments of this disclosure, the first query engine is configured with a priority.

[0141] According to embodiments of this disclosure, the fifth determining unit includes a third determining subunit and a fourth determining subunit.

[0142] The third determining subunit is used to sort at least one first query engine in descending order of priority to obtain a query engine sequence.

[0143] The fourth determining subunit is used to determine the first query engine in the query engine sequence as the first target query engine.

[0144] According to embodiments of this disclosure, the data query device 600 further includes a third determining module, a second processing module, a fourth determining module, and a second constructing module.

[0145] The third determining module is used to determine the preset resource consumption of the first target query engine when the first target query engine is a preset query engine.

[0146] The second processing module is used to parse the query plan based on the first query statement to obtain the predicted resource consumption.

[0147] The fourth determination module is used to determine the second target query engine from the query engine sequence when the predicted resource consumption is greater than the preset resource consumption.

[0148] The second construction module is used to construct a second query statement for the field to be queried, based on the second target query engine.

[0149] According to embodiments of this disclosure, the data query device 600 further includes a third processing module and a storage module.

[0150] The third processing module is used to serialize the first target data to obtain serialized data.

[0151] The storage module is used to store the field to be queried as the cache primary key and the serialized data as the cache value.

[0152] According to embodiments of this disclosure, the data query device 600 further includes a generation module, a query module, and a fourth processing module.

[0153] The generation module is used to respond to the second query request and generate the target cache primary key based on the target query field carried in the second query request.

[0154] The query module is used to query the cache using the target cache primary key and obtain the query results.

[0155] The fourth processing module is used to deserialize the target cache value when the query result indicates that a target cache value corresponding to the target cache primary key exists in the cache, so as to obtain the second target data associated with the target query field.

[0156] Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as hardware circuitry, such as Field Programmable Gate Arrays (FPGAs), Programmable Logic Arrays (PLAs), Systems-on-Chip, Systems-on-Substrate, Systems-on-Package, Application-Specific Integrated Circuits (ASICs), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0157] For example, any plurality of the first acquisition module 610, the first determination module 620, the second determination module 630, the first construction module 640, and the first processing module 650 can be combined into one module / unit / subunit, or any one of these modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functionality of one or more of these modules / units / subunits can be combined with at least part of the functionality of other modules / units / subunits and implemented in one module / unit / subunit. According to embodiments of this disclosure, at least one of the first acquisition module 610, the first determination module 620, the second determination module 630, the first construction module 640, and the first processing module 650 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the first acquisition module 610, the first determination module 620, the second determination module 630, the first construction module 640, and the first processing module 650 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0158] It should be noted that the data query device part in the embodiments of this disclosure corresponds to the data query method part in the embodiments of this disclosure. The specific description of the data query device part is referred to in the data query method part, and will not be repeated here.

[0159] Figure 7 A block diagram of an electronic device suitable for implementing a data query method according to an embodiment of the present disclosure is shown schematically. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0160] like Figure 7 As shown, a computer electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0161] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0162] According to embodiments of this disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0163] According to embodiments of this disclosure, the method flow according to embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by processor 701, it performs the functions defined in the system of embodiments of this disclosure. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0164] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0165] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0166] For example, according to embodiments of this disclosure, a computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0167] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the data query method provided in the embodiments of this disclosure.

[0168] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0169] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0170] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0171] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not expressly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0172] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A data query method, comprising: in response to a first query request, obtaining a field to be queried carried in the first query request; determining a target field information table corresponding to the field to be queried from a data source; based on the target field information table, determining a first target query engine associated with the field to be queried; based on the first target query engine, constructing a first query statement; and executing the first query statement to obtain first target data associated with the field to be queried; wherein the first query request further carries query period information, and the data source comprises a dictionary information table and a plurality of field information tables; wherein the determination of the target field information table corresponding to the field to be queried from the data source comprises: based on the query period and response time information of the first query request, determining a target query type of the first query request; based on the dictionary information table, determining a reference field associated with the field to be queried; and based on the target query type and the reference field, determining the target field information table from the plurality of field information tables. the determination of the first target query engine associated with the field to be queried based on the target field information table comprises:

2. The method of claim 1, wherein, based on the target field information table, determining at least one first query engine associated with the reference field; and determining the first target query engine from the at least one first query engine. The field information table is configured with a query engine attribute value; 3. The method of claim 2, wherein, wherein the determination of the at least one first query engine associated with the reference field based on the target field information table comprises: parsing the query engine attribute value of the target field information table to determine the at least one first query engine. 4.The method of claim 2, further comprising: determining a target query dimension of the reference field; determining at least one second query engine related to the target query dimension; and based on the at least one first query engine and the at least one second query engine, determining at least one third query engine; wherein the determination of the first target query engine from the at least one first query engine comprises: determining the first target query engine from the at least one third query engine. The first query engine is configured with a priority; wherein the determination of the first target query engine from the at least one first query engine comprises:

5. The method of claim 2, wherein, sorting the at least one first query engine in order of priority from high to low to obtain a query engine sequence; and determining the first query engine at the first position in the query engine sequence as the first target query engine. 6.The method of claim 5, further comprising: in the case that the first target query engine is a preset query engine, determining a preset resource consumption of the first target query engine; performing query plan analysis based on the first query statement to obtain a predicted resource consumption; in the case that the predicted resource consumption is greater than the preset resource consumption, determining a second target query engine from the query engine sequence; and ​ ​ ​ construct a second query statement for the to-be-queried field based on the second target query engine.

7. The method of claim 1, further comprising: serializing the first target data to obtain serialized data; and storing the to-be-queried field and the serialized data into a cache, with the to-be-queried field as a cache primary key and the serialized data as a cache value.

8. The method of claim 7, further comprising: in response to a second query request, generating a target cache primary key based on a target query field carried in the second query request; querying the cache using the target cache primary key to obtain a query result; and in a case where the query result indicates that there is a target cache value corresponding to the target cache primary key in the cache, deserializing the target cache value to obtain second target data associated with the target query field.

9. A data query apparatus, comprising: a first acquisition module configured to acquire a to-be-queried field carried in a first query request in response to the first query request; a first determination module configured to determine a target field information table corresponding to the to-be-queried field from a data source; a second determination module configured to determine a first target query engine associated with the to-be-queried field based on the target field information table; a first construction module configured to construct a first query statement based on the first target query engine; and a first processing module configured to execute the first query statement to obtain first target data associated with the to-be-queried field; wherein the first query request further carries query time period information, and the data source includes a dictionary information table and a plurality of field information tables; the first determination module is further configured to: determine a target query type of the first query request based on the query time period and response time information of the first query request; determine a reference field associated with the to-be-queried field based on the dictionary information table; and determine the target field information table from the plurality of field information tables based on the target query type and the reference field.

10. An electronic device, comprising: one or more processors; a memory configured to store one or more instructions, wherein the one or more instructions, when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1 to 8.

11. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to implement the method of any one of claims 1 to 8.

12. A computer program product comprising computer executable instructions that, when executed, implement the method of any one of claims 1 to 8. ​

Images