Statement execution method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application relate to a sentence execution method and device, electronic equipment and storage medium. The method comprises: obtaining a to-be-executed sentence, wherein the to-be-executed sentence represents a sentence composed of a structured query language; determining a target analysis engine from a predetermined set of analysis engines, wherein the target analysis engine has the ability to parse the to-be-executed sentence; determining a data table indicated by the to-be-executed sentence; determining a target execution engine from a predetermined set of execution engines based on the target analysis engine and the data table, wherein the target execution engine is used to execute the to-be-executed sentence; and executing the to-be-executed sentence through the target execution engine. Thus, the period for determining the execution engine is shortened, and the complexity of determining the execution engine is reduced.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a statement execution method, apparatus, electronic device, and storage medium. Background Technology

[0002] As internet companies expand, they may establish data centers in multiple geographically dispersed regions, hosting massive amounts of data. Simultaneously, with the development of global operations, companies are facing the need to process and analyze big data across data centers.

[0003] Currently, the most common approach is to use data replication and synchronization technologies to synchronize and replicate data distributed across different Availability Zones (AZs) to meet the needs of computing scenarios.

[0004] However, the above solutions typically require configuring cluster and execution engine information, which results in a long and complex process for determining the execution engine. Summary of the Invention

[0005] In view of this, in order to solve some or all of the above-mentioned technical problems, embodiments of this application provide a statement execution method, apparatus, electronic device and storage medium.

[0006] In a first aspect, embodiments of this application provide a statement execution method, the method comprising:

[0007] Obtain the statement to be executed, wherein the statement to be executed represents a statement composed of Structured Query Language;

[0008] From a pre-determined set of analysis engines, a target analysis engine is determined, wherein the target analysis engine has the ability to parse the statement to be executed;

[0009] Determine the data table to be operated on by the statement to be executed;

[0010] Based on the target analysis engine and the data table, a target execution engine is determined from a pre-determined set of execution engines, wherein the target execution engine is used to execute the statement to be executed;

[0011] The statement to be executed is executed through the target execution engine.

[0012] In one possible implementation, before determining the target analytics engine from a predetermined set of analytics engines, the method further includes:

[0013] Determine the abstract syntax tree of the statement to be executed; and

[0014] The step of determining the target analysis engine from a pre-determined set of analysis engines includes:

[0015] Based on the abstract syntax tree, the target analysis engine is determined from a pre-defined set of analysis engines.

[0016] In one possible implementation, determining the data table indicating the operation of the statement to be executed includes:

[0017] Based on the abstract syntax tree, determine whether the statement to be executed conforms to the preset semantic rules;

[0018] If the statement to be executed conforms to the preset semantic rules, the data table to be operated on by the statement to be executed is determined.

[0019] In one possible implementation, determining the target execution engine from a predetermined set of execution engines based on the target analysis engine and the data table includes:

[0020] Determine the clusters of data distribution in the data table to obtain a cluster set, wherein each cluster includes at least one execution engine;

[0021] From the set of clusters, a target cluster is determined, wherein the target cluster is determined based on the storage space occupied by the data table, the storage cost of the cluster, and the inter-cluster transmission cost;

[0022] The target cluster includes at least one execution engine as a pre-determined set of execution engines, and the target execution engine is determined from the set of execution engines based on the target analysis engine.

[0023] In one possible implementation, determining the target execution engine from the set of execution engines includes:

[0024] The operation indicated by the statement to be executed is determined, and the target operation is obtained;

[0025] Based on the target operation, a target execution engine is determined from the set of execution engines.

[0026] In one possible implementation, determining the target execution engine from the set of execution engines based on the target operation includes:

[0027] Determine the priority and execution scenario of the target operation;

[0028] Determine the engine distribution information of the execution engine set;

[0029] Based on the priority, the execution scenario, and the engine distribution information, the target execution engine is determined from the set of execution engines.

[0030] In one possible implementation, obtaining the statement to be executed includes:

[0031] Obtain the statement to be executed sent by the target terminal; and

[0032] After executing the statement to be executed through the target execution engine, the method further includes:

[0033] Determine the execution result of the target execution engine in executing the statement to be executed;

[0034] The execution result is returned to the target terminal.

[0035] Secondly, embodiments of this application provide a statement execution apparatus, the apparatus comprising:

[0036] The acquisition unit is used to acquire the statement to be executed, wherein the statement to be executed represents a statement composed of a structured query language.

[0037] The first determining unit is configured to determine a target analysis engine from a pre-determined set of analysis engines, wherein the target analysis engine has the ability to parse the statement to be executed;

[0038] The second determining unit is used to determine the data table of the operation indicated by the statement to be executed;

[0039] The third determining unit is used to determine a target execution engine from a pre-determined set of execution engines based on the target analysis engine and the data table, wherein the target execution engine is used to execute the statement to be executed;

[0040] An execution unit is used to execute the statement to be executed through the target execution engine.

[0041] In one possible implementation, before determining the target analysis engine from a predetermined set of analysis engines, the apparatus further includes:

[0042] The fourth determining unit is used to determine the abstract syntax tree of the statement to be executed; and

[0043] The step of determining the target analysis engine from a pre-determined set of analysis engines includes:

[0044] The fifth determining unit is used to determine the target analysis engine from a pre-determined set of analysis engines based on the abstract syntax tree.

[0045] In one possible implementation, determining the data table indicating the operation of the statement to be executed includes:

[0046] Based on the abstract syntax tree, determine whether the statement to be executed conforms to the preset semantic rules;

[0047] If the statement to be executed conforms to the preset semantic rules, the data table to be operated on by the statement to be executed is determined.

[0048] In one possible implementation, determining the target execution engine from a predetermined set of execution engines based on the target analysis engine and the data table includes:

[0049] Determine the clusters of data distribution in the data table to obtain a cluster set, wherein each cluster includes at least one execution engine;

[0050] From the set of clusters, a target cluster is determined, wherein the target cluster is determined based on the storage space occupied by the data table, the storage cost of the cluster, and the inter-cluster transmission cost;

[0051] The target cluster includes at least one execution engine as a pre-determined set of execution engines, and the target execution engine is determined from the set of execution engines based on the target analysis engine.

[0052] In one possible implementation, determining the target execution engine from the set of execution engines includes:

[0053] The operation indicated by the statement to be executed is determined, and the target operation is obtained;

[0054] Based on the target operation, a target execution engine is determined from the set of execution engines.

[0055] In one possible implementation, determining the target execution engine from the set of execution engines based on the target operation includes:

[0056] Determine the priority and execution scenario of the target operation;

[0057] Determine the engine distribution information of the execution engine set;

[0058] Based on the priority, the execution scenario, and the engine distribution information, the target execution engine is determined from the set of execution engines.

[0059] In one possible implementation, obtaining the statement to be executed includes:

[0060] Obtain the statement to be executed sent by the target terminal; and

[0061] After the statement to be executed is executed by the target execution engine, the apparatus further includes:

[0062] The sixth determining unit is used to determine the execution result of the target execution engine executing the statement to be executed;

[0063] The return unit is used to return the execution result to the target terminal.

[0064] Thirdly, embodiments of this application provide an electronic device, including:

[0065] Memory, used to store computer programs;

[0066] A processor is configured to execute a computer program stored in the memory, wherein, when the computer program is executed, it implements the method of any embodiment of the statement execution method of the first aspect of this application.

[0067] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the method of any embodiment of the statement execution method of the first aspect described above.

[0068] Fifthly, embodiments of this application provide a computer program comprising computer-readable code, which, when executed on a device, causes a processor in the device to implement the method of any embodiment of the statement execution method of the first aspect described above.

[0069] The statement execution method provided in this application embodiment can obtain a statement to be executed, wherein the statement to be executed represents a statement composed of a structured query language. Then, a target analysis engine is determined from a pre-determined set of analysis engines, wherein the target analysis engine has the ability to parse the statement to be executed. Next, a data table indicating the operation of the statement to be executed is determined. Subsequently, based on the target analysis engine and the data table, a target execution engine is determined from a pre-determined set of execution engines, wherein the target execution engine is used to execute the statement to be executed. Finally, the statement to be executed is executed through the target execution engine. Therefore, the execution engine for executing the statement to be executed can be determined based on the target analysis engine and the data table, and the statement to be executed is executed through that execution engine. Thus, a suitable execution engine can be determined without configuring cluster and execution engine information, shortening the cycle of determining the execution engine and reducing the complexity of determining the execution engine. Attached Figure Description

[0070] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0071] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0072] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0073] Figure 1 A flowchart illustrating a statement execution method provided in an embodiment of this application;

[0074] Figure 2 A flowchart illustrating another statement execution method provided in an embodiment of this application;

[0075] Figure 3A A flowchart illustrating yet another statement execution method provided in an embodiment of this application;

[0076] Figure 3B This application provides an embodiment of a statement execution method that illustrates the execution process of a statement to be executed.

[0077] Figure 3C A system block diagram relating to a statement execution method provided in an embodiment of this application;

[0078] Figure 4 This is a schematic diagram of the structure of a statement execution device provided in an embodiment of this application;

[0079] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0080] Various exemplary embodiments of this application will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this application.

[0081] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of this application are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor do they indicate the logical order between them.

[0082] It should also be understood that in this embodiment, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0083] It should also be understood that any component, data or structure mentioned in the embodiments of this application can generally be understood as one or more unless explicitly defined or given contrary guidance in the context.

[0084] Furthermore, the term "and / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects have an "or" relationship.

[0085] It should also be understood that the description of the various embodiments in this application emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0086] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0087] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0088] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0089] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. To facilitate understanding of the embodiments of this application, the application will be described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0090] To address the technical problem of long and complex cycles in determining the execution engine in existing technologies, this application provides a statement execution method that shortens the cycle of determining the execution engine and reduces its complexity.

[0091] Figure 1This is a flowchart illustrating a statement execution method provided in an embodiment of this application. This method can be applied to one or more electronic devices, such as servers. Furthermore, the execution subject of this method can be hardware or software. When the execution subject is hardware, it can be one or more of the aforementioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the execution subject is software, this method can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are imposed here.

[0092] like Figure 1 As shown, the method specifically includes:

[0093] Step 101: Obtain the statement to be executed, wherein the statement to be executed represents a statement composed of Structured Query Language.

[0094] In this embodiment, Structured Query Language (SQL) can be used for database data querying. The statement to be executed can be used to perform tasks such as adding, deleting, modifying, and querying data in the database.

[0095] Step 102: Determine the target analysis engine from a pre-defined set of analysis engines, wherein the target analysis engine has the ability to parse the statement to be executed.

[0096] In this embodiment, the analysis engine set can be a collection of analysis engines used to interface with the statements to be executed. Typically, the analysis engine set can include multiple analysis engines. For example, the analysis engines may include an OLAP (Online Analytical Processing) engine. Each analysis engine can correspond to at least one execution engine.

[0097] Here, the statement to be executed can be sent to each set of analysis engines in the analysis engine set. If the analysis engine does not generate any errors or other abnormalities after receiving the statement to be executed, then it can be determined that the analysis engine has the ability to parse the statement to be executed.

[0098] In addition, the number of target analysis engines can be one or more.

[0099] Step 103: Determine the data table for the operation indicated by the statement to be executed.

[0100] In this embodiment, since the statement to be executed is used to perform a data query on the database, the statement to be executed contains the table name of the data table to which it indicates the operation.

[0101] Step 104: Based on the target analysis engine and the data table, determine the target execution engine from a pre-determined set of execution engines, wherein the target execution engine is used to execute the statement to be executed.

[0102] In this embodiment, the aforementioned predetermined set of execution engines may be a set of execution engines corresponding to the target analysis engine. Alternatively, the predetermined set of execution engines may also be a set of execution engines corresponding to one or more data centers.

[0103] The engine is an important component of a database system. Its main responsibility is to generate an efficient execution plan from the SQL statements input by the application under the current load scenario, playing a crucial role in the efficient execution of SQL statements.

[0104] As an example, if the predetermined set of execution engines is the set of execution engines corresponding to the target analysis engine, the target execution engine can be determined based on at least one of the distribution of the data table among Availability Zones (AZs) and the storage space occupied by the data table.

[0105] As another example, if the predetermined set of execution engines is a set of execution engines corresponding to one or more data centers, the execution engine corresponding to the target analysis engine can be determined first from the predetermined set of execution engines, and then the target execution engine can be determined from it.

[0106] Step 105: Execute the statement to be executed through the target execution engine.

[0107] In this embodiment, after the statement to be executed is submitted to the target execution engine, the target execution engine can execute the statement to be executed.

[0108] In some optional implementations of this embodiment, the abstract syntax tree of the statement to be executed can also be determined before performing step 102 above.

[0109] An abstract syntax tree (AST) is a tree-like representation of the abstract syntactic structure of the source code (i.e., the statements to be executed mentioned above). Each node in the tree represents a structure in the source code. It's called abstract because the AST doesn't represent every detail of the actual syntax; for example, nested parentheses are implicit in the tree structure and not presented as nodes. The AST doesn't depend on the syntax of the source language, meaning it doesn't rely on the context-agnostic grammar used in the parsing phase. This is because when writing grammars, equivalent transformations are often performed (eliminating left recursion, backtracking, ambiguity, etc.), which introduces redundant components into grammar analysis, negatively impacting subsequent stages and potentially causing confusion. Many compilers often construct their own parse trees to establish a clear interface between the front-end and back-end.

[0110] Based on this, the target analysis engine can be determined from a pre-defined set of analysis engines using the following method:

[0111] Based on the abstract syntax tree, the target analysis engine is determined from a pre-defined set of analysis engines.

[0112] Here, after receiving the SQL task (i.e. the statement to be executed mentioned above) and performing lexical and syntactic parsing, an abstract syntax tree can be obtained. During this process, it will be verified whether the connected analysis engine has parsing capabilities, and candidate analysis engines will be selected, that is, the target analysis engine will be obtained.

[0113] It is understandable that among the above optional implementation methods, the statement to be executed can be analyzed through an abstract syntax tree to obtain the target analysis engine. In this way, through subsequent steps, a more suitable execution engine for processing the statement to be executed can be determined.

[0114] In some application scenarios of the above optional implementation methods, the data table to be operated on by the statement to be executed can be determined in the following way:

[0115] First, based on the abstract syntax tree, it is determined whether the statement to be executed conforms to the preset semantic rules.

[0116] As an example, the aforementioned preset semantic rules can be used to determine whether tables and fields exist in the statement to be executed.

[0117] Subsequently, if the statement to be executed conforms to the preset semantic rules, the data table indicated by the statement to be executed is determined.

[0118] It is understandable that in the above application scenario, the data table to be operated on by the statement to be executed can be determined only if the statement to be executed conforms to the preset semantic rules, and no subsequent steps need to be executed if the statement to be executed does not conform to the preset semantic rules.

[0119] In some optional implementations of this embodiment, the statement to be executed can be obtained in the following manner:

[0120] Obtain the statement to be executed sent by the target terminal.

[0121] The target terminal can generate and send statements to be executed through the client running on it.

[0122] Based on this, after the statement to be executed is executed through the target execution engine, the following steps can also be performed:

[0123] First, determine the execution result of the target execution engine in executing the statement to be executed.

[0124] Then, the execution result is returned to the target terminal.

[0125] Here, after the target execution engine executes the statement to be executed, if an execution result is generated, the execution result can be returned to the target terminal.

[0126] Optionally, after the target terminal collects the execution results, the task connection with the server can be closed.

[0127] It is understandable that, among the above optional implementation methods, the execution results obtained by the execution engine can be fed back to the target terminal to ensure the complete execution of the task indicated by the statement to be executed.

[0128] The statement execution method provided in this application can obtain a statement to be executed, wherein the statement to be executed represents a statement composed of a structured query language. Then, a target analysis engine is determined from a pre-determined set of analysis engines, wherein the target analysis engine has the ability to parse the statement to be executed. Next, a data table indicating the operation of the statement to be executed is determined. Subsequently, based on the target analysis engine and the data table, a target execution engine is determined from a pre-determined set of execution engines, wherein the target execution engine is used to execute the statement to be executed. Finally, the statement to be executed is executed through the target execution engine. Therefore, the execution engine for executing the statement to be executed can be determined based on the target analysis engine and the data table, and the statement to be executed is executed through that execution engine. Thus, a suitable execution engine can be determined without configuring cluster and execution engine information, shortening the cycle of determining the execution engine and reducing the complexity of determining the execution engine.

[0129] Figure 2 This is a flowchart illustrating another statement execution method provided in an embodiment of this application.

[0130] like Figure 2 As shown, the method specifically includes:

[0131] Step 201: Obtain the statement to be executed, wherein the statement to be executed represents a statement composed of Structured Query Language.

[0132] In this embodiment, step 201 and Figure 1 Step 101 in the corresponding embodiment is basically the same, and will not be repeated here.

[0133] Step 202: Determine the target analysis engine from a pre-defined set of analysis engines, wherein the target analysis engine has the ability to parse the statement to be executed.

[0134] In this embodiment, step 202 and Figure 1 Step 102 in the corresponding embodiment is basically the same, and will not be repeated here.

[0135] Step 203: Determine the data table for the operation indicated by the statement to be executed.

[0136] In this embodiment, step 203 and Figure 1 Step 103 in the corresponding embodiment is basically the same, and will not be repeated here.

[0137] Step 204: Determine the clusters of data distribution in the data table to obtain a cluster set, wherein each cluster includes at least one execution engine.

[0138] In this embodiment, the cluster set can be a cluster containing data from the data table.

[0139] Step 205: Determine the target cluster from the cluster set, wherein the target cluster is determined based on the storage space occupied by the data table, the storage cost of the cluster, and the inter-cluster transmission cost.

[0140] In this embodiment, the target cluster can be determined from the set of clusters based on the storage space occupied by the data table, the storage cost of the cluster, and the inter-cluster transmission cost.

[0141] Here, the goal can be to minimize the usage cost of the final target cluster. Based on the storage space occupied by the data table, the storage cost of the cluster, and the transmission cost between clusters, the target cluster can be determined from the set of clusters.

[0142] Furthermore, in cases where cross-cluster transmission is required, it can be determined which direction consumes the least dedicated line resources and has a relatively low time consumption, thereby deciding on the cluster to which the statement to be executed will be submitted, i.e., the target cluster.

[0143] Step 206: Take at least one execution engine included in the target cluster as a pre-determined set of execution engines, and determine a target execution engine from the set of execution engines based on the target analysis engine, wherein the target execution engine is used to execute the statement to be executed.

[0144] In this embodiment, when selecting an execution engine, the hierarchical status of the current user's task (e.g., priority), the task's usage scenario, the engine distribution in the current cluster, and the target analysis engine obtained above can be comprehensively considered to determine the target execution engine from the set of execution engines.

[0145] Step 207: Execute the statement to be executed through the target execution engine.

[0146] In this embodiment, step 207 and Figure 1 Step 105 in the corresponding embodiment is basically the same, and will not be repeated here.

[0147] In some optional implementations of this embodiment, the target execution engine can be determined from the set of execution engines in the following manner:

[0148] First, determine the operation to be executed as indicated by the statement to be executed, and obtain the target operation.

[0149] The target operation can be the operation that the statement to be executed indicates.

[0150] Then, based on the target operation, the target execution engine is determined from the set of execution engines.

[0151] As an example, the target execution engine can be determined from the set of execution engines with the goal of minimizing the execution cost of the final determined target execution engine, based on at least one of the priority of the target operation and the execution scenario.

[0152] It is understandable that among the above optional implementation methods, different target execution engines can be determined for different target operations (such as operations with different priorities or operations with different execution scenarios). In this way, the determined target execution engine can be more suitable for processing the statement to be executed.

[0153] In some application scenarios of the above-mentioned optional implementation methods, the target execution engine can be determined from the set of execution engines based on the target operation in the following manner:

[0154] First, determine the priority and execution scenario of the target operation.

[0155] Next, the engine distribution information of the execution engine set is determined.

[0156] The engine distribution information mentioned above can represent two execution engines.

[0157] Then, based on the priority, the execution scenario, and the engine distribution information, the target execution engine is determined from the set of execution engines.

[0158] Specifically, the selected target execution engine can be located at a short distance from the cluster containing the data in the data table, with short transmission latency and low execution cost.

[0159] It is understandable that in the above application scenarios, the priority of the statement to be executed, the execution scenario, and the engine distribution information can be comprehensively considered to determine the target execution engine. In this way, the selection of the target execution engine can be made to match the execution priority and execution scenario of the statement to be executed. Furthermore, by combining the engine distribution information, the execution cost of the statement to be executed can be reduced.

[0160] It should be noted that, in addition to the contents described above, this embodiment may also include... Figure 1 The corresponding technical features described in the corresponding embodiments, thereby achieving Figure 1 For details on the technical effects of the statement execution method shown, please refer to [link / reference]. Figure 1 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0161] The statement execution method provided in this application embodiment can determine the target execution engine based on the storage space occupied by the data table, the storage cost of the cluster, and the transmission cost between clusters, thereby reducing the execution cost of the statement to be executed.

[0162] The embodiments of this application are described below by way of example. However, it should be noted that the embodiments of this application may have the features described below, but the following description does not constitute a limitation on the protection scope of the embodiments of this application.

[0163] As internet companies expand, they may establish data centers in multiple geographically dispersed regions, hosting massive amounts of data. Simultaneously, with the development of global operations, companies face the need to process and analyze big data across data centers. Currently, companies often employ data replication and synchronization technologies to synchronize and replicate data distributed across different Availability Zones (AZs) to meet computing needs. This approach incurs high cross-data center network bandwidth and costs. Such solutions do not fully consider data distribution and the differences in storage costs across different AZs. In this case, transmitting full data can lead to network bandwidth bottlenecks between data centers, potentially resulting in high network transmission and data storage costs.

[0164] Furthermore, in traditional solutions, users need to pre-configure cluster and execution engine information and submit SQL tasks for execution through the Pilot service. Pilot's primary function is to manage and configure sidecar agent instances deployed within a specific Istio service mesh. It manages routing traffic rules between sidecar agents and configures fault recovery features such as timeouts, retries, and circuit breakers. For cross-AZ analysis requirements, data distributed across other AZs needs to be synchronized to the local cluster before task execution to participate in subsequent computation and analysis.

[0165] Specifically, traditional cross-AZ analysis methods include the following steps:

[0166] Step 1: Analyze the cross-AZ analysis requirements and use third-party data migration tools to complete the cross-AZ data synchronization.

[0167] Step 2: Submit SQL tasks using the unified entry client. You need to configure the cluster where the task is located and the computing engine used.

[0168] Step 3: Collect the execution results (if any) on the client side and close the task connection with the server.

[0169] In this method, users do not need to configure cluster or execution engine information when submitting a task. The Pilot Parser module will automatically parse the SQL, obtain the OLAP engine that can participate in task execution through syntax analysis, obtain the tables and their distribution in the task through semantic analysis based on the unified metadata service, and determine which cluster and engine is most suitable for the SQL task based on cost analysis.

[0170] In this approach, each Availability Zone (AZ) may include at least one OLAP engine (i.e., the analytics engine described above). Each AZ may include at least one cluster (server). Each OLAP engine may include one or more execution engines.

[0171] Figure 3A This is a flowchart illustrating another statement execution method provided in an embodiment of this application.

[0172] like Figure 3A As shown, this method may include the following steps:

[0173] Step 1: Call the SQL task submitted by the entry client (i.e., the statement to be executed mentioned above).

[0174] Step 2: The unified entry service receives the SQL task and performs lexical and syntactic parsing to obtain an abstract syntax tree. During this process, it verifies whether the connected OLAP engine has parsing capabilities, selects candidate OLAP engines, and thus obtains the target analysis engine mentioned above.

[0175] Step 3: Perform semantic analysis on the abstract syntax tree by associating unified metadata. On the one hand, check whether the SQL conforms to the semantic specification (that is, the above-mentioned preset semantic rules, such as whether the table and fields exist). On the other hand, analyze the distribution of the tables in the Availability Zones.

[0176] Step 4: During cost optimization (CBO), combine custom feature analysis to determine costs. First, consider the storage space occupied by tables appearing in the SQL task, the storage cost of the cluster, and which direction consumes the least dedicated line resources and has the lowest latency if cross-cluster transmission is required. This determines the cluster Availability Zone (AZ) where the SQL task will be submitted (i.e., the target cluster mentioned above). When selecting the execution engine, comprehensively consider the task's hierarchy, the task's usage scenario, the engine distribution in the current cluster, and the candidate OLAP engines obtained earlier, thus determining the execution engine to be used by the SQL task, i.e., the target execution engine.

[0177] Step 5: Submit the current SQL task to the cluster after the decision and submit it to the specific engine for execution.

[0178] Step 6: Collect the execution results (if any) on the client side and close the task connection with the server.

[0179] For example, see Figure 3B , Figure 3B This is an illustration of the execution process of a statement to be executed in a statement execution method provided in this application embodiment.

[0180] like Figure 3B As shown, if a user submits a complex batch task, its execution process may include the following steps:

[0181] Step 1: The user develops a complex SQL task using an information platform (such as an advertising platform) and submits it for execution through a unified entry client.

[0182] Step 2: After receiving the SQL task, the unified entry service schedules it to the Parser module for cross-AZ query analysis.

[0183] Step 3: The Parser module performs lexical and syntactic parsing on the SQL, performs semantic analysis by associating it with unified metadata, and combines data storage / computing costs, dedicated line resources, task tiering, and other comprehensive analysis to determine the best scheduling Availability Zone (AZ) and execution engine for the SQL task.

[0184] Step 4: The Parser feeds back the decision information to the unified entry server, and then schedules the SQL task to be executed on the cluster.

[0185] Step 5: Collect the execution results (if any) on the client side and close the task connection with the server.

[0186] In addition, please refer to Figure 3C , Figure 3C This is a system block diagram related to a statement execution method provided in an embodiment of this application.

[0187] It should be noted that, in addition to the contents described above, this embodiment may also include the technical features described in the above embodiments, thereby achieving the technical effect of the statement execution method shown above. Please refer to the above description for details. For the sake of brevity, it will not be elaborated here.

[0188] The statement execution method provided in this application embodiment can be integrated into the unified entry service Pilot and deployed on the upper layer of the OLAP engine. It uses the Apache Calcite data management framework as the core and extends it for development. Taking into account factors such as data / task classification, response latency, storage / computing costs, and dedicated line resources, it distributes SQL tasks to the AZ cluster with the lowest cost and selects the most suitable execution engine to run the task in the scenario.

[0189] Figure 4 This is a schematic diagram of the structure of a statement execution device provided in an embodiment of this application. Specifically, it includes:

[0190] The acquisition unit 401 is used to acquire the statement to be executed, wherein the statement to be executed represents a statement composed of a structured query language.

[0191] The first determining unit 402 is used to determine a target analysis engine from a pre-determined set of analysis engines, wherein the target analysis engine has the ability to parse the statement to be executed;

[0192] The second determining unit 403 is used to determine the data table of the operation indicated by the statement to be executed;

[0193] The third determining unit 404 is used to determine a target execution engine from a predetermined set of execution engines based on the target analysis engine and the data table, wherein the target execution engine is used to execute the statement to be executed;

[0194] The execution unit 405 is used to execute the statement to be executed through the target execution engine.

[0195] In one possible implementation, before determining the target analysis engine from a predetermined set of analysis engines, the apparatus further includes:

[0196] The fourth determining unit (not shown in the figure) is used to determine the abstract syntax tree of the statement to be executed; and

[0197] The step of determining the target analysis engine from a pre-determined set of analysis engines includes:

[0198] The fifth determining unit (not shown in the figure) is used to determine the target analysis engine from a predetermined set of analysis engines based on the abstract syntax tree.

[0199] In one possible implementation, determining the data table indicating the operation of the statement to be executed includes:

[0200] Based on the abstract syntax tree, determine whether the statement to be executed conforms to the preset semantic rules;

[0201] If the statement to be executed conforms to the preset semantic rules, the data table to be operated on by the statement to be executed is determined.

[0202] In one possible implementation, determining the target execution engine from a predetermined set of execution engines based on the target analysis engine and the data table includes:

[0203] Determine the clusters of data distribution in the data table to obtain a cluster set, wherein each cluster includes at least one execution engine;

[0204] From the set of clusters, a target cluster is determined, wherein the target cluster is determined based on the storage space occupied by the data table, the storage cost of the cluster, and the inter-cluster transmission cost;

[0205] The target cluster includes at least one execution engine as a pre-determined set of execution engines, and the target execution engine is determined from the set of execution engines based on the target analysis engine.

[0206] In one possible implementation, determining the target execution engine from the set of execution engines includes:

[0207] The operation indicated by the statement to be executed is determined, and the target operation is obtained;

[0208] Based on the target operation, a target execution engine is determined from the set of execution engines.

[0209] In one possible implementation, determining the target execution engine from the set of execution engines based on the target operation includes:

[0210] Determine the priority and execution scenario of the target operation;

[0211] Determine the engine distribution information of the execution engine set;

[0212] Based on the priority, the execution scenario, and the engine distribution information, the target execution engine is determined from the set of execution engines.

[0213] In one possible implementation, obtaining the statement to be executed includes:

[0214] Obtain the statement to be executed sent by the target terminal; and

[0215] After the statement to be executed is executed by the target execution engine, the apparatus further includes:

[0216] The sixth determining unit (not shown in the figure) is used to determine the execution result of the target execution engine executing the statement to be executed;

[0217] A return unit (not shown in the figure) is used to return the execution result to the target terminal.

[0218] The statement execution device provided in this embodiment can be as follows: Figure 4 The statement execution device shown can execute all the steps of the statement execution methods described above, thereby achieving the technical effects of the statement execution methods described above. For details, please refer to the relevant descriptions above. For the sake of brevity, it will not be elaborated here.

[0219] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 The illustrated electronic device 500 includes at least one processor 501, a memory 502, at least one network interface 504, and other user interfaces 503. The various components in the electronic device 500 are coupled together via a bus system 505. It is understood that the bus system 505 is used to implement communication between these components. In addition to a data bus, the bus system 505 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 5 The general designated all buses as Bus System 505.

[0220] The user interface 503 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0221] It is understood that the memory 502 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0222] In some implementations, memory 502 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 5021 and application program 5022.

[0223] The operating system 5021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 5022 includes various applications, such as a media player and a browser, used to implement various application functions. Programs implementing the methods of this application embodiment can be included in application program 5022.

[0224] In this embodiment, by calling the program or instructions stored in memory 502, specifically the program or instructions stored in application program 5022, processor 501 executes the method steps provided in each method embodiment, including, for example:

[0225] Obtain the statement to be executed, wherein the statement to be executed represents a statement composed of Structured Query Language;

[0226] From a pre-determined set of analysis engines, a target analysis engine is determined, wherein the target analysis engine has the ability to parse the statement to be executed;

[0227] Determine the data table to be operated on by the statement to be executed;

[0228] Based on the target analysis engine and the data table, a target execution engine is determined from a pre-determined set of execution engines, wherein the target execution engine is used to execute the statement to be executed;

[0229] The statement to be executed is executed through the target execution engine.

[0230] The methods disclosed in the embodiments of this application can be applied to or implemented by processor 501. Processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 501 or by instructions in the form of software. The processor 501 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 502. Processor 501 reads the information in memory 502 and, in conjunction with its hardware, completes the steps of the above method.

[0231] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described above in this application, or combinations thereof.

[0232] For software implementation, the techniques described herein can be implemented by units that perform the functions described above. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or external to the processor.

[0233] The electronic device provided in this embodiment may be as follows: Figure 5 The electronic device shown can execute all the steps of the statement execution methods described above, thereby achieving the technical effects of the statement execution methods described above. For details, please refer to the relevant descriptions above. For the sake of brevity, further details are not provided here.

[0234] This application also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; it may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and it may also include combinations of the above types of memory.

[0235] When one or more programs in the storage medium can be executed by one or more processors to implement the statement execution method described above on the electronic device side.

[0236] The processor described above is used to execute a statement execution program stored in memory to implement the following steps of a statement execution method executed on the electronic device side:

[0237] Obtain the statement to be executed, wherein the statement to be executed represents a statement composed of Structured Query Language;

[0238] From a pre-determined set of analysis engines, a target analysis engine is determined, wherein the target analysis engine has the ability to parse the statement to be executed;

[0239] Determine the data table to be operated on by the statement to be executed;

[0240] Based on the target analysis engine and the data table, a target execution engine is determined from a pre-determined set of execution engines, wherein the target execution engine is used to execute the statement to be executed;

[0241] The statement to be executed is executed through the target execution engine.

[0242] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0243] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0244] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0245] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for executing statements, characterized in that, The method includes: Obtain the statement to be executed, wherein the statement to be executed represents a statement composed of Structured Query Language; Determine the abstract syntax tree of the statement to be executed; Determining a target analysis engine from a pre-defined set of analysis engines includes: determining the target analysis engine from the pre-defined set of analysis engines based on the abstract syntax tree; wherein the target analysis engine has the ability to parse the statement to be executed; Determining the data table for the operation indicated by the statement to be executed includes: determining whether the statement to be executed conforms to a preset semantic rule based on the abstract syntax tree; and determining the data table for the operation indicated by the statement to be executed if the statement to be executed conforms to the preset semantic rule. Based on the target analysis engine and the data table, determining a target execution engine from a predetermined set of execution engines includes: determining clusters of data distribution in the data table to obtain a cluster set, wherein each cluster includes at least one execution engine; determining a target cluster from the cluster set, wherein the target cluster is determined based on the storage space occupied by the data table, the storage cost of the cluster, and the inter-cluster transmission cost; using at least one execution engine included in the target cluster as a predetermined set of execution engines; and determining a target execution engine from the set of execution engines based on the target analysis engine, wherein the target execution engine is used to execute the statement to be executed; the predetermined set of execution engines is: a set of execution engines corresponding to the target analysis engine; or, a predetermined set of execution engines; or, a set of execution engines corresponding to one or more data centers; The statement to be executed is executed through the target execution engine.

2. The method according to claim 1, characterized in that, The step of determining the target execution engine from the set of execution engines includes: The operation indicated by the statement to be executed is determined, and the target operation is obtained; Based on the target operation, a target execution engine is determined from the set of execution engines.

3. The method according to claim 2, characterized in that, The step of determining the target execution engine from the set of execution engines based on the target operation includes: Determine the priority and execution scenario of the target operation; Determine the engine distribution information of the execution engine set; Based on the priority, the execution scenario, and the engine distribution information, the target execution engine is determined from the set of execution engines.

4. The method according to any one of claims 1-3, characterized in that, The process of obtaining the statement to be executed includes: Obtain the statement to be executed sent by the target terminal; and After executing the statement to be executed through the target execution engine, the method further includes: Determine the execution result of the target execution engine in executing the statement to be executed; The execution result is returned to the target terminal.

5. A statement execution device, characterized in that, The device includes: The acquisition unit is used to acquire the statement to be executed, wherein the statement to be executed represents a statement composed of a structured query language. The fourth determining unit is used to determine the abstract syntax tree of the statement to be executed; The first determining unit is configured to determine a target analysis engine from a pre-determined set of analysis engines, wherein the target analysis engine has the ability to parse the statement to be executed; The fifth determining unit is used to determine the target analysis engine from a predetermined set of analysis engines based on the abstract syntax tree; The second determining unit is used to determine the data table indicating the operation of the statement to be executed, including: determining whether the statement to be executed conforms to a preset semantic rule based on the abstract syntax tree; and determining the data table indicating the operation of the statement to be executed if the statement to be executed conforms to the preset semantic rule. The third determining unit is configured to determine a target execution engine from a pre-determined set of execution engines based on the target analysis engine and the data table. Specifically, it is configured to: determine the clusters of data distribution in the data table to obtain a cluster set, wherein each cluster includes at least one execution engine; determine a target cluster from the cluster set, wherein the target cluster is determined based on the storage space occupied by the data table, the storage cost of the cluster, and the inter-cluster transmission cost; use at least one execution engine included in the target cluster as a pre-determined set of execution engines; and determine a target execution engine from the set of execution engines based on the target analysis engine, wherein the target execution engine is used to execute the statement to be executed; the pre-determined set of execution engines is: a set of execution engines corresponding to the target analysis engine; or, a pre-determined set of execution engines; or, a set of execution engines corresponding to one or more data centers; An execution unit is used to execute the statement to be executed through the target execution engine.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing a computer program stored in the memory, wherein when the computer program is executed, it implements the method described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-4.

Images