Method and device for processing structured query statements in database multi-cluster mode

Abstract

The present invention provides a method, device, equipment and storage medium for processing structured query statements in a database multi-cluster mode, which can be applied to the field of database technology. The method for processing structured query statements in a database multi-cluster mode includes: in response to receiving a structured query statement, parsing the structured query statement, generating an original execution plan of the structured query statement, wherein the original execution plan includes multiple original execution steps; determining a target original execution step from the multiple original execution steps, wherein the target original execution step includes a custom function; in response to determining that the first cluster for executing the target original execution step does not include the custom function, adding a function creation step before the target original execution step to obtain a target execution plan, wherein the function creation step is used to create a temporary function in the first cluster according to the custom function; and executing the target execution plan.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and more specifically to a method, device, equipment and medium for processing structured query statements in a database multi-cluster mode. Background Art

[0002] With the widespread application of databases, the inherent functions in databases can no longer meet the complex data processing needs of users. Therefore, databases usually allow users to create custom functions based on data processing needs.

[0003] However, in the multi-cluster mode of the database, users can only create custom functions in a single cluster, which means that users cannot use custom functions when performing queries involving multiple clusters. Summary of the invention

[0004] In view of the above problems, the present invention provides a method, apparatus, device, medium and program product for processing structured query statements in a database multi-cluster mode.

[0005] According to a first aspect of the present invention, a method for processing structured query statements in a database multi-cluster mode is provided, comprising: in response to receiving a structured query statement, parsing the structured query statement to generate an original execution plan for the structured query statement, wherein the structured query statement includes a custom function and the original execution plan includes multiple original execution steps; determining a target original execution step from the multiple original execution steps, wherein the target original execution step includes the custom function; in response to determining that the first cluster used to execute the target original execution step does not include the custom function, adding a function creation step before the target original execution step to obtain a target execution plan, wherein the function creation step is used to create a temporary function in the first cluster according to the custom function; and executing the target execution plan.

[0006] According to an embodiment of the present invention, the above-mentioned generation of the original execution plan for the above-mentioned structured query statement includes: obtaining definition information of the above-mentioned custom function and a cluster identifier of a second cluster for storing the above-mentioned custom function; generating a function call instance according to the above-mentioned definition information and the cluster identifier of the above-mentioned second cluster; and in response to determining that the above-mentioned field includes the custom function, generating original execution steps corresponding to the above-mentioned field based on the above-mentioned function call instance.

[0007] According to an embodiment of the present invention, the method further includes: when it is determined that the cluster identifier of the first cluster is the same as the cluster identifier of the second cluster, determining that the first cluster includes the custom function; and when it is determined that the cluster identifier of the first cluster is different from the cluster identifier of the second cluster, determining that the first cluster does not include the custom function.

[0008] According to an embodiment of the present invention, the above-mentioned adding a function creation step before the above-mentioned original execution step to obtain the target execution plan includes: adding the above-mentioned function creation step before the first original execution step among the above-mentioned multiple original execution steps; adding a function destruction step after the last original execution step among the above-mentioned multiple original execution steps, wherein the above-mentioned function destruction step is used to destroy the above-mentioned temporary function; and obtaining the above-mentioned target execution plan by combining the above-mentioned multiple original execution steps, the above-mentioned function creation step and the above-mentioned function destruction step.

[0009] According to an embodiment of the present invention, the executing the target execution plan includes: in response to detecting that an exception occurs during the execution of the function destroying step, not throwing the exception.

[0010] According to an embodiment of the present invention, the method further includes: in response to triggering a timing event, deleting a temporary function created in the first cluster within a predetermined period of time.

[0011] According to an embodiment of the present invention, the method further includes: determining a second function identifier of the temporary function according to the first cluster identifier of the first cluster, the first function identifier of the custom function, the query identifier of the structured query statement and the creation time of the temporary function.

[0012] The second aspect of the present invention provides a structured query statement processing device in a database multi-cluster mode, including: a parsing module, which is used to parse the structured query statement in response to receiving a structured query statement, and generate an original execution plan for the structured query statement, wherein the structured query statement includes a custom function, and the original execution plan includes multiple original execution steps; a determination module, which is used to determine a target original execution step from the multiple original execution steps, wherein the target original execution step includes the custom function; an adding module, which is used to add a function creation step before the target original execution step in response to determining that the first cluster used to execute the target original execution step does not include the custom function, to obtain a target execution plan, wherein the function creation step is used to create a temporary function in the first cluster according to the custom function; and an execution module, which is used to execute the target execution plan.

[0013] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0014] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0015] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0016] According to an embodiment of the present invention, by adding a function creation step for creating a temporary function in the first cluster in the original execution plan, the first cluster can use the temporary function to complete the target original execution step, thereby ensuring the normal execution of the structured query statement and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0018] Figure 1 An application scenario diagram is shown of a method, apparatus, device, medium, and program product for processing structured query statements in a database multi-cluster mode according to an embodiment of the present invention.

[0019] Figure 2 A flow chart of a method for processing structured query statements in a database multi-cluster mode according to an embodiment of the present invention is shown.

[0020] Figure 3 A flowchart of generating a target execution plan according to an embodiment of the present invention is shown.

[0021] Figure 4 A schematic diagram of obtaining a target execution plan according to an embodiment of the present invention is shown.

[0022] Figure 5 A structural block diagram of a structured query statement processing device in a database multi-cluster mode according to an embodiment of the present invention is shown.

[0023] Figure 6 A block diagram of an electronic device suitable for implementing a method for processing structured query statements in a database multi-cluster mode according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "include", "comprises", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0029] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present invention provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0030] The multi-cluster mode of a database is usually achieved by dividing multiple computing nodes into multiple computing clusters, each of which includes multiple computing nodes. In actual applications, database users are usually associated with one of the multiple computing clusters. For example, employees in a business department are usually associated with the computing cluster used by the business department. Therefore, when users create custom functions, they can usually only create custom functions under the computing cluster associated with the user. However, when users use structured query statements to query other computing clusters, because other computing clusters do not include the definition of custom functions, other computing clusters cannot recognize the custom functions in the structured query statements, and errors will be generated, which will prevent users from using custom functions under other clusters.

[0031] An embodiment of the present invention provides a method for processing structured query statements in a database multi-cluster mode, comprising: in response to receiving a structured query statement, parsing the structured query statement, generating an original execution plan for the structured query statement, wherein the structured query statement includes a custom function, and the original execution plan includes multiple original execution steps; determining a target original execution step from the multiple original execution steps, wherein the target original execution step includes a custom function; in response to determining that the first cluster used to execute the target original execution step does not include the custom function, adding a function creation step before the target original execution step to obtain a target execution plan, wherein the function creation step is used to create a temporary function in the first cluster according to the custom function; and executing the target execution plan. This solution adds a function creation step when creating an execution plan, so that when the first cluster executes the target execution plan, it can create a temporary function equivalent to the custom function in the first cluster according to the function creation step, and call the temporary function to complete the query.

[0032] Figure 1 An application scenario diagram is shown of a method, apparatus, device, medium, and program product for processing structured query statements in a database multi-cluster mode according to an embodiment of the present invention.

[0033] like Figure 1As shown, the application scenario 100 according to this embodiment may include a terminal device 110, a network 120 and a database cluster 130, and the database cluster 130 includes a scheduling cluster 131, a management cluster 132, a first computing cluster 133_1 and a second computing cluster 133_2. The scheduling cluster 131 may include multiple scheduling nodes, such as scheduling node 1 and scheduling node 2. The management cluster 132 may include multiple management nodes, such as management node 1 and management node 2. The first computing cluster 133_1 may include multiple computing nodes, such as computing node 1 and computing node 2. The second computing cluster 133_2 may include multiple computing nodes, such as computing node 3 and computing node 4.

[0034] The network 120 is used to provide a medium for a communication link between the terminal device 110 and the database cluster 130. The network 120 may include various connection types, such as wired or wireless communication links or optical fiber cables.

[0035] The user can use the terminal device 110 to interact with the database cluster 130 through the network 120 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 110, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0036] The terminal device 110 may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0037] The scheduling cluster 131 is used to parse, optimize, generate an execution plan for the structured query statement sent by the terminal device 110 , and schedule the first computing cluster 133_1 and the second computing cluster 133_2 .

[0038] The management cluster 132 is used to realize information sharing between the scheduling node 1 and the scheduling node 2 in the scheduling cluster 131, including information such as computing node status and computing node resource status.

[0039] Computing nodes 1 and 2 in the first computing cluster 131_1 and computing nodes 3 and 4 in the second computing cluster 131_2 are used to store data, receive and execute the decomposed execution plan from the scheduling cluster 131 , and return the execution result to the scheduling cluster 131 .

[0040] It should be noted that the structured query statement processing method under the database multi-cluster mode provided in the embodiment of the present invention can generally be executed by the database cluster 130. Accordingly, the structured query statement processing device under the database multi-cluster mode provided in the embodiment of the present invention can generally be set in the database cluster 130. The structured query statement processing method under the database multi-cluster mode provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the database cluster 130 and can communicate with the terminal device 110 and / or the database cluster 130. Correspondingly, the structured query statement processing device under the database multi-cluster mode provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the database cluster 130 and can communicate with the terminal device 110 and / or the database cluster 130.

[0041] It should be understood that Figure 1 The number of terminal devices, networks, databases, scheduling nodes, management nodes, computing clusters, and computing nodes in the embodiment is only for illustration purposes. Any number of terminal devices, networks, databases, scheduling nodes, management nodes, computing clusters, and computing nodes may be provided as required.

[0042] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The structured query statement processing method in the database multi-cluster mode according to an embodiment of the present invention is described in detail.

[0043] Figure 2 A flow chart of a method for processing structured query statements in a database multi-cluster mode according to an embodiment of the present invention is shown.

[0044] like Figure 2 As shown, the method for processing structured query statements in the database multi-cluster mode of this embodiment includes operations S210 to S240.

[0045] In operation S210 , in response to receiving the structured query statement, the structured query statement is parsed to generate an original execution plan of the structured query statement.

[0046] According to an embodiment of the present invention, a structured query statement includes a custom function, which is usually created by a database user to expand the functionality of the database. Specifically, in a multi-cluster mode of a database, the database includes multiple computing clusters. When a user creates a custom function, the database usually stores the definition of the custom function in a computing cluster associated with the user.

[0047] According to an embodiment of the present invention, the original execution plan includes a plurality of original execution steps. The process of parsing the structured query statement and generating the original execution plan is usually completed by a scheduling node in a scheduling cluster. When parsing the structured query statement, the scheduling node needs to determine at least one computing cluster involved in the structured query statement and determine the original execution steps of each computing cluster to obtain the original execution plan.

[0048] In operation S220, a target original execution step is determined from among the plurality of original execution steps.

[0049] According to an embodiment of the present invention, the target original execution step may be an original execution step including a user-defined function.

[0050] According to an embodiment of the present invention, after the original execution plan is generated, the scheduling node traverses each original execution step in the original execution plan, and when it is determined that the current original execution step includes a custom function, the original execution step is determined to be a target original execution step.

[0051] In operation S230 , in response to determining that the first cluster used to execute the target original execution step does not include the custom function, a function creation step is added before the target original execution step to obtain a target execution plan.

[0052] According to an embodiment of the present invention, when the target original execution step involves multiple clusters, the cluster used to execute the target original execution step is usually determined first, and data of other clusters in the multiple clusters are sent to the cluster used to execute the target original execution step, so that the cluster executes the target original execution step.

[0053] According to an embodiment of the present invention, the first cluster may be a computing cluster for executing the target original execution step. Since the custom function is not defined in all computing clusters, after determining the target original execution step, the scheduling node needs to determine whether the first cluster for executing the target original execution step includes the custom function, so as to avoid meaningless calls caused by the first cluster not including the custom function.

[0054] According to an embodiment of the present invention, the function creation step is used to create a temporary function in the first cluster according to the custom function. Since the temporary function is created according to the custom function, the temporary function has the same function as the custom function.

[0055] In operation S240 , the target execution plan is executed.

[0056] According to an embodiment of the present invention, when executing the target execution plan, since the function creation step is before the target original execution step, when the first cluster executes the target original execution step, it has already created a temporary function with the same function as the custom function through the function creation step, and can use the temporary function to execute the target original execution step.

[0057] According to an embodiment of the present invention, after executing the target execution plan, a query result of the structured query statement is obtained, and the query result is returned to the client that issued the structured query statement.

[0058] According to an embodiment of the present invention, by adding a function creation step for creating a temporary function in the first cluster in the original execution plan, the first cluster can use the temporary function to complete the target original execution step, thereby ensuring the normal execution of the structured query statement and improving the user experience.

[0059] According to an embodiment of the present invention, generating an original execution plan for a structured query statement includes: obtaining definition information of a custom function and a cluster identifier of a second cluster for storing the custom function; generating a function call instance according to the definition information and the cluster identifier of the second cluster; and in response to determining that a field includes the custom function, generating original execution steps corresponding to the field based on the function call instance.

[0060] According to an embodiment of the present invention, the definition information of the custom function and the cluster identifier of the second cluster for storing the custom function can be stored in a system table of the database, and specifically, the system table can be stored in the management cluster. When obtaining the definition information of the custom function, a query can be performed in the system table based on the function identifier of the custom function in the structured query statement.

[0061] According to an embodiment of the present invention, the function call class includes at least definition information of the user-defined function and a cluster identifier of the second cluster.

[0062] According to an embodiment of the present invention, a structured query statement may include multiple fields. When generating original execution steps for each field, if the field includes a custom function, the function call instance may be recorded in the original execution steps of the field.

[0063] According to an embodiment of the present disclosure, a function call instance may be created according to a function call class based on definition information and a cluster identifier of a second cluster, and the generated function call instance corresponds to a field.

[0064] According to an embodiment of the present invention, in the process of generating an original execution plan, a function call instance is written into an original execution step, thereby generating an original execution step corresponding to a field.

[0065] According to an embodiment of the present disclosure, by traversing the original execution plan based on a function call instance, it is convenient to determine whether the original execution steps include a custom function based on the function call instance, and implement the function call using the function call instance during the plan execution phase.

[0066] According to an embodiment of the present invention, before adding a function creation step before the target original execution step, it also includes: when it is determined that the cluster identifier of the first cluster is the same as the cluster identifier of the second cluster, determining that the first cluster includes the custom function; and when it is determined that the cluster identifier of the first cluster is different from the cluster identifier of the second cluster, determining that the first cluster does not include the custom function.

[0067] According to an embodiment of the present invention, when generating an original execution plan, it is also necessary to determine the first cluster that executes the original execution step, and record the cluster identifier of the first cluster used to execute the original execution step.

[0068] According to an embodiment of the present invention, since the function call class has been written into the target original execution step, and since the function call class includes the cluster identifier of the second cluster for storing the custom function, the cluster identifier of the first cluster and the cluster identifier of the second cluster can be obtained from the target original execution step.

[0069] According to an embodiment of the present invention, when the cluster identifier of the first cluster and the cluster identifier of the second cluster are the same, it indicates that the first cluster and the second cluster are actually the same cluster, so it can be determined that the first cluster includes the custom function.

[0070] According to an embodiment of the present invention, when the cluster identifier of the first cluster is different from the cluster identifier of the second cluster, it indicates that the first cluster and the second cluster are two different clusters, and since the custom function is only stored in the second cluster, it can be determined that the first cluster does not include the custom function.

[0071] According to an embodiment of the present invention, by judging the identifiers of the first cluster and the second cluster, it is determined whether the custom function is called in the cluster defined for storing the custom function, thereby avoiding call failure and improving user experience.

[0072] Figure 3 A flowchart of generating a target execution plan according to an embodiment of the present invention is shown.

[0073] like Figure 3 As shown, generating a target execution plan includes operations S310 to S350.

[0074] In operation S310 , a target original execution step including a user-defined function is determined from among a plurality of original execution steps.

[0075] According to an embodiment of the present invention, after the original execution plan is generated, the original execution plan is traversed to determine a plurality of original execution steps including user-defined functions as target original execution steps.

[0076] In operation S320 , a cluster identifier of the first cluster and a cluster identifier of the second cluster are obtained from the target original execution step.

[0077] According to an embodiment of the present invention, the first cluster is a cluster that executes the original execution steps of the target, and the second cluster is a cluster that includes a user-defined function.

[0078] In operation S330, it is determined whether the cluster identifier of the first cluster is the same as the cluster identifier of the second cluster. If the cluster identifier of the first cluster is the same as the cluster identifier of the second cluster, operation S350 is performed, otherwise operation S340 is performed.

[0079] According to an embodiment of the present invention, when the cluster identifier of the first cluster and the cluster identifier of the second cluster are the same, it indicates that the first cluster used to execute the target original execution step includes a custom function, and there is no need to create a temporary function. Therefore, there is no need to add a function creation step based on the original execution plan, and the original execution plan is directly determined as the target execution plan.

[0080] According to an embodiment of the present invention, when the cluster identifier of the first cluster is different from the cluster identifier of the second cluster, it indicates that the first cluster used to execute the target original execution step does not include a custom function and a temporary function needs to be created. Therefore, a function creation step needs to be added based on the original execution plan.

[0081] In operation S340, a function creation step is added.

[0082] According to an embodiment of the present invention, a function creation step is added to the original execution plan to obtain a target execution plan.

[0083] In operation S350 , a target execution plan is obtained.

[0084] According to an embodiment of the present invention, adding a function creation step before an original execution step to obtain a target execution plan includes: adding a function creation step before the first original execution step among multiple original execution steps; adding a function destruction step after the last original execution step among the multiple original execution steps; and obtaining a target execution plan by combining the multiple original execution steps, the function creation step and the function destruction step.

[0085] According to an embodiment of the present invention, the function creation step is used to create a temporary function in the first cluster, and the function destruction step is used to destroy the temporary function created in the first cluster.

[0086] According to an embodiment of the present invention, since the utilization rate of the temporary function in the first cluster is low, a function destruction step for destroying the temporary function may be added to prevent the temporary function from occupying a large amount of storage resources.

[0087] According to an embodiment of the present invention, in order to avoid affecting multiple original execution steps in the original execution plan, the function creation step is added before the first original execution step in the original execution plan, and the function destruction step is added after the last original execution step in the original execution plan to ensure that the data redistribution logic is not changed.

[0088] According to an embodiment of the present disclosure, a function creation step, a plurality of original execution steps, and a function destruction step are combined to obtain a target execution plan.

[0089] According to an embodiment of the present invention, by adding a function creation step and a function destruction step before and after the original execution plan respectively, the impact on the data redistribution logic is avoided.

[0090] Figure 4 A schematic diagram of obtaining a target execution plan according to an embodiment of the present invention is shown.

[0091] like Figure 4 As shown, the original execution plan includes original execution step 1, original execution step 2 and original execution step 3.

[0092] According to an embodiment of the present invention, a function creation step is added before the original execution plan, and a function destruction step is added after the original execution plan to obtain a target execution plan, wherein the target execution plan includes a function creation step, original execution step 1, original execution step 2, original execution step 3 and a function destruction step.

[0093] According to an embodiment of the present invention, executing the target execution plan includes: in response to detecting that an exception occurs during the process of executing the function destruction step, not throwing an exception.

[0094] According to an embodiment of the present invention, the target execution plan includes a function creation step, an original execution step and a function destruction step. When executing the target execution plan, each step needs to be sent to a cluster for executing the step so as to execute the step in the cluster.

[0095] According to an embodiment of the present invention, during the execution of the target execution plan, the function creation step is first executed to create a temporary function in the first cluster, and then the original execution step is executed to perform operations such as data query and processing. For the target original execution step in the original execution step, it is necessary to call the temporary function that has been created in the first cluster to perform operations such as data processing, and finally the function destruction step is executed to destroy the temporary function in the first cluster.

[0096] According to an embodiment of the present invention, during the execution of the function destruction step, an exception may occur, such as a function destruction failure. However, since the original execution step has been successfully executed, that is, the execution of the structured query statement has actually been successfully completed, the exception that occurs in the function destruction step will not affect the query result of the structured query statement, and there is no need to throw the exception.

[0097] According to an embodiment of the present invention, when an exception is detected in a function destruction step, no exception is thrown, thereby achieving the purpose of making the user unaware of the temporary function creation and destruction process, thereby improving the user experience.

[0098] According to an embodiment of the present invention, the query statement processing method further includes: in response to triggering a timing event, deleting a temporary function created in the first cluster within a predetermined period of time.

[0099] According to an embodiment of the present invention, a timer event may be used to delete a temporary function created in the first cluster.

[0100] According to an embodiment of the present invention, since an exception may occur in the function destruction step in the target execution plan, resulting in the failure of destruction of the temporary function, in order to avoid the temporary function that failed to be destroyed occupying a large amount of storage resources, the temporary function that failed to be destroyed can be deleted from the first cluster by using a timed event.

[0101] According to an embodiment of the present invention, the predetermined period is determined according to the frequency of the timed event. Specifically, the user can set the frequency of the timed event. For example, if the frequency is set to 10 days, temporary functions generated in the first cluster within the 10 days are cleared every 10 days.

[0102] According to an embodiment of the present invention, determining the function identifier of the temporary function includes: determining the second function identifier of the temporary function according to the first cluster identifier of the first cluster, the first function identifier of the custom function, the query identifier of the structured query statement and the creation time of the temporary function.

[0103] According to an embodiment of the present invention, the first function identifier may be a function identifier of a user-defined function, and the second function identifier may be a function identifier of a temporary function.

[0104] According to an embodiment of the present invention, in order to ensure the uniqueness of the function identifier of a temporary function and avoid conflicts between multiple temporary functions or between a temporary function and a user-defined function, an identifier naming rule for the temporary function may be set.

[0105] According to an embodiment of the present invention, the second function identifier may be determined based on the first cluster identifier, the first function identifier, the query identifier, and the creation time.

[0106] According to an embodiment of the present invention, by setting the first cluster identifier in the second function identifier, it is possible to distinguish temporary functions from other clusters; by setting the first function identifier in the second function identifier, it is possible to distinguish temporary functions from other temporary functions in the first cluster; by setting the query identifier in the second function identifier, it is possible to distinguish temporary functions from other query statements; by setting the creation time in the second function identifier, it is possible to distinguish temporary functions from other times.

[0107] According to an embodiment of the present invention, by setting an identifier naming rule for a temporary function, a second function identifier of the temporary function is determined so that the second function identifier is unique and conflicts between temporary function identifiers are avoided.

[0108] Based on the above-mentioned structured query statement processing method in the database multi-cluster mode, the present invention also provides a structured query statement processing device in the database multi-cluster mode. Figure 5 The device is described in detail.

[0109] Figure 5 A structural block diagram of a structured query statement processing device in a database multi-cluster mode according to an embodiment of the present invention is shown.

[0110] like Figure 5 As shown, the structured query statement processing device 500 in the database multi-cluster mode of this embodiment includes a parsing module 510 , a determining module 520 , an adding module 530 and an executing module 540 .

[0111] The parsing module 510 is used to parse the structured query statement in response to receiving the structured query statement, and generate an original execution plan of the structured query statement, wherein the structured query statement includes a custom function, and the original execution plan includes multiple original execution steps. In one embodiment, the parsing module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0112] The determination module 520 is used to determine a target original execution step from a plurality of original execution steps, wherein the target original execution step includes a user-defined function. In one embodiment, the determination module 520 can be used to perform the operation S220 described above, which will not be described in detail herein.

[0113] The adding module 530 is used for adding a function creation step before the target original execution step in response to determining that the first cluster used to execute the target original execution step does not include the custom function, so as to obtain a target execution plan, wherein the function creation step is used to create a temporary function in the first cluster according to the custom function. In one embodiment, the adding module 530 can be used to perform the operation S230 described above, which will not be described in detail here.

[0114] The execution module 540 is used to execute the target execution plan. In one embodiment, the execution module 540 can be used to execute the operation S240 described above, which will not be described in detail here.

[0115] According to an embodiment of the present invention, the structured query statement includes fields, and the parsing module 510 includes an information acquisition submodule, a call generation submodule, and a step generation submodule.

[0116] The information acquisition submodule is used to acquire definition information of the user-defined function and a cluster identifier of a second cluster for storing the user-defined function.

[0117] The call generation submodule is used to generate a function call instance according to the definition information and the cluster identifier of the second cluster.

[0118] The step generation submodule is used for generating original execution steps corresponding to the field based on the function call instance in response to determining that the field includes a custom function.

[0119] According to an embodiment of the present invention, the structured query statement processing device 500 in the database multi-cluster mode further includes a first determination module and a second determination submodule.

[0120] The first determining submodule is used to determine that the first cluster includes the user-defined function when it is determined that the cluster identifier of the first cluster is the same as the cluster identifier of the second cluster.

[0121] The second determining submodule is used to determine that the first cluster does not include the user-defined function when it is determined that the cluster identifier of the first cluster is different from the cluster identifier of the second cluster.

[0122] According to an embodiment of the present invention, the adding module 530 includes a creating adding submodule, a destroying adding submodule and a plan generating submodule.

[0123] The creation and addition submodule is used to add a function creation step before the first original execution step among multiple original execution steps.

[0124] The destruction adding submodule is used to add a function destruction step after the last original execution step among multiple original execution steps, wherein the function destruction step is used to destroy a temporary function.

[0125] The plan generation submodule is used to combine a plurality of original execution steps, function creation steps and function destruction steps to obtain a target execution plan.

[0126] According to an embodiment of the present invention, the execution module 540 includes an exception submodule.

[0127] The exception submodule is used to respond to detecting an exception in the process of executing the function destruction step without throwing an exception.

[0128] According to an embodiment of the present invention, the structured query statement processing device 500 in the database multi-cluster mode further includes a deletion module.

[0129] The deletion module is used to delete the temporary function created in the first cluster within a predetermined period in response to triggering a timing event.

[0130] According to an embodiment of the present invention, the structured query statement processing apparatus 500 further includes an identification determination module.

[0131] The identifier determination module is used to determine a second function identifier of the temporary function according to the first cluster identifier of the first cluster, the first function identifier of the user-defined function, the query identifier of the structured query statement, and the creation time of the temporary function.

[0132] According to an embodiment of the present invention, any multiple modules of the parsing module 510, the determination module 520, the addition module 530 and the execution module 540 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the parsing module 510, the determination module 520, the addition module 530 and the execution module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the parsing module 510, the determination module 520, the addition module 530 and the execution module 540 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0133] Figure 6 A block diagram of an electronic device suitable for implementing a method for processing structured query statements in a database multi-cluster mode according to an embodiment of the present invention is shown.

[0134] like Figure 6As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0135] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.

[0136] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0137] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0138] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0139] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program includes a program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the structured query statement processing in the database multi-cluster mode provided by the embodiment of the present invention.

[0140] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0141] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0142] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0143] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0144] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0145] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0146] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for processing structured query statements in a database multi-cluster mode, characterized in that: Applied to a scheduling node in a scheduling cluster, the method comprises: In response to receiving a structured query statement, parsing the structured query statement to generate an original execution plan of the structured query statement, wherein the structured query statement includes a custom function, the original execution plan includes a plurality of original execution steps, and the custom function is created in the second cluster; Determine a target original execution step from the plurality of original execution steps, wherein the target original execution step includes the user-defined function; In response to determining that the first cluster used to execute the target original execution step does not include the custom function, adding a function creation step before a first original execution step among the multiple original execution steps, wherein the function creation step is used to create a temporary function in the first cluster according to the custom function; After the last original execution step among the multiple original execution steps, adding a function destruction step, wherein the function destruction step is used to destroy the temporary function; Combining the plurality of original execution steps, the function creation step, and the function destruction step to obtain a target execution plan; and Sending the function creation step, the target original execution step, and the function destruction step to the first cluster, so that the first cluster executes the function creation step to create a temporary function having the same function as the custom function created in the second cluster, calls the temporary function created in the first cluster to perform data processing during the execution of the target original execution step, and causes the first cluster to execute the function destruction step to destroy the temporary function in the first cluster; The method further comprises: In a case where it is determined that the cluster identifier of the first cluster is the same as the cluster identifier of the second cluster, determining that the first cluster includes the user-defined function; and When it is determined that the cluster identifier of the first cluster is different from the cluster identifier of the second cluster, it is determined that the first cluster does not include the user-defined function.

2. The method according to claim 1, characterized in that The structured query statement includes fields, and the original execution plan for generating the structured query statement includes: Obtaining definition information of the user-defined function and a cluster identifier of a second cluster used to store the user-defined function; generating a function call instance according to the definition information and the cluster identifier of the second cluster; and In response to determining that the field includes the custom function, an original execution step corresponding to the field is generated based on the function call instance.

3. The method according to claim 1, characterized in that: The executing the target execution plan includes: In response to detecting that an exception occurs during the execution of the function destroying step, the exception is not thrown.

4. The method according to claim 1, characterized in that: The method further comprises: In response to triggering a timing event, deleting temporary functions created in the first cluster within a predetermined period of time.

5. The method according to claim 1, characterized in that The method further comprises: A second function identifier of the temporary function is determined according to the first cluster identifier of the first cluster, the first function identifier of the user-defined function, the query identifier of the structured query statement, and the creation time of the temporary function.

6. A structured query statement processing device in a database multi-cluster mode, characterized in that: Applied to a scheduling node in a scheduling cluster, the device comprises: a parsing module, configured to parse the structured query statement in response to receiving the structured query statement, and generate an original execution plan of the structured query statement, wherein the structured query statement includes a custom function, the original execution plan includes a plurality of original execution steps, and the custom function is created in the second cluster; A determination module, configured to determine a target original execution step from the plurality of original execution steps, wherein the target original execution step includes the user-defined function; an adding module, configured to, in response to determining that the first cluster used to execute the target original execution step does not include the custom function, add a function creation step before the target original execution step to obtain a target execution plan, wherein the function creation step is used to create a temporary function in the first cluster according to the custom function; and an execution module, configured to send the function creation step, the target original execution step, and the function destruction step to the first cluster, so that the first cluster executes the function creation step to create a temporary function having the same function as the custom function created in the second cluster, and in the process of executing the target original execution step, calls the temporary function created in the first cluster to perform data processing, and enables the first cluster to execute the function destruction step to destroy the temporary function in the first cluster; The device also includes: a first determining submodule, configured to determine that the first cluster includes the user-defined function when it is determined that the cluster identifier of the first cluster is the same as the cluster identifier of the second cluster; and a second determining submodule, configured to determine that the first cluster does not include the user-defined function when it is determined that the cluster identifier of the first cluster is different from the cluster identifier of the second cluster; The adding module comprises: Creating an adding submodule, used for adding the function creation step before the first original execution step among the multiple original execution steps; A destruction adding submodule, used for adding a function destruction step after the last original execution step among the multiple original execution steps, wherein the function destruction step is used for destroying the temporary function; The plan generation submodule is used to combine the target execution plan according to the multiple original execution steps, the function creation step and the function destruction step.

7. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images