SQL Statement Processing Method, Device, Computer Equipment and Storage Medium

Abstract

The present invention discloses an SQL statement processing method, apparatus, computer device, and storage medium, belonging to the technical field of databases. The SQL statement processing method can group first SQL statements in an SQL statement set that have the same operation item type, the same data source, and the same logical item according to a preset rule, so as to combine all the first SQL statements in the same group to generate a combined second SQL statement, reduce the task volume of the SQL statements, and obtain corresponding task results by executing the second SQL statement, thereby shortening the time for executing tasks and achieving the purpose of improving execution efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of databases, and in particular to a method and apparatus for processing SQL statements, a computer device, and a storage medium. Background Art

[0002] Data quality detection in a database is an indispensable link. Through data quality inspection, data with quality problems can be effectively screened out for rectification. The existing data quality inspection process is as follows:

[0003] (1) Configure data quality inspection rules. The configuration content of the quality inspection rules is: [table, field, inspection rule]. Taking an example that there is a field FIELD_1 in table TABLE_A and a non-null check is performed on this field, the configuration rule is: [TABLE_A, FIELD_1, non-null verification];

[0004] (2) Generate corresponding SQL statements according to the configured rules;

[0005] (3) Submit the SQL statements to the SPARK computing engine for execution to obtain the results returned after the SPARK computing engine executes the SQL statements.

[0006] In the actual implementation process, it is found that there is a performance bottleneck in "submitting SQL statements to the SPARK computing engine for execution". When an SQL statement is submitted to the SPARK computing engine for execution, the computing engine needs to apply to the cluster resource manager for the resources required to execute this task, allocate, create tasks, perform task scheduling, etc. Generally, it takes about 2 seconds to calculate and allocate task execution (if there are too many tasks and insufficient resources, it is also necessary to wait for the previous tasks to complete and release resources before the subsequent tasks can be executed, and the waiting time will be longer). Taking the example of configuring 8000 configuration rules for 720 tables: 8000 SQL statements are generated according to 8000 configuration rules, and these SQL statements are submitted to the SPARK computing engine for execution. The SPARK computing engine performs corresponding operations on the received 8000 tasks. During the execution process, due to too many tasks, the efficiency of data quality inspection will be reduced, and even some SQL statements will time out and report errors during execution, and the execution results cannot be correctly returned. Summary of the Invention

[0007] Aiming at the problems of low efficiency and long time consumption when executing a large number of SQL statements, the present invention provides a method and apparatus for processing SQL statements, a computer device, and a storage medium, which are designed to improve the execution efficiency of SQL statements.

[0008] To achieve the above object, the present invention provides a method for processing SQL statements, including:

[0009] Obtain a set of SQL statements, where the set of SQL statements includes at least two first SQL statements, and the data structure of the first SQL statements includes: an operation item, a data source, and a logic item;

[0010] Group the first SQL statements in the set of SQL statements according to a preset rule, where the preset rule includes that the operation item types, data sources, and logic items in at least two of the first SQL statements are the same;

[0011] Combine all the first SQL statements in the same group to generate a second SQL statement;

[0012] Execute the second SQL statement to obtain a task result.

[0013] Optionally, the operation item includes a field element and at least one operation function;

[0014] The same operation item type is used to indicate that the structures of the operation items are the same and the types of the field elements are the same.

[0015] Optionally, each of the first SQL statements is associated with a label;

[0016] The combining all the first SQL statements in the same group to generate a second SQL statement includes:

[0017] Use a string concatenation function to concatenate the labels of each of the first SQL statements in the same group and separate each of the labels with a delimiter to generate a label combination item;

[0018] According to the concatenation order of the labels, use the string concatenation function to concatenate the operation functions with the same structure in each of the first SQL statements in the same group and separate the same operation functions with the delimiter to generate an operation combination item;

[0019] Combine the label combination item, all the operation combination items, the data source and the logic item of the first SQL statements in the same group to generate the second SQL statement.

[0020] Optionally, each of the first SQL statements is associated with a label;

[0021] The combining all the first SQL statements in the same group to generate a second SQL statement includes:

[0022] Use a string concatenation function to concatenate the labels of each of the first SQL statements in the same group and separate each of the labels with a delimiter to generate a label combination item;

[0023] The operation functions with the same structure in the same group and the same parameters in the operation functions are regarded as unified operation functions;

[0024] For the operation functions with the same structure in the same group and different field elements in the operation functions, according to the concatenation order of the tags, the operation functions with the same structure in each of the first SQL statements in the same group are concatenated by using the string concatenation function, and the same operation functions are separated by the delimiter to generate operation combination items;

[0025] The tag combination items, the unified operation functions, all the operation combination items, the data sources and logical items of the first SQL statements in the same group are combined to generate the second SQL statement.

[0026] Optionally, executing the second SQL statement to obtain a task result includes:

[0027] Sending the second SQL statement to a computing engine, where the computing engine calculates the task time and resource space, applies for resource allocation from a resource manager according to the task time and the resource space, creates a task, sends the task to the corresponding resource, and executes the task operation to obtain the task result.

[0028] Optionally, the task result includes a tag combination item and a result item, the tag combination includes at least two of the tags, the tags are concatenated by the delimiter, the result item includes at least one operation result combination item, the operation result combination item includes at least two result sub-items, and the result sub-items are concatenated by the delimiter;

[0029] Further included are:

[0030] Splitting the task result into at least two sub-task results according to the delimiter;

[0031] The sub-task result includes a tag and a result sub-item.

[0032] To achieve the above object, the present invention also provides an SQL statement processing device, including:

[0033] An acquisition unit, configured to acquire a set of SQL statements, the set of SQL statements includes at least two first SQL statements, and the data structure of the first SQL statement includes: an operation item, a data source, and a logical item;

[0034] A grouping unit, configured to group the first SQL statements in the set of SQL statements according to a preset rule, and the preset rule includes that the operation item types, data sources, and logical items in at least two of the first SQL statements are the same;

[0035] A combination unit, configured to combine all the first SQL statements in the same group to generate a second SQL statement;

[0036] An execution unit, configured to execute the second SQL statement to obtain a task result.

[0037] Optionally, the operation item includes a field element and at least one operation function;

[0038] The same operation item types are used to indicate that the structures of the operation items are the same, and the types of the field elements are the same.

[0039] To achieve the above object, the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0040] To achieve the above object, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0041] The SQL statement processing method, device, computer device, and storage medium provided by the present invention can group the first SQL statements with the same operation item types, the same data sources, and the same logical items in a SQL statement set according to a preset rule, so as to combine all the first SQL statements in the same group to generate a combined second SQL statement, reduce the task volume of the SQL statement, and obtain the corresponding task result by executing the second SQL statement, thereby shortening the time for executing the task and achieving the purpose of improving the execution efficiency. Description of the Drawings

[0042] Figure 1 It is a flowchart of an embodiment of the SQL statement processing method described in the present invention;

[0043] Figure 2 It is a flowchart of another embodiment of the SQL statement processing method described in the present invention;

[0044] Figure 3 It is a flowchart of an embodiment for generating a second SQL statement;

[0045] Figure 4 It is a flowchart of another embodiment for generating a second SQL statement;

[0046] Figure 5 It is a flowchart of another embodiment of the SQL statement processing method described in the present invention;

[0047] Figure 6Module diagram of an embodiment of the SQL statement processing device according to the present invention;

[0048] Figure 7 Module diagram of the combination unit described in the present invention;

[0049] Figure 8 Schematic diagram of the hardware architecture of an embodiment of the computer device according to the present invention. Detailed implementation manners

[0050] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0051] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0052] The SQL statement processing method, device, computer device and storage medium provided by the present invention can be applied to business fields such as insurance, finance, intelligent medical care, leasing, etc. The present invention can group the first SQL statements with the same operation item type, the same data source and the same logic item in the SQL statement set, so as to combine all the first SQL statements in the same group to generate the combined second SQL statement, reduce the task volume of the SQL statement, and obtain the corresponding task result by executing the second SQL statement, thereby shortening the time for executing the task and achieving the purpose of improving the execution efficiency.

[0053] Embodiment 1

[0054] Please refer to Figure 1 , a SQL statement processing method in this embodiment includes the following steps:

[0055] S1. Obtain a SQL statement set.

[0056] Wherein, the SQL statement set includes at least two first SQL statements, and the data structure of the first SQL statement includes: an operation item, a data source and a logic item; the operation item includes a field element and at least one operation function.

[0057] It should be noted that each of the first SQL statements is associated with a label. The field element can be a certain field or a field range including multiple fields.

[0058] In practical applications, the data source is the table name in the database. The operation items can be composed of the structured query language of SQL, such as: SELECT (retrieve data from the database table), UPDATE (update the data in the database table), DELETE (delete data from the database table), INSERT INTO (insert data into the database table), etc. The logical items are configuration rules, such as: non-empty verification, dictionary value check, etc.

[0059] The above database can use a server, and the server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0060] In a possible implementation, the data source is medical data, such as personal health records, prescriptions, inspection reports, and other data.

[0061] In a possible implementation, the data source is medical text, and the medical text can be an Electronic Healthcare Record, an electronic personal health record, including a series of electronic records such as medical records, electrocardiograms, and medical images that have the value of being preserved for future reference.

[0062] Information query has become a channel for users to quickly obtain the required information in many scenarios. For example, in the medical field, the required medical record information of users can be queried from a large number of electronic medical records based on an artificial intelligence model, which helps to provide medical record references for users.

[0063] As an example rather than a limitation, the configuration data is:

TABLE_A, FIELD_1, non-empty verification

[0064]

[0065]

[0066] Among them, SELECT……FROM is the operation item, "TABLE_A" after FROM is the data source, and "is_valid = 1" after WHERE is the logical item.

[0067] The configuration data is:

TABLE_A, FIELD_2, non-empty verification

[0068]

[0069] Among them, SELECT... FROM is the operation item, "TABLE_A" after FROM is the data source, and "is_valid = 1" after WHERE is the logical item.

[0070] In one embodiment, referring to Figure 2 as shown, before performing step S1, it may further include:

[0071] A. Obtain all configuration data, respectively generate corresponding first SQL statements according to each of the configuration data, and obtain the SQL statement set composed of all the first SQL statements.

[0072] Among them, each piece of the configuration data includes a data source, a field element, and event rule data.

[0073] In this embodiment, each piece of configuration data will generate a first SQL statement. If there are multiple pieces of configuration data, multiple first SQL statements will be generated. For example: The configuration data for performing a non-null check on the FIELD_1 field of the TABLE_A table is: [TABLE_A, FIELD_1, non-null check]; The configuration data for performing a non-null check on the FIELD_2 field of the TABLE_A table is: [TABLE_A, FIELD_2, non-null check]; Among them, TABLE_A is the data source (table name), FIELD_1 and FIELD_2 are both field elements, and non-null check is the event rule data.

[0074] S2. Group the first SQL statements in the SQL statement set according to a preset rule.

[0075] Among them, the preset rule includes that the operation item types in at least two of the first SQL statements are the same, the data sources are the same, and the logical items are the same.

[0076] In this embodiment, the same operation item type is used to indicate that the structures of the operation items are the same and the types of the field elements are the same.

[0077] Further, step S2 may include:

[0078] S21. Analyze each of the first SQL statements in the SQL statement set to determine the operation item, data source, and logical item in each first SQL statement.

[0079] S22. Take all the first SQL statements with the same operation item type, the same data source, and the same logical item as a group.

[0080] In this embodiment, the operation item structure corresponds to the result structure after the execution of the first SQL statement. Therefore, the same structure of the operation item can also be understood as the same result structure after the execution of the first SQL statement. For example, the first SQL statement with label ID 1 is:

[0081]

[0082] The result after execution is:

[0083] Label ID total statis result 1 50 25 0.5

[0084] The first SQL statement with label ID 2 is:

[0085]

[0086] The result after execution is:

[0087]

[0088]

[0089] As mentioned above, the operation item structures of the first SQL statements with label ID 1 and label ID 2 are the same, the result structures after execution are also the same, the types of field elements (FIELD_1 field and FIELD_2 field) are the same, the data sources (TABLE_A) are the same, and the logical items (is_valid = 1) are the same. Therefore, the two first SQL statements can be grouped together.

[0090] S3. Combine all the first SQL statements in the same group to generate a second SQL statement.

[0091] In a preferred embodiment, referring to Figure 3 as shown, step S3 may include the following steps:

[0092] S301. Use a string concatenation function to concatenate the labels of each of the first SQL statements in the same group, and split each of the labels with a delimiter to generate a label combination item;

[0093] In this embodiment, the delimiter is a symbol that does not conflict with the SQL syntax and does not have a special combination with the content of the data source. For example, the delimiter can be '#_#'.

[0094] S302. Use the operation functions with the same structure and the same parameters in the operation functions in the same group as a unified operation function;

[0095] S303. For operation functions with the same structure in the same group and different field elements in the operation functions, according to the concatenation order of the tags, use the string concatenation function to concatenate the operation functions with the same structure in each of the first SQL statements in the same group, and separate the same operation functions with the delimiter to generate operation combination items;

[0096] S304. Combine the tag combination items, the unified operation function, all the operation combination items, the data sources and logical items of the first SQL statements in the same group to generate the second SQL statement.

[0097] Taking the first SQL statement with tag ID 1 and the first SQL statement with tag ID 2 as examples: The combined second SQL statement is:

[0098]

[0099]

[0100] Among them, the tag combination item is: CONCAT(1, ‘#_#’, 2) AS id; the unified operation function is: count(1) AS total; the operation combination items are: CONCAT(count(case when FIELD_1 is null then 1 end), ‘#_#’, count(case when FIELD_2 is null then 1 end)) AS statis, CONCAT(count(case when FIELD_1 is null then 1 end) / count(1), ‘#_#’, count(case when FIELD_2 is null then 1 end) / count(1)) AS result.

[0101] In another preferred embodiment, refer to Figure 4 as shown, step S3 may include the following steps:

[0102] S311. Use the string concatenation function to concatenate the tags of each of the first SQL statements in the same group, and separate each tag with the delimiter to generate tag combination items;

[0103] S312. According to the concatenation order of the tags, use the string concatenation function to concatenate the operation functions with the same structure in each of the first SQL statements in the same group, and separate the same operation functions with the delimiter to generate operation combination items;

[0104] S313. Combine the label combination item, all the operation combination items, the data source and the logical item of the first SQL statement in the same group to generate the second SQL statement.

[0105] S4. Execute the second SQL statement to obtain the task result.

[0106] Further, step S4 may include: sending the second SQL statement to a computing engine, the computing engine calculating the task time and resource space, applying to a resource manager for resource allocation according to the task time and the resource space, creating a task, sending the task to the corresponding resource, and performing task operations to obtain the task result.

[0107] In this embodiment, when the second SQL statement is submitted to the SPARK computing engine for execution, the SPARK computing engine needs to calculate the task time and resource space, so as to apply to the cluster resource manager for the resources required to execute this task, allocate, create tasks, and perform task scheduling operations. For example, 8000 first SQL statements are combined to obtain 720 second SQL statements. When the second SQL statement is executed through the SPARK computing engine, (8000 - 720)×2s = 14560s (about 4.04h) can be saved. It can be seen that before executing the SQL statement, by combining the first SQL statements that meet the preset rules and then executing the combined second SQL statement, the execution time can be effectively reduced and the execution efficiency can be improved.

[0108] In this embodiment, the task result may include a label combination item and a result item. The label combination includes at least two of the labels, and the labels are concatenated by the delimiter. The result item includes at least one operation result combination item, and the operation result combination item includes at least two result sub-items, and the result sub-items are concatenated by the delimiter.

[0109] Taking the second SQL statement as an example, which is composed of the first SQL statement with label ID 1 and the first SQL statement with label ID 2, its task result A is:

[0110] Label ID total statis result 1‘#_#’2 50 25‘#_#’20 0.5‘#_#’0.4

[0111] Among them, 1‘#_#’2 is the label combination, 50, 25‘#_#’20 and 0.5‘#_#’0.4 jointly form the result item, 25‘#_#’20 and 0.5‘#_#’0.4 are both operation result combination items, and 25, 20, 0.5, and 0.4 are all result sub-items.

[0112] In another embodiment, referring to Figure 5 as shown, the SQL statement processing method may further include:

[0113] S5. Split the task result into at least two sub - task results according to the delimiter.

[0114] Among them, the sub - task result includes a label and a result sub - item.

[0115] In this embodiment, according to the order of label combination, the task result is split into several sub - task results.

[0116] Taking task result A as an example, the splitting result is as follows:

[0117] Label ID total statis result 1 50 25 0.5 2 50 20 0.4

[0118] In this embodiment, the SQL statement processing method can group the first SQL statements with the same operation item type, the same data source, and the same logical item in the SQL statement set according to preset rules, so as to combine all the first SQL statements in the same group to generate the combined second SQL statement, reduce the task volume of the SQL statement, and obtain the corresponding task result by executing the second SQL statement, thereby shortening the time for task execution and achieving the purpose of improving execution efficiency.

[0119] In practical applications, before optimizing the combination of SQL statements, when validating the data quality of 720 tables stored in the ods (operational storage data) layer of the data warehouse, 8000 first SQL statements need to be generated. Through the SPARK computing engine, among them, more than about 80% of the first SQL statements will fail because the SPARK resources cannot be allocated and they have been queuing for a long time, and only less than 20% of the SQL statements can be executed successfully. After performing dynamic SQL statement combination optimization, 8000 first SQL statements can be combined into 720 second SQL statements (about 720 tables). After combination, only 720 tasks are submitted to the SPARK computing engine for execution, greatly reducing the number of tasks, and all 720 tasks can be executed successfully. The execution time is about 2.5 hours, solving the performance bottleneck problem in the data quality analysis link and improving the execution efficiency.

[0120] Embodiment 2

[0121] Please refer to Figure 6 , a SQL statement processing device 1 in this embodiment includes: an acquisition unit 11, a grouping unit 12, a combination unit 13, and an execution unit 14.

[0122] The acquisition unit 11 is used to acquire a SQL statement set.

[0123] Among them, the SQL statement set includes at least two first SQL statements, and the data structure of the first SQL statement includes: an operation item, a data source, and a logical item; the operation item includes field elements and at least one operation function.

[0124] It should be noted that each of the first SQL statements is associated with a tag. The field element can be a certain field or a field range including multiple fields.

[0125] In practical applications, the data source is the table name in the database. The operation item can be composed of the structured query language of SQL, such as: SELECT (retrieving data from a database table), UPDATE (updating data in a database table), DELETE (deleting data from a database table), INSERT INTO (inserting data into a database table), etc. The logic item is a configuration rule, such as: non-empty verification, dictionary value check, etc.

[0126] The above database can adopt a server, and the server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0127] In a possible implementation manner, the data source is medical data, such as personal health records, prescriptions, inspection reports, and other data.

[0128] In a possible implementation manner, the data source is medical text, and the medical text can be an Electronic Healthcare Record, an electronic personal health record, including a series of electronic records such as medical records, electrocardiograms, and medical images that have the value of being saved for future reference.

[0129] Information query has become a channel for users to quickly obtain the required information in many scenarios. For example, in the medical field, it is possible to query the medical record information required by users from a large number of electronic medical records based on an artificial intelligence model, which helps to provide medical record references for users.

[0130] The grouping unit 12 is used to group the first SQL statements in the SQL statement set according to a preset rule.

[0131] Among them, the preset rule includes that the operation item types, data sources, and logic items in at least two of the first SQL statements are the same.

[0132] In this embodiment, the same operation item type is used to indicate that the structures of the operation items are the same and the types of the field elements are the same.

[0133] Further, the grouping unit 12 is further configured to parse each of the first SQL statements in the SQL statement set, determine the operation item, data source, and logical item in each of the first SQL statements; the grouping unit 12 is further configured to group all the first SQL statements with the same operation item type, the same data source, and the same logical item into one group.

[0134] In this embodiment, the operation item structure corresponds to the result structure after the execution of the first SQL statement. Therefore, the same structure of the operation item can also be understood as the same result structure after the execution of the first SQL statement.

[0135] The combining unit 13 is configured to combine all the first SQL statements in the same group to generate a second SQL statement.

[0136] In a preferred embodiment, referring to Figure 7 As shown, the combining unit 13 may include: a first generation module 131, a processing module 132, a second generation module 133, and a combining module 134.

[0137] The first generation module 131 is configured to concatenate the tags of each of the first SQL statements in the same group by using a string concatenation function, and split each of the tags by a delimiter to generate a tag combination item.

[0138] In this embodiment, the delimiter is a symbol that does not conflict with the SQL syntax and does not have a special combination with the content of the data source. For example, the delimiter may adopt '#_#'.

[0139] The processing module 132 is configured to use the operation functions with the same structure and the same parameters in the operation functions in the same group as a unified operation function.

[0140] The second generation module 133 is configured to, for the operation functions with the same structure and different field elements in the operation functions in the same group, concatenate the operation functions with the same structure in each of the first SQL statements in the same group by using the string concatenation function according to the concatenation order of the tags, and separate the same operation functions by the delimiter to generate an operation combination item.

[0141] The combining module 134 is configured to combine the tag combination item, the unified operation function, all the operation combination items, the data source and the logical item of the first SQL statement in the same group to generate the second SQL statement.

[0142] In another preferred embodiment, the combining unit 13 may include: a first generation module 131, a second generation module 133, and a combining module 134.

[0143] The first generation module 131 is configured to concatenate the tags of each of the first SQL statements in the same group by using a string concatenation function, and split each of the tags by a delimiter to generate a tag combination item;

[0144] The second generation module 133 is configured to concatenate the operation functions with the same structure in each of the first SQL statements in the same group by using the string concatenation function according to the concatenation order of the tags, and separate the same operation functions by the delimiter to generate an operation combination item;

[0145] The combination module 134 is configured to combine the tag combination item, all the operation combination items, the data source and the logic item of the first SQL statement in the same group to generate the second SQL statement.

[0146] The execution unit 14 is configured to execute the second SQL statement to obtain a task result.

[0147] Further, the execution unit 14 may send the second SQL statement to a computing engine, and the computing engine calculates the task time and the resource space, applies to a resource manager for resource allocation according to the task time and the resource space, creates a task, sends the task to the corresponding resource, and executes the task operation to obtain the task result.

[0148] In this embodiment, when the second SQL statement is submitted to the SPARK computing engine for execution, the SPARK computing engine needs to calculate the task time and the resource space, so as to apply to the cluster resource manager for the resources required to execute the task, allocate, create tasks, and perform task scheduling operations. For example, 8000 first SQL statements are combined to obtain 720 second SQL statements. When the second SQL statement is executed by the SPARK computing engine, (8000 - 720)×2s = 14560s (about 4.04h) can be saved. It can be seen that before executing the SQL statement, by combining the first SQL statements that meet the preset rules and then executing the combined second SQL statement, the execution time can be effectively reduced and the execution efficiency can be improved.

[0149] In this embodiment, the task result may include a tag combination item and a result item. The tag combination includes at least two of the tags, and the tags are concatenated by the delimiter. The result item includes at least one operation result combination item, and the operation result combination item includes at least two result sub-items, and the result sub-items are concatenated by the delimiter.

[0150] In another embodiment, the SQL statement processing device 1 may further include:

[0151] A splitting unit, configured to split the task result into at least two subtask results according to the delimiter.

[0152] Wherein, the subtask result includes a label and a result sub-item.

[0153] In this embodiment, according to the order of label combination, the task result is split into several subtask results.

[0154] In this embodiment, the SQL statement processing device 1 can group the first SQL statements with the same operation item type, the same data source, and the same logical item in the SQL statement set according to a preset rule, so as to combine all the first SQL statements in the same group to generate a combined second SQL statement, reduce the task volume of the SQL statement, and obtain the corresponding task result by executing the second SQL statement, thereby shortening the time for executing the task and achieving the purpose of improving the execution efficiency.

[0155] In a preferred embodiment, the SQL statement processing device 1 may further include: a generating unit.

[0156] The generating unit is configured to obtain all configuration data, and respectively generate corresponding first SQL statements according to each piece of the configuration data, so as to obtain the SQL statement set composed of all the first SQL statements.

[0157] Wherein, each piece of the configuration data includes a data source, a field element, and event rule data.

[0158] In this embodiment, each piece of configuration data generates a first SQL statement. If there are multiple pieces of configuration data, multiple first SQL statements will be generated. For example: the configuration data for performing a non-null check on the FIELD_1 field of the TABLE_A table is: [TABLE_A, FIELD_1, non-null check]; the configuration data for performing a non-null check on the FIELD_2 field of the TABLE_A table is: [TABLE_A, FIELD_2, non-null check]; wherein, TABLE_A is the data source (table name), FIELD_1 and FIELD_2 are both field elements, and non-null check is the event rule data.

[0159] Embodiment Three

[0160] To achieve the above object, the present invention further provides a computer device 2, which includes a plurality of computer devices 2. The components of the SQL statement processing device 1 in Embodiment 2 can be distributed in different computer devices 2. The computer device 2 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a cabinet server (including an independent server or a server cluster composed of multiple servers) that executes programs, etc. The computer device 2 in this embodiment at least includes, but is not limited to: a memory 21, a processor 23, a network interface 22 and an SQL statement processing device 1 that can communicate with each other through a system bus (refer to Figure 8 ). It should be noted that Figure 8 only the computer device 2 with components - is shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0161] In this embodiment, the memory 21 at least includes one type of computer-readable storage medium. The readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disc, etc. In some embodiments, the memory 21 can be an internal storage unit of the computer device 2, such as the hard disk or memory of the computer device 2. In other embodiments, the memory 21 can also be an external storage device of the computer device 2, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the computer device 2. Of course, the memory 21 can also include both the internal storage unit and the external storage device of the computer device 2. In this embodiment, the memory 21 is usually used to store the operating system and various application software installed on the computer device 2, such as the program code of the SQL statement processing method in Embodiment 1. In addition, the memory 21 can also be used to temporarily store various data that have been output or will be output.

[0162] In some embodiments, the processor 23 may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 23 is generally used to control the overall operation of the computer device 2, such as performing control and processing related to data interaction or communication with the computer device 2. In this embodiment, the processor 23 is used to run the program code stored in the memory 21 or process data, such as running the SQL statement processing device 1, etc.

[0163] The network interface 22 may include a wireless network interface or a wired network interface. The network interface 22 is generally used to establish a communication connection between the computer device 2 and other computer devices 2. For example, the network interface 22 is used to connect the computer device 2 to an external terminal through a network, and establish a data transmission channel and a communication connection between the computer device 2 and the external terminal. The network may be an enterprise internal network (Intranet), the Internet, a Global System of Mobile communication (GSM), a Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0164] It should be noted that Figure 8 Only the computer device 2 with components 21-23 is shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components may be implemented alternatively.

[0165] In this embodiment, the SQL statement processing device 1 stored in the memory 21 may also be divided into one or more program modules. The one or more program modules are stored in the memory 21 and executed by one or more processors (the processor 23 in this embodiment) to complete the present invention.

[0166] Embodiment Four

[0167] To achieve the above object, the present invention further provides a computer-readable storage medium, which includes a plurality of storage media, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, server, App application store, etc., on which a computer program is stored, and when the program is executed by the processor 23, corresponding functions are implemented. The computer-readable storage medium of this embodiment is used to store the SQL statement processing device 1, and when executed by the processor 23, it implements the SQL statement processing method of the first embodiment.

[0168] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0169] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0170] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A method for processing SQL statements, characterized in that, Including: Obtain a set of SQL statements, where the set of SQL statements includes at least two first SQL statements. The data structure of the first SQL statement includes: an operation item, a data source, and a logic item; the operation item includes a field element and at least one operation function; the types of the operation items are the same, indicating that the structures of the operation items are the same, and the types of the field elements are the same; Group the first SQL statements in the set of SQL statements according to a preset rule, where the preset rule includes that the operation item types, data sources, and logic items in at least two of the first SQL statements are the same; Combine all the first SQL statements in the same group to generate a second SQL statement; each of the first SQL statements is associated with a label; the combining all the first SQL statements in the same group to generate a second SQL statement includes: using a string concatenation function to concatenate the labels of each of the first SQL statements in the same group, and separating each of the labels with a delimiter to generate a label combination item; regarding the operation functions with the same structure and the same parameters in the operation functions in the same group as a unified operation function; for the operation functions with the same structure and different field elements in the operation functions in the same group, according to the concatenation order of the labels, using the string concatenation function to concatenate the operation functions with the same structure in each of the first SQL statements in the same group, and separating the same operation functions with the delimiter to generate an operation combination item; combining the label combination item, the unified operation function, all the operation combination items, the data source and the logic item of the first SQL statements in the same group to generate the second SQL statement; Execute the second SQL statement to obtain a task result.

2. The SQL statement processing method according to claim 1, wherein Each of the first SQL statements is associated with a label; The combining all the first SQL statements in the same group to generate a second SQL statement includes: Using a string concatenation function to concatenate the labels of each of the first SQL statements in the same group, and separating each of the labels with a delimiter to generate a label combination item; According to the concatenation order of the labels, using the string concatenation function to concatenate the operation functions with the same structure in each of the first SQL statements in the same group, and separating the same operation functions with the delimiter to generate an operation combination item; Combining the label combination item, all the operation combination items, the data source and the logic item of the first SQL statements in the same group to generate the second SQL statement.

3. The SQL statement processing method according to claim 1, wherein The executing the second SQL statement to obtain a task result includes: Sending the second SQL statement to a computing engine, where the computing engine calculates the task time and resource space, applies to a resource manager for resource allocation according to the task time and the resource space, creates a task, sends the task to the corresponding resource, and executes the task operation to obtain the task result.

4. The SQL statement processing method according to claim 1 or 2, characterized in that The task result includes label combination items and result items. The label combination includes at least two of the labels, and the labels are concatenated by the delimiter. The result item includes at least one operation result combination item, and the operation result combination item includes at least two result sub-items, and the result sub-items are concatenated by the delimiter; It further includes: Splitting the task result into at least two sub-task results according to the delimiter; The sub-task result includes a label and a result sub-item.

5. A SQL statement processing device, characterized in that, It includes: An acquisition unit for acquiring a set of SQL statements, the set of SQL statements including at least two first SQL statements, and the data structure of the first SQL statement including: an operation item, a data source, and a logic item; the operation item includes a field element and at least one operation function; the same type of operation item is used to indicate that the structures of the operation items are the same, and the types of the field elements are the same; A grouping unit for grouping the first SQL statements in the set of SQL statements according to a preset rule, the preset rule including that the types of operation items, the data sources, and the logic items in at least two of the first SQL statements are the same; A combination unit for combining all the first SQL statements in the same group to generate a second SQL statement; each of the first SQL statements is associated with a label; combining all the first SQL statements in the same group to generate a second SQL statement includes: concatenating the labels of the first SQL statements in the same group using a string concatenation function, and separating the labels by a delimiter to generate a label combination item; regarding the operation functions with the same structure and the same parameters in the operation functions in the same group as a unified operation function; for the operation functions with the same structure but different field elements in the operation functions in the same group, concatenating the operation functions with the same structure in the first SQL statements in the same group in the order of the concatenation of the labels using the string concatenation function, and separating the same operation functions by the delimiter to generate an operation combination item; combining the label combination item, the unified operation function, all the operation combination items, the data source and the logic item of the first SQL statement in the same group to generate the second SQL statement; An execution unit for executing the second SQL statement to obtain a task result.

6. The SQL statement processing device according to claim 5, wherein The operation item includes a field element and at least one operation function; The same type of operation item is used to indicate that the structures of the operation items are the same, and the types of the field elements are the same.

7. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images