Dynamic updating method, device and equipment for SQL statement template and storage medium

Abstract

The invention discloses an SQL statement template dynamic updating method and device, equipment and a storage medium, and relates to the field of databases, and the method comprises the following steps: constructing a to-be-matched template sequence of a to-be-processed SQL statement; determining a longest common subsequence between each statement template sequence and the to-be-matched template sequence, and determining a target statement template sequence from all statement template sequences based on all the longest common subsequences; if the sequence length of the longest common subsequence corresponding to the target statement template sequence is greater than or equal to a preset sequence length threshold value, updating the target statement template sequence into the corresponding longest common subsequence in a dynamic statement template sequence library; if the sequence length of the longest common subsequence corresponding to the target statement template sequence is smaller than a preset sequence length threshold value, the template sequence to be matched serves as a new statement template sequence to be added into the dynamic statement template sequence library. According to the method and the device, the dynamic statement template sequence library of the SQL statement is updated.

Description

Technical Field

[0001] The present invention relates to the field of databases, and in particular to a method, device, equipment and storage medium for dynamically updating a SQL statement template. Background Art

[0002] SQL (Structured Query Language) statement clustering and deduplication refers to grouping multiple related SQL statements together to perform a set of operations. At present, in order to achieve clustering and deduplication of SQL statements, conditional parameter traversal or tree structure recursion is often used. Among them, conditional parameter traversal uses simple conditional clauses for traversal matching, and traverses and matches each conditional clause of the input SQL statement. However, since the format and content of conditional clauses are very complex and diverse, manual code adaptation is difficult to cover them comprehensively, and the nested traversal matching parsing efficiency is low, which is only suitable for template matching of small-scale and simple-structured SQL statements; The tree structure recursive method is an SQL parsing based on AST (Abstract Syntax Tree), which parses the SQL query into an abstract syntax tree, and each node represents an operation or expression in the SQL query. However, recursive traversal of the tree structure requires a lot of additional memory. If a SQL statement is too long, it may fail to parse successfully due to stack overflow exceptions due to too many recursive layers.

[0003] In summary, how to implement simple, efficient and stable parsing of various SQL statements to complete clustering and deduplication of SQL statements is a problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method, device, equipment and storage medium for dynamically updating SQL statement templates, which can realize simple, efficient and stable parsing of various SQL statements to complete clustering and deduplication of SQL statements. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a method for dynamically updating a SQL statement template, comprising:

[0006] Construct a template sequence to be matched for the SQL statements to be processed obtained in real time;

[0007] Determine the longest common subsequence between each statement template sequence in the dynamic statement template sequence library and the template sequence to be matched based on the longest common subsequence algorithm, and determine the target statement template sequence corresponding to the template sequence to be matched from all the statement template sequences based on all the longest common subsequences;

[0008] If the sequence length of the longest common subsequence between the target statement template sequence and the to-be-matched template sequence is greater than or equal to a preset sequence length threshold, a first dynamic update operation is performed on the dynamic statement template sequence library to update the target statement template sequence to the longest common subsequence between the target statement template sequence and the to-be-matched template sequence;

[0009] If the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, a second dynamic update operation is performed on the dynamic statement template sequence library to add the template sequence to be matched as a new statement template sequence to the dynamic statement template sequence library.

[0010] Optionally, before constructing the to-be-matched template sequence of the SQL statement to be processed acquired in real time, the process further includes:

[0011] Determine whether the number of SQL statements in the current preset SQL statement storage space meets the preset number requirement;

[0012] If the number of SQL statements in the current preset SQL statement storage space does not meet the preset number requirement, then trigger the batch acquisition operation of inputting new SQL statements in a streaming input manner so that the number of SQL statements in the preset SQL statement storage space meets the preset number requirement, so as to obtain the SQL statements to be processed from the preset SQL statement storage space.

[0013] Optionally, the step of constructing a sequence of templates to be matched for the SQL statements to be processed acquired in real time includes:

[0014] Determine a target parameter in the SQL statement to be processed, and replace a parameter value of the target parameter in the SQL statement to be processed with a preset placeholder to obtain a target SQL statement;

[0015] The target SQL statement is split based on a preset separator to obtain tokens corresponding to the target SQL statement, and a corresponding sequence is constructed based on all the tokens to obtain the template sequence to be matched.

[0016] Optionally, splitting the target SQL statement based on a preset delimiter to obtain tokens corresponding to the target SQL statement includes:

[0017] Classifying the target SQL statement based on a type keyword in the target SQL statement to determine a statement type corresponding to the target SQL statement;

[0018] Determine a target keyword in the target SQL statement based on the statement type, and determine fixed content and non-fixed content in the SQL statement to be classified based on the target keyword;

[0019] The fixed content is taken as a Token, and the non-fixed content is split based on the preset separator to obtain tokens corresponding to the non-fixed content.

[0020] Optionally, the determining, from all the statement template sequences based on all the longest common subsequences, a target statement template sequence corresponding to the to-be-matched template sequence comprises:

[0021] Determine each of the longest common subsequences with the largest sequence length from all of the longest common subsequences, and determine each of the statement template sequences corresponding to each of the longest common subsequences with the largest sequence length from all of the statement template sequences;

[0022] The sentence template sequence with the least number of tokens is selected from the sentence template sequences corresponding to the longest common subsequences with the largest sequence length to obtain the target sentence template sequence.

[0023] Optionally, the preset sequence length threshold is a sequence length threshold determined based on the ratio of the number of word units in the template sequence to be matched and a preset threshold.

[0024] Optionally, the SQL statement template dynamic update method further includes:

[0025] Determine the parsing template sequence corresponding to the SQL statement to be processed from the dynamic statement template sequence library, and insert the ID corresponding to the SQL statement to be processed into the ID list of the parsing template sequence to update the dynamic statement template sequence library;

[0026] Generate a parsing result corresponding to the SQL statement to be processed based on the updated dynamic statement template sequence library, and save the parsing result.

[0027] In a second aspect, the present application discloses a device for dynamically updating a SQL statement template, comprising:

[0028] A sequence acquisition module is used to construct a template sequence to be matched for the SQL statement to be processed obtained in real time;

[0029] A template sequence matching module, used to determine the longest common subsequence between each statement template sequence in the dynamic statement template sequence library and the template sequence to be matched based on the longest common subsequence algorithm, and determine the target statement template sequence corresponding to the template sequence to be matched from all the statement template sequences based on all the longest common subsequences;

[0030] A first template updating module, configured to perform a first dynamic update operation on the dynamic statement template sequence library to update the target statement template sequence to the longest common subsequence between the target statement template sequence and the template sequence to be matched, if the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is greater than or equal to a preset sequence length threshold;

[0031] The second template updating module is used to perform a second dynamic update operation on the dynamic statement template sequence library if the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, so as to add the template sequence to be matched as a new statement template sequence to the dynamic statement template sequence library.

[0032] In a third aspect, the present application discloses an electronic device, comprising:

[0033] Memory, used to store computer programs;

[0034] The processor is used to execute the computer program to implement the aforementioned SQL statement template dynamic update method.

[0035] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program implements the aforementioned SQL statement template dynamic update method when executed by a processor.

[0036] In the present application, when clustering and deduplicating SQL statements, a to-be-matched template sequence of the SQL statements to be processed that is acquired in real time is constructed; based on the longest common subsequence algorithm, the longest common subsequence between each statement template sequence in the dynamic statement template sequence library and the template sequence to be matched is determined, and based on all the longest common subsequences, a target statement template sequence corresponding to the template sequence to be matched is determined from all the statement template sequences; if the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is greater than or equal to a preset sequence length threshold, a first dynamic update operation is performed on the dynamic statement template sequence library to update the target statement template sequence to the longest common subsequence between the target statement template sequence and the template sequence to be matched; if the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, a second dynamic update operation is performed on the dynamic statement template sequence library to add the template sequence to be matched as a new statement template sequence to the dynamic statement template sequence library. It can be seen that the present application uses the longest common subsequence algorithm to identify and extract the template corresponding to the SQL statement to be processed, so as to be adapted to clustering and deduplication of various complex SQL statements, and when the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, the template sequence to be matched itself is determined as the parsing template corresponding to the SQL statement to be processed, and then the template sequence to be matched is added to the dynamic statement template sequence library as a new statement template sequence, thereby realizing dynamic maintenance of the statement template sequence in the dynamic statement template sequence library, adapting to the constantly changing SQL data stream, so as to realize simple, efficient and stable parsing of various SQL statements, and complete the clustering and deduplication of SQL statements. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0038] Figure 1 A flow chart of a method for dynamically updating a SQL statement template disclosed in this application;

[0039] Figure 2 A schematic diagram of a SQL statement parsing process disclosed in this application;

[0040] Figure 3 A schematic diagram of a SQL statement parsing workflow disclosed in this application;

[0041] Figure 4 A flowchart of a specific SQL statement template dynamic update method disclosed in this application;

[0042] Figure 5 This is a schematic diagram of the structure of a SQL statement template dynamic update device disclosed in this application;

[0043] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] At present, in order to achieve clustering and deduplication of SQL statements, conditional parameter traversal or tree structure recursion is often used. Among them, conditional parameter traversal is performed by using simple conditional clauses for traversal matching, and each conditional clause of the input SQL statement is traversed and matched. However, since the format and content of the conditional clauses are very complex and diverse, it is difficult for manual code adaptation to cover them comprehensively, and the nested traversal matching parsing efficiency is low, which is only suitable for template matching of small-scale and simple-structured SQL statements; The tree structure recursive method is an SQL parsing based on AST (Abstract Syntax Tree), which parses the SQL query into an abstract syntax tree, and each node represents an operation or expression in the SQL query, but recursive traversal of the tree structure requires a lot of additional memory. If a SQL statement is too long, it may fail to parse successfully due to a stack overflow exception due to too many recursive layers. In order to solve the above technical problems, the present application discloses a method for dynamically updating SQL statement templates, which can realize simple, efficient and stable parsing of various SQL statements to complete clustering and deduplication of SQL statements.

[0046] See also Figure 1 As shown, an embodiment of the present invention discloses a method for dynamically updating a SQL statement template, comprising:

[0047] Step S11: construct a template sequence to be matched for the SQL statement to be processed obtained in real time.

[0048] In this embodiment, before constructing a to-be-processed template sequence corresponding to the to-be-processed SQL statement obtained in real time, it is necessary to determine whether the number of SQL statements in the current preset SQL statement storage space meets the preset number requirement; if the number of SQL statements in the current preset SQL statement storage space does not meet the preset number requirement, then trigger a batch acquisition operation to input new SQL statements in a streaming input manner, so that the number of SQL statements in the preset SQL statement storage space meets the preset number requirement, so as to obtain the to-be-processed SQL statement from the preset SQL statement storage space.

[0049] In this embodiment, after obtaining the SQL statement to be processed, it is necessary to process the SQL statement to be processed to construct its corresponding template sequence to be matched, so that the template sequence to be matched can be used to determine the template sequence corresponding to the SQL statement later. Constructing the template sequence to be matched of the SQL statement to be processed obtained in real time can specifically include: determining the target parameter in the SQL statement to be processed, and replacing the parameter value of the target parameter in the SQL statement to be processed with a preset placeholder to obtain the target SQL statement; splitting the target SQL statement based on the preset delimiter to obtain each Token corresponding to the target SQL statement, and constructing a corresponding sequence based on all Tokens to obtain the template sequence to be matched. In other words, after obtaining the fixed parameter value, such as a number or a parameter value in quotation marks, etc., it can be replaced with the placeholder "[*]" in advance, and then the target SQL statement is split based on the preset delimiter to obtain each Token corresponding to the target SQL statement and a corresponding sequence is constructed based on all Tokens to obtain the template sequence to be matched, thereby improving the subsequent parsing efficiency.

[0050] In order to reduce the amount of calculation, in a specific implementation, the target SQL statement is split based on a preset delimiter to obtain the tokens corresponding to the target SQL statement, which may specifically include: classifying the target SQL statement based on the type keyword in the target SQL statement to determine the statement type corresponding to the target SQL statement; determining the target keyword in the target SQL statement based on the statement type, and determining the fixed content and non-fixed content in the SQL statement to be classified based on the target keyword; taking the fixed content as a token, and splitting the non-fixed content based on the preset delimiter to obtain the tokens corresponding to the non-fixed content. In other words, the input SQL statement can be basically distinguished by type according to type keywords such as "SELECT" and "UPDATE" to ensure the accuracy of the tokens obtained by splitting the target SQL statement, and in the following example, Figure 2In the process shown, the fixed statement content before the "WHERE" keyword or the "SET" keyword is divided into a complete token, that is, the prefix "SELECT id, name FROM table_user" is processed as a token, so as to avoid excessive word segmentation and increase the amount of calculation. Then the remaining part of the SQL statement (that is, the non-fixed content) is split according to the preset delimiter to obtain the tokens corresponding to the non-fixed content. In other words, when there is a new SQL statement entry Input, parsed into a sequence of tokens according to conventional text delimiters (such as spaces, commas, colons, etc.) , that is, the template sequence to be matched.

[0051] Step S12: determine the longest common subsequence between each statement template sequence in the dynamic statement template sequence library and the template sequence to be matched based on the longest common subsequence algorithm, and determine the target statement template sequence corresponding to the template sequence to be matched from all the statement template sequences based on all the longest common subsequences.

[0052] In this embodiment, the longest common subsequence between each statement template sequence in the dynamic statement template sequence library and the template sequence to be matched is first determined based on the longest common subsequence algorithm, and then the target statement template sequence corresponding to the template sequence to be matched is determined from all the obtained longest common subsequences. Wherein, determining the target statement template sequence corresponding to the template sequence to be matched from all statement template sequences based on all the longest common subsequences may specifically include: determining each longest common subsequence with the largest sequence length from all the longest common subsequences, and determining each statement template sequence corresponding to each longest common subsequence with the largest sequence length from all statement template sequences; and filtering out the statement template sequence with the least number of tokens from each statement template sequence corresponding to each longest common subsequence with the largest sequence length, so as to obtain the target statement template sequence. That is to say, if there are multiple sentence template sequences and the LCS (Longest Common Subsequence) between the to-be-matched template sequence are equal and the maximum value, then the sentence template sequence with the least number of tokens (that is, the shortest sequence length) is selected from the sentence template sequences corresponding to the LCS with the largest sequence length as the target sentence template sequence, that is, the sentence template sequence in the dynamic sentence template sequence library with the highest matching degree with the to-be-matched template sequence.

[0053] In this embodiment, Figure 3As shown, the dynamic statement template sequence library can be a bitmap named LCSMap, which is used to store all LCSObject lists obtained by parsing SQL statements. The data structure LCSObject contains two attributes, sql_template and sql_id_list. sql_template is used to store the statement template sequence of the SQL statement, and sql_id_list is used to store the ID list of the SQL statement item corresponding to the sql_template. Compare with the sql_template of all LCSObjects in the current LCSMap to confirm Whether it can match one of the existing sql_templates. If the LCS obtained by matching the statement template sequence of an existing LCSObject with the new SQL statement meets the conditions, the ID (Identity Document) of the new SQL statement can be inserted into the corresponding sql_id_list. If any existing statement template sequence cannot match the new SQL statement entry to obtain an LCS that meets the conditions, a new LCSObject is created, and the current new SQL statement entry is directly used as the sql_template and inserted into the LCSMap. Based on this, the continuous input of SQL statement data streams can be supported.

[0054] like Figure 3As shown in the figure, for the first SQL statement "SELECT id, name FROM table userWHERE id = '001' and age = 20" input, since it is uncertain which variable parameter is and there is no statement template sequence in LCSMap, this SQL statement will be directly inserted into LCSMap as its own statement template sequence, thereby obtaining LCSObject_1; when the second SQL statement "SELECT id, name FROM table user WHERE id ='002' and age = 20" is input, the LCS algorithm can determine that the numbers after "=" in the two SQL statements are variable parameters, and the LCS is "SELECT id, name FROM table user WHERE id = [*] and age = [*]", then the LCS is used to overwrite the original sql_template in LCSObject_1, and the id corresponding to the SQL statement is also added to the corresponding sql_id_list. Among them, "[*]" is a preset placeholder used to replace the variable parameter. When the third SQL statement "SHOWTABLE STATUS" is input, since the sql_template in LCSObject_1 has no LCS with the third SQL statement, "SHOW TABLE STATUS" is directly inserted into LCSMap as its own statement template sequence, thereby obtaining LCSObject_2. This realizes the dynamic maintenance of the statement template sequence in the dynamic statement template sequence library.

[0055] Step S13: If the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is greater than or equal to a preset sequence length threshold, a first dynamic update operation is performed on the dynamic statement template sequence library to update the target statement template sequence to the longest common subsequence between the target statement template sequence and the template sequence to be matched.

[0056] In this embodiment, the preset sequence length threshold is a sequence length threshold determined based on the ratio of the number of word units in the template sequence to be matched and the preset threshold. In order to ensure the validity and accuracy of the matching, the product of the sequence length of the template sequence s to be matched and the preset threshold ratio can be determined as the preset sequence length threshold . Specifically, since the situation that the total length of variable parameters in the SQL statement exceeds half of the length of the SQL statement itself basically does not exist, the preset threshold ratio can be set to 50%. In addition, the preset threshold ratio can be adjusted according to the actual situation. If the matching accuracy is required to be higher, the preset threshold ratio can be appropriately increased; if the matching coverage is required to be higher, the preset threshold ratio can be appropriately reduced. If the sequence length of the LCS between the target statement template sequence and the template sequence to be matched is greater than or equal to the preset sequence length threshold, it means that the LCS between the target statement template matching sequence and the template sequence to be matched is the parsed template of the template sequence to be matched. At this time, the first dynamic update operation can be performed on the dynamic statement template sequence library to update the target statement template sequence to the longest common subsequence between the target statement template sequence and the template sequence to be matched.

[0057] Step S14: If the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, a second dynamic update operation is performed on the dynamic statement template sequence library to add the template sequence to be matched as a new statement template sequence to the dynamic statement template sequence library.

[0058] In this embodiment, if the sequence length of the LCS calculated by a to-be-matched template sequence and a statement template sequence is less than , it means that the matching degree between the SQL statement template sequence and the new sequence is too low and may not accurately represent the real structure of the new sequence. If the sequence length of the LCS between the target statement template sequence and the template sequence to be matched is also less than , it means that there is no statement template sequence matching the to-be-matched template sequence in the dynamic statement template sequence library. At this time, the to-be-matched template sequence itself is used as its corresponding statement template sequence, and a second dynamic update operation is performed on the dynamic statement template sequence library to add the to-be-matched template sequence as a new statement template sequence to the dynamic statement template sequence library.

[0059] In this embodiment, after obtaining a new dynamic statement template sequence library, the parsing result corresponding to the SQL statement to be processed can be obtained based on the dynamic statement template sequence library. The specific process of obtaining the parsing result corresponding to the SQL statement to be processed may include: determining the parsing template sequence corresponding to the SQL statement to be processed from the dynamic statement template sequence library, and inserting the ID corresponding to the SQL statement to be processed into the ID list of the parsing template sequence to update the dynamic statement template sequence library; generating the parsing result corresponding to the SQL statement to be processed based on the updated dynamic statement template sequence library, and saving the parsing result. The next time a new SQL statement to be processed is processed, the parsing result can be read from the cache, and then the newly input SQL statement to be processed can be parsed and the existing dynamic statement template sequence library can be updated, and the corresponding parsing result can be obtained at the same time, which significantly improves the parsing speed, thereby easily meeting the parsing needs of large amounts of data.

[0060] It can be seen that the present application uses the longest common subsequence algorithm to identify and extract the template corresponding to the SQL statement to be processed, so as to be adapted to clustering and deduplication of various complex SQL statements, and when the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, the template sequence to be matched itself is determined as the parsing template corresponding to the SQL statement to be processed, and then the template sequence to be matched is added to the dynamic statement template sequence library as a new statement template sequence, thereby realizing dynamic maintenance of the statement template sequence in the dynamic statement template sequence library, adapting to the constantly changing SQL data stream, so as to utilize the dynamic statement template sequence library to realize simple, efficient and stable parsing of various SQL statements, and complete clustering and deduplication of SQL statements.

[0061] Based on the previous embodiment, the present application discloses a method for dynamically updating SQL statement templates, which can realize dynamic maintenance of statement template sequences in a dynamic statement template sequence library, adapt to the ever-changing SQL data stream, and use the dynamic statement template sequence library to realize simple, efficient and stable parsing of various SQL statements, and complete clustering and deduplication of SQL statements. Next, the specific SQL statement template dynamic update process will be described.

[0062] See also Figure 4As shown, the present application discloses a specific SQL statement template dynamic update method flow chart. After reading the original SQL statement, the input original SQL statement is first preprocessed to improve the subsequent parsing efficiency. The specific preprocessing process may include: replacing fixed parameter values ​​such as numbers or parameter values ​​in quotation marks with placeholders "[*]" in advance and making basic type distinctions for the input original SQL statements according to keywords such as "SELECT" and "UPDATE" to ensure the accuracy of the subsequent word segmentation method. Then, based on the type of the original SQL statement, the word segmenter divides the fixed statement content before the "WHERE" keyword or the "SET" keyword into a complete Token, that is, the prefix of the original SQL statement, and splits the remaining non-fixed part according to the preset delimiter, thereby converting the entire SQL statement into a Token sequence, that is, a template sequence to be matched, and inputs the parser.

[0063] When determining the statement template sequence corresponding to the to-be-matched template sequence, the LCS algorithm is used to identify and extract the statement template sequence template corresponding to the to-be-matched template sequence. The statement template sequence with the highest matching degree with the to-be-matched template sequence is determined by calculating the longest common subsequence between each statement template sequence in the dynamic statement template sequence library and the to-be-matched template sequence, that is, the target statement template sequence, and the target statement template sequence is determined according to a preset sequence length threshold. Specifically, given the original SQL statement After token splitting, the template sequence s to be matched is obtained. The current LCSMap is traversed. For the i-th LCSObject, assuming that its sql_template is , whose value is Length . Traverse LCSMap and keep The index of the LCSObject corresponding to the maximum value. After the traversal is completed, if Greater than threshold , then it is believed that has the same statement template as the template sequence to be matched, and their LCS is the largest LCS among all LCSObjects in LCSMap, and The length of is greater than or equal to the preset sequence length threshold, such as half of the length of s. In addition, if there are multiple LCSObject Calculated with s are equal and both are maximum values, then choose The one with the shortest length is taken as the successfully matched LCSObject because it has a higher similarity with sequence s.

[0064] After the above process, LCS may have changed. This change will be more frequent when the amount of SQL statements entered is relatively small. Generate a new sql_template to overwrite the old one to represent all SQL statement entries and Finally, the sql_template of the jth LCSObject is Updated to , and the SQL statement The ID is added to the sql_id_list of the jth LCSObject. , can successfully match s under the LCS condition that the length is greater than or equal to the preset sequence length threshold, then in LCSMap Create a new LCSObject and set s itself as the new sql_template. This completes the input of a SQL statement. The entire parsing process is completed, and the final SQL statement parsing result list is the result of clustering and deduplication.

[0065] It can be seen that the present application uses the longest common subsequence algorithm to identify and extract the template corresponding to the SQL statement to be processed, so as to be adapted to clustering and deduplication of various complex SQL statements, and when the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, the template sequence to be matched itself is determined as the parsing template corresponding to the SQL statement to be processed, and then the template sequence to be matched is added to the dynamic statement template sequence library as a new statement template sequence, thereby realizing dynamic maintenance of the statement template sequence in the dynamic statement template sequence library, adapting to the constantly changing SQL data stream, so as to utilize the dynamic statement template sequence library to realize simple, efficient and stable parsing of various SQL statements, and complete clustering and deduplication of SQL statements.

[0066] See also Figure 5 As shown, the embodiment of the present application discloses a schematic diagram of a structure of a device for dynamically updating a SQL statement template, including:

[0067] The sequence acquisition module 11 is used to construct a template sequence to be matched of the SQL statement to be processed acquired in real time;

[0068] The template sequence matching module 12 is used to determine the longest common subsequence between each statement template sequence in the dynamic statement template sequence library and the template sequence to be matched based on the longest common subsequence algorithm, and determine the target statement template sequence corresponding to the template sequence to be matched from all the statement template sequences based on all the longest common subsequences;

[0069] A first template updating module 13 is configured to perform a first dynamic update operation on the dynamic statement template sequence library to update the target statement template sequence to the longest common subsequence between the target statement template sequence and the template sequence to be matched, if the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is greater than or equal to a preset sequence length threshold;

[0070] The second template updating module 14 is used to perform a second dynamic update operation on the dynamic statement template sequence library if the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, so as to add the template sequence to be matched as a new statement template sequence to the dynamic statement template sequence library.

[0071] It can be seen that the present application uses the longest common subsequence algorithm to identify and extract the template corresponding to the SQL statement to be processed, so as to be adapted to clustering and deduplication of various complex SQL statements, and when the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, the template sequence to be matched itself is determined as the parsing template corresponding to the SQL statement to be processed, and then the template sequence to be matched is added to the dynamic statement template sequence library as a new statement template sequence, thereby realizing dynamic maintenance of the statement template sequence in the dynamic statement template sequence library, adapting to the constantly changing SQL data stream, so as to utilize the dynamic statement template sequence library to realize simple, efficient and stable parsing of various SQL statements, and complete clustering and deduplication of SQL statements.

[0072] In a specific embodiment, the device may further include:

[0073] A statement quantity determination module is used to determine whether the number of SQL statements in the current preset SQL statement storage space meets the preset quantity requirement;

[0074] A statement acquisition module is used to trigger the operation of batch acquisition and inputting new SQL statements in a streaming input manner if the number of SQL statements in the current preset SQL statement storage space does not meet the preset number requirement, so as to make the number of SQL statements in the preset SQL statement storage space meet the preset number requirement, so as to obtain the SQL statements to be processed from the preset SQL statement storage space.

[0075] In a specific embodiment, the sequence acquisition module 11 may specifically include:

[0076] A parameter value replacement submodule, used to determine a target parameter in the SQL statement to be processed, and replace the parameter value of the target parameter in the SQL statement to be processed with a preset placeholder to obtain a target SQL statement;

[0077] The sequence construction submodule is used to split the target SQL statement based on a preset separator to obtain each Token corresponding to the target SQL statement, and construct a corresponding sequence based on all the Tokens to obtain the template sequence to be matched.

[0078] In a specific embodiment, the sequence construction submodule may specifically include:

[0079] a type determination unit, configured to classify the target SQL statement based on a type keyword in the target SQL statement to determine a statement type corresponding to the target SQL statement;

[0080] A statement classification unit, configured to determine a target keyword in the target SQL statement based on the statement type, and determine fixed content and non-fixed content in the SQL statement to be classified based on the target keyword;

[0081] The content splitting unit is used to take the fixed content as a Token and split the non-fixed content based on the preset separator to obtain tokens corresponding to the non-fixed content.

[0082] In a specific embodiment, the template sequence matching module 12 may specifically include:

[0083] a target sentence template sequence screening unit, used to determine each of the longest common subsequences with the largest sequence length from all the longest common subsequences, and to determine each of the sentence template sequences corresponding to each of the longest common subsequences with the largest sequence length from all the sentence template sequences;

[0084] The target sentence template sequence determination unit is used to select the sentence template sequence with the least number of tokens from the sentence template sequences corresponding to the longest common subsequences with the largest sequence length to obtain the target sentence template sequence.

[0085] In a specific embodiment, the device further includes:

[0086] a third template updating module, configured to determine the parsing template sequence corresponding to the SQL statement to be processed from the dynamic statement template sequence library, and insert the ID corresponding to the SQL statement to be processed into the ID list of the parsing template sequence to update the dynamic statement template sequence library;

[0087] The parsing result saving module is used to generate the parsing result corresponding to the SQL statement to be processed based on the updated dynamic statement template sequence library, and save the parsing result.

[0088] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0089] Figure 6 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the method for dynamically updating the SQL statement template disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0090] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0091] In addition, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0092] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the SQL statement template dynamic update method executed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0093] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed SQL statement template dynamic update method is implemented. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiment, and will not be repeated here.

[0094] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0095] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0096] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0097] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0098] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for dynamically updating a SQL statement template, characterized in that: include: Construct a template sequence to be matched for the SQL statements to be processed obtained in real time; Determine the longest common subsequence between each statement template sequence in the dynamic statement template sequence library and the template sequence to be matched based on the longest common subsequence algorithm, and determine the target statement template sequence corresponding to the template sequence to be matched from all the statement template sequences based on all the longest common subsequences; If the sequence length of the longest common subsequence between the target statement template sequence and the to-be-matched template sequence is greater than or equal to a preset sequence length threshold, a first dynamic update operation is performed on the dynamic statement template sequence library to update the target statement template sequence to the longest common subsequence between the target statement template sequence and the to-be-matched template sequence; If the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, a second dynamic update operation is performed on the dynamic statement template sequence library to add the template sequence to be matched as a new statement template sequence to the dynamic statement template sequence library.

2. The method for dynamically updating SQL statement templates according to claim 1, characterized in that: Before constructing the template sequence to be matched of the SQL statement to be processed obtained in real time, the process further includes: Determine whether the number of SQL statements in the current preset SQL statement storage space meets the preset number requirement; If the number of SQL statements in the current preset SQL statement storage space does not meet the preset number requirement, then trigger the batch acquisition operation of inputting new SQL statements in a streaming input manner so that the number of SQL statements in the preset SQL statement storage space meets the preset number requirement, so as to obtain the SQL statements to be processed from the preset SQL statement storage space.

3. The method for dynamically updating SQL statement templates according to claim 1, characterized in that: The step of constructing a template sequence to be matched of the SQL statement to be processed obtained in real time includes: Determine a target parameter in the SQL statement to be processed, and replace a parameter value of the target parameter in the SQL statement to be processed with a preset placeholder to obtain a target SQL statement; The target SQL statement is split based on a preset separator to obtain tokens corresponding to the target SQL statement, and a corresponding sequence is constructed based on all the tokens to obtain the template sequence to be matched.

4. The method for dynamically updating SQL statement templates according to claim 3, characterized in that: The step of splitting the target SQL statement based on a preset delimiter to obtain tokens corresponding to the target SQL statement includes: Classifying the target SQL statement based on a type keyword in the target SQL statement to determine a statement type corresponding to the target SQL statement; Determine a target keyword in the target SQL statement based on the statement type, and determine fixed content and non-fixed content in the SQL statement to be classified based on the target keyword; The fixed content is taken as a Token, and the non-fixed content is split based on the preset separator to obtain tokens corresponding to the non-fixed content.

5. The method for dynamically updating SQL statement templates according to claim 1, characterized in that: The determining, based on all the longest common subsequences, a target statement template sequence corresponding to the to-be-matched template sequence from all the statement template sequences comprises: Determine each of the longest common subsequences with the largest sequence length from all of the longest common subsequences, and determine each of the statement template sequences corresponding to each of the longest common subsequences with the largest sequence length from all of the statement template sequences; The sentence template sequence with the least number of tokens is selected from the sentence template sequences corresponding to the longest common subsequences with the largest sequence length to obtain the target sentence template sequence.

6. The method for dynamically updating SQL statement templates according to claim 1, characterized in that: The preset sequence length threshold is a sequence length threshold determined based on the ratio of the number of word units in the template sequence to be matched and a preset threshold.

7. The method for dynamically updating a SQL statement template according to any one of claims 1 to 6, characterized in that: Also includes: Determine the parsing template sequence corresponding to the SQL statement to be processed from the dynamic statement template sequence library, and insert the ID corresponding to the SQL statement to be processed into the ID list of the parsing template sequence to update the dynamic statement template sequence library; Generate a parsing result corresponding to the SQL statement to be processed based on the updated dynamic statement template sequence library, and save the parsing result.

8. A SQL statement template dynamic update device, characterized in that: include: A sequence acquisition module is used to construct a template sequence to be matched for the SQL statement to be processed obtained in real time; A template sequence matching module, used to determine the longest common subsequence between each statement template sequence in the dynamic statement template sequence library and the template sequence to be matched based on the longest common subsequence algorithm, and determine the target statement template sequence corresponding to the template sequence to be matched from all the statement template sequences based on all the longest common subsequences; A first template updating module, configured to perform a first dynamic update operation on the dynamic statement template sequence library to update the target statement template sequence to the longest common subsequence between the target statement template sequence and the template sequence to be matched, if the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is greater than or equal to a preset sequence length threshold; The second template updating module is used to perform a second dynamic update operation on the dynamic statement template sequence library if the sequence length of the longest common subsequence between the target statement template sequence and the template sequence to be matched is less than the preset sequence length threshold, so as to add the template sequence to be matched as a new statement template sequence to the dynamic statement template sequence library.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is used to execute the computer program to implement the SQL statement template dynamic update method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method for dynamically updating a SQL statement template according to any one of claims 1 to 7 is implemented.

Images