Structured query language similarity detection method, device and equipment

Abstract

The application discloses a structured query statement similarity detection method, device and equipment, obtains a first structured query statement and a second structured query statement, and constructs corresponding first and second abstract syntax trees. The correlation degree of the root node in the abstract syntax tree with the query result of the statement is higher than that of other nodes with the query result of the statement. The nodes in the first abstract syntax tree are traversed, the traversed nodes are matched with nodes in the same level in the second abstract syntax tree, and a matching result of the nodes is obtained, which indicates whether there is a node in the same level in the second abstract syntax tree matched with the node in the first abstract syntax tree. The statement similarity of the first structured query statement and the second structured query statement is determined in combination with the matching result of the root node in the first abstract syntax tree, the matching result of the child nodes of the root node and respective corresponding weights. In this way, the statement does not need to be executed in cooperation with a database, and the detection cost is lower.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a structured query statement similarity detection method, device and equipment. BACKGROUND

[0002] The structured query statement similarity detection is mainly used for detecting whether the semantics expressed by multiple structured query statements are the same. The structured query statement similarity detection can be applied to query rewriting, statement duplication detection and the like scenes.

[0003] At present, multiple structured query statements can be directly executed, the query result of each structured query statement is obtained, and the similarity of the multiple structured query statements is determined by judging whether the multiple query results are consistent. However, the structured query statement needs to be executed in cooperation with a database. In many scenes, a database cannot be provided, or if a database can be provided, the cost of database query can be high, so that the detection cost of the detection method is high. SUMMARY

[0004] Therefore, the present application provides a structured query statement similarity detection method, device and equipment, which does not need to execute the structured query statement in cooperation with a database, so that the detection cost of the method is low.

[0005] To solve the above problems, the technical scheme provided by the present application is as follows:

[0006] In a first aspect, the present application provides a structured query statement similarity detection method, which comprises:

[0007] Obtaining a first structured query statement and a second structured query statement, and constructing a first abstract syntax tree of the first structured query statement and a second abstract syntax tree of the second structured query statement; the abstract syntax tree is used to represent the abstract syntax structure of the structured query statement, and the root node in the abstract syntax tree has a higher correlation degree with the query result of the structured query statement than other nodes;

[0008] Traversing the nodes in the first abstract syntax tree, matching the nodes in the first abstract syntax tree traversed with the nodes in the same level of the second abstract syntax tree, and obtaining the matching result of the nodes in the first abstract syntax tree; the matching result of the nodes in the first abstract syntax tree indicates whether there is a node in the same level of the second abstract syntax tree matched with the node in the first abstract syntax tree;

[0009] The statement similarity between the first structured query statement and the second structured query statement is determined according to the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node.

[0010] In a second aspect, the present application provides a structured query statement similarity detection device, the device comprising:

[0011] A construction unit is configured to obtain a first structured query statement and a second structured query statement, and construct a first abstract syntax tree of the first structured query statement and a second abstract syntax tree of the second structured query statement. The abstract syntax tree is used to represent the abstract syntax structure of the structured query statement. The root node in the abstract syntax tree has a higher correlation degree with the query result of the structured query statement than other nodes.

[0012] A traversal unit is configured to traverse a node in the first abstract syntax tree, match the node in the first abstract syntax tree that is traversed with a node in the same level in the second abstract syntax tree, and obtain a matching result of the node in the first abstract syntax tree. The matching result of the node in the first abstract syntax tree indicates whether there is a node in the same level in the second abstract syntax tree that matches the node in the first abstract syntax tree.

[0013] A first determination unit is configured to determine the statement similarity between the first structured query statement and the second structured query statement according to the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node.

[0014] In a third aspect, the present application provides an electronic device comprising:

[0015] One or more processors;

[0016] A storage device having one or more programs stored thereon,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the structured query statement similarity detection method according to the first aspect.

[0018] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the structured query statement similarity detection method according to the first aspect.

[0019] Therefore, the application has the following beneficial effects:

[0020] The application provides a structured query statement similarity detection method, device and equipment. A first structured query statement and a second structured query statement which need to be detected for similarity are obtained, and a first abstract syntax tree of the first structured query statement and a second abstract syntax tree of the second structured query statement are constructed. The abstract syntax tree is used to represent an abstract syntax structure of the structured query statement, wherein a root node in the abstract syntax tree has a higher correlation degree with a query result of the structured query statement than other nodes. Nodes in the first abstract syntax tree are traversed, and the traversed nodes in the first abstract syntax tree are matched with nodes in the same level in the second abstract syntax tree, and a matching result of the nodes in the first abstract syntax tree is obtained. The matching result of the nodes indicates whether there is a node in the same level in the second abstract syntax tree which matches the nodes in the first abstract syntax tree, and if there is, the matching result is matching, that is, a syntax structure represented by the nodes in the first structured query statement is the same in semantics as a syntax structure represented by the matching nodes in the same level in the second structured query statement. Furthermore, different weights are assigned to the matching result of the root node in the first abstract syntax tree and the matching result of the child nodes of the root node, and the different weights represent different influence degrees on the statement similarity. Therefore, the statement similarity of the first structured query statement and the second structured query statement can be determined in combination with the matching result of the root node in the first abstract syntax tree, the matching result of the child nodes of the root node and the respective corresponding weights. It can be known that in the structured query statement similarity detection method, the structured query statement does not need to be executed in cooperation with a database, and the detection cost is reduced to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A framework schematic diagram of an exemplary application scenario provided by the embodiment of the application is shown in the figure.

[0022] Figure 2 A flowchart of a structured query statement similarity detection method provided by the embodiment of the application is shown in the figure.

[0023] Figure 3 A schematic diagram of an abstract syntax tree provided by the embodiment of the application is shown in the figure.

[0024] Figure 4a A schematic diagram of a first abstract syntax tree provided by the embodiment of the application is shown in the figure.

[0025] Figure 4b A schematic diagram of a second abstract syntax tree provided by the embodiment of the application is shown in the figure.

[0026] Figure 4c A schematic diagram of a matching result provided by the embodiment of the application is shown in the figure.

[0027] Figure 4d Another schematic diagram of a matching result provided by an embodiment of the present application is provided.

[0028] Figure 5a Another schematic diagram of a second abstract syntax tree provided by an embodiment of the present application is provided.

[0029] Figure 5b Another schematic diagram of a matching result provided by an embodiment of the present application is provided.

[0030] Figure 6 A schematic diagram of a score of a root node provided by an embodiment of the present application is provided.

[0031] Figure 7 A schematic diagram of a structured query statement similarity detection device provided by an embodiment of the present application is provided.

[0032] Figure 8 A schematic diagram of a basic structure of an electronic device provided by an embodiment of the present application is provided. DETAILED DESCRIPTION

[0033] In order to make the above objectives, characteristics and advantages of the present application more apparent, comprehensible and easy to understand, the embodiments of the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0034] In order to facilitate understanding and explaining the technical solutions provided by the embodiments of the present application, the background art of the present application will be described first.

[0035] Structured query statement similarity detection is mainly used to detect whether the semantics expressed by multiple structured query statements (i.e., SQL statements) are the same. Structured query statement similarity is used to analyze and quantify the degree of semantic similarity between multiple SQL statements. The similarity of SQL statements can be applied to scenarios such as repeatability detection, query rewriting, database query performance optimization, code completion task evaluation, etc.

[0036] At present, the similarity detection of SQL statements can be realized by two matching modes, i.e., accurate set matching and execution matching. In the accurate set matching mode, two SQL statements (multiple SQL statements are similar) are first divided into multiple sentences respectively, it is judged whether the texts of the sentences under the two SQL statements are consistent, so as to determine the similarity of the sentences, and finally the similarities of the sentences are summarized to obtain the similarity of the two SQL statements. The accurate set matching mode makes the matching between the sentences more accurate, but it only considers the accurate matching of the texts and does not consider the case that the texts of two sentences may be different but the semantics are the same. For example, the texts of the SQL statement select count(*) from table and the SQL statement select count(1) from table are different, but the semantics of the statements are the same.

[0037] In the execution matching manner, the query results are obtained by directly executing two SQL statements. Then, whether the query results are consistent is compared to determine the statement similarity of the two SQL statements. It can be known that the query results reflect whether the semantics of the two SQL statements are the same to a certain extent. However, this manner usually involves a database query operation with high execution cost, so that the detection cost of the SQL statement similarity is high.

[0038] Based on this, an embodiment of the present application provides a structured query statement similarity detection method, device and equipment. A first structured query statement and a second structured query statement which need to be detected for similarity are obtained, and a first abstract syntax tree of the first structured query statement and a second abstract syntax tree of the second structured query statement are constructed. The abstract syntax tree is used to represent the abstract syntax structure of the structured query statement, wherein the correlation degree of the root node in the abstract syntax tree with the query result of the structured query statement is higher than the correlation degree of other nodes with the query result of the structured query statement. The nodes in the first abstract syntax tree are traversed, the nodes in the first abstract syntax tree traversed are matched with the nodes in the same level of the second abstract syntax tree, and the matching result of the nodes in the first abstract syntax tree is obtained. The matching result of the node indicates whether there is a node in the same level in the second abstract syntax tree which is matched with the node in the first abstract syntax tree, if there is, the matching result is matched, that is, the syntax structure represented by the node in the first structured query statement is the same in semantics with the syntax structure represented by the matched node in the same level in the second structured query statement. Then, different weights are assigned to the matching result of the root node in the first abstract syntax tree and the matching result of the child nodes of the root node, and the different weights represent different influence degrees on the statement similarity. Thus, the statement similarity of the first structured query statement and the second structured query statement can be determined in combination with the matching result of the root node in the first abstract syntax tree, the matching result of the child nodes of the root node and the respective corresponding weights. It can be known that in the structured query statement similarity detection method, the structured query statement does not need to be executed in combination with the database, so that the detection cost of the method is low to a certain extent.

[0039] It can be understood that the defects of the above solutions are the results obtained by the applicant after practice and careful study. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of the present application to solve the above problems in the following should be the contributions made by the applicant to the embodiments of the present application in the process of the present application.

[0040] In order to facilitate understanding of the structured query statement similarity detection method provided by the embodiments of the present application, the following will be described in combination with the scene examples shown in Figure 1 Figure 1 ​As shown in the figure, the figure is a framework schematic diagram of an exemplary application scenario provided by an embodiment of the present application. The method can be executed by an electronic device, such as a terminal device and / or a server, etc., which is not limited here and can be determined according to actual application scenarios.

[0041] As shown in the figure, Figure 1 As shown in the figure, the input of the framework schematic diagram is a first structured query statement and a second structured query statement that need to be detected for similarity. The first structured query statement can be represented as p_sql, and the second structured query statement can be represented as g_sql. After obtaining the first structured query statement and the second structured query statement, the first structured query statement and the second structured query statement can be standardized first, so that the syntax structures with different texts but the same semantics in the first structured query statement and the second structured query statement are uniformly represented after standardization, so as to facilitate semantic matching. The first structured query statement and the second structured query statement after standardization can be represented as p_std_sql and g_std_sql respectively.

[0042] A first abstract syntax tree of the first structured query statement p_std_sql after standardization and a second abstract syntax tree of the second structured query statement g_std_sql after standardization are constructed. The abstract syntax tree is used to represent the abstract syntax structure of the structured query statement, and the root node in the abstract syntax tree has a higher degree of association with the query result of the structured query statement than other nodes. The first abstract syntax tree and the second abstract syntax tree can be represented as p_tree and g_tree respectively.

[0043] The nodes in the first abstract syntax tree p_tree are traversed, and the nodes in the first abstract syntax tree p_tree that are traversed are matched with the nodes in the second abstract syntax tree g_tree at the same level, and the matching result of the nodes in the first abstract syntax tree p_tree is obtained. The matching result of the node indicates whether there is a node at the same level in the second abstract syntax tree g_tree that matches the node in the first abstract syntax tree p_tree. If there is, the matching result is matching, otherwise, the matching result is not matching. The first abstract syntax tree in which the matching result of the node is identified can be represented as p_matched_tree.

[0044] Finally, the statement similarity of the first structured query statement and the second structured query statement can be determined in combination with the matching result of the root node in the p_matched_tree, the matching result of the child node of the root node, and the respective corresponding weights, and the statement similarity is output.

[0045] Those skilled in the art can understand, Figure 1The illustrated framework diagram is merely one example in which embodiments of the present application can be implemented. The scope of applicability of embodiments of the present application is not limited by any aspect of this framework.

[0046] To facilitate understanding of the present application, a structured query statement similarity detection method provided by an embodiment of the present application is described below in conjunction with the accompanying drawings.

[0047] Referring to Figure 2 As shown in the figure, it is a flowchart of a structured query statement similarity detection method provided by an embodiment of the present application, which can include S201-S203: Figure 2 As shown in the figure, the method can include S201-S203:

[0048] S201: Obtain a first structured query statement and a second structured query statement, and construct a first abstract syntax tree of the first structured query statement and a second abstract syntax tree of the second structured query statement; the abstract syntax tree is used to represent the abstract syntax structure of the structured query statement, and the root node in the abstract syntax tree has a higher correlation degree with the query result of the structured query statement than other nodes.

[0049] The first structured query statement and the second structured query statement are structured query statements that need to be detected for similarity. The first structured query statement and the second structured query statement are both composed of multiple syntax structures, including variables, functions, keywords, table names, column names, conditions, operators, subqueries, etc. As an optional example, before constructing the corresponding abstract syntax trees for the first structured query statement and the second structured query statement, the first structured query statement and the second structured query statement can be standardized first, so that the syntax structures with different texts but the same semantics in the first structured query statement and the second structured query statement are uniformly represented after standardization, which facilitates eliminating the diversity of semantic expressions and speeding up the subsequent semantic matching process.

[0050] Specifically, the standardization of the structured query statements can be performed through a self-defined standardization process, which is not limited here. For example, the first structured query statement before standardization is: select * from (select product_name, price from product where type == 'toy' group by product_name, price;) where price > 17. The first structured query statement after standardization is: select product_name, price from product where type == 'toy' and price > 17 group by product_name, price. The second structured query statement before standardization is: select t.product_name from (select product_name, price from product where type == 'toy' group by product_name, price;) as t; where t.price > 17. The second structured query statement after standardization is: select product_name from product where type == 'toy' and price > 17 group by product_name, price. As can be seen from the above examples, the first structured query statement and the second structured query statement after standardization are unified in terms of syntax structure.

[0051] Further, an Abstract Syntax Tree (AST) corresponding to the first structured query statement after standardization is constructed, which can be referred to as a first AST. An AST corresponding to the second structured query statement after standardization is constructed, which can be referred to as a second AST. The AST is used to represent the abstract syntax structure of the structured query statement. For example, the AST can be constructed using SQL Parser. Specifically, the first structured query statement and the second structured query statement are parsed into the first AST and the second AST using SQL Parser.

[0052] The abstract syntax tree is a tree-shaped data structure composed of nodes. Each node in the tree represents a syntax structure in the structured query statement. The nodes include a root node and child nodes, and the child nodes include a clause type node and other attribute nodes. The syntax structure represented by the clause type node can be a keyword (such as "select", "from", "where", and "group") in the structured query statement, and the syntax structure represented by the attribute node can be a table name, a column name, an operator, and the like in the structured query statement. Each node in the abstract syntax tree corresponds to a node type and a node attribute value. Generally, the node attribute values of the root node and the clause type node are empty. The abstract syntax tree is divided into multiple levels from top to bottom starting from the root node, and the abstract syntax tree is formed through the hierarchical relationship between multiple nodes.

[0053] Referring to Figure 3 , Figure 3 An abstract syntax tree provided by an embodiment of the present application is shown in the figure. The standardized structured query statement is select product_name, color from product where type == 'toy' group by product_name, color. The abstract syntax tree constructed based on the structured query statement is shown in Figure 3 , and includes four levels, which can be the first level to the fourth level from top to bottom. The root node of the first level is a Query Type query type, and the query type is a root type, indicating the query operation of the structured query statement. The next level node of the root node is four clause type nodes, and the node types of the nodes are Select Type, From Type, Where Type, and Group Type, respectively, indicating the keywords "select", "from", "where", and "group" in the structured query statement. The next level nodes of the clause type nodes include six attribute nodes, for example, the node type of the first attribute node is Field Type, indicating the column name in the structured query statement, and the node attribute value is column: product_name, that is, the column name is product_name. The node type of the third attribute node is Table Type, indicating the table name in the structured query statement, and the node attribute value is name: product, that is, the table name is product. The product_name is the column name of the column in the product data table. The node type of the fourth attribute node is Operator Type, indicating the operator in the structured query statement, and the node attribute value is type: eq, that is, the operator is "equal". The fourth level includes two attribute nodes, which will not be described here.

[0054] Typically, the syntax structures represented by hierarchical nodes farther from the root node are executed first in a structured query, while those represented by hierarchical nodes closer to the root node are executed later. The syntax structures represented by leaf nodes in the abstract syntax tree are executed first in a structured query. For example... Figure 3 As shown, the root node represents the query type, signifying the entire query operation of the structured query statement. After the query operation, the query result of the structured query statement can be obtained. Therefore, the correlation between the root node of the query type and the query result of the structured query statement can be considered higher than the correlation between other nodes and the query result of the structured query statement. For example, if the structured query statement is "1+1", the root node in its generated abstract syntax tree is the "add" operation, and both child nodes are "1". Since the query result of the structured query statement can be obtained after executing the "add" operation, the correlation between the root node "add" and the query result of the structured query statement is considered higher than the correlation between the child node "1" and the query result of the structured query statement.

[0055] Understandably, after constructing the corresponding abstract syntax tree based on the structured query statement, the abstract syntax tree can represent the abstract syntactic structure of the structured query statement. Therefore, based on the abstract syntax tree, the keywords, table names, column names, conditions, operators, etc., in the structured query statement can be determined relatively accurately. Thus, detecting the statement similarity between the first and second structured query statements is transformed into detecting the similarity between the first and second abstract syntax trees.

[0056] S202: Traverse the nodes in the first abstract syntax tree, match the traversed nodes in the first abstract syntax tree with the same level nodes in the second abstract syntax tree, and obtain the matching results of the nodes in the first abstract syntax tree; the matching results of the nodes in the first abstract syntax tree indicate whether there are any same level nodes in the second abstract syntax tree that match the nodes in the first abstract syntax tree.

[0057] For example, the similarity between the first abstract syntax tree (AST) and the second abstract syntax tree can be determined by the matching results of nodes in the first and second ASTs. Specifically, the nodes in the first AST are traversed. Since nodes at the same level in different trees are more likely to match successfully, the nodes in the first AST are matched with the nodes at the same level in the second AST to obtain the matching results of the nodes in the first AST. The number of nodes at the same level in the second AST can be one or more.

[0058] The matching result of the node indicates whether a node at the same level in the second abstract syntax tree matches the node in the first abstract syntax tree. Further, 1 can be assigned to a node in the first abstract syntax tree that is successfully matched, and 0 can be assigned to a node in the first abstract syntax tree that is not successfully matched, to distinguish the matching results of the nodes. Generally, the more nodes that can be matched, the higher the similarity between the first abstract syntax tree and the second abstract syntax tree, and the higher the statement similarity between the first structured query statement and the second structured query statement.

[0059] In a possible implementation, the embodiment of the present application provides a specific implementation of traversing a node in a first abstract syntax tree, matching the node in the first abstract syntax tree that is traversed with a node at the same level in a second abstract syntax tree, and obtaining a matching result of the node in the first abstract syntax tree, including:

[0060] A1: traversing nodes in the first abstract syntax tree in a hierarchical manner according to a bottom-up hierarchical structure of the first abstract syntax tree.

[0061] Since a syntax structure represented by a hierarchical node far from the root node is executed first in a structured query statement, and a syntax structure represented by a hierarchical node close to the root node is executed later in the structured query statement, the leaf nodes of the first abstract syntax tree are traversed first, the nodes are traversed in a hierarchical manner from bottom to top, and the root node of the first abstract syntax tree is traversed last.

[0062] A2: determining a first node in the first abstract syntax tree that is traversed, and a second node at the same level as the first node in the second abstract syntax tree.

[0063] Each node in the first abstract syntax tree that is traversed each time is referred to as a first node, and a node at the same level as the first node in the second abstract syntax tree is referred to as a second node. The number of second nodes can be one or more.

[0064] A3: judging whether the node types of the first node and the second node are the same and whether the node attribute values of the first node and the second node satisfy an attribute value matching rule, and obtaining a matching result of the first node according to a judgment result.

[0065] Each node in the abstract syntax tree corresponds to a node type and a node attribute value. The matching result of the node can be determined by matching the node type and the node attribute value. Specifically, when the node types of the first node and the second node are the same and the node attribute values of the first node and the second node satisfy the attribute value matching rule, it is determined that the matching result of the first node is matching, that is, the first node and the second node match. Otherwise, it is determined that the matching result of the first node is not matching, that is, the first node and the second node do not match.

[0066] As an optional example, the attribute value matching rule is that the node attribute values of the first node and the second node are the same or the node attribute values of the first node and the second node are the same in semantics. It can be understood that texts of different syntax structures can be different but the same in semantics, and the syntax structures are similar in semantics, and the corresponding nodes can be considered to be matched. The rule for judging whether the node attribute values of the first node and the second node are the same in semantics can be implemented by self-definition, which is not limited here and can be flexibly extended. For example, the self-defined rule includes various node attribute values that are the same in semantics, for example, the semantics of BJ and beijing are the same.

[0067] It can be understood that when the node attribute value of the node is empty, only whether the node types of the first node and the second node are the same can be judged.

[0068] A4: When the number of the second nodes is multiple, the matching results of the multiple first nodes are compared, and the matching result of the first node is re-determined.

[0069] It can be understood that when the number of the second nodes is multiple, the first node needs to be matched with each second node at the same level to obtain the matching results of the multiple first nodes. Further, the matching results can be quantified, and the quantification manner is not limited here, for example, score quantification. Thus, the matching results of the multiple first nodes are compared, and the matching result with the highest matching degree (i.e., the highest score) is selected as the final matching result of the first node.

[0070] As can be seen from A1-A4, whether the first node and the second node are matched can be determined from the node type and the node attribute value of the node. If matched, it indicates that the node types of the first node and the second node are the same, and the node attribute values are the same in semantics.

[0071] In addition, in a possible implementation, after A3 and before A4, the method further includes:

[0072] B1: determining a first child node at a neighboring level of the first node and a second child node at a neighboring level of the second node; the second child node is at the same level as the first child node.

[0073] B2: judging whether the node types of the first child node and the second child node are the same and whether the node attribute values of the first child node and the second child node satisfy the attribute value matching rule, and obtaining the matching result of the first child node according to the judgment result.

[0074] It can be known that the technical implementation of steps B1 and B2 is similar to that of steps A2 and A3, which is not described here.

[0075] B3: Redetermine the matching result of the first node according to the matching result of the first node, the weight corresponding to the matching result of the first node, the matching result of the first child node, and the weight corresponding to the matching result of the child node of the first node.

[0076] The sum of the weight corresponding to the matching result of the first node and the weight corresponding to the matching result of the child node of the first node is 1.

[0077] That is, when determining the matching result of the first node, not only the matching result of the first node itself is considered, but also the matching result of the child node of the first node in the adjacent level is considered, so that the matching result of the first node determined again is more accurate. When the weight corresponding to the matching result of the first node is higher than the weight corresponding to the matching result of the child node of the first node, it indicates that the matching result of the first node itself is more important.

[0078] In a possible implementation, the embodiment of the application provides a specific implementation of redetermining the matching result of the first node according to the matching result of the first node, the weight corresponding to the matching result of the first node, the matching result of the first child node, and the weight corresponding to the matching result of the child node of the first node, which comprises the following steps.

[0079] B31: Determine the score of the first node according to the matching result of the first node.

[0080] The first node in the first abstract syntax tree can be represented as n, and each second node in the same level in the second abstract syntax tree can be represented as n i . i ∈ {0, 1,..., l}, and l represents the number of second nodes in the same level. The matching result of the first node is obtained through the matching of the first node n and the second nodes n i in the same level, and the corresponding score of the first node can be represented as Specifically,

[0081]

[0082] type(n)==type(n i ) indicates that the node types of the first node and the second node are the same, (val(n)==val(n i ) ∨ custom_fn(n, n i ) indicates that the node attribute values of the first node and the second node satisfy the attribute value matching rule. val(n)==val(n i ) indicates that the node attribute values of the first node and the second node are the same, and custom_fn(n, n i) represents that the node attribute values of the first node and the second node are semantically identical. Then, when the node types of the first node and the second node are identical and the node attribute values of the first node and the second node satisfy the attribute value matching rule, it is determined that the matching result of the first node is matching, at this time, the score of the first node is 1, otherwise, it is 0. It can be understood that the specific score of the first node is not limited here, and other scores can also be used.

[0083] B32: determining the score of the first sub-node according to the matching result of the first sub-node.

[0084] The score of the first sub-node can be represented as for quantifying the matching result between the first sub-node and the second sub-node.

[0085] In actual application, when the first sub-node has no next adjacent level sub-node, the determination process of the score of the first sub-node is the same as the acquisition process of m in B31. When the first sub-node has a next adjacent level sub-node, the score of the first sub-node is determined according to the matching result of the first sub-node and the matching result of the next adjacent level sub-node of the first sub-node. That is, the acquisition process of m i in B34. In the determination of the score of the first sub-node, the weight corresponding to the matching result of the first sub-node and the weight corresponding to the sub-node matching result of the first sub-node can also use α and (1-α) below, which are not limited here. The weight corresponding to the sub-node matching result of the first sub-node is used to represent the influence degree of the matching result of the next adjacent level sub-node of the first sub-node on the score of the first sub-node.

[0086] B33: when the number of the first sub-nodes is multiple, the average value of the scores of the multiple first sub-nodes is determined as the sub-node score of the first node.

[0087] B34: the score of the first node, the weight corresponding to the matching result of the first node, the sub-node score of the first node, and the weight corresponding to the sub-node matching result of the first node are weighted and summed to reacquire the score of the first node.

[0088] The reacquired score of the first node can be represented as m i for quantifying the matching result between the first sub-node n and the second node n i of the same level. The reacquired score m i of the first node can be represented as:

[0089]

[0090] wherein α is an adjustment factor, which can also be referred to as the weight corresponding to the matching result of the first node. (1-α) is the weight corresponding to the sub-node matching result of the first node.

[0091] B35: determining the matching result of the first node based on the score of the first node.

[0092] For example, when the score of the first node is greater than the score threshold, the matching result of the first node is determined as matching, otherwise, as non-matching. Here, the matching threshold is not limited and can be determined according to actual conditions. For example, when the full score is 1, the score threshold can be 0.5.

[0093] In addition, when the matching result of the first node is multiple, the scores of the multiple first nodes can be obtained, and the highest score m is determined by comparing the scores of the multiple first nodes, that is, m = max(m i ). The highest score indicates the highest matching degree after quantization of the matching result. The second node with the highest matching degree can be regarded as the node in the second abstract syntax tree that finally matches the first node.

[0094] Referring to Figure 4a and Figure 4b , Figure 4a is a schematic diagram of a first abstract syntax tree provided by an embodiment of the present application; Figure 4b is a schematic diagram of a second abstract syntax tree provided by an embodiment of the present application. Figure 4a The standardized first structured query statement represented by the first abstract syntax tree in Figure 4b The standardized first structured query statement represented by the second abstract syntax tree in Figure 4c and Figure 4d , Figure 4c is a schematic diagram of a matching result provided by an embodiment of the present application; Figure 4d is another schematic diagram of a matching result provided by an embodiment of the present application. Figure 4c shows the matching result of the node of the first abstract syntax tree when α is less than 0.5, Figure 4d shows the matching result of the node of the first abstract syntax tree when α is greater than 0.5. As shown in Figure 4c and Figure 4d , the matching result of the node is different when α is less than 0.5 or α is greater than 0.5. Figure 4c and Figure 4dThe node in the dark background color represents that the matching result of the node is not matched (i.e., there is no node in the same level in the second abstract syntax tree matched with the node), and the node in the light background color represents that the matching result of the node is matched (i.e., there is a node in the same level in the second abstract syntax tree matched with the node). Thus, a suitable adjustment factor can be selected according to actual requirements. For example, if it is desired to emphasize the influence of the matching result of the first node on the score of the first node, the corresponding adjustment factor can be increased.

[0095] Based on B1-B3, the matching result of the first node can be determined in combination with the matching result of the first node itself and the matching result of the adjacent level child node of the first node, so that the matching result of the first node is more accurate.

[0096] Based on B31-B35, the matching result of the first node can be quantified in quantity, and the matching result of the first node can be determined by the score of the first node, so that the matching result of the first node can be more intuitively represented.

[0097] S203: Determine the sentence similarity of the first structured query statement and the second structured query statement in combination with the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node.

[0098] The weight corresponding to the matching result of the root node and the weight corresponding to the matching result of the child node of the root node are different, indicating that the matching result of the root node in the first abstract syntax tree and the matching result of the child node of the root node have different influence degrees on the sentence similarity, and the respective corresponding weights can be determined according to actual conditions.

[0099] The sum of the weight corresponding to the matching result of the root node and the weight corresponding to the matching result of the child node of the root node is 1. The different weights indicate different influence degrees on the sentence similarity of the first structured query statement and the second structured query statement.

[0100] In a possible implementation, the embodiment of the present application provides a specific implementation of determining the sentence similarity of the first structured query statement and the second structured query statement in combination with the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node, including:

[0101] C1: Determine the score of the root node in the first abstract syntax tree according to the matching result of the root node in the first abstract syntax tree.

[0102] After determining the matching result (match or no match) of each node in the first abstract syntax tree, the matching result of the node in the first abstract syntax tree can be quantified by using a score.

[0103] For example, the score of the node can be represented as When the matching result is match, the score of the node can be determined as 1, otherwise 0. It can be seen that the specific score is not limited here, and is only used as an example for illustration.

[0104] C2: According to the matching result of the adjacent level child node of the root node, the score of the adjacent level child node of the root node is determined, and the average value of the scores of the plurality of adjacent level child nodes is determined as the child node score of the root node.

[0105] The child node score of the node is represented as Wherein, j is each adjacent level child node of the node, M is the number of adjacent level child nodes of the node, s j is the score of each adjacent level child node of the node.

[0106] C3: The score of the root node, the weight corresponding to the matching result of the root node, the child node score of the root node, and the weight corresponding to the matching result of the child node of the root node are weighted and summed to reacquire the score of the root node.

[0107] The reacquired score s of the node can be represented as Wherein, β represents the weight corresponding to the matching result of the node, (1-β) represents the weight corresponding to the matching result of the child node of the node. Wherein, when the node attribute value corresponding to the node is empty, the weight corresponding to the matching result of the node is 0.

[0108] It can be seen that when the node is the root node, the various formulas in C1-C3 above are applicable. In actual application, the score of the leaf node can be calculated first, then the score of each node in each level can be calculated from bottom to top, and finally the score of the root node can be obtained.

[0109] C4: Based on the score of the root node, the sentence similarity of the first structured query statement and the second structured query statement is determined.

[0110] Since the correlation degree of the root node and the query result of the structured query statement is higher than that of other nodes and the query result of the structured query statement, in order to adapt to the execution order of the nodes in the abstract syntax tree in the code, the score of the root node is used to determine the sentence similarity of the first structured query statement and the second structured query statement. When the score of the root node is higher, the sentence similarity of the first structured query statement and the second structured query statement is higher.

[0111] Referring to Figure 5a - Figure 5b ,Figure 5a Another schematic diagram of a second abstract syntax tree provided for an embodiment of the present application, Figure 5b Another schematic diagram of a matching result provided for an embodiment of the present application. Take "select product_name, color from product where type == 'toy' group by product_name, color" as a first structured query statement, a first abstract syntax tree obtained is as shown in Figure 3 Take "select product_name from product where type == 'clothes'" as a second structured query statement, a second abstract syntax tree obtained is as shown in Figure 5a The matching result of each node in the first abstract syntax tree obtained after step S202 is as shown in Figure 5b When the node is in dark color background, it means that the matching result of the node is mismatch, and when the node is in light color background, it means that the matching result of the node is match. See Figure 6 Figure 6 A schematic diagram of a score of a root node provided for an embodiment of the present application. As shown in Figure 6 The score of a leaf node is 1 because the leaf node has no child node. Take an Operator Type node in the third level as an example, it has two child nodes, and set β as 0.5, then the score of the node is 0.5x1+0.5x((1+0) / 2)=0.75. Since the node attribute value of the Where Type node in the second level is empty, the weight corresponding to the matching result of the node is 0, and the weight corresponding to the matching result of the child node is 1, then the score of the node is 1x0.75=0.75. That is, when the node attribute value of a node is empty and the number of child nodes of the node is one, the score of the node is equal to the score of the child node. The score of the root node is 0.5625, and the statement similarity of the first structured query statement and the second structured query statement can be evaluated according to the score.

[0112] ​Based on the related content of S201-S203, the first structured query statement and the second structured query statement requiring similarity detection are obtained, and the first abstract syntax tree of the first structured query statement and the second abstract syntax tree of the second structured query statement are constructed. The abstract syntax tree is used to represent the abstract syntax structure of the structured query statement, wherein the root node in the abstract syntax tree has a higher correlation degree with the query result of the structured query statement than other nodes. By traversing the abstract syntax tree, syntax analysis, semantic analysis, and execution operations can be performed. The nodes in the first abstract syntax tree are traversed, and the nodes in the first abstract syntax tree that are traversed are matched with nodes in the same level in the second abstract syntax tree, and a matching result of the nodes in the first abstract syntax tree is obtained. The matching result of the node indicates whether there is a node in the same level in the second abstract syntax tree that matches the node in the first abstract syntax tree, if there is, the matching result is matching, that is, the syntax structure represented by the node in the first structured query statement is semantically the same as the syntax structure represented by the matching node in the same level in the second structured query statement. Further, different weights are assigned to the matching result of the root node in the first abstract syntax tree and the matching result of the child node of the root node, and the different weights represent different degrees of influence on the statement similarity. Therefore, the statement similarity between the first structured query statement and the second structured query statement can be determined in combination with the matching result of the root node in the first abstract syntax tree, the matching result of the child node of the root node, and the respective corresponding weights. It can be seen that this method does not need to execute a database, has a lower detection cost, and ensures the matching of semantic similarity. In addition, partial node matching can also be performed, and partial scores can also be obtained for the code that is not completely correct, and the partial node matching is quantified by the partial scores.

[0113] In a possible implementation, the embodiment of the present application provides a specific implementation of S203 for determining the statement similarity between the first structured query statement and the second structured query statement in combination with the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node, including:

[0114] D1: determining the first similarity between the first structured query statement and the second structured query statement in combination with the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node.

[0115] That is, the statement similarity determined in S203 can be referred to as the first similarity between the first structured query statement and the second structured query statement.

[0116] D2: traversing the node in the second abstract syntax tree, matching the node in the second abstract syntax tree traversed with the node in the same level in the first abstract syntax tree, obtaining the matching result of the node in the second abstract syntax tree; the matching result of the node in the second abstract syntax tree indicates whether there is a node in the same level in the first abstract syntax tree matched with the node in the second abstract syntax tree.

[0117] D3: determining the second similarity between the first structured query statement and the second structured query statement according to the matching result of the root node in the second abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node.

[0118] It can be understood that if the first abstract syntax tree is a subtree of the second abstract syntax tree, the nodes in the first abstract syntax tree can be matched with the nodes in the second abstract syntax tree, but the first abstract syntax tree and the second abstract syntax tree are not the same. In order to avoid this situation, so that the statement similarity between the first structured query statement and the second structured query statement is more accurate, the second abstract syntax tree can also be taken as a reference, and the second similarity between the first structured query statement and the second structured query statement can be obtained by executing D2-D3. The technical implementation of D2-D3 is similar to S202-S203, which will not be repeated here.

[0119] Among them, the first similarity can be regarded as the degree of matching the second abstract syntax tree to the first abstract syntax tree, and the second similarity can be regarded as the degree of matching the first abstract syntax tree to the second abstract syntax tree.

[0120] D4: determining the statement similarity between the first structured query statement and the second structured query statement based on the first similarity and the second similarity.

[0121] Among them, determining the statement similarity between the first structured query statement and the second structured query statement based on the first similarity and the second similarity includes D41 or D42:

[0122] D41: taking the average result of the first similarity and the second similarity as the statement similarity between the first structured query statement and the second structured query statement.

[0123] In an optional example, the average similarity of the first similarity and the second similarity can be directly taken, and the average similarity is taken as the statement similarity between the first structured query statement and the second structured query statement.

[0124] For example, when the first similarity and the second similarity are both represented by scores, the average score of the two scores can be taken to evaluate the statement similarity between the first structured query statement and the second structured query statement.

[0125] D42: calculate F β1 score based on F β1 score to determine the sentence similarity of the first structured query statement and the second structured query statement.

[0126] In another optional example, F β1 score is calculated by F β1 score based on F β1 score to determine the sentence similarity of the first structured query statement and the second structured query statement. β1 The higher the F

[0127] wherein F β1 score is calculated by the following formula:

[0128]

[0129] wherein β1 is an adjustment factor, PRECISION represents the precision, and RECALL represents the recall. It can be understood that the size of β1 can be adjusted according to actual needs.

[0130] In actual applications, the method for detecting the similarity of structured query statements based on abstract syntax trees provided by the embodiments of the present application can facilitate database application development and database management, improve the efficiency of detecting the similarity of different structured query statements, and at the same time provide timely feedback for developers to help improve the SQL statements in the program. In addition, the method can be applied to the tuning of hyperparameters of large language models, such as the tuning of Prompts, temperature, and Prefix, to improve the performance of large models on traditional NL2SQL tasks. For example, when there is a function for developing SQL statements in a large model, the quality of the developed SQL statements can be detected by using the method for detecting the similarity of structured query statements provided by the embodiments of the present application. When the similarity of the SQL statements is high, it indicates that the development quality of the SQL is low, and thus the parameters of the large model can be adjusted to improve the development quality of the SQL statements. That is, the tuning of the parameters of the large model is performed by the development quality of the SQL statements. In addition, the method for detecting the similarity of SQL statements based on abstract syntax trees provided by the embodiments of the present application can also be applied to model prediction problems, and the model outputs a predicted SQL statement to detect the prediction effect of the model by the similarity of the predicted SQL statement and the standard SQL statement. In this application, when F β1When the similarity of the predicted SQL statement to the standard SQL statement is evaluated by the score, and the predicted SQL statement corresponds to the first structured query statement and the standard SQL statement corresponds to the second structured query statement, if more attention is paid to whether the semantic structure of the standard SQL statement is matched by the predicted SQL statement, then the second similarity, i.e., the recall, is relatively emphasized. At this time, β1 can be increased a little.

[0131] Those skilled in the art can understand that, in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0132] Based on the structured query statement similarity detection method provided in the above method embodiment, the embodiment of the application further provides a structured query statement similarity detection device. The structured query statement similarity detection device will be described below with reference to the accompanying drawings. Since the principle of solving problems in the device in the embodiment of the disclosure is similar to the above-mentioned structured query statement similarity detection method of the embodiment of the application, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described herein.

[0133] Referring to Figure 7 As shown in the figure, the figure is a structural schematic diagram of a structured query statement similarity detection device provided by an embodiment of the application. As Figure 7 As shown in the figure, the structured query statement similarity detection device comprises:

[0134] The construction unit 701 is configured to obtain a first structured query statement and a second structured query statement, and construct a first abstract syntax tree of the first structured query statement and a second abstract syntax tree of the second structured query statement. The abstract syntax tree is used to represent the abstract syntax structure of the structured query statement. The root node in the abstract syntax tree has a higher degree of association with the query result of the structured query statement than other nodes.

[0135] The traversal unit 702 is configured to traverse a node in the first abstract syntax tree, match the node in the first abstract syntax tree that is traversed with a node in the same level in the second abstract syntax tree, and obtain a matching result of the node in the first abstract syntax tree. The matching result of the node in the first abstract syntax tree indicates whether there is a node in the same level in the second abstract syntax tree that is matched with the node in the first abstract syntax tree.

[0136] The first determining unit 703 is configured to determine the statement similarity between the first structured query statement and the second structured query statement in combination with the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node.

[0137] In a possible implementation, the traversal unit 702 includes:

[0138] The first traversal subunit is configured to traverse the nodes in the first abstract syntax tree in a hierarchical manner according to the bottom-up hierarchical structure of the first abstract syntax tree.

[0139] The first determining subunit is configured to determine a first node in the first abstract syntax tree and a second node in the second abstract syntax tree that are at the same level as the first node.

[0140] The judging subunit is configured to judge whether the node types of the first node and the second node are the same and whether the node attribute values of the first node and the second node satisfy the attribute value matching rule, and to obtain the matching result of the first node according to the judgment result.

[0141] The second determining subunit is configured to compare the matching results of the plurality of first nodes when the number of the second nodes is a plurality, and to redetermine the matching result of the first node.

[0142] In a possible implementation, the apparatus further includes:

[0143] The second determining unit is configured to determine a first child node at the adjacent level of the first node and a second child node at the adjacent level of the second node before redetermining the matching result of the first node by comparing the matching results of the plurality of first nodes when the number of the second nodes is a plurality, after obtaining the matching result of the first node according to the judgment result.

[0144] The judging unit is configured to judge whether the node types of the first child node and the second child node are the same and whether the node attribute values of the first child node and the second child node satisfy the attribute value matching rule, and to obtain the matching result of the first child node according to the judgment result.

[0145] The third determining unit is configured to redetermine the matching result of the first node according to the matching result of the first node, the weight corresponding to the matching result of the first node, the matching result of the first child node, and the weight corresponding to the matching result of the first node.

[0146] In a possible implementation, the third determining unit comprises:

[0147] The third determining sub-unit is configured to determine a score of the first node according to the matching result of the first node.

[0148] The fourth determining sub-unit is configured to determine a score of the first sub-node according to the matching result of the first sub-node.

[0149] The fifth determining sub-unit is configured to, when the number of the first sub-nodes is greater than one, determine an average value of the scores of the first sub-nodes as a sub-node score of the first node.

[0150] The first obtaining sub-unit is configured to perform weighted summation on the score of the first node, a weight corresponding to the matching result of the first node, the sub-node score of the first node, and a weight corresponding to the matching result of the sub-node of the first node, and reobtain the score of the first node.

[0151] The sixth determining sub-unit is configured to determine the matching result of the first node based on the score of the first node.

[0152] In a possible implementation, the attribute value matching rule is that the node attribute values of the first node and the second node are the same or the node attribute values of the first node and the second node are the same in semantics.

[0153] In a possible implementation, each node in the abstract syntax tree corresponds to a node type and a node attribute value.

[0154] The first determining unit 703 comprises:

[0155] The seventh determining sub-unit is configured to determine a score of a root node in the first abstract syntax tree according to a matching result of the root node.

[0156] The eighth determining sub-unit is configured to determine scores of adjacent level sub-nodes of the root node according to matching results of the adjacent level sub-nodes, determine an average value of the scores of the adjacent level sub-nodes as a sub-node score of the root node, and determine the sub-node score of the root node.

[0157] The second obtaining sub-unit is configured to perform weighted summation on the score of the root node, a weight corresponding to the matching result of the root node, the sub-node score of the root node, and a weight corresponding to the matching result of the sub-node of the root node, and reobtain the score of the root node.

[0158] The ninth determining sub-unit is configured to determine a statement similarity between the first structured query statement and the second structured query statement based on the score of the root node.

[0159] When the node attribute value corresponding to the root node is empty, the weight corresponding to the matching result of the root node is 0, and the weight corresponding to the matching result of the child node of the root node is 1.

[0160] In a possible implementation, the first determining unit 703 includes:

[0161] The tenth determining sub-unit is configured to determine a first similarity between the first structured query statement and the second structured query statement in combination with the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node.

[0162] The second traversal sub-unit is configured to traverse the nodes in the second abstract syntax tree, match the nodes in the second abstract syntax tree that are traversed with the nodes in the first abstract syntax tree at the same level, and obtain the matching result of the nodes in the second abstract syntax tree. The matching result of the nodes in the second abstract syntax tree indicates whether there is a node at the same level in the first abstract syntax tree that matches the node in the second abstract syntax tree.

[0163] The eleventh determining sub-unit is configured to determine a second similarity between the first structured query statement and the second structured query statement according to the matching result of the root node in the second abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node.

[0164] The twelfth determining sub-unit is configured to determine a statement similarity between the first structured query statement and the second structured query statement based on the first similarity and the second similarity.

[0165] In a possible implementation, the twelfth determining sub-unit is specifically configured to:

[0166] take the average similarity of the first similarity and the second similarity as the statement similarity between the first structured query statement and the second structured query statement.

[0167] Alternatively,

[0168] take the average similarity of the first similarity and the second similarity as the statement similarity between the first structured query statement and the second structured query statement.

[0169] Alternatively,

[0170] The first similarity is taken as a precision rate, and the second similarity is taken as a recall rate, to calculate an F β1 score, based on the F β1 score to determine a sentence similarity of the first structured query statement and the second structured query statement.

[0171] On the basis of the implementation manners of the above aspects provided by the present application, further combinations can be made to provide more implementation manners.

[0172] It should be noted that the specific implementation of each unit in the present embodiment can be referred to the related description in the above method embodiments. The division of units in the present embodiment is illustrative, and is only a logical function division. Another division manner can be used in actual implementation. Each functional unit in the present embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. For example, in the above embodiments, the processing unit and the sending unit can be the same unit, or can be different units. The integrated unit can be realized in the form of hardware, or in the form of a software functional unit.

[0173] Based on the structured query statement similarity detection method provided in the above method embodiments, the present application further provides an electronic device, including: one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the structured query statement similarity detection method described in any of the above embodiments.

[0174] Reference will be made to the following Figure 8 , which shows a structural schematic diagram of an electronic device 800 suitable for implementing the embodiments of the present application. The terminal device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (portable android devices), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 8 The electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0175] As Figure 8As shown, the electronic device 800 can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 802 or loaded from a storage device 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0176] Generally, the following devices can be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication devices 809 can allow the electronic device 800 to communicate wirelessly or wired with other devices to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that not all of the shown devices are required to be implemented or present. More or fewer devices can alternatively be implemented or present.

[0177] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 809, or installed from the storage devices 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the methods of embodiments of the present application are performed.

[0178] The electronic device provided by the embodiments of the present application and the structured query statement similarity detection method provided by the above-mentioned embodiments belong to the same inventive concept, and the technical details not described in detail in the present embodiment can be referred to the above-mentioned embodiments, and the present embodiment has the same beneficial effects as the above-mentioned embodiments.

[0179] Based on the structured query statement similarity detection method provided by the above-mentioned method embodiment, the present embodiment provides a computer readable medium, which stores a computer program, wherein the program is executed by a processor to realize the structured query statement similarity detection method as described in any of the above-mentioned embodiments.

[0180] It should be noted that the computer readable medium in the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in connection with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a RF (radio frequency) or the like, or any suitable combination of the above.

[0181] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0182] The computer readable medium described above can be contained in the electronic device described above; or can exist separately without being assembled into the electronic device.

[0183] The computer readable medium described above carries one or more programs, which when executed by the electronic device, cause the electronic device to perform the structured query statement similarity detection method described above.

[0184] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0185] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0186] The units related in the embodiments of the present application can be implemented by software, or by hardware. Among them, the name of the unit / module does not constitute a limitation to the unit itself in some cases. For example, the voice data acquisition module can also be described as a "data acquisition module".

[0187] The functions described above in the specification of the present application can be performed at least in part by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0188] In the context of this application, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0189] It should be noted that the various embodiments described in the specification are progressive and each embodiment focuses on the differences from other embodiments. The same and similar parts between embodiments can be referred to each other. For the system or device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the relevant part can be referred to the method part.

[0190] It should be understood that in this application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, "A and / or B" can represent: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0191] It is also to be noted that, as used in the specification and the appended claims, the singular forms "a," "an" and "the" include plural referents unless otherwise indicated. Furthermore, to the extent that the terms "including," "includes," "having," "has," "with," or "contains" are used in either the detailed description and the claims, such terms are intended to be inclusive in a manner similar to the term "comprising" as an open transition term without precluding any additional or other elements.

[0192] The embodiments disclosed herein can each be implemented as a method, apparatus, or article of manufacture using programming instructions. The embodiments disclosed herein can be implemented using software, firmware, hardware, or a combination thereof. The various elements of the disclosed embodiments, as well as the embodiments themselves, can be constructed from any combination of hardware, software, and / or firmware. The software implementation can be implemented by one or more software modules using object-oriented design methodology, among other techniques. The software modules can be stored on any computer-readable medium, including RAM, ROM, EEPROM, flash memory, or a hard disk, to name a few. The software modules can include one or more routines.

[0193] The above description of disclosed embodiments provides enough information to enable those with ordinary skill in the art to make and use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Accordingly, the application is not to be restricted based on the specific embodiments illustrated and described, but is instead to be defined based on the claims and their equivalents.

Claims

1. A method for detecting similarity of structured query statements, characterized by, The method comprises: obtaining a first structured query statement and a second structured query statement, constructing a first abstract syntax tree of the first structured query statement and a second abstract syntax tree of the second structured query statement; the abstract syntax tree is used to represent the abstract syntax structure of the structured query statement, the root node in the abstract syntax tree has a higher degree of association with the query result of the structured query statement than other nodes; wherein each node in the abstract syntax tree corresponds to a node type and a node attribute value; traverse the nodes in the first abstract syntax tree, match the nodes in the first abstract syntax tree traversed with the nodes in the same level of the second abstract syntax tree in terms of node type and node attribute value, and obtain the matching result of the nodes in the first abstract syntax tree; the matching result of the nodes in the first abstract syntax tree indicates whether there is a node in the same level in the second abstract syntax tree that matches the nodes in the first abstract syntax tree; determine the statement similarity between the first structured query statement and the second structured query statement by combining the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child nodes of the root node, and the weight corresponding to the matching result of the child nodes of the root node, wherein the matching result of the root node of the first abstract syntax tree and the matching result of the child nodes of the root node are assigned different weights.

2. The method of claim 1, wherein, The traversal of the nodes in the first abstract syntax tree, the matching of the nodes in the first abstract syntax tree traversed with the nodes in the same level of the second abstract syntax tree, and the obtaining of the matching result of the nodes in the first abstract syntax tree, comprise: traverse the nodes in the first abstract syntax tree in a hierarchical manner according to the hierarchical structure of the first abstract syntax tree from bottom to top; determine a first node in the first abstract syntax tree traversed and a second node in the second abstract syntax tree which is in the same level as the first node; determine whether the node types of the first node and the second node are the same and whether the node attribute values of the first node and the second node satisfy the attribute value matching rule, and obtain the matching result of the first node according to the determination result; when the number of the second nodes is more than one, compare the matching results of the plurality of first nodes, and re-determine the matching result of the first node.

3. The method of claim 2, wherein, After the matching result of the first node is obtained according to the determination result, before the matching result of the first node is re-determined by comparing the matching results of the plurality of first nodes when the number of the second nodes is more than one, the method further comprises: determine a first child node adjacent to the first node and a second child node adjacent to the second node; the second child node is in the same level as the first child node; determining whether the node types of the first child node and the second child node are same and whether the node attribute values of the first child node and the second child node satisfy an attribute value matching rule, and obtaining a matching result of the first child node according to a determination result; redetermining the matching result of the first node according to the matching result of the first node, a weight corresponding to the matching result of the first node, the matching result of the first child node, and a weight corresponding to the matching result of the child node of the first node.

4. The method of claim 3, wherein, The redetermining the matching result of the first node according to the matching result of the first node, a weight corresponding to the matching result of the first node, the matching result of the first child node, and a weight corresponding to the matching result of the child node of the first node includes: determining a score of the first node according to the matching result of the first node; determining a score of the first child node according to the matching result of the first child node; when the number of the first child nodes is multiple, determining an average value of the scores of the multiple first child nodes as a child node score of the first node; performing weighted summation on the score of the first node, the weight corresponding to the matching result of the first node, the child node score of the first node, and the weight corresponding to the matching result of the child node of the first node, and redetermining the score of the first node; determining the matching result of the first node based on the score of the first node.

5. The method of claim 2, wherein, The attribute value matching rule is that the node attribute values of the first node and the second node are same or the node attribute values of the first node and the second node are same in semantics.

6. The method of claim 1, wherein, Each node in the abstract syntax tree corresponds to a node type and a node attribute value. The determining the statement similarity between the first structured query statement and the second structured query statement according to the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching results of the child nodes of the root node, and the weights corresponding to the matching results of the child nodes of the root node includes: determining a score of the root node in the first abstract syntax tree according to the matching result of the root node in the first abstract syntax tree; determining scores of adjacent level child nodes of the root node according to the matching results of the adjacent level child nodes of the root node, and determining an average value of the scores of the multiple adjacent level child nodes as a child node score of the root node; performing weighted summation on the score of the root node, the weight corresponding to the matching result of the root node, the child node score of the root node, and the weight corresponding to the matching result of the child node of the root node, and redetermining the score of the root node; determining the statement similarity between the first structured query statement and the second structured query statement based on the score of the root node; when the node attribute value corresponding to the root node is empty, the weight corresponding to the matching result of the root node is 0, and the weight corresponding to the matching result of the child node of the root node is 1.

7. The method of claim 1, wherein, The matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the child node matching result of the root node are combined to determine the statement similarity of the first structured query statement and the second structured query statement, including: The matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the child node matching result of the root node are combined to determine the first similarity of the first structured query statement and the second structured query statement; Traverse the nodes in the second abstract syntax tree, match the nodes in the second abstract syntax tree traversed with the nodes in the first abstract syntax tree at the same level, and obtain the matching result of the nodes in the second abstract syntax tree; The matching result of the node in the second abstract syntax tree indicates whether there is a node at the same level in the first abstract syntax tree that matches the node in the second abstract syntax tree; According to the matching result of the root node in the second abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the child node matching result of the root node, determine the second similarity of the first structured query statement and the second structured query statement; Based on the first similarity and the second similarity, the statement similarity of the first structured query statement and the second structured query statement is determined.

8. The method of claim 7, wherein, The first similarity and the second similarity are combined to determine the statement similarity of the first structured query statement and the second structured query statement, including: The average similarity of the first similarity and the second similarity is taken as the statement similarity of the first structured query statement and the second structured query statement; Or, The first similarity is taken as a precision, and the second similarity is taken as a recall, to calculate a score based on the score to determine a sentence similarity of the first structured query statement and the second structured query statement.

9. A structured query language similarity detection apparatus, characterized by, The device includes: A construction unit is configured to obtain a first structured query statement and a second structured query statement, and construct a first abstract syntax tree of the first structured query statement and a second abstract syntax tree of the second structured query statement; The abstract syntax tree is used to represent the abstract syntax structure of the structured query statement, and the root node in the abstract syntax tree has a higher degree of association with the query result of the structured query statement than other nodes; Wherein, each node in the abstract syntax tree corresponds to a node type and a node attribute value; A traversal unit is configured to traverse the nodes in the first abstract syntax tree, match the nodes in the first abstract syntax tree traversed with the nodes in the second abstract syntax tree at the same level in terms of the node type and the node attribute value, and obtain the matching result of the nodes in the first abstract syntax tree; The matching result of the node in the first abstract syntax tree indicates whether there is a node at the same level in the second abstract syntax tree that matches the node in the first abstract syntax tree; The first determining unit is configured to determine the statement similarity between the first structured query statement and the second structured query statement by combining the matching result of the root node in the first abstract syntax tree, the weight corresponding to the matching result of the root node, the matching result of the child node of the root node, and the weight corresponding to the matching result of the child node of the root node. The matching result of the root node of the first abstract syntax tree and the matching result of the child node of the root node are assigned different weights.

10. An electronic device, comprising: The method comprises: one or more processors; a storage device having stored thereon one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the structured query statement similarity detection method according to any one of claims 1-8.

11. A computer readable storage medium, characterized in that, a computer program is stored thereon, and the computer program is executed by a processor to implement the structured query statement similarity detection method according to any one of claims 1-8.

Images