SQL statement processing method, device, equipment and storage medium

Abstract

This invention discloses a method, apparatus, device, and storage medium for processing SQL statements. Embodiments of this invention relate to the field of natural language processing. The method includes: acquiring a SQL statement to be detected, a set of N historical SQL statements, and 2N models, where N is a positive integer greater than or equal to 2; determining a target SQL statement set based on the SQL statement to be detected, the set of N historical SQL statements, and the 2N models, wherein the target SQL statement set includes: the first M historical SQL statements in the historical SQL statement set whose similarity scores are ranked highest among the SQL statements to be detected, where M is a positive integer greater than or equal to 3; acquiring the target SQL statement with the highest similarity score in the target SQL statement set, and displaying the target SQL statement. Through the technical solution of this invention, similarity detection of SQL statements can be achieved before creating data items by running SQL statements, quickly and accurately identifying similar SQL statements and ensuring the uniqueness of data item creation.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of natural language processing, and in particular to a SQL statement processing method, apparatus, device and storage medium. Background Art

[0002] In project development, we often need to use SQL statements to perform some operations on the database, such as adding, deleting, modifying, and querying. In a specific scenario of creating new data items, because the uniqueness of the data items must be guaranteed, but it is not allowed to check whether the items are created repeatedly after the data items are created, it is necessary to perform similarity detection of SQL statements before running SQL statements to create data items, so as to prevent the repeated creation of data items with the same meaning.

[0003] In the existing technical solutions, some simple text string comparison detection solutions are used to calculate the string similarity. For similar statements such as a=b, b=a, the similarity cannot be correctly judged. In fact, they are similar under certain conditions, but the simple text string comparison method will judge them as dissimilar, which will reduce the accuracy of similarity detection. The existing technology also uses classification algorithms to judge the similarity of SQL statements. Of course, this method can generally find similarities, which does not meet the accuracy requirements.

[0004] Deep learning is also widely used in the field of natural language processing. However, there are many problems when using these methods to deduplicate SQL statements. The collation of a large amount of data in different forms and the training of models will consume a lot of time. In addition, the high complexity of the model will most likely lead to low efficiency of post-launch detection and poor portability. Summary of the invention

[0005] The embodiments of the present invention provide a SQL statement processing method, apparatus, device and storage medium to implement similarity detection of SQL statements before creating data items, thereby preventing repeated creation of data items with the same meaning.

[0006] In a first aspect, an embodiment of the present invention provides a method for processing an SQL statement, comprising:

[0007] Obtain a SQL statement to be detected, a set of N historical SQL statements, and 2N models, where N is a positive integer greater than or equal to 2;

[0008] Determine a target SQL statement set according to the SQL statement to be detected, N historical SQL statement sets and 2N models, wherein the target SQL statement set includes: the first M historical SQL statements in the historical SQL statement set that are ranked by similarity values ​​with the SQL statement to be detected, where M is a positive integer greater than or equal to 3;

[0009] The target SQL statement with the highest similarity in the target SQL statement set is obtained, and the target SQL statement is displayed.

[0010] In a second aspect, an embodiment of the present invention further provides a SQL statement processing device, the device comprising:

[0011] An acquisition module, used to acquire a SQL statement to be detected, a set of N historical SQL statements, and 2N models, where N is a positive integer greater than or equal to 2;

[0012] A determination module is used to determine a target SQL statement set according to the SQL statement to be detected, N historical SQL statement sets and 2N models, wherein the target SQL statement set includes: the first M historical SQL statements in the historical SQL statement set that are ranked by similarity values ​​with the SQL statement to be detected, where M is a positive integer greater than or equal to 3;

[0013] The display module is used to obtain the target SQL statement with the highest similarity in the target SQL statement set and display the target SQL statement.

[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an SQL statement processing method as described in any one of the embodiments of the present invention is implemented.

[0015] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the SQL statement processing method as described in any one of the embodiments of the present invention is implemented.

[0016] The embodiment of the present invention obtains a SQL statement to be detected, a set of N historical SQL statements, and 2N models; determines a target SQL statement set according to the SQL statement to be detected, the set of N historical SQL statements, and the 2N models; obtains the target SQL statement with the highest similarity in the target SQL statement set, and displays the target SQL statement. Compared with the prior art, the invention solves the problem that a large amount of time is consumed by using a deep learning network to learn to build and train a complex model for similarity detection, the model has high computational complexity, and the method has poor portability, and achieves the effect of quickly and accurately finding similar SQL statements, ensuring the uniqueness of data item establishment, and also has great inspiration for similarity detection of other script statements. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 This is a flow chart of a SQL statement processing method provided by Embodiment 1 of the present invention;

[0019] Figure 2 It is a structural diagram of a SQL statement processing device provided by Embodiment 2 of the present invention;

[0020] Figure 3 is a schematic diagram of the structure of an electronic device provided by Embodiment 3 of the present invention;

[0021] Figure 4 It is a structural diagram of a computer-readable storage medium containing a computer program provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without conflict.

[0023] Embodiment 1

[0024] Figure 1 is a flowchart of a SQL statement processing method provided in the first embodiment of the present invention. This embodiment is applicable to the case of detecting the similarity of SQL statements. The method can be executed by a SQL statement processing device in the embodiment of the present invention. The device can be implemented in software and / or hardware. Figure 1 As shown, the method specifically comprises the following steps:

[0025] S101. Obtain a SQL statement to be detected, a set of N historical SQL statements, and 2N models.

[0026] Wherein, N is a positive integer greater than or equal to 2.

[0027] It should be explained that the SQL statement to be detected can be understood as an SQL statement newly input by the user and requiring similarity detection.

[0028] It should be noted that the historical SQL statement set refers to a set of SQL statements that are already stored in the database. Corresponding SQL statements are collected from the system and stored to form a historical SQL statement data set.

[0029] In this embodiment, the model can be understood as a detection model established to detect the similarity of the SQL statements to be detected. After obtaining the historical SQL statement set, a model is established based on the historical SQL statement set. For example, it can be that the first model is generated based on the historical SQL operation statement set and the BoW word bag model; the second model is generated based on the historical SQL definition statement set and the BoW word bag model; the third model is generated based on the historical SQL control statement set and the BoW word bag model; the fourth model is generated based on the historical SQL embedded statement set and the BoW word bag model; the fifth model is generated based on the historical SQL remaining statement set and the BoW word bag model; the sixth model is generated based on the historical SQL operation statement set and the TF-IDF model; the seventh model is generated based on the historical SQL definition statement set and the TF-IDF model; the eighth model is generated based on the historical SQL control statement set and the TF-IDF model; the ninth model is generated based on the historical SQL embedded statement set and the TF-IDF model; the tenth model is generated based on the historical SQL remaining statement set and the TF-IDF model. In the embodiment of the present invention, N models are generated based on the BoW word bag model and the historical SQL statement set, and N models are generated based on the TF-IDF model and the historical SQL statement set, and a total of 2N models are obtained.

[0030] The SQL statement to be detected input by the user, the N historical SQL statement sets existing in the database, and the 2N models established for SQL statement similarity detection are obtained for subsequent SQL statement similarity detection.

[0031] Specifically, obtaining a set of N historical SQL statements includes:

[0032] a. Get historical SQL statements.

[0033] In this embodiment, the historical SQL statements can be understood as the existing SQL statements stored in the database. Generally speaking, the database has the function of providing historical SQL statements, and the historical SQL statements can be directly exported from the database.

[0034] b. Create a first SQL statement set based on historical SQL statements.

[0035] The first SQL statement set may be the first collected historical SQL statement stored first, which may be assumed to exist in set A, and set A is the first SQL statement set.

[0036] Specifically, creating a first SQL statement set according to historical SQL statements includes:

[0037] The first historical SQL statement is stored in the first SQL statement set.

[0038] In this embodiment, the first historical SQL statement can be understood as the first historical SQL statement collected.

[0039] The non-first historical SQL statement different from the SQL statement in the first SQL statement set is stored in the first SQL statement set.

[0040] The non-first historical SQL statement may be understood as an SQL statement collected after the first historical SQL statement.

[0041] During the actual operation, after the first historical SQL statement is stored in the first SQL statement set, each time a new SQL statement is collected, it is determined whether there is an identical SQL statement in the first SQL statement set. If so, the next SQL statement is collected; if not, it is stored in the first SQL statement set.

[0042] c. Classify the SQL statements in the first SQL statement set according to SQL statement keywords to obtain N historical SQL statement sets.

[0043] The SQL statement keywords refer to the key information in the SQL statement except for some Chinese or English comment information.

[0044] In this embodiment, the classification does not refer to classification based on the actual business functions of the SQL statements, but classification of the SQL statements from the perspective of the SQL statement keywords.

[0045] Specifically, the SQL statements in the first SQL statement set are classified according to the SQL statement keywords to obtain N historical SQL statement sets, including:

[0046] If the number of occurrences of a first SQL statement keyword in a first SQL statement in the first SQL statement set is greater than the number of occurrences of other SQL statement keywords, the first SQL statement is added to a first historical SQL statement set corresponding to the first SQL statement keyword.

[0047] If the number of times a first SQL statement keyword appears in the first SQL statement in the first SQL statement set is equal to the number of times other SQL statement keywords appear, the first SQL statement is randomly added to the first historical SQL statement set corresponding to the first SQL statement keyword or the second historical SQL statement set corresponding to other SQL statement keywords.

[0048] The second historical SQL statement set may be understood as another set relative to the first historical SQL statement set.

[0049] If the SQL statement keyword pair does not exist in the first SQL statement, the first SQL statement is added to the third historical SQL statement set.

[0050] The third historical SQL statement set may be understood as another set relative to the first historical SQL statement set and the second historical SQL statement set.

[0051] The first historical SQL statement set, the second historical SQL statement set and the third historical SQL statement set are different sets.

[0052] Scan each SQL statement in the historical SQL statement collection, scan the keywords of the SQL statements and count them accordingly. If a certain keyword appears the most times, put this SQL statement into the collection corresponding to the keyword; when the number of multiple keywords is the same, put them randomly; when there are no keywords of the above types, put the SQL statement into other collections.

[0053] Specifically, the N historical SQL statement sets include: a historical SQL operation statement set, a historical SQL definition statement set, a historical SQL control statement set, a historical SQL embedded statement set, and a historical SQL remaining statement set.

[0054] For example, the historical SQL statement set can be divided into five categories according to operation statements, definition statements, control statements, embedded statements and other statements: historical SQL operation statement set, historical SQL definition statement set, historical SQL control statement set, historical SQL embedded statement set and historical SQL remaining statement set, which can be named Set1, Set2, Set3, Set4 and Set5 respectively.

[0055] Specifically, the SQL statement keywords for determining the first four types of SQL statement sets are shown in Table 1:

[0056] Table 1

[0057]

[0058] Accordingly, obtaining N models includes:

[0059] The first model is generated based on the historical SQL operation statement set and the BoW bag-of-words model.

[0060] Among them, the Bag of Words (BoW) model was originally used in text classification to represent documents as feature vectors. Its basic idea is to assume that for a text, its word order, grammar, and syntax are ignored, and it is only regarded as a collection of words, and each word in the text is independent. Simply put, each document is regarded as a bag (because it is filled with words, so it is called a bag of words, hence the name Bag of words), and then see what words are in the bag and classify them.

[0061] In this embodiment, the first model can be understood as a detection model constructed by using the BoW bag-of-words model for the historical SQL operation statement set. For example, it can be recorded as M_11.

[0062] A second model is generated based on a set of historical SQL definition statements and the BoW bag-of-words model.

[0063] In this embodiment, the second model can be understood as a detection model constructed using the BoW bag-of-words model for the historical SQL definition statement set. For example, it can be recorded as M_12.

[0064] The third model is generated based on the historical SQL control statement set and the BoW bag-of-words model.

[0065] In this embodiment, the third model can be understood as a detection model constructed using the BoW bag-of-words model for the historical SQL control statement set. For example, it can be recorded as M_13.

[0066] The fourth model is generated based on the historical SQL embedding statement set and the BoW bag-of-words model.

[0067] In this embodiment, the fourth model can be understood as a detection model constructed using the BoW bag-of-words model for the historical SQL embedded statement set. For example, it can be recorded as M_14.

[0068] The fifth model is generated based on the remaining historical SQL statement set and the BoW bag-of-words model.

[0069] In this embodiment, the fifth model can be understood as a detection model constructed using the BoW bag-of-words model for the historical SQL remaining statement set. For example, it can be recorded as M_15.

[0070] Taking the generation of the first model M_11 as an example, the following operations are performed for each statement in Set1:

[0071] 1. Perform word segmentation on all the sentences in Set1;

[0072] 2. Build a corpus after word segmentation;

[0073] 3. Use the corpus to represent each sentence in Set1 as a vector and add an index number;

[0074] 4. Save the vector persistently in a file format (such as a .pkl file).

[0075] The sixth model is generated based on the historical SQL operation statement set and the TF-IDF model.

[0076] Among them, the TF-IDF (term frequency-inverse document frequency) model is a commonly used weighted technology model for information retrieval and data mining. TF in TF-IDF stands for term frequency, which is the number of times a word appears in an article or sentence; IDF stands for inverse document frequency, which is the weight of the word. Multiplying TF and IDF gives the TF-IDF value of a word, which is proportional to the number of times the word appears in a document and inversely proportional to the number of documents containing the word.

[0077] In this embodiment, the sixth model can be understood as a detection model constructed using the TF-IDF model for the historical SQL operation statement set. For example, it can be recorded as M_21.

[0078] The seventh model is generated based on the historical SQL definition statement set and the TF-IDF model.

[0079] In this embodiment, the seventh model can be understood as a detection model constructed using the TF-IDF model for the historical SQL definition statement set. For example, it can be recorded as M_22.

[0080] The eighth model is generated based on the historical SQL control statement set and the TF-IDF model.

[0081] In this embodiment, the eighth model can be understood as a detection model constructed using the TF-IDF model for the historical SQL control statement set. For example, it can be recorded as M_23.

[0082] The ninth model is generated based on the historical SQL embedding statement set and the TF-IDF model.

[0083] In this embodiment, the ninth model can be understood as a detection model constructed using the TF-IDF model for the historical SQL embedded statement set. For example, it can be recorded as M_24.

[0084] The tenth model is generated based on the remaining historical SQL statement set and the TF-IDF model.

[0085] In this embodiment, the tenth model can be understood as a detection model constructed using the TF-IDF model for the historical SQL remaining statement set. For example, it can be recorded as M_25.

[0086] Taking the generation of the sixth model M_21 as an example, the following operations are performed for each statement in Set1:

[0087] 1. Perform word segmentation processing as in step 1 above (or directly use the above word segmentation results);

[0088] 2. The word frequency of each word set is vectorized and an index number is added;

[0089] 3. Save the vector persistently in a file format (such as a .pkl file).

[0090] Furthermore, before the SQL statements in the first SQL statement set are classified according to the SQL statement keywords to obtain N historical SQL statement sets, the method further includes:

[0091] Remove comments from SQL statements based on regular expressions.

[0092] Among them, regular expression, also known as regular expression, is a concept in computer science. Regular expression is usually used to retrieve and replace texts that meet a certain pattern (rule). Regular expression is a logical formula for string operation, which is to use some pre-defined specific characters and the combination of these specific characters to form a "regular string". This "regular string" is used to express a filtering logic for the string.

[0093] Specifically, the comment information may be some Chinese or English comment information in the SQL statement that does not affect the similarity detection.

[0094] and / or;

[0095] If the length of a target word constituting an SQL statement in the first SQL statement set is greater than a length threshold, the target word is converted into a target identifier, wherein the target word is not a SQL statement keyword and is not a number.

[0096] The target word can be understood as a word that is collected after the user inputs the SQL statement to be detected and constitutes the SQL statement in the first SQL statement set. The target word is not a SQL statement keyword and is not a number.

[0097] In this embodiment, the length threshold may be a value set by the user to determine whether the length of an SQL statement is too long. For example, the length threshold may be 20, and when the length value of an SQL statement is greater than 20, the SQL statement is considered to be too long.

[0098] It should be noted that the target identifier can be understood as a word of an SQL statement with an identifier into which the target word is converted.

[0099] Further, the target word is converted into a target identifier, including:

[0100] Convert the target word into a binary number.

[0101] For example, when the character length of the target word is greater than 20, it can be converted into a 16-bit binary number. For example, the target word:

[0102] TMP_TP01_PRVT_CST_MAGIN_BSC_TENMP is converted to the binary number 1010000000000000F. The conversion result can start from 0000000000000001F and increase in sequence.

[0103] The target identity is determined based on the binary number and the preset identifier.

[0104] Specifically, the preset identifier may be a letter or other symbol with a symbolic function preset by the user, such as F.

[0105] In actual operation, a conversion method of binary number plus one character (F) is used to process the target words of SQL statements that are too long. Of course, other better conversion methods can also be used to simplify the length of SQL statements. The principle of simplification is not to affect the subsequent similarity detection. Binary data conversion can greatly reduce the length of SQL statements. SQL statements that remove redundant information can greatly reduce the calculation time of the model, and to a certain extent can speed up the search for similar statements and improve performance.

[0106] S102: Determine a target SQL statement set according to the SQL statement to be detected, N historical SQL statement sets, and 2N models.

[0107] Among them, the target SQL statement set can be understood as a set of historical SQL statements with high similarity to the SQL statement to be detected, including: the first historical SQL statement in the historical SQL statement set ranked before M in similarity value with the SQL statement to be detected, where M is a positive integer greater than or equal to 3. The following takes M as 3 as an example for explanation.

[0108] Specifically, the first historical SQL statement refers to a historical SQL statement in the historical SQL statement set that has the highest similarity to the SQL statement to be detected.

[0109] Furthermore, before determining the target SQL statement set according to the SQL statement to be detected, the N historical SQL statement sets and the 2N models, the method further includes:

[0110] The ratio of the number of historical SQL statements in the historical SQL operation statement set to the number of historical SQL statements in the first SQL statement set is determined as the weight of the first model and the weight of the sixth model.

[0111] The weight of the first model refers to the weight of the detection model constructed using the BoW bag-of-words model for the historical SQL operation statement set.

[0112] The weight of the sixth model refers to the weight of the detection model constructed using the TF-IDF model for the historical SQL operation statement set.

[0113] Because the first model and the sixth model are both models established for historical SQL operation statements, the weights of the first model and the sixth model are the same, which can also be called the weight of the historical SQL operation statement set, for example, can be recorded as W1.

[0114] The ratio of the number of historical SQL statements in the historical SQL definition statement set to the number of historical SQL statements in the first SQL statement set is determined as the weight of the second model and the weight of the seventh model.

[0115] The weight of the second model refers to the weight of the detection model constructed using the BoW bag-of-words model for the historical SQL definition statement set.

[0116] The weight of the seventh model refers to the weight of the detection model constructed using the TF-IDF model for the historical SQL definition statement set.

[0117] Because the second model and the seventh model are both models established for historical SQL definition statements, the weights of the second model and the seventh model are the same, which can also be called the weight of the historical SQL definition statement set, for example, can be recorded as W2.

[0118] The ratio of the number of historical SQL statements in the historical SQL control statement set to the number of historical SQL statements in the first SQL statement set is determined as the weight of the third model and the weight of the eighth model.

[0119] The weight of the third model refers to the weight of the detection model constructed using the BoW bag-of-words model for the historical SQL control statement set.

[0120] Among them, the weight of the eighth model refers to the weight of the detection model constructed using the TF-IDF model for the historical SQL control statement set.

[0121] Because the third model and the eighth model are both models established for historical SQL control statements, the weights of the third model and the eighth model are the same, which can also be called the weight of the historical SQL control statement set, for example, can be recorded as W3.

[0122] The ratio of the number of historical SQL statements in the historical SQL embedded statement set to the number of historical SQL statements in the first SQL statement set is determined as the weight of the fourth model and the weight of the ninth model.

[0123] Among them, the weight of the fourth model refers to the weight of the detection model constructed using the BoW bag-of-words model for the historical SQL embedding statement set.

[0124] Among them, the weight of the ninth model refers to the weight of the detection model constructed using the TF-IDF model for the historical SQL embedding statement set.

[0125] Because the fourth model and the ninth model are both models established for historical SQL embedded statements, the weights of the fourth model and the ninth model are the same, which can also be called the weight of the historical SQL embedded statement set, for example, can be recorded as W4.

[0126] The ratio of the number of historical SQL statements in the historical SQL remaining statement set to the number of historical SQL statements in the first SQL statement set is determined as the weight of the fifth model and the weight of the tenth model.

[0127] The weight of the fifth model refers to the weight of the detection model constructed using the BoW bag-of-words model for the remaining set of historical SQL statements.

[0128] Among them, the weight of the tenth model refers to the weight of the detection model constructed using the TF-IDF model for the remaining set of historical SQL statements.

[0129] Because the fifth model and the tenth model are both models established for the remaining historical SQL statements, the weights of the fifth model and the tenth model are the same, which can also be called the weight of the set of remaining historical SQL statements, for example, can be recorded as W5.

[0130] In actual operation, assuming that the number of historical SQL statement sets is NUM, the number of historical SQL statements in the historical SQL operation statement set Set1 is num1, the number of historical SQL statements in the historical SQL definition statement set Set2 is num2, the number of historical SQL statements in the historical SQL control statement set Set3 is num3, the number of historical SQL statements in the historical SQL embedded statement set Set4 is num4, and the number of historical SQL statements in the historical SQL other statement set Set5 is num5, then the weight Wi of each set is:

[0131] Wi=numi / NUM(i=1,2,3,4,5)

[0132] The use of the set weight parameter Wi can prevent the influence of similarity values ​​caused by random allocation of the same statements during SQL classification.

[0133] Furthermore, a target SQL statement set is determined according to the SQL statement to be detected, the N historical SQL statement sets, and the 2N models, including:

[0134] A first set is obtained according to a first model.

[0135] The first set includes: the top M historical SQL statements in the historical SQL operation statement set in order of similarity values ​​with the SQL statement to be detected.

[0136] A second set is obtained according to the second model.

[0137] The second set includes: the top M historical SQL statements in the historical SQL definition statement set in order of similarity values ​​with the SQL statement to be detected.

[0138] A third set is obtained according to the third model.

[0139] The third set includes: the top M historical SQL statements in the historical SQL control statement set that are ranked by similarity values ​​with the SQL statement to be detected.

[0140] A fourth set is obtained according to the fourth model.

[0141] The fourth set includes: the top M historical SQL statements in the historical SQL embedded statement set ranked by similarity values ​​with the SQL statement to be detected.

[0142] A fifth set is obtained according to the fifth model.

[0143] The fifth set includes: the top M historical SQL statements in the historical SQL remaining statement set that are ranked by similarity values ​​with the SQL statement to be detected.

[0144] A sixth set is obtained according to the sixth model.

[0145] The sixth set includes: the top M historical SQL statements in the historical SQL operation statement set in order of similarity values ​​with the SQL statement to be detected.

[0146] Obtain the seventh set according to the seventh model.

[0147] The seventh set includes: the top M historical SQL statements in the historical SQL definition statement set in order of similarity values ​​with the SQL statement to be detected.

[0148] An eighth set is obtained according to the eighth model.

[0149] The eighth set includes: the top M historical SQL statements in the historical SQL control statement set in order of similarity values ​​with the SQL statement to be detected.

[0150] Get the ninth set according to the ninth model.

[0151] The ninth set includes: the top M historical SQL statements in the historical SQL embedded statement set ranked by similarity values ​​with the SQL statement to be detected.

[0152] Get the tenth set according to the tenth model.

[0153] The tenth set includes: the top M historical SQL statements in the historical SQL remaining statement set that are ranked by similarity values ​​with the SQL statement to be detected.

[0154] In the actual operation process, models M_11, M_12, ​​M_13, M_14 and M_15 are used to calculate the top M statements most similar to the input SQL statement to be detected from sets Set1, Set2, Set3, Set4 and Set5 respectively, and record the statement text and statement similarity value.

[0155] Among them, M can be adjusted according to the business, and an example is given below in which M is 3.

[0156] After calculation, the sets are formed respectively: Sim_11, Sim_12, Sim_13, Sim_14 and Sim_15.

[0157] Sim_11: [statement, similarity value][statement, similarity value][statement, similarity value]

[0158] Sim_12: [statement, similarity value][statement, similarity value][statement, similarity value]

[0159] Sim_13: [statement, similarity value][statement, similarity value][statement, similarity value]

[0160] Sim_14: [statement, similarity value][statement, similarity value][statement, similarity value]

[0161] Sim_15: [statement, similarity value][statement, similarity value][statement, similarity value]

[0162] Similarly, models M_21, M_22, M_23, M_24 and M_25 are used to calculate the top three statements most similar to the input SQL statement to be detected from sets Set1, Set2, Set3, Set4 and Set5 respectively, and the statement texts and statement similarity values ​​are recorded to form sets: Sim_21, Sim_22, Sim_23, Sim_24 and Sim_25 respectively.

[0163] Sim_21: [statement, similarity value][statement, similarity value][statement, similarity value]

[0164] Sim_22: [statement, similarity value][statement, similarity value][statement, similarity value]

[0165] Sim_23: [statement, similarity value][statement, similarity value][statement, similarity value]

[0166] Sim_24: [statement, similarity value][statement, similarity value][statement, similarity value]

[0167] Sim_25: [statement, similarity value][statement, similarity value][statement, similarity value]

[0168] The similarity values ​​obtained in the above sets are all the results of corresponding multiplication with the weight (Wi) of the corresponding set.

[0169] The first weight and the second weight are determined according to the similarity of historical SQL statements in the first set, the similarity of historical SQL statements in the second set, the similarity of historical SQL statements in the third set, the similarity of historical SQL statements in the fourth set, the similarity of historical SQL statements in the fifth set, the similarity of historical SQL statements in the sixth set, the similarity of historical SQL statements in the seventh set, the similarity of historical SQL statements in the eighth set, the similarity of historical SQL statements in the ninth set, and the similarity of historical SQL statements in the tenth set.

[0170] In actual operation, the output similarity results of the model can be fused as follows:

[0171] Model_A=W1*M_11+W2*M_12+W3*M_13+W4*M_14+W5*M_15

[0172] Model_B=W1*M_21+W2*M_22+W3*M_23+W4*M_24+W5*M_25

[0173] Model_R=WA*Model_A+WB*Model_B

[0174] Where Wi is the weight Wi of each set generated in the above steps. Here W1*M_11 represents the similarity calculation result between the word vector M_11 and the input sentence. The word vector technology converts words into dense vectors, and for similar words, their corresponding word vectors are also similar.

[0175] After the above steps, 15 similarities and similar sentences were obtained using the BoW bag of words model, and the corresponding results were also obtained using the TF-IDF model. The 10 similarity values ​​obtained in the same set, such as Set_1, are counted, assuming that they are shown in Table 2:

[0176] Table 2

[0177]

[0178] First, initialize the values ​​of two counting variables BoW_sum and TF-IDF_sum to 0. Then determine the size of BoW_sim_Set_11 and TF-IDF_sim_Set_11. If the left side is larger than the right side, add 1 to BoW_sum, otherwise add 1 to the right side. Then perform the corresponding ratio operation on other sets Set_2, Set_3, Set_4 and Set_5. After comparison, get the weight value:

[0179] WA=BoW_sum / 30

[0180] WB = TF-IDF_sum / 30

[0181] To prevent the result from being 0, rewrite the formula as follows:

[0182] WA=(BoW_sum+1) / 30

[0183] WB=(TF-IDF_sum+1) / 30

[0184] Through the above steps, the first weight and the second weight can be determined. The use of the model fusion parameters WA and WB combines the prediction results of the two models to a certain extent, preventing one model from playing a decisive role due to a special situation.

[0185] Selecting a first target SQL statement from the first set and the sixth set according to the first weight, the second weight, the weight of the first model, and the weight of the sixth model;

[0186] Selecting a second target SQL statement from the second set and the seventh set according to the first weight, the second weight, the weight of the second model, and the weight of the seventh model;

[0187] Selecting a third target SQL statement from the third set and the eighth set according to the first weight, the second weight, the weight of the third model, and the weight of the eighth model;

[0188] Selecting a fourth target SQL statement from the fourth set and the ninth set according to the first weight, the second weight, the weight of the fourth model, and the weight of the ninth model;

[0189] A fifth target SQL statement is selected from the fifth set and the tenth set according to the first weight, the second weight, the weight of the fifth model, and the weight of the tenth model.

[0190] After determining the weights, determine the similarity statements that are output at the end. Take determining the most similar statements in Set_1 as an example:

[0191] The WA and WB weights obtained above are multiplied by the corresponding sentence similarities, as shown in Table 3:

[0192] Table 3

[0193] WA*BoW_sim_Set_11 WB*TF-IDF_sim_Set_11 WA*BoW_sim_Set_12 WB*TF-IDF_sim_Set_12 WA*BoW_sim_Set_13 WB*TF-IDF_sim_Set_13 WA*BoW_sim_Set_14 WB*TF-IDF_sim_Set_14 WA*BoW_sim_Set_15 WB*TF-IDF_sim_Set_15

[0194] Compare the data in Table 3 accordingly:

[0195] When WA*BoW_sim_Set_11>=WB*TF-IDF_sim_Set_11, the Set_1 statement of BoW is taken, otherwise the Set_1 statement of TF-IDF is taken.

[0196] Furthermore, a target SQL statement set is determined according to the SQL statement to be detected, the N historical SQL statement sets, and the 2N models, including:

[0197] If the number of times the first SQL statement keyword appears in the SQL statement to be detected is greater than the number of times other SQL statement keywords appear, the top M historical SQL statements in the first historical SQL statement set corresponding to the first SQL statement keyword and ranked by similarity values ​​with the SQL statement to be detected are determined as the target SQL statement set.

[0198] In the actual operation process, after determining the first historical SQL statement set, model A is generated according to the first historical SQL statement set and the BoW bag of words model, and then model B is generated according to the first historical SQL statement set and the TF-IDF model. According to model A, the top M historical SQL statements in the similarity value ranking are selected from the first historical SQL statement set, and then according to model B, the top M historical SQL statements in the similarity value ranking are selected from the first historical SQL statement set. Here, take M as 3 as an example for explanation. Three similarities and similar statements are obtained in set A, and similarly, three similarities and similar statements are also obtained in set B. After obtaining the first three similarities and similar statements of set A and the first three similarities and similar statements of set B, the weights WA and WB of set A and set B are obtained. The method of obtaining WA and WB is the same as the method of obtaining WA and WB in the above step S101, which will not be repeated here. Multiply the 3 similarities in model A by the weight WA to obtain 3 values, such as A1, A2, and A3, and multiply the 3 similarities in model B by the weight WB to obtain 3 values, such as B1, B2, and B3. Then, compare A1 and B1 to select the SQL statement with the maximum similarity, compare A2 and B2 to select the SQL statement with the maximum similarity, and compare A3 and B3 to select the SQL statement with the maximum similarity. Finally, the selected 3 SQL statements are determined as the target SQL statement set.

[0199] S103: Obtain the target SQL statement with the highest similarity in the target SQL statement set, and display the target SQL statement.

[0200] The target SQL statement may be a SQL statement in the target SQL statement set that has the highest similarity to the SQL statement to be detected.

[0201] Furthermore, the target SQL statement with the highest similarity in the target SQL statement set is obtained, and the target SQL statement is displayed, including:

[0202] Get the target SQL statement with the highest similarity in each target SQL statement set.

[0203] Determine each target SQL statement set, and for each set, take the statement corresponding to the maximum similarity value.

[0204] We judge 5 sets and 30 sentences in turn, and finally get the 5 sentences with the highest similarity: Sent_1, Sent_2, Sent_3, Sent_4 and Sent_5. Of course, the similarity between these sentences and the input sentence is sorted from high to low.

[0205] If the similarity of any target SQL statement is greater than the similarity threshold, a duplicate prompt message is generated, and the target SQL statement and the duplicate prompt message are displayed.

[0206] The similarity threshold may be a threshold set by the user to determine the similarity of SQL statements. A threshold may be set, for example, 0.9, and when the similarity value of the target SQL statement is greater than 0.9, it is considered to be a duplicate.

[0207] Specifically, the repeated prompt information can be understood as prompt information generated by the system after detecting that the similarity between the SQL statement to be detected input by the user and the historical SQL statements in the database is greater than a similarity threshold.

[0208] Furthermore, after obtaining the target SQL statement with the highest similarity in the target SQL statement set and displaying the target SQL statement, the method further includes:

[0209] If a storage instruction input by the user is received, the SQL statement to be detected is stored in the database.

[0210] In this embodiment, the storage instruction may be an instruction input by the user to store the SQL statement in the database. The user inputs the SQL statement to be detected, and after detecting the similarity of the SQL statement, if the SQL statement to be detected is not a duplicate statement, the user inputs the storage instruction, and the system receives the instruction and stores the SQL statement to be detected in the database.

[0211] The technical solution of this embodiment obtains the SQL statement to be detected, obtains N historical SQL statement sets and performs binary conversion to remove redundancy, classification, determine weights, and use the BoW bag of words model and TF-IDF model to build 2N detection models and model fusion; determines the target SQL statement set according to the SQL statement to be detected, the N historical SQL statement sets and the 2N models; obtains the target SQL statement with the highest similarity in the target SQL statement set, generates duplicate prompt information and displays the target SQL statement. Compared with the prior art, it solves the problem that it takes a lot of time to use deep learning networks to learn to build and train complex models for similarity detection, and the model has high computational complexity and poor portability of the method. It achieves the effect of quickly and accurately finding similar SQL statements, ensures the uniqueness of data item establishment, and has great inspiration for similarity detection of other script statements.

[0212] Embodiment 2

[0213] Figure 2Schematic diagram of the structure of a SQL statement processing device provided in the second embodiment of the present invention. This embodiment is applicable to the case of detecting the similarity of SQL statements. The device can be implemented in software and / or hardware. The device can be integrated in any device that provides the function of SQL statement processing, such as Figure 2 As shown, the SQL statement processing device specifically includes: an acquisition module 210, a determination module 220 and a display module 230.

[0214] The acquisition module 210 is used to acquire the SQL statement to be detected, N historical SQL statement sets and 2N models, where N is a positive integer greater than or equal to 2;

[0215] A determination module 220 is used to determine a target SQL statement set according to the SQL statement to be detected, N historical SQL statement sets and 2N models, wherein the target SQL statement set includes: the first M historical SQL statements in the historical SQL statement set that are ranked by similarity values ​​with the SQL statement to be detected, where M is a positive integer greater than or equal to 3;

[0216] The display module 230 is used to obtain the target SQL statement with the highest similarity in the target SQL statement set and display the target SQL statement.

[0217] The above-mentioned product can execute the SQL statement processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0218] The embodiment of the present invention obtains a SQL statement to be detected, a set of N historical SQL statements, and 2N models; determines a target SQL statement set according to the SQL statement to be detected, the set of N historical SQL statements, and the 2N models; obtains the target SQL statement with the highest similarity in the target SQL statement set, and displays the target SQL statement. Compared with the prior art, the invention solves the problem that a large amount of time is consumed by using a deep learning network to learn to build and train a complex model for similarity detection, the model has high computational complexity, and the method has poor portability, and achieves the effect of quickly and accurately finding similar SQL statements, ensuring the uniqueness of data item establishment, and also has great inspiration for similarity detection of other script statements.

[0219] Optionally, the acquisition module 210 includes:

[0220] The first acquisition unit is used to acquire historical SQL statements.

[0221] A creating unit, used to create a first SQL statement set according to historical SQL statements;

[0222] The classification unit is used to classify the SQL statements in the first SQL statement set according to the SQL statement keywords to obtain N historical SQL statement sets.

[0223] Furthermore, the creation unit includes:

[0224] The first storage subunit is used to store the first historical SQL statement into the first SQL statement set.

[0225] The second storage subunit is used to store non-first historical SQL statements different from the SQL statements in the first SQL statement set into the first SQL statement set.

[0226] Further, the taxonomic units include:

[0227] The first adding subunit is used to add the first SQL statement to the first historical SQL statement set corresponding to the first SQL statement keyword if the number of times the first SQL statement keyword appears in the first SQL statement in the first SQL statement set is greater than the number of times other SQL statement keywords appear.

[0228] The second adding sub-unit is used to randomly add the first SQL statement to the first historical SQL statement set corresponding to the first SQL statement keyword or the second historical SQL statement set corresponding to other SQL statement keywords if the number of times the first SQL statement keyword appears in the first SQL statement in the first SQL statement set is equal to the number of times other SQL statement keywords appear.

[0229] The third adding subunit is used to add the first SQL statement to a third historical SQL statement set if there is no SQL statement keyword pair in the first SQL statement, wherein the first historical SQL statement set, the second historical SQL statement set and the third historical SQL statement set are different sets.

[0230] Furthermore, the N historical SQL statement sets include: a historical SQL operation statement set, a historical SQL definition statement set, a historical SQL control statement set, a historical SQL embedded statement set, and a historical SQL remaining statement set.

[0231] Accordingly, the obtaining of N models includes:

[0232] Generate a first model based on a historical SQL operation statement set and a BoW bag-of-words model;

[0233] Generate a second model based on the historical SQL definition statement set and the BoW bag of words model;

[0234] Generate a third model based on the historical SQL control statement set and the BoW bag-of-words model;

[0235] Generate the fourth model based on the historical SQL embedding statement set and the BoW bag-of-words model;

[0236] Generate the fifth model based on the remaining historical SQL statement set and the BoW bag-of-words model;

[0237] Generate the sixth model based on the historical SQL operation statement set and the TF-IDF model;

[0238] Generate the seventh model based on the historical SQL definition statement set and the TF-IDF model;

[0239] Generate the eighth model based on the historical SQL control statement set and the TF-IDF model;

[0240] Generate the ninth model based on the historical SQL embedding statement set and the TF-IDF model;

[0241] The tenth model is generated based on the historical SQL remaining statement set and the TF-IDF model.

[0242] Furthermore, before the SQL statements in the first SQL statement set are classified according to the SQL statement keywords to obtain N historical SQL statement sets, the method further includes:

[0243] Eliminate comments in SQL statements based on regular expressions;

[0244] and / or;

[0245] If the length of a target word constituting an SQL statement in the first SQL statement set is greater than a length threshold, the target word is converted into a target identifier, wherein the target word is not a SQL statement keyword and is not a number.

[0246] Further, the target word is converted into a target identifier, including:

[0247] Convert the target word into a binary number;

[0248] The target identity is determined based on the binary number and the preset identifier.

[0249] Optionally, the display module 230 includes:

[0250] The second acquisition unit is used to acquire the target SQL statement with the highest similarity in each target SQL statement set.

[0251] The display unit is used to generate duplicate prompt information if the similarity of any target SQL statement is greater than a similarity threshold, and to display the target SQL statement and the duplicate prompt information.

[0252] Furthermore, before determining the target SQL statement set according to the SQL statement to be detected, the N historical SQL statement sets and the 2N models, the method further includes:

[0253] Determine the ratio of the number of historical SQL statements in the historical SQL operation statement set to the number of historical SQL statements in the first SQL statement set as the weight of the first model and the weight of the sixth model;

[0254] Determine the ratio of the number of historical SQL statements in the historical SQL definition statement set to the number of historical SQL statements in the first SQL statement set as the weight of the second model and the weight of the seventh model;

[0255] Determine the ratio of the number of historical SQL statements in the historical SQL control statement set to the number of historical SQL statements in the first SQL statement set as the weight of the third model and the weight of the eighth model;

[0256] Determine the ratio of the number of historical SQL statements in the historical SQL embedded statement set to the number of historical SQL statements in the first SQL statement set as the weight of the fourth model and the weight of the ninth model;

[0257] The ratio of the number of historical SQL statements in the historical SQL remaining statement set to the number of historical SQL statements in the first SQL statement set is determined as the weight of the fifth model and the weight of the tenth model.

[0258] Furthermore, a target SQL statement set is determined according to the SQL statement to be detected, the N historical SQL statement sets, and the 2N models, including:

[0259] Acquire a first set according to the first model, wherein the first set includes: historical SQL statements in the historical SQL operation statement set ranked by similarity values ​​with the SQL statement to be detected, which are the top M;

[0260] Acquire a second set according to the second model, wherein the second set includes: the top M historical SQL statements in the historical SQL definition statement set that are ranked by similarity values ​​with the SQL statement to be detected;

[0261] Acquire a third set according to the third model, wherein the third set includes: the top M historical SQL statements in the historical SQL control statement set that are ranked by similarity values ​​with the SQL statement to be detected;

[0262] Acquire a fourth set according to the fourth model, wherein the fourth set includes: the top M historical SQL statements in the historical SQL embedded statement set that are ranked by similarity values ​​with the SQL statement to be detected;

[0263] Acquire a fifth set according to the fifth model, wherein the fifth set includes: the top M historical SQL statements in the historical SQL remaining statement set that are ranked by similarity values ​​with the SQL statement to be detected;

[0264] Acquire a sixth set according to the sixth model, wherein the sixth set includes: the top M historical SQL statements in the historical SQL operation statement set that are ranked by similarity values ​​with the SQL statement to be detected;

[0265] Acquire a seventh set according to the seventh model, wherein the seventh set includes: the top M historical SQL statements in the historical SQL definition statement set that are ranked by similarity values ​​with the SQL statement to be detected;

[0266] An eighth set is obtained according to the eighth model, wherein the eighth set includes: the top M historical SQL statements in the historical SQL control statement set that are ranked by similarity values ​​with the SQL statement to be detected;

[0267] Acquire a ninth set according to the ninth model, wherein the ninth set includes: the top M historical SQL statements in the historical SQL embedded statement set that are ranked by similarity values ​​with the SQL statement to be detected;

[0268] A tenth set is obtained according to the tenth model, wherein the tenth set includes: the top M historical SQL statements in the historical SQL remaining statement set that are ranked by similarity values ​​with the SQL statement to be detected;

[0269] determining a first weight and a second weight according to the similarity of historical SQL statements in the first set, the similarity of historical SQL statements in the second set, the similarity of historical SQL statements in the third set, the similarity of historical SQL statements in the fourth set, the similarity of historical SQL statements in the fifth set, the similarity of historical SQL statements in the sixth set, the similarity of historical SQL statements in the seventh set, the similarity of historical SQL statements in the eighth set, the similarity of historical SQL statements in the ninth set, and the similarity of historical SQL statements in the tenth set;

[0270] Selecting a first target SQL statement from the first set and the sixth set according to the first weight, the second weight, the weight of the first model, and the weight of the sixth model;

[0271] Selecting a second target SQL statement from the second set and the seventh set according to the first weight, the second weight, the weight of the second model, and the weight of the seventh model;

[0272] Selecting a third target SQL statement from the third set and the eighth set according to the first weight, the second weight, the weight of the third model, and the weight of the eighth model;

[0273] Selecting a fourth target SQL statement from the fourth set and the ninth set according to the first weight, the second weight, the weight of the fourth model, and the weight of the ninth model;

[0274] A fifth target SQL statement is selected from the fifth set and the tenth set according to the first weight, the second weight, the weight of the fifth model, and the weight of the tenth model.

[0275] Optionally, the determination module 220 is specifically configured to:

[0276] If the number of times the first SQL statement keyword appears in the SQL statement to be detected is greater than the number of times other SQL statement keywords appear, the top M historical SQL statements in the first historical SQL statement set corresponding to the first SQL statement keyword and ranked by similarity values ​​with the SQL statement to be detected are determined as the target SQL statement set.

[0277] Embodiment 3

[0278] Figure 3 It is a structural schematic diagram of an electronic device provided in Embodiment 3 of the present invention. Figure 3 A block diagram of an electronic device 312 suitable for implementing embodiments of the present invention is shown. Figure 3 The electronic device 312 shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. The device 312 is a typical computing device for trajectory fitting functions.

[0279] like Figure 3 As shown, the electronic device 312 is in the form of a general purpose computing device. The components of the electronic device 312 may include, but are not limited to: one or more processors 316, a storage device 328, and a bus 318 connecting different system components (including the storage device 328 and the processor 316).

[0280] Bus 318 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0281] The electronic device 312 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 312, including volatile and non-volatile media, removable and non-removable media.

[0282] The storage device 328 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 330 and / or cache memory 332. The electronic device 312 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 334 may be used to read and write non-removable, non-volatile magnetic media ( Figure 3 not shown, usually called a "hard drive"). Although Figure 3 Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a read-only optical disk (Compact Disc-Read Only Memory, CD-ROM), a digital video disk (Digital Video Disc-Read Only Memory, DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 318 via one or more data medium interfaces. The storage device 328 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0283] A program 336 having a set (at least one) of program modules 326 may be stored, for example, in a storage device 328, such program modules 326 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 326 generally perform the functions and / or methods of the embodiments described herein.

[0284] The electronic device 312 may also communicate with one or more external devices 314 (e.g., keyboard, pointing device, camera, display 324, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 312, and / or communicate with any device that enables the electronic device 312 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 322. In addition, the electronic device 312 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter 320. As shown, the network adapter 320 communicates with other modules of the electronic device 312 through a bus 318. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 312, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) systems, tape drives, and data backup storage systems.

[0285] The processor 316 executes various functional applications and data processing by running the programs stored in the storage device 328, such as implementing the SQL statement processing method provided in the above embodiment of the present invention.

[0286] Embodiment 4

[0287] Figure 4 6 is a schematic diagram of the structure of a computer-readable storage medium containing a computer program provided in Embodiment 4 of the present invention. The embodiment of the present invention provides a computer-readable storage medium 61 on which a computer program 610 is stored. When the program is executed by one or more processors, the SQL statement processing method provided in all the embodiments of the present invention is implemented:

[0288] Obtain a SQL statement to be detected, a set of N historical SQL statements, and 2N models, where N is a positive integer greater than or equal to 2;

[0289] Determine a target SQL statement set according to the SQL statement to be detected, N historical SQL statement sets and 2N models, wherein the target SQL statement set includes: the first M historical SQL statements in the historical SQL statement set that are ranked by similarity values ​​with the SQL statement to be detected, where M is a positive integer greater than or equal to 3;

[0290] The target SQL statement with the highest similarity in the target SQL statement set is obtained, and the target SQL statement is displayed.

[0291] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media or any combination of the above two. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0292] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0293] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0294] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0295] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0296] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0297] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0298] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0299] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0300] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for processing SQL statements, characterized in that: include: Obtain the SQL statement to be detected, N historical SQL statement sets and 2N models, where N is a positive integer greater than or equal to 2; wherein, N models are generated based on the BoW bag of words model and the historical SQL statement set, and N models are generated based on the TF-IDF model and the historical SQL statement set, and a total of 2N models are obtained; Determine a target SQL statement set according to the SQL statement to be detected, N historical SQL statement sets and 2N models, wherein the target SQL statement set includes: the first M historical SQL statements in the historical SQL statement set that are ranked by similarity values ​​with the SQL statement to be detected, where M is a positive integer greater than or equal to 3; The target SQL statement with the highest similarity in the target SQL statement set is obtained, and the target SQL statement is displayed.

2. The method according to claim 1, characterized in that: After obtaining the target SQL statement with the highest similarity in the target SQL statement set and displaying the target SQL statement, the method further includes: If a storage instruction input by the user is received, the SQL statement to be detected is stored in the database.

3. The method according to claim 1, characterized in that Obtaining a set of N historical SQL statements includes: Get historical SQL statements; Creating a first SQL statement set according to the historical SQL statements; The SQL statements in the first SQL statement set are classified according to SQL statement keywords to obtain N historical SQL statement sets.

4. The method according to claim 3, characterized in that: Creating a first SQL statement set according to the historical SQL statements includes: Store the first historical SQL statement into the first SQL statement set; The non-first historical SQL statement different from the SQL statement in the first SQL statement set is stored in the first SQL statement set.

5. The method according to claim 3, characterized in that: The SQL statements in the first SQL statement set are classified according to SQL statement keywords to obtain N historical SQL statement sets, including: If the number of occurrences of a first SQL statement keyword in a first SQL statement in the first SQL statement set is greater than the number of occurrences of other SQL statement keywords, adding the first SQL statement to a first historical SQL statement set corresponding to the first SQL statement keyword; If the number of occurrences of a first SQL statement keyword in a first SQL statement in the first SQL statement set is equal to the number of occurrences of other SQL statement keywords, randomly adding the first SQL statement to a first historical SQL statement set corresponding to the first SQL statement keyword or a second historical SQL statement set corresponding to the other SQL statement keywords; If the first SQL statement does not contain an SQL statement keyword pair, the first SQL statement is added to a third historical SQL statement set, wherein the first historical SQL statement set, the second historical SQL statement set and the third historical SQL statement set are different sets.

6. The method according to claim 1, characterized in that Obtaining the target SQL statement with the highest similarity in the target SQL statement set and displaying the target SQL statement, including: Obtain the target SQL statement with the highest similarity in each target SQL statement set; If the similarity of any target SQL statement is greater than a similarity threshold, a duplicate prompt message is generated, and the target SQL statement and the duplicate prompt message are displayed.

7. The method according to claim 3, characterized in that Before classifying the SQL statements in the first SQL statement set according to the SQL statement keywords to obtain N historical SQL statement sets, the method further includes: Eliminate comment information in the SQL statement based on a regular expression; and / or; If the length of a target word constituting an SQL statement in the first SQL statement set is greater than a length threshold, the target word is converted into a target identifier, wherein the target word is not a SQL statement keyword and is not a number.

8. The method according to claim 7, characterized in that Converting the target word into a target identifier includes: Convert the target word into a binary number. The target identifier is determined according to the binary number and a preset identifier.

9. The method according to claim 3, characterized in that: The N historical SQL statement sets include: a historical SQL operation statement set, a historical SQL definition statement set, a historical SQL control statement set, a historical SQL embedded statement set, and a historical SQL remaining statement set.

10. The method according to claim 1, characterized in that The target SQL statement set is determined based on the SQL statement to be detected, N historical SQL statement sets, and 2N models, including: If the number of times a first SQL statement keyword appears in the SQL statement to be detected is greater than the number of times other SQL statement keywords appear, the top M historical SQL statements in the first historical SQL statement set corresponding to the first SQL statement keyword and ranked by similarity values ​​with the SQL statement to be detected are determined as the target SQL statement set.

11. A SQL statement processing device, characterized in that: include: An acquisition module is used to acquire the SQL statement to be detected, N historical SQL statement sets and 2N models, where N is a positive integer greater than or equal to 2; wherein N models are generated based on the BoW bag of words model and the historical SQL statement set, and N models are generated based on the TF-IDF model and the historical SQL statement set, so that a total of 2N models are obtained; A determination module is used to determine a target SQL statement set according to the SQL statement to be detected, N historical SQL statement sets and 2N models, wherein the target SQL statement set includes: the first M historical SQL statements in the historical SQL statement set that are ranked by similarity values ​​with the SQL statement to be detected, where M is a positive integer greater than or equal to 3; The display module is used to obtain the target SQL statement with the highest similarity in the target SQL statement set and display the target SQL statement.

12. The device according to claim 11, characterized in that Also includes: The storage module is used to store the SQL statement to be detected in a database if a storage instruction input by a user is received after obtaining the target SQL statement with the highest similarity in the target SQL statement set and displaying the target SQL statement.

13. The device according to claim 11, characterized in that The acquisition modules include: A first acquisition unit is used to acquire historical SQL statements; A creating unit, configured to create a first SQL statement set according to the historical SQL statements; The classification unit is used to classify the SQL statements in the first SQL statement set according to SQL statement keywords to obtain N historical SQL statement sets.

14. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the processors are enabled to implement the method according to any one of claims 1 to 10.

15. A computer-readable storage medium containing a computer program, wherein the computer program is stored thereon, characterized in that: When the program is executed by one or more processors, the method according to any one of claims 1 to 10 is implemented.

Images