A power data security monitoring system code clone detection method and device

Abstract

The application discloses a kind of electric power data security monitoring system code clone detection method and device, the method includes: according to partition index mechanism, the electric power data security monitoring system software source code to be detected is partitioned, and is searched according to partition acquisition method;Based on the size of method, the first type and the second type clone pair possibly are screened out;After eliminating the first type and the second type clone pair screened, based on the similarity of calling method, the third type clone pair possibly is screened out;After eliminating the three clone pairs screened above, for the fourth clone pair, code semantic representation is obtained based on code control flow and data flow analysis;Classification is carried out using a pre-trained deep learning model, and semantic level clone code is detected.The application effectively improves the effectiveness of large-scale code clone detection in the process of electric power data security monitoring system research and development under multi-version iteration, and guarantees the safe and stable operation of system.

Description

TECHNICAL FIELD

[0001] The present application relates to a power data security monitoring system code clone detection method and device, which is applied to the security test in the software research and development process of the power data security monitoring system and the risk analysis of the software research and development execution code. BACKGROUND

[0002] The power system, as a system for the synthesis, transmission, distribution and consumption of electric energy, plays an increasingly powerful role in people's life and work. With the development of computer technology, the application of computers and software has deeply penetrated into the power industry. With the vigorous development of the digital transformation of the Southern Power Grid, data presents an "explosive" growth and accumulation, and the application of power big data has become a strategic commanding point for enterprise development, and data security has been raised to a new height. In view of the code data security caused by code clone in the research and development process of the power data security monitoring system and the potential risk analysis problem in the software execution process, the software defects are mainly detected by designing test cases, the high-quality operation of the software is effectively guaranteed through the repair of defects, and the real-time monitoring business operation of the power data security is supported.

[0003] In the research and development process of the power data security monitoring system, in order to improve the programming efficiency, it is a common phenomenon for the developers to copy, paste and modify code fragments. The repeated code or similar code is also called code clone, which mainly refers to two or more identical or similar source code fragments existing in the code library. The moderate use of code clone plays a positive role in the software life cycle, aiming to reduce errors and improve software development efficiency. However, the abuse of code clone is a great challenge in the process of software development, testing, maintenance and redevelopment, which brings a series of problems:

[0004] (1) In the aspect of software research and development, code clone may cause code data redundancy and scale increase in the software system, and thus increase the demand for information resources and maintenance costs;

[0005] (2) In the aspect of code security, by copying and pasting a piece of code containing unknown vulnerabilities, the number of vulnerabilities will increase, and new vulnerabilities may also be introduced in the cloning process, which will have immeasurable consequences on the security of code data.

[0006] Cloning detection is a process of locating exact or similar code fragments within or between software systems, and effective cloning detection method becomes an important part of software testing and analysis. At present, according to the detection target and granularity, it is divided into four categories from easy to detect to difficult to detect, namely text similarity, lexical similarity, syntactic similarity and semantic similarity. Among them, text similarity mainly refers to the same code fragments, except for the differences in blank, layout and annotation; lexical similarity mainly refers to the same code fragments, except for the differences in identifier name and literal value, and text difference; syntactic similarity mainly refers to the code fragments with similar syntax at the statement level, and in addition to the differences in text similarity and lexical similarity, the fragments also add, modify and / or delete the statements related to each other; for semantic similarity, it mainly refers to the code fragments with different syntax but the same function. The mainstream method of existing cloning code detection is carried out under the condition of converting source code into intermediate representation, and for the completely same or almost identical code cloning detection, the existing algorithm and tool can detect well. The power data security monitoring system can monitor the data security in real time under the complex network environment, and then comprehensively improve the security of the power network. Its business process is complex and has high requirements for the safety of the system itself. In view of the data security of the underlying code in the iteration process of the system, the existing method has problems such as low recognition rate, high computational complexity, high false positive rate and the like for the difference code cloning with certain similarity in text and the difference code cloning without similarity in text. How to intelligently detect different types of cloning codes through effective technical means is of great significance to ensure the software code quality of the power data security monitoring system itself. SUMMARY

[0007] The purpose of the present application is to provide a code cloning detection method and device for the research and development process version iteration of a power data security monitoring system, and at least partially solve the problems in the background art.

[0008] Technical scheme: In a first aspect, a power data security monitoring system code cloning detection method comprises the following steps:

[0009] After eliminating the embedded code that is impossible to be a cloning pair, the software source code of the power data security monitoring system to be detected is partitioned according to a partition index mechanism, so that the acquisition method can be searched according to the partition index. The partition index mechanism comprises: determining the number of partitions, determining the partition key based on the name and version of the code method, mapping the partition key to a specific partition number by using a hash function, and distributing the code to the corresponding partition according to the hash function, so that the corresponding partition can be located by the partition key when the code is detected for cloning, and the methods are searched and accessed at the same time in different partitions;

[0010] The source code lines of the methods in different partitions are converted into a series of tokens, wherein the token is a specified keyword, an identifier, a metric based on the size of the method, and a possible first type and second type clone pair is screened out, wherein the first type clone pair is a clone pair with the same code segment, and the second type clone pair is a clone pair with the same code structure or code syntax;

[0011] After the first type and the second type clone pair are screened out, the corresponding partition is located through the partition key, the token sliding window is set, the similarity between the tokens of the two methods is measured through the overlap similarity, and a possible third type clone pair is screened out according to the similarity, wherein the third type clone pair is a method clone pair with the same size, similar operation and similar call;

[0012] After the first type, the second type and the third type clone pair are screened out, for the remaining code segment, the dependence between the basic blocks and the processes is captured through the control flow, the flow of data values along the program path and the operation is captured through the data flow graph, and a semantic representation of the code behavior is obtained;

[0013] Based on the semantic representation of the code, a pre-trained deep learning model is used for classification to detect a fourth type clone pair, wherein the fourth type clone pair is a semantic level clone code.

[0014] Preferably, for the determination of the number of partitions, the program size, the data volume, the business logic and the hardware resources are comprehensively considered to determine the number of partitions.

[0015] Preferably, the metric of the method size includes that: a method M1 has x tokens, a method M2 has y tokens, a given similarity threshold T is between 0 and 1, when x*T≤y≤x / T is satisfied, it is determined that the method M1 and the method M2 belong to a clone pair.

[0016] Preferably, the calculation method of the similarity between the tokens of the two methods through the overlap similarity is as follows:

[0017]

[0018] Wherein, sim<M1,M2> represents the similarity between the method M1 and the method M2, A and B are respectively the token set of the method M1 and the method M2, each element of the token set A of the method M1 is defined as <ta i ,freqta i ,s,Na> and each element of the token set B of the method M2 is defined as <tb j ,freqtb j, s, Nb>, 1≤i≤j≤n; s is the token sliding window set; Na is the total number of rows of method M1, Nb is the total number of rows of method M2; freqta i is the ta of token set A in method M1 i is the number of occurrences under the sliding window, ta i is the i-th token of token set A in method M1; freqtb j is the tb of token set B in method M2 j is the number of occurrences under the sliding window, tb j is the j-th token of token set B in method M2; α is the sliding window sequence number after the token sliding window is divided.

[0019] Preferably, when obtaining the semantic representation of the code behavior, the control flow and data flow information are encoded by variable features, basic block features, and relationship features between variables and basic blocks, and the encoding rules are as follows:

[0020] The variable feature is mainly obtained by extracting each variable, and the type feature includes data type V(t), modifier V(m) and other information V(o), wherein V(t)={bool, byte, char, double, float, int, long, short, void, JDK class, user-defined type}, V(m)={final, static, none}, V(o)={basic, array, pointer, reference}, and the above variable features are encoded into an 18d binary feature vector V={V(t), V(m), V(o)};

[0021] The basic block feature is the feature of each basic block, mainly considering the following seven types: normal, loop, loop body, if, if body, switch, and switch body, and the type information is encoded into a 7-dimensional one-hot feature vector D;

[0022] The relationship features between variables and basic blocks, mainly referring to encoding data flow and control flow as operations between variables and control jumps between basic blocks, a total of 38 main operation types, addition: ADD; subtraction: SUB; multiplication: MUL; division: DIV; remainder: REM; bitwise AND: AND; bitwise OR: OR; bitwise XOR: XOR; left shift: SHL; right shift: SHR; unsigned right shift: USHR; negation: NEG; logical NOT: NOT; bitwise NOT: BITNOT; equal to: EQ; not equal to: NE; greater than: GT; greater than or equal to: GE; less than: LT; less than or equal to: LE; type checking: INSTANCEOF; type conversion: CHECKCAST; conditional branch: COND_BRANCH; jump: GOTO; type conversion: CONVERSION; array length: ARRAY_LENGTH; Phi function: PHI; method call: INVOKE; get caught exception: GET_CAUGHT_EXCEPTION; abstract method call: ABSTRACT_INVOKE; checkpoint: CHECKPOINT; conditional switch: COND_SWITCH; field assignment: PUT; field get: GET; get address: ADDRESSOF; return: RETURN; branch: BRANCH; identity: IDENTITY, using a 38-dimensional one-hot feature vector M to represent;

[0023] In order to preserve the information of each operation between variables in the data flow graph, an 18*2+38=74d feature vector T={V option1 , V option2 , M} is generated, where V option1 and V option2 represent the feature vectors of variable option1 and option2 in the semantic matrix encoding respectively, in order to maintain the control flow, T is extended to {V option1 , V option2 , M, D} with a size of 81d.

[0024] Preferably, the semantic representation of code behavior is generated as a semantic feature matrix E according to the encoding rule, each row and each column in E is a variable or a basic block, the order of which corresponds to the order of code instructions and basic blocks, E(i,j)=T(i,j) is a binary feature vector with a size of 81, representing the relationship between row i and column j, i,j=1,2…n, where n=n v +b b is the sum of the variable count and the basic block count.

[0025] Preferably, the deep learning model is established by the following way:

[0026] Based on the sparse binary feature vector encoding data flow and control flow, it is mapped to a hidden state h0 of size c, c is a constant; then each row of the semantic feature matrix is converted into a vector of length c·k, k is the specified input matrix size; then add two fully connected layers, take the weakened row feature vector as input, to extract all information related to the variables or basic blocks represented by each row; finally, add a pooling layer to summarize all row feature vectors, connect the latent representation vectors of the semantic matrices of the two methods in different orders, apply a fully connected layer on the two concatenated vectors with shared weights W fc , perform another average pooling on the output state, and finally add a classification layer.

[0027] In a second aspect, a power data security monitoring system code clone detection device comprises:

[0028] A code partitioning module is configured to eliminate embedded code that cannot be a clone pair, and then partition the power data security monitoring system software source code to be detected according to a partition index mechanism, so that it can be searched and obtained by partition index search method. The partition index mechanism comprises: determining the number of partitions, determining the partition key based on the name and version of the code method, mapping the partition key to a specific partition number using a hash function, and distributing the code to the corresponding partition according to the hash function, so that the corresponding partition can be located by the partition key when the code is detected for cloning, and the methods are searched and accessed simultaneously in different partitions;

[0029] A method size detection module is configured to convert the source code lines of the methods in different partitions into a series of tokens, where token is a specified keyword or identifier, and filter out possible first type and second type clone pairs based on the size of the method. The first type of clone pair is a clone pair with identical code fragments, and the second type of clone pair is a clone pair with identical code structure or syntax.

[0030] A method similarity detection module is configured to locate the corresponding partition by the partition key after eliminating the first type and second type clone pairs filtered out, set a token sliding window, measure the similarity between the two method tokens by overlapping similarity, and filter out possible third type clone pairs according to the similarity. The third type of clone pair is a method clone pair with the same size, similar operation and similar call.

[0031] A code semantic representation construction module is configured to, after eliminating the first type, second type and third type clone pairs filtered out, for the remaining code fragments, capture the dependency relationship between code basic blocks and procedures through control flow, and capture the flow of data values along program paths and operations through data flow graph, to obtain the semantic representation of code behavior.

[0032] The semantic clone detection module is configured to utilize a pre-trained deep learning model to classify based on a code semantic representation, and detect a fourth type of clone pair, which is a semantic level cloned code.

[0033] In a third aspect, the present application also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs, when executed by the processors, implement the steps of the power data security monitoring system code clone detection method as described above.

[0034] In a fourth aspect, the present application also provides a computer readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the steps of the power data security monitoring system code clone detection method as described above.

[0035] Beneficial effects: The present application takes the code fragment set to be detected as a starting point, and performs clone detection in partitions, effectively detects cloned codes with the same text, vocabulary and syntax and high similarity through method size and method similarity analysis techniques, and for the remaining code fragments, in order to make the code semantic representation capture more code semantic information, the control flow and data flow are effectively encoded into a semantic feature matrix through code control flow and data flow encoding techniques, and high-level features are learned from the semantic matrix based on a deep learning model, and semantic similar cloned code detection is effectively performed, thereby realizing efficient and reliable detection of large-scale cloned codes. Compared with the prior art, the present application performs clone pair screening layer by layer, improves the timeliness, effectiveness and accuracy of large-scale code clone detection in the multi-version iteration of the power data security monitoring system development process, and guarantees the safe and stable operation of the system. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a power data security monitoring system code clone detection method flowchart of the present application;

[0037] Figure 2 is a classification detection process schematic diagram of the present application;

[0038] Figure 3 is a deep learning model schematic diagram of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings.

[0040] Referring to Figure 1 The present application proposes a power data security monitoring system code clone detection method, which comprises the following steps:

[0041] Step S1, according to the partition index mechanism, the data security monitoring system software source code to be detected is partitioned, and the partition index is obtained.

[0042] When detecting the real code clone of the data security monitoring system software source code, most methods are not clones of each other. However, clone detection needs to process all possible method pairs in order to find out which are clones. For the system, a large-scale code detection of multiple versions, it is not very likely that two methods of very different sizes will implement the same or similar functions.

[0043] First, eliminate unlikely clone pairs. For embedded code, remove code by isolating partitions. Mainly includes deleting SQL statements embedded in the code segment based on java program. SQL statements mainly include connecting to a database, creating a SQL insert statement, executing a SQL insert statement, and closing a database connection. If the code covers methods such as DriverManager.getConnection(), PreparedStatement, executeUpdate(), etc., delete this part of the code.

[0044] Next, divide the code into different partitions for management based on the partition index acquisition method. Effective partitioning of the code can limit the impact within the partition, thereby reducing the number of problems and the cost of handling problems, and effectively improving the efficiency of indexing for cloned code. Partitioning mainly considers two principles: first, method names with the same prefix need to be divided into the same partition as much as possible; second, when dividing the code set into multiple partitions, the search quality needs to be effectively guaranteed under the premise of effectively reducing the large-scale query search cost. For the determination of the number of partitions, the program size, data volume, business logic, hardware resources, etc. are considered; then determine the partition key based on the code method name, version, etc. At the same time, based on the determined partition key, a hash function is implemented to map the partition key to a specific partition number; the code is allocated to the corresponding partition according to the hash function; when the code data is cloned, the corresponding partition is quickly located through the partition key, and the methods in different partitions are searched and accessed at the same time, effectively performing clone detection.

[0045] S2, based on the size of the method, screen out possible first type and second type clone pairs.

[0046] The present application is based on the consideration of method size, in order to ensure that the source code information is fully utilized, the source code lines of the methods in different partitions are converted into a series of tokens. The method name and the number of tokens are used as the measurement of the method size, wherein the token is a keyword, an identifier, etc. The keywords mainly include Auto, break, case, char, const, continue, default, etc.; the identifiers mainly include special symbols such as __*__ and __*__. According to the embodiment of the present application, given a similarity threshold T between 0 and 1, if a method M1 has x tokens, and a method M2 is a textual and lexical type clone of M1, then the number of tokens y of M2 should satisfy the inequality x × T ≤ y ≤ x / T. In the present application, T = 0.9 is set. Through the above comparison method, first, the first type and the second type of clone pairs are screened out.

[0047] Code cloning mainly includes textual similarity and functional similarity. For textual similarity, the code clone pairs are divided into three types:

[0048] Type one: the code fragments are completely the same, except for spaces, layout formats or comments of the code;

[0049] Type two: the structure or syntax of the code is the same, except for user-defined variables, constant values, types, layouts or comments;

[0050] Type three: the copied and pasted code fragments are modified. In addition to the changes in type two, the program itself may add, delete or modify part of the statements;

[0051] Functional similarity is divided into type four: the functions of the two code fragments are similar or completely the same.

[0052] Type one and type two can be detected by the method size measurement of the present step.

[0053] S3, after the first type and the second type of clone pairs screened out in step S2 are removed, the possible third type of clone pairs are effectively screened out based on the method similarity analysis method.

[0054] If two methods perform the same function, even if these methods are different in text and vocabulary, but they can call the same library method and refer to the same object attribute, so the similarity of the function calls of these methods can be used to detect the cloning.

[0055] The application positions to the corresponding partition through partition key, and sets token sliding window to effectively perform code clone detection. i ,freqta i ,s,Na> and each element of token set A in method M1 is defined as <ta j ,freqtb j ,s,Nb> and each element of token set B in method M2 is defined as <tb i ,freqtb i is the number of occurrences of the i-th token in token set A in method M1, and freqtb i is the number of occurrences of the j-th token in token set B in method M2. j is the i-th token in token set A in method M1, and tb j is the j-th token in token set B in method M2. j is the number of occurrences of the i-th token in token set A in method M1, and freqtb option1 is the number of occurrences of the j-th token in token set B in method M2. option2 The similarity between the tokens of the two methods is measured by using overlapping similarity, and the calculation method is as follows: option1 option2 option1 option2 v Wherein, alpha is the sliding window number after the token sliding window is divided. b v Based on the measurement value, it is determined whether it is a clone or not. For example, if the similarity calculation result of the two methods is greater than a specified threshold value, it is considered to be a method clone pair. b v Through the above use method, the method clone pair with the same size, similar operation and software similar call can be effectively found out. b b S4, if the measurement value is determined to be non-clone, the candidate object is paired with the partition query, and a feature vector is created for the candidate object. Then, based on the trained deep learning model, semantic similarity analysis is performed on the candidate pairs to predict whether the pair is a clone pair. b v After further eliminating the third type of clone pairs screened in step S3, the code semantic representation is obtained based on code control flow and data flow analysis. b

[0063] The code semantics represented by control flow and data flow is encoded into a single semantic matrix for the input of the code snippet which has been detected by text, vocabulary, and syntax similarity clone detection. CFG is a kind of directed graph used to represent all execution paths in the process of program execution. The dependence between basic blocks and procedures is captured by control flow, and the flow of data values along program paths and operations is captured by DFG. The information obtained from control flow analysis and data flow analysis is used as code behavior representation. Finally, a matrix containing control flow information and data flow information is obtained.

[0064] For semantic matrix encoding, the application encodes control flow and data flow information through variable features, basic block features, and relationship features between variables and basic blocks.

[0065] Variable features are mainly obtained by extracting each variable, and its type features include data type V(t), modifier V(m), and other information V(o).

[0066] V(t) = {bool, byte, char, double, float, int, long, short, void, JDK class, user-defined type};

[0067] V(m) = {final, static, none};

[0068] V(o) = {basic, array, pointer, reference}.

[0069] The variable features mentioned above are encoded into an 18d binary feature vector V = {V(t), V(m), V(o)}.

[0070] Basic block features are features for each basic block, mainly considering the following seven types: normal, loop, loop body, if, if body, switch, and switch body. At the same time, this type of information is encoded into a 7-dimensional one-hot feature vector D.

[0071] The relationship features between variables and basic blocks, mainly refer to encoding data flow and control flow as operations between variables and control jumps between basic blocks. A total of 38 different operation types are included, addition: ADD; subtraction: SUB; multiplication: MUL; division: DIV; remainder: REM; bitwise AND: AND; bitwise OR: OR; bitwise XOR: XOR; left shift: SHL; right shift: SHR; unsigned right shift: USHR; negation: NEG; logical NOT: NOT; bitwise NOT: BITNOT; equal: EQ; not equal: NE; greater than: GT; greater than or equal to: GE; less than: LT; less than or equal to: LE; type check: INSTANCEOF; type conversion: CHECKCAST; conditional branch: COND_BRANCH; jump: GOTO; type conversion: CONVERSION; array length: ARRAY_LENGTH; Phi function: PHI; method call: INVOKE; get caught exception: GET_CAUGHT_EXCEPTION; abstract method call: ABSTRACT_INVOKE; checkpoint: CHECKPOINT; conditional switch: COND_SWITCH; field assignment: PUT; field get: GET; get address: ADDRESSOF; return: RETURN; branch: BRANCH; identity: IDENTITY. A 38-dimensional one-hot feature vector M is used. In order to retain the information of each operation between variables in the data flow DFG, an 18*2+38=74d feature vector T={V option1 , V option2 , M} is generated, where V option1 and V option2 represent the feature vectors of variables option1 and option2 in the semantic matrix encoding respectively. In order to maintain the control flow, T is extended to {V option1 , V option2 , M, D} with a size of 81d.

[0072] For the construction of the semantic feature matrix E, based on the given code snippet, the control flow and data flow information are generated by the above encoding rules to capture them. Each row and each column in E is a variable or a basic block, and the order corresponds to the code instructions and the basic block order. E(i,j)=T(i,j) is a binary feature vector with a size of 81, representing the relationship between row i and column j, i,j=1,2…n, where n=n v +b bis the sum of variable count and basic block count. The semantic feature matrix represents the way that is not to encode all the information in the program dependence graph PDG, but it represents all the dependencies through the relationship features between variables and basic blocks. In this way, the semantic feature matrix can simplify the problem of finding isomorphic subgraphs to detecting similar patterns in the matrix.

[0073] S6, based on the code semantic representation, using a pre-trained deep learning model for classification, detecting semantic level cloned code.

[0074] As shown in Figure 2 and Figure 3 , for this deep learning model, a feedforward neural network is first trained, which generates high-level features from the semantic matrix of each code pair through the feedforward neural network, and uses the classification layer at the end of the neural network to classify them as functionally similar or dissimilar. In this way, the definition of a separate distance metric for learning features is eliminated, and the model can be fine-tuned in a supervised manner to learn better representations from the input by backpropagating the label signals of the classification layer. The way the feedforward neural network generates high-level features requires a fixed-size input semantic feature matrix, which is first processed to a fixed size k x k, k = 128 in the embodiment. For smaller methods, the feature matrix is padded with zero values; for larger methods, the matrix is truncated by retaining all CFG information and discarding DFG information beyond the matrix size, the original feature matrix size n = n v +b b , the truncated k = n v ′+n b ′ = 128, where n b ′ = n b , n v ′ = 128-n b .

[0075] The present application selects the ELU function as the activation function, ELU(c) = {c, if c ≥ 0; e c -1, if c < 0}, c is a constant. Based on encoding data flow and control flow into 81d sparse binary feature vectors, which are mapped to hidden states h0 with a size of c = 6. Then each row of E is converted into a vector with a length of c k, followed by two fully connected layers, which take the weakened row feature vector as input to extract all the information related to the variable or basic block represented by each row. Finally, a pooling layer is added to aggregate all the row feature vectors. The latent representation vectors h 13 and h 23 are connected in different orders: [h 13 , h 23 ], [h23 , h 13 ]m1, m2 are semantic matrices generated from methods M1 and M2 respectively.

[0076] Then apply a fully connected layer on two concatenated vectors with shared weights W fc . Perform another average pooling on the output states and finally add a classification layer. Cross-entropy is used as the cost function:

[0077]

[0078] Minimize the cost function to optimize the whole model, S (i) is the true label of sample i; S (i) is the predicted label; N is the number of input samples. In this way, the model can learn the similarity between each pair of code with different syntax.

[0079] In order to alleviate the impact of code statement reordering, the present application ignores the order of arguments or basic blocks, only retains the order of variables or basic blocks with dependencies between them, and uses average pooling instead of dispersing the whole matrix, so that the pooling layer reduces the input size from c x k x k to c x k without any additional parameters, effectively reducing the complexity of the model and improving the training and prediction efficiency. The first average pooling effectively calculates the average state of all rows of the matrix, reducing the side effects caused by statement reordering; the second average pooling calculates the average state of two different concatenations of the final latent representation, ensuring the symmetry of code similarity.

[0080] The present application starts with a set of code fragments to be detected, and performs partitioned clone detection. Through method size and method similarity analysis techniques, it effectively detects cloned code with the same text, vocabulary and syntax, and high similarity. For the remaining code fragments, in order to make the code semantic representation capture more code semantic information, the control flow and data flow are effectively encoded into a semantic feature matrix through code control flow and data flow encoding techniques. Based on the deep learning model, high-level features are learned from the semantic matrix, and semantic similarity clone code detection is effectively performed, and then efficient and reliable detection of large-scale cloned code is realized.

[0081] Based on the same inventive concept as the method embodiment, the present application also provides a power data security monitoring system code clone detection device, comprising:

[0082] The code partition module is configured to eliminate embedded code that is not likely to be a clone pair, and partition the software source code of the power data security monitoring system to be detected according to a partition index mechanism, so that the partition index search method can be used to obtain the partition index mechanism, which includes determining the number of partitions, determining the partition key based on the name and version of the code method, mapping the partition key to a specific partition number using a hash function, and distributing the code to the corresponding partition according to the hash function, so that the corresponding partition can be located through the partition key when the code is detected for cloning, and the methods are searched and accessed simultaneously in different partitions;

[0083] The method size detection module is configured to convert the source code lines of the methods in different partitions into a series of tokens, where the token is a specified keyword or identifier, and filter out possible first-type and second-type clone pairs based on the size of the method, where the first-type clone pair is a clone pair with identical code fragments, and the second-type clone pair is a clone pair with identical code structure or syntax;

[0084] The method similarity detection module is configured to locate the corresponding partition through the partition key after eliminating the first-type and second-type clone pairs, set a token sliding window, measure the similarity between the two method tokens through overlapping similarity, and filter out possible third-type clone pairs according to the similarity, where the third-type clone pair is a method clone pair with the same size, similar operations, and similar calls;

[0085] The code semantic representation construction module is configured to, after eliminating the first-type, second-type, and third-type clone pairs, obtain a semantic representation of the code behavior by capturing the dependency relationship between basic blocks and procedures through control flow, and capturing the flow of data values along the program path and operations through a data flow graph for the remaining code fragments;

[0086] The semantic clone detection module is configured to use a pre-trained deep learning model based on the code semantic representation to perform classification and detect fourth-type clone pairs, where the fourth-type clone pair is a semantic-level cloned code.

[0087] It should be understood that the power data security monitoring system code clone detection device in the embodiments of the present application can implement all the technical solutions in the above method embodiments, and the functions of each functional module can be implemented according to the methods in the above method embodiments, and the specific implementation process can be referred to the related description in the above embodiments, which will not be described here.

[0088] The application further provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs, when executed by the processors, implement the steps of the power data security monitoring system code clone detection method.

[0089] The application further provides a computer readable storage medium, having stored thereon a computer program, the computer program, when executed by a processor, implementing the steps of the power data security monitoring system code clone detection method.

[0090] Those skilled in the art will understand that the embodiments of the application can be provided as methods, apparatus (systems), computer devices or computer program products. Therefore, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0091] The application is described with reference to flowcharts of methods according to embodiments of the application. It will be understood that each block of the flowcharts, and combinations of blocks in the flowcharts, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowcharts block or blocks. Figure 1 Apparatus for implementing each block or a combination of blocks in the flowcharts.

[0092] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowcharts block or blocks. Figure 1 Apparatus for implementing each block or a combination of blocks in the flowcharts.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowcharts block or blocks. Figure 1 Apparatus for implementing each block or a combination of blocks in the flowcharts.

Claims

1. A power data security monitoring system code clone detection method, characterized in that, The method comprises the following steps: After eliminating the embedded code that is impossible to be a clone pair, the software source code of the power data security monitoring system to be detected is partitioned according to a partition index mechanism, so that the partition index search method can be used to obtain the partition index mechanism, which comprises the following steps: determining the number of partitions, determining the partition key based on the name and version of the code method, mapping the partition key to a specific partition number by using a hash function, and distributing the code to the corresponding partition according to the hash function, so that the corresponding partition can be located by the partition key when the code is subjected to clone detection, and the methods in different partitions can be searched and accessed at the same time; The source code lines of the methods in different partitions are converted into a series of tokens, wherein the token is a specified keyword or identifier, and the possible first type and second type clone pairs are screened out based on the size of the method, wherein the first type clone pair is a clone pair with the same code segment, and the second type clone pair is a clone pair with the same code structure or code syntax; the size of the method comprises the following steps: a method M1 has x tokens, a method M2 has y tokens, a similarity threshold T is given between 0 and 1, when x*T≤y≤x / T is satisfied, the method M1 and the method M2 are determined to belong to a clone pair; After the first type and the second type clone pairs screened out are removed, the corresponding partition is located by the partition key, a token sliding window is set, the similarity between the tokens of the two methods is measured by overlapping similarity, and the possible third type clone pair is screened out according to the similarity, wherein the third type clone pair is a method clone pair with the same size, similar operation and similar call; the calculation method of the overlapping similarity is as follows: Among them, sim<M1,M2> represents the similarity between method M1 and method M2. A and B are the token sets in method M1 and method M2 respectively. Each element of token set A in method M1 is defined as <ta i ,freqta i ,s,Na>, each element of the token set B in method M2 is defined as <tb j ,freqtb j ,s,Nb>;s is the token sliding window set; Na is the total number of lines of method M1, Nb is the total number of lines of method M2; freqta i is the token set A in method M1 i The number of times it appears in this sliding window, ta i is the i-th token in token set A in method M1; freqtb j is tb of token set B in method M2 j The number of times it appears in this sliding window, tb j is the jth token in token set B in method M2; α is the sliding window sequence number after the token sliding window is divided; After the first type, the second type and the third type clone pairs screened out are removed, for the remaining code segment, the dependence relationship between the code basic blocks and the processes is captured by controlling the flow, the flow of data values along the program path and the operation is captured by the data flow graph, and the semantic representation of the code behavior is obtained; Based on the semantic representation of the code, a pre-trained deep learning model is used for classification to detect the fourth type clone pair, which is a semantic level clone code.

2. The method of claim 1, wherein, For the determination of the number of partitions, the program size, data volume, business logic and hardware resources are comprehensively considered to determine the number of partitions.

3. The method of claim 1, wherein, When obtaining the semantic representation of the code behavior, the control flow and the data flow information are encoded by the variable features, the basic block features and the relationship features between the variables and the basic blocks, and the encoding rules are as follows: Variable features are mainly extracted from each variable, and its type features include data type V(t), modifier V(m) and other information V(o), where V(t) = {bool, byte, char, double, float, int, long, short, void, JDK class, user-defined type}, V(m) = {final, static, none}, V(o) = {basic, array, pointer, reference}, and the above variable features are encoded into an 18-dimensional binary feature vector V = {V(t), V(m), V(o)}; Basic block features are features of each basic block, mainly considering the following seven types: normal, loop, loopbody, if, if body, switch, switch body, and this type information is encoded into a 7-dimensional one-hot feature vector D; The relationship between variables and basic blocks is mainly to encode data flow and control flow into operations between variables and control jumps between basic blocks, including 38 main operation types: addition: ADD; subtraction: SUB; Multiplication: MUL; division: DIV; remainder: REM; bitwise AND: AND; bitwise OR: OR; bitwise XOR: XOR; left shift: SHL; Right shift: SHR; unsigned right shift: USHR; negation: NEG; logical NOT: NOT; bitwise NOT: BITNOT; equal: EQ; not equal: NE; greater than: GT; greater than or equal: GE; less than: LT; less than or equal: LE; type check: INSTANCEOF; type conversion: CHECKCAST; conditional branch: COND_BRANCH; Jump: GOTO; type conversion: CONVERSION; array length: ARRAY_LENGTH; Phi function: PHI; method call: INVOKE; get caught exception: GET_CAUGHT_EXCEPTION; abstract method call: ABSTRACT_INVOKE; checkpoint: CHECKPOINT; conditional switch: COND_SWITCH; field assignment: PUT; field get: GET; get address: ADDRESSOF; return: RETURN; Branch: BRANCH; Identity: IDENTITY, represented by a 38-dimensional one-hot feature vector M; To preserve the information of each operation between variables in the dataflow graph, an 18*2+38 = 74 dimensional feature vector T = {V option1 , V option2 , M} is generated, where V option1 and V option2 represent the feature vectors of variables option1 and option2 in the semantic matrix encoding, respectively. To maintain the control flow, T is extended to size 81 dimensional {V option1 , V option2 , M, D}.

4. The method of claim 3, wherein, The semantic representation of code behavior is generated as a semantic feature matrix E according to the encoding rules, each element in the semantic feature matrix E is a variable or a basic block, and the order corresponds to the order of code instructions and basic blocks, E(i,j) = T(i,j) is a binary feature vector with a size of 81, representing the relationship between row i and column j, i,j = 1,2…n, where n = n v +b b is the sum of the variable count n v and the basic block count b b .

5. The method of claim 4, wherein, The deep learning model is established in the following way: Based on the sparse binary feature vector encoding data flow and control flow, it is mapped to a hidden state h0 of size c, c is a constant; then each row of the semantic feature matrix is converted into a vector of length c k, k is the specified input matrix size; then add two fully connected layers, take the weakened row feature vector as input, to extract all the information related to the variable or basic block represented by each row; finally, add a pooling layer to summarize all the row feature vectors, connect the latent representation vectors of the semantic matrices of the two methods in different orders, apply a fully connected layer on the two concatenated vectors with shared weights W fc , perform another average pooling on the output state, and finally add a classification layer.

6. An electric power data security monitoring system code clone detection apparatus characterized by comprising: Including: The code partition module is configured to eliminate embedded code that is not likely to be a clone pair, and to partition the software source code of the power data security monitoring system to be detected according to a partition index mechanism, so that the partition index search method can be used, the partition index mechanism comprising: determining the number of partitions, determining the partition key based on the name and version of the code method, mapping the partition key to a specific partition number using a hash function, and distributing the code to the corresponding partition according to the hash function, so that when the code is detected for cloning, the corresponding partition can be located through the partition key, and the methods in different partitions are searched and accessed at the same time; The method size detection module is configured to convert the source code lines of the methods in different partitions into a series of tokens, wherein the token is a specified keyword or identifier, and to filter out possible first-type and second-type clone pairs based on the size of the method, wherein the first-type clone pair is a clone pair with identical code fragments, and the second-type clone pair is a clone pair with identical code structure or syntax; the size of the method comprises: method M1 has x tokens, method M2 has y tokens, and a similarity threshold T is given between 0 and 1, when x*T≤y≤x / T is satisfied, method M1 and method M2 are determined to belong to a clone pair; The method similarity detection module is configured to locate the corresponding partition through the partition key after removing the first-type and second-type clone pairs filtered out, set a token sliding window, measure the similarity between the tokens of the two methods through overlapping similarity, and filter out possible third-type clone pairs according to the similarity, wherein the third-type clone pair is a method clone pair with the same size, similar operations, and similar calls; the calculation method of the overlapping similarity is as follows: Among them, sim<M1,M2> represents the similarity between method M1 and method M2. A and B are the token sets in method M1 and method M2 respectively. Each element of token set A in method M1 is defined as <ta i ,freqta i ,s,Na>, each element of the token set B in method M2 is defined as <tb j ,freqtb j ,s,Nb>;s is the token sliding window set; Na is the total number of lines of method M1, Nb is the total number of lines of method M2; freqta i is the token set A in method M1 i The number of times it appears in this sliding window, ta i is the i-th token in token set A in method M1; freqtb j is tb of token set B in method M2 j The number of times it appears in this sliding window, tb j is the jth token in token set B in method M2; α is the sliding window sequence number after the token sliding window is divided; The code semantic representation construction module is configured to, after removing the first-type, second-type, and third-type clone pairs filtered out, obtain the semantic representation of the code behavior by capturing the dependency relationship between basic blocks and procedures through control flow and capturing the flow of data values along program paths and operations through data flow graphs for the remaining code fragments; The semantic clone detection module is configured to use a pre-trained deep learning model based on the code semantic representation to classify and detect fourth-type clone pairs, wherein the fourth-type clone pair is a semantic-level cloned code.

7. A computer device, comprising: comprise: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs, when executed by the processor, implement the steps of the power data security monitoring system code clone detection method according to any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the power data security monitoring system code clone detection method according to any one of claims 1-5.

Images