A code search system and method based on graph serialization

Abstract

This invention provides a code search system and method based on graph sequencer, relating to the field of code search technology. The system and apparatus of this invention learn the control and data dependencies of code fragments through a graph sequencer (G2SC), obtaining a program dependency graph sequence rich in code fragment graph structure information, and is more suitable for code graph structures with fewer nodes than graph neural networks. During feature extraction, an attention mechanism is used to fuse method name feature vectors, token feature vectors, and program dependency graph feature vectors extracted by G2SC, enabling the semantic and structural information of the code to be fully expressed, resulting in more complete feature extraction and effectively improving the accuracy of code search. Using G2SC and a bidirectional long short-term memory network to map the program dependency graph information of code fragments into a lower-dimensional feature space can save a significant amount of computing power consumed in deep learning.

Description

Technical Field

[0001] This invention relates to the field of code search technology, and more specifically, to a code search system and method based on graph serialization. Background Technology

[0002] With the development of internet technology, the number of open-source code snippets is constantly increasing. Code search can greatly improve coding efficiency in the software development process. How to effectively help developers directly and accurately search for code snippets related to specific programming tasks directly from code databases using the existing massive amounts of code snippets has become a hot topic in the field of software engineering. In the early research stages of code search, most code search methods treated code snippets as plain text sequences and used keyword matching to measure the similarity between code snippets and natural language. However, since user queries and code snippets are heterogeneous information, the accuracy of keyword matching-based code search methods is unsatisfactory. Therefore, with the significant achievements of deep learning technology in the field of natural language processing, researchers have gradually considered studying code search tasks from the perspective of deep learning. They use deep learning technology to effectively learn the semantic information expressed by user queries and the structured information contained in code snippets to improve the accuracy of code search methods.

[0003] One existing deep learning-based code search method, DeepCS (Deep Code Search), uses deep learning to learn the features of code snippets and natural language, mapping the feature vectors of code snippets and natural language to the same high-dimensional space, and using cosine similarity to determine the degree of similarity between the code snippets and natural language. However, this method only considers the textual information of the code snippets during feature extraction, ignoring the graph structure information unique to code, resulting in incomplete code snippet features and thus reducing the accuracy of the code search method. Another improved method of DeepCS, MMAN (Multi-modal attention network), uses graph neural networks to learn the control flow graph information of the code during feature extraction to make the code snippet features more comprehensive. However, MMAN does not make good use of the graph structure information of the code. On the one hand, graph neural networks are suitable for graphs with many nodes, while the control flow graph of code snippets usually has few nodes, and using graph neural networks cannot effectively learn the graph structure information of the code. On the other hand, since the control flow graph of code only contains control dependencies and not data dependencies, using only the control flow graph ignores the data dependencies of the code. Therefore, the accuracy of MMAN is not effectively improved. Summary of the Invention

[0004] To address the shortcomings of the existing technologies, this invention proposes a code search system and method based on graph serialization, aiming to overcome the problem of low code search accuracy caused by the failure of existing technologies to fully utilize the effective information contained in the code graph structure.

[0005] The technical solution of this invention is as follows:

[0006] The first aspect of this invention provides a code search system based on graph serialization, the system comprising:

[0007] The data acquisition module is used to collect multiple raw code snippets to form a set of raw code snippets;

[0008] The code snippet preprocessing module is used to extract relevant features for each code snippet in the original code snippet set, including the method name sequence of the code snippet, the token sequence of the code snippet, and the program dependency graph (PDG) of the code snippet;

[0009] Graph Sequence Converter G2SC: Used to learn the control dependencies and data dependencies of code snippets based on the program dependency graph PDG extracted by the code snippet preprocessing module, generate a program dependency graph sequence rich in code graph structure information, and send the program dependency graph sequence to the graph feature extraction module;

[0010] The method name feature extraction module is used to input the method name sequence extracted by the code snippet preprocessing module into the bidirectional long short-term memory network Bi-LSTM to extract the features contained in the method name sequence, obtain the method name feature vector, and send it to the attention embedding module.

[0011] The Token feature extraction module is used to input the Token sequence extracted by the code snippet preprocessing module into the multilayer perceptron (MLP) to extract the features contained in the Token sequence, obtain the Token feature vector, and send it to the attention embedding module.

[0012] The graph feature extraction module is used to input the program dependency graph sequence into the bidirectional long short-term memory network Bi-LSTM to extract the features contained in the program dependency graph sequence, obtain the program dependency graph feature vector, and send it to the attention embedding module.

[0013] The attention embedding module is used to receive the method name feature vector sent by the method name feature extraction module, the token feature vector sent by the token feature extraction module, and the program dependency graph feature vector sent by the graph feature extraction module, and to assign different attention coefficients to the method name feature vector, the token feature vector and the program dependency graph feature vector to obtain the fused feature vector of the code fragment and send it to the similarity calculation module.

[0014] The natural language preprocessing module is used to segment the user-input query statement using a word segmentation tool, filter out special symbols contained therein, obtain a natural language sequence, and send it to the natural language feature extraction module.

[0015] The natural language feature extraction module is used to input the natural language sequence extracted by the natural language preprocessing module into the bidirectional long short-term memory network Bi-LSTM to extract the features contained in the natural language sequence, obtain the natural language feature vector, and send the natural language feature vector to the similarity calculation module.

[0016] The similarity calculation module receives the fused feature vector of the code snippet output by the attention embedding module and the natural language feature vector output by the natural language feature extraction module, calculates the cosine similarity between the natural language feature vector and the fused feature vector of all code snippets, and returns the code snippets corresponding to the fused feature vectors with the highest cosine similarity as code search results to the user.

[0017] A second aspect of the present invention provides a code search method based on graph serialization, the method comprising the following steps:

[0018] Step 1: Collect code snippets and their corresponding natural language comments;

[0019] Step 2: Use a word segmentation tool to segment the natural language annotations of each code segment and filter out special symbols in the segmentation results to obtain the natural language annotation sequence D for each code segment;

[0020] Step 3: Extract the method name sequence M, token sequence T, and program dependency graph G for each code snippet. PDG ;

[0021] Step 4: Create a program dependency graph G for each code snippet. PDG The input is fed into the graph sequence converter G2SC, which learns the control and data dependencies of the program dependency graph, generating a program dependency graph sequence P rich in code graph structure information. N represents the set of all words in the program dependency graph sequence. P The total number of words in the program dependency graph sequence;

[0022] Step 5: For each code snippet, use the method name feature extraction module to extract the features contained in the method name sequence to obtain the method name feature vector V. M The token feature vector V is obtained by extracting the features contained in the token sequence using the token feature extraction module. T And by using the graph feature extraction module to extract features contained in the program dependency graph sequence to obtain the program dependency graph feature vector V.P And use an attention mechanism to V M V T and V P The fusion feature vector V is obtained by combining the features. C ;

[0023] Step 6: Extract the features of the natural language annotation sequence D for each code segment using the natural language feature extraction module to obtain the natural language annotation feature vector V. D ;

[0024] Step 7: Based on the fused feature vector V of each code segment C and natural language annotation feature vector V D Calculate the similarity loss function L(θ), and optimize and update the parameters in the method name feature extraction module, the token feature extraction module, the graph feature extraction module, the attention embedding module, and the natural language feature extraction module based on the similarity loss.

[0025] Step 8: Repeat steps 5-7 until the similarity loss L(θ) converges, and obtain the final fused feature vector for each code segment, as well as the method name feature extraction module with optimal parameters, the token feature extraction module with optimal parameters, the graph feature extraction module with optimal parameters, the attention embedding module with optimal parameters, and the natural language feature extraction module with optimal parameters.

[0026] Step 9: When the user enters a query statement q, the query statement is segmented using a word segmentation tool and special symbols are filtered out to obtain a natural language sequence Q;

[0027] Step 10: Extract the features contained in the natural language sequence through the natural language feature extraction module to obtain the natural language feature vector V. Q ;

[0028] Step 11: Calculate the natural language feature vector V Q The code snippets corresponding to the cosine similarity between the final fused feature vectors of all code snippets are used as the code search results and returned to the user.

[0029] Furthermore, according to the aforementioned graph-based code search method, the method for generating a program dependency graph sequence P rich in code graph structure information by utilizing G2SC to learn the control dependencies and data dependencies of the program dependency graph includes the following steps:

[0030] Step 4-1: For the program dependency graph G PDGInitialize the program dependency graph sequence P = {}, set v1 as the first element p1 of the program dependency graph sequence P, and set the node corresponding to the last element of the program dependency graph sequence P as e. end ;

[0031] Step 4-2: Recursively select one edge from the candidate edges according to the edge selection rules to expand until all edges have been traversed;

[0032] Step 4-3: If the traversed edge is a control dependency edge, then only add the attribute information corresponding to the start and end points of the edge to the program dependency graph sequence P. If the traversed edge is a data dependency edge, then add the attribute information corresponding to the start point, data dependency edge, and end point of the data dependency edge to the program dependency graph sequence P in sequence.

[0033] Step 4-4: Repeat steps 4-1 to 4-3 to traverse all directed edges and obtain the program dependency graph sequence P.

[0034] Furthermore, according to the graph serialization-based code search method described above, the candidate edge refers to an edge starting with e. end An untraversed directed edge originating from a vertex.

[0035] Furthermore, according to the graph serialization-based code search method described above, the edge selection rules are as follows:

[0036] Rule a: Backtracking edges are selected before forward edges. Backtracking edges are directed edges whose endpoints have been traversed, while forward edges are directed edges whose endpoints have not been traversed.

[0037] Rule b: When there are multiple backtracking edges or multiple forward edges, select either control dependency edges or data dependency edges according to the preset priority.

[0038] Rule c: If the edge still cannot be determined after rule b, then select the edge with the highest priority corresponding to the endpoint node as the priority edge to be expanded.

[0039] Rule d, if it does not exist with e end For directed edges that have not been traversed and originating from point e, continue traversing to the next point e. end Untraversed directed edges whose nearest ancestor node is the nearest ancestor node.

[0040] Furthermore, according to the graph-sequence-based code search method, the method name feature vector V is extracted using the method name feature extraction module. M The method is as follows: in the method name feature extraction module, extract the sequence of method names from the code snippet. Each word in the sequence is converted into a word vector and input into a bidirectional long short-term memory network (Bi-LSTM) that can learn temporal relationships. The forward hidden layer vector of the method name sequence M at time i in the forward LSTM is obtained according to formula (1). The reverse hidden layer vector of the method name sequence M at time i in the reverse LSTM is obtained according to formula (2). Finally, according to formula (3), the forward and backward hidden layer vectors of the method name sequence M at all time points are integrated to obtain the method name feature vector V. M The specific formula is as follows;

[0041]

[0042]

[0043]

[0044] Where, m i Let N be the i-th word in the sequence of method names. M ω is the total number of words in the sequence of method names M; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors; and The forward and backward LSTM neural network models representing the sequence of method names, along with their associated parameters; and These represent the forward and backward hidden layer vectors at time i for the forward and backward LSTM neural network models of the method name sequence, respectively; CONCAT() represents the concatenation function of the two vectors; maxpooling() represents the max pooling function; V M This is the final feature vector of the method name.

[0045] Furthermore, according to the graph serialization-based code search method, the token feature vector V is extracted using the token feature extraction module. T The method is as follows: in the Token feature extraction module, the token sequence of the code snippet is extracted. Each word in the sequence is converted into a word vector and input into a multilayer perceptron (MLP). The hidden layer embedding vector of each word in the token sequence T in the MLP is obtained according to formula (4). Finally, the hidden layer embedding vectors of all words in the Token sequence T are integrated according to formula (5) to obtain the Token feature vector V. T The specific formula is as follows;

[0046]

[0047]

[0048] Among them, t j Let N be the j-th word in the token sequence T; T ω is the total number of words in the token sequence T; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors; W T These are the weight parameters of the MLP; tanh() represents the tanh function. It is t j Hidden layer embedding vectors in MLP; maxpooling() represents the max pooling function; V T It is the final obtained token feature vector.

[0049] Furthermore, according to the aforementioned graph-based code search method, the graph feature extraction module extracts the program dependency graph feature vector V. P The method is as follows: in the graph feature extraction module, the program dependency graph sequence of the code snippet is extracted. Each word in the sequence is converted into a word vector and input into a bidirectional long short-term memory network (Bi-LSTM) that can learn temporal relationships. The forward hidden layer vector of the program dependency graph sequence P at time k in the forward LSTM is obtained according to formula (6). The reverse hidden layer vector of the program dependency graph sequence P at time k in the reverse LSTM is obtained according to formula (7). Finally, the forward and backward hidden layer vectors of the program dependency graph sequence P at all time points are integrated according to formula (8) to obtain the program dependency graph feature vector V. P The specific formula is as follows;

[0050]

[0051]

[0052]

[0053] Where, p k N is the k-th word of the program dependency graph sequence P; P ω is the total number of words in the program dependency graph sequence P; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors; and The forward and backward LSTM neural network models and related parameters represent the program-dependent graph sequence P; and Let V represent the forward and backward hidden layer vectors of the forward and backward LSTM neural network models of the program-dependent graph sequence P at time k, respectively; CONCAT() represents the concatenation function of the two vectors; maxpooling() represents the max pooling function; V P It is the feature vector of the final program dependency graph.

[0054] Furthermore, according to the graph-sequence-based code search method, the natural language annotation feature vector V is extracted through the natural language feature extraction module. D The method is as follows: in the natural language feature extraction module, the natural language annotation sequence of the code snippet is extracted. Each word in the sequence is converted into a word vector and input into a bidirectional long short-term memory network (Bi-LSTM) that can learn temporal relationships. The forward hidden layer vector of the natural language annotation sequence D at time t in the forward LSTM is obtained according to formula (10). The inverse hidden layer vector of the natural language annotation sequence D at time t in the inverse LSTM is obtained according to formula (11). Finally, the forward and backward hidden layer vectors of the natural language annotation sequence D at all time points are integrated according to formula (12) to obtain the natural language annotation feature vector V. D The specific formula is as follows;

[0055]

[0056]

[0057]

[0058] Where, d r N represents the r-th word in the natural language annotation sequence. D ω is the total number of words in the natural language annotation sequence D; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors. and The forward and backward LSTM neural network models representing the natural language annotation sequence D, and their related parameters; and Let V represent the forward and backward hidden layer vectors of the forward and backward LSTM neural network models at time r, respectively; CONCAT() represents the concatenation function of the two vectors; maxpooling() represents the max pooling function; V D It is the final natural language annotation feature vector obtained.

[0059] Compared with the prior art, the technical solution proposed in this invention has the following beneficial effects:

[0060] This invention proposes a graph-to-sequence converter (G2SC) to convert the program dependency graph of code snippets into a sequence. G2SC can learn both the control dependencies and data dependencies of code snippets and is more suitable for code graph structures with fewer nodes than graph neural networks. Through G2SC, a program dependency graph sequence rich in code snippet graph structure information can be obtained. Simultaneously, during feature extraction, the system and method use an attention mechanism to fuse the method name feature vector, token feature vector, and the program dependency graph feature vector extracted by G2SC, enabling the semantic and structural information of the code to be fully expressed, resulting in more complete feature extraction and effectively improving the accuracy of code search. Finally, the system and method use G2SC and a bidirectional long short-term memory network to map the information of the program dependency graph of code snippets into a lower-dimensional feature space, saving a significant amount of computational power consumed in deep learning. Attached Figure Description

[0061] To more clearly illustrate the specific methods in the embodiments of the present invention, the relevant drawings involved in the embodiments will be briefly described below. The drawings below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative changes.

[0062] Figure 1 This is a schematic diagram of the structure of the code search system based on graph serialization in this embodiment;

[0063] Figure 2 This is a flowchart illustrating the code search method based on graph serialization in this embodiment.

[0064] Figure 3 This is a schematic diagram of the method for generating a program dependency graph sequence using G2SC in this embodiment. Detailed Implementation

[0065] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings. Preferred embodiments of this application are shown in the drawings. However, this application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0066] Figure 1 This is a schematic diagram of the structure of the code search system based on graph serialization in this embodiment. For example... Figure 1 As shown, the graph serialization-based code search system includes:

[0067] The data acquisition module is used to collect code snippets. In this embodiment, it first obtains Java files from Java open-source projects using a code data acquisition tool such as the GitHub API, then extracts Java methods from the Java files, with each Java method serving as a raw code snippet. The raw code snippets corresponding to all Java methods are then extracted from the Java files to form a raw code snippet dataset.

[0068] The code snippet preprocessing module is used to extract relevant features from the code snippets collected by the data acquisition module, including the method name sequence of the code snippet, the token sequence of the code snippet, and the program dependency graph (PDG) of the code snippet.

[0069] The Graph to Sequence Converter (G2SC) is used to learn the control dependencies and data dependencies of code snippets based on the program dependency graph (PDG) extracted by the code snippet preprocessing module, generate a program dependency graph sequence rich in code graph structure information, and send the program dependency graph sequence to the graph feature extraction module.

[0070] The method name feature extraction module is used to input the method name sequence extracted by the code snippet preprocessing module into the Bi-directional Long Short-Term Memory Network (Bi-LSTM) to extract the features contained in the method name sequence, obtain the method name feature vector, and send the method name feature vector to the attention embedding module.

[0071] The Token feature extraction module is used to input the Token sequence extracted by the code snippet preprocessing module into the Multilayer Perceptron (MLP) to extract the features contained in the Token sequence, obtain the Token feature vector, and send the Token feature vector to the attention embedding module.

[0072] The graph feature extraction module is used to input the program dependency graph sequence into the bidirectional long short-term memory network Bi-LSTM to extract the features contained in the program dependency graph sequence, obtain the program dependency graph feature vector, and send the program dependency graph feature vector to the attention embedding module.

[0073] The attention embedding module is used to receive the method name feature vector sent by the method name feature extraction module, the token feature vector sent by the token feature extraction module, and the program dependency graph feature vector sent by the graph feature extraction module, and to assign different attention coefficients to the method name feature vector, the token feature vector, and the program dependency graph feature vector to obtain the fused feature vector of the code fragment and send it to the similarity calculation module.

[0074] The natural language preprocessing module is used to segment the user-input query using a word segmentation tool such as jieba, filter out special symbols contained therein, obtain a natural language sequence, and send it to the natural language feature extraction module.

[0075] The natural language feature extraction module is used to input the natural language sequence extracted by the natural language preprocessing module into the bidirectional long short-term memory network Bi-LSTM to extract the features contained in the natural language sequence, obtain the natural language feature vector, and send the natural language feature vector to the similarity calculation module.

[0076] The similarity calculation module receives the fused feature vector of the code snippet output by the attention embedding module and the natural language feature vector output by the natural language feature extraction module, calculates the cosine similarity between the natural language feature vector and the fused feature vector of all code snippets, and returns the code snippets corresponding to the fused feature vectors with the highest cosine similarity as code search results to the user.

[0077] Figure 2 This is a flowchart illustrating the code search method based on graph serialization in this embodiment. Figure 2 As shown, the code search method based on graph serialization includes the following steps:

[0078] Step 1: Collect code snippets and their corresponding natural language comments;

[0079] Specifically, in this implementation, firstly, Java files from Java open-source projects released between August 2016 and September 2020 are obtained using code data acquisition tools such as the GitHub API. The STAR parameter threshold for Java open-source projects is limited to 10 to ensure the quality of the obtained Java files. Then, all Java methods are extracted from the Java files as raw code snippets. Java methods without comments or with non-English comments are filtered out, and the first line of the comment for each remaining Java method is extracted as the natural language comment for that code snippet. The final obtained code data pairs are then divided into an 8:2 ratio to serve as the training dataset and the evaluation dataset, respectively. Each code data pair consists of a code snippet and its natural language comment.

[0080] Step 2: Use a word segmentation tool to segment the natural language annotations of each code segment and filter out special symbols in the segmentation results to obtain the natural language annotation sequence D for each code segment.

[0081] In this embodiment, specifically, the natural language annotations of each code segment are segmented using a word segmentation tool such as jieba, and special symbols contained in the segmentation results are filtered out to obtain the natural language annotation sequence D for each code segment, wherein, N represents the set of all words in the natural language annotation sequence. D This represents the total number of words.

[0082] Step 3: Extract the method name sequence M, token sequence T, and program dependency graph G for each code snippet. PDG .

[0083] In this embodiment of the invention, firstly, since the method names of Java code snippets are named using camelCase, the method names of the code snippets are segmented using camelCase to obtain the method name sequence M of the code snippets, where... N represents the set of all words in the sequence of method names. M The total number of words in the method name sequence is given. Then, for all text contained in the code snippet, tokenization is performed using the StanfordCore NLP method to obtain the tokenized result of the code snippet. Furthermore, since Java keywords and stop words frequently appear in the source code and cannot effectively distinguish different code snippets, Java keywords and stop words are removed from the tokenized result of the code snippet, resulting in the token sequence T of the code snippet. N represents the set of all words in the token sequence. T The total number of words in the token sequence; finally, the program dependency graph G is extracted from the code snippet using the program dependency graph extraction tool TinyPDG. PDG In a program dependency graph, edge dependencies include two types: control dependencies and data dependencies. Edges with control dependencies are called control dependency edges, and edges with data dependencies are called data dependency edges. For G... PDG ={V,V A E C E D E DA}, N represents the set of nodes in the program dependency graph. V The total number of nodes; This indicates that the program depends on the attribute information of each node in the graph. The attribute information of each node refers to the statement corresponding to the node. N represents the set of control dependency edges in the program dependency graph. ec To control the total number of dependent edges, N represents the set of data dependency edges in the program dependency graph. ed This represents the total number of data-dependent edges. This represents the attribute information of each data dependency edge in the program dependency graph. The attribute of each data dependency edge refers to the set of variable names that generate the data dependency by the data dependency edge.

[0084] Step 4: Create a program dependency graph G from the code snippet. PDG The input is fed into the graph sequence converter G2SC, which learns the control and data dependencies of the program dependency graph, generating a program dependency graph sequence P rich in code graph structure information. N represents the set of all words in the program dependency graph sequence. P The total number of words in the program dependency graph sequence;

[0085] like Figure 3 As shown, the method for generating a program dependency graph sequence P rich in code graph structure information by using G2SC to learn the control dependency and data dependency relationships of the program dependency graph includes the following steps:

[0086] Step 4-1: For the program dependency graph G PDG Initialize the program dependency graph sequence P = {}, set v1 as the first element p1 of the program dependency graph sequence P, and set the node corresponding to the last element of the program dependency graph sequence P as e. end ;

[0087] Step 4-2: Recursively select one edge from the candidate edges according to the edge selection rules to expand until all edges have been traversed. Candidate edges refer to edges starting with e. end For directed edges that have not been traversed and originate from a given vertex, the selection rules are as follows:

[0088] Rule a: Backtracking edges are selected before forward edges. Backtracking edges are directed edges whose endpoints have been traversed, while forward edges are directed edges whose endpoints have not been traversed.

[0089] Rule b: When multiple backtracking edges or multiple forward edges exist, control dependency edges or data dependency edges are selected according to a preset priority. It should be noted that the priority between control dependency edges and data dependency edges does not affect the uniqueness of the generated program dependency graph sequence P. In this embodiment, to determine a unique program dependency graph sequence, control dependency edges are given higher priority and are selected first during the edge selection process. When multiple backtracking edges or multiple forward edges exist, control dependency edges are selected instead of data dependency edges.

[0090] Rule c: If an edge still cannot be determined after rule b, then the edge with the highest priority corresponding to the endpoint node is selected as the priority edge to be expanded. In this embodiment of the invention, the priority of a node is determined by the order of the nodes. For example, for the program dependency graph G... PDG Node set v1 has higher priority than v2;

[0091] Rule d, if it does not exist with e end For directed edges that have not been traversed and originating from point e, continue traversing to the next point e. end Untraversed directed edges whose nearest ancestor node is the nearest ancestor node.

[0092] Step 4-3: If the traversed edge is a control dependency edge, then only add the attribute information corresponding to the start and end points of the edge to the program dependency graph sequence P. If the traversed edge is a data dependency edge, then add the attribute information corresponding to the start point, data dependency edge, and end point of the data dependency edge to the program dependency graph sequence P in sequence.

[0093] In this embodiment of the invention, when generating the program dependency graph sequence P, the information contained in the control dependency edges is not considered, because the control dependency edges only represent the control dependency relationship between two nodes and have no other practical meaning; while the data dependency edges contain the transit relationship of variables in the code fragments, so the attribute information contained in the data dependency edges is retained when generating the program dependency graph sequence P.

[0094] Step 4-4: Repeat steps 4-1 to 4-3 to traverse all directed edges and obtain the program dependency graph sequence P.

[0095] Step 5: For each code snippet, use the method name feature extraction module to extract the features contained in the method name sequence to obtain the method name feature vector V. M The token feature vector V is obtained by extracting the features contained in the token sequence using the token feature extraction module. T And by using the graph feature extraction module to extract features contained in the program dependency graph sequence to obtain the program dependency graph feature vector V. P And use an attention mechanism to V M V T and V P The fusion feature vector V is obtained by combining the features. C ;

[0096] Step 5-1: For each code snippet, use the method name feature extraction module to extract the features contained in the method name sequence M to obtain the method name feature vector V. M ;

[0097] In this embodiment, since the method name sequence has a temporal relationship, the method name sequence of the code fragment is extracted in the method name feature extraction module. Each word in the sequence is converted into a word vector and input into a bidirectional long short-term memory network (Bi-LSTM) that can learn temporal relationships. The forward hidden layer vector of the method name sequence M at time i in the forward LSTM is obtained according to formula (1). The reverse hidden layer vector of the method name sequence M at time i in the reverse LSTM is obtained according to formula (2). Finally, according to formula (3), the forward and backward hidden layer vectors of the method name sequence M at all time points are integrated to obtain the method name feature vector V. M The specific formula is as follows;

[0098]

[0099]

[0100]

[0101] Where, m i Let N be the i-th word in the sequence of method names. M ω is the total number of words in the sequence M of method names; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors; and The forward and backward LSTM neural network models representing the sequence of method names, along with their associated parameters; and These represent the forward and backward hidden layer vectors at time i for the forward and backward LSTM neural network models of the method name sequence, respectively; CONCAT() represents the concatenation function of the two vectors; maxpooling() represents the max pooling function; V M This is the final feature vector of the method name.

[0102] Step 5-2: For each code segment, use the Token feature extraction module to extract the features contained in the Token sequence T to obtain the Token feature vector V. T ;

[0103] In this embodiment, since the token sequence does not have a temporal relationship, the token sequence of the code fragment is extracted in the token feature extraction module. Each word in the sequence is converted into a word vector and input into a multilayer perceptron (MLP). The hidden layer embedding vector of each word in the token sequence T in the MLP is obtained according to formula (4). Finally, the hidden layer embedding vectors of all words in the Token sequence T are integrated according to formula (5) to obtain the Token feature vector V. T The specific formula is as follows;

[0104]

[0105]

[0106] Among them, t j Let N be the j-th word in the token sequence T; T ω is the total number of words in the token sequence T; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors; W T These are the weight parameters of the MLP; tanh() represents the tanh function. It is t j Hidden layer embedding vectors in MLP; maxpooling() represents the max pooling function; V T It is the final token feature vector.

[0107] Step 5-3: For each code segment, use the graph feature extraction module to extract the features contained in the program dependency graph sequence P to obtain the program dependency graph feature vector V. P ;

[0108] In this embodiment, since the program dependency graph sequence has a temporal relationship, the program dependency graph sequence of the code segment is extracted in the graph feature extraction module. Each word in the sequence is converted into a word vector and input into a bidirectional long short-term memory network (Bi-LSTM) that can learn temporal relationships. The forward hidden layer vector of the program dependency graph sequence P at time k in the forward LSTM is obtained according to formula (6). The reverse hidden layer vector of the program dependency graph sequence P at time k in the reverse LSTM is obtained according to formula (7). Finally, the forward and backward hidden layer vectors of the program dependency graph sequence P at all time points are integrated according to formula (8) to obtain the program dependency graph feature vector V. P The specific formula is as follows;

[0109]

[0110]

[0111]

[0112] Where pk is the k-th word of the program dependency graph sequence P; N Pω is the total number of words in the program dependency graph sequence P; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors; and The forward and backward LSTM neural network models and related parameters represent the program-dependent graph sequence P; and Let V represent the forward and backward hidden layer vectors of the forward and backward LSTM neural network models of the program-dependent graph sequence P at time k, respectively; CONCAT() represents the concatenation function of the two vectors; maxpooling() represents the max pooling function; V P It is the feature vector of the final program dependency graph.

[0113] Step 5-4: Create a feature vector V for the method name of each code snippet using the attention embedding module. m Token feature vector V T and program dependency graph feature vector V P Assign different attention coefficients and use attention mechanisms to fuse V M V T and V P Obtain the fused feature vector V of the code snippet C .

[0114] V C =α m ×V M +α t ×VT+α p ×V P (9)

[0115] Where, α m The feature vector V is the method name. M Attention coefficient; α t For the token feature vector V T Attention coefficient; α p The feature vector V of the program dependency graph P Attention coefficient; V C This is the fused feature vector of the code snippet.

[0116] Step 6: Extract the features of the natural language annotation sequence D for each code segment using the natural language feature extraction module to obtain the natural language annotation feature vector V. D ;

[0117] In this embodiment, since the natural language annotation sequence has a temporal relationship, the natural language annotation sequence of the code segment is extracted in the natural language feature extraction module. Each word in the sequence is converted into a word vector and input into a bidirectional long short-term memory network (Bi-LSTM) that can learn temporal relationships. The forward hidden layer vector of the natural language annotation sequence D at time t in the forward LSTM is obtained according to formula (10). The inverse hidden layer vector of the natural language annotation sequence D at time t in the inverse LSTM is obtained according to formula (11). Finally, the forward and backward hidden layer vectors of the natural language annotation sequence D at all time points are integrated according to formula (12) to obtain the natural language annotation feature vector V. D The specific formula is as follows;

[0118]

[0119]

[0120]

[0121] Where, d r N represents the r-th word in the natural language annotation sequence. D ω is the total number of words in the natural language annotation sequence D; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors. and The forward and backward LSTM neural network models representing the natural language annotation sequence D, and their related parameters; and Let V represent the forward and backward hidden layer vectors of the forward and backward LSTM neural network models at time r, respectively; CONCAT() represents the concatenation function of the two vectors; maxpooling() represents the max pooling function; V D It is the final natural language annotation feature vector obtained.

[0122] Step 7: Based on the fused feature vector V of each code segment C and natural language annotation feature vector V D Calculate the similarity loss function L(θ), and optimize and update the parameters in the method name feature extraction module, the token feature extraction module, the graph feature extraction module, the attention embedding module, and the natural language feature extraction module based on the similarity loss.

[0123] The specific formula for the similarity loss function is as follows:

[0124] L(θ) = max(0, ∈ -cos(V) c V d )+cos(V c Vd -)) (13)

[0125] Where θ represents the sum of the parameters in the method name feature extraction module, the token feature extraction module, the graph feature extraction module, the attention embedding module, and the natural language feature extraction module; ∈ is a commonly used slack variable in machine learning, usually set to 0.05; V C It is the fused feature vector of the code snippet, V D Is with V C The corresponding natural language annotation feature vector, V D - is obtained by negative sampling and is related to V C For non-corresponding natural language annotation feature vectors, max() is the maximum value function.

[0126] Step 8: Repeat steps 5-7 until the similarity loss L(θ) converges, and obtain the final fused feature vector for each code segment, as well as the method name feature extraction module with optimal parameters, the token feature extraction module with optimal parameters, the graph feature extraction module with optimal parameters, the attention embedding module with optimal parameters, and the natural language feature extraction module with optimal parameters.

[0127] In this embodiment of the invention, when training the parameters of the method name feature extraction module, token feature extraction module, graph feature extraction module, attention embedding module, and natural language feature extraction module, the batch size is set to 64. The maximum lengths of the method name sequence M, token sequence T, program dependency graph sequence P, and natural language annotation sequence D for each code segment are set to 6, 20, 80, and 50 respectively. Sequences shorter than the maximum length are specially marked. <pad>Pad up to the maximum length; for bidirectional LSTM units, the hidden layer size is set to 256; for MLP, the embedding vector dimension is set to 512; update parameters using the ADAM optimizer.

[0128] Step 9: When the user enters a query statement q, the query statement is segmented using a word segmentation tool and special symbols are filtered out to obtain a natural language sequence Q;

[0129] Step 10: Extract the features contained in the natural language sequence through the natural language feature extraction module to obtain the natural language feature vector V. Q ;

[0130] Step 11: Calculate the natural language feature vector V Q The code snippets corresponding to the cosine similarity between the final fused feature vectors of all code snippets are used as the code search results and returned to the user.

[0131] In this embodiment, the natural language feature vector V is used. Q The code snippets corresponding to the 10 fused feature vectors with the highest cosine similarity are returned to the user as code retrieval results.

[0132] To evaluate the accuracy of the code search method proposed in this invention, two widely used evaluation metrics for code search are employed in this embodiment: Mean Reciprocal Rank (MRR) and R@w (success percentage at w).

[0133] MRR is an internationally recognized metric for evaluating search algorithms. A higher MRR value indicates higher code search accuracy. The formula for calculating MRR is as follows:

[0134]

[0135] Where q represents a user query, Q all It is the set of all queries q, |Q all | represents Q all The number of queries q included, Frank q This represents the Frank value of query q; the Frank value of query q refers to the position of the first occurrence of a result that matches query q in the code search results returned by query q, also known as the best hit ranking.

[0136] R@w represents the percentage of the top w code search results that may contain multiple correct results. A higher R@w value indicates higher code search accuracy. The formula for calculating R@w is as follows:

[0137]

[0138] Where q represents a user query, Q all It is the set of all queries q, |Q all | represents Q all The number of queries q included, Frank q This indicates that the query returns the Frank value of q; F() represents the discrimination function. When the condition in F() is true, F() returns 1; when the condition in F() is false, F() returns 0.

[0139] To demonstrate the superior accuracy of the code search method proposed in this invention, DeepCS, MMAN, and the proposed code search method were evaluated on the same dataset. The evaluation datasets included two sets: the existing public dataset CodeSearchNet and the evaluation dataset obtained in step 1. The evaluation results for the code search on the CodeSearchNet dataset are shown in Table 1; the evaluation results for the evaluation dataset obtained in step 1 are shown in Table 2. The results show that the proposed code search method exhibits superior accuracy compared to DeepCS and MMAN.

[0140] Table 1. Evaluation results of code search on the CodeSearchNet dataset.

[0141]

[0142] Table 1 shows that on the CodeSearchNet dataset, the code search algorithm proposed in this invention improves upon DeepCS by 81%, 70%, 61%, and 29% in terms of MRR, R@1, R@5, and R@10 evaluation metrics, and improves upon MMAN by 62%, 55%, 45%, and 24% in terms of MRR, R@1, R@5, and R@10 evaluation metrics.

[0143] Table 2. Evaluation results of code search on the evaluation dataset obtained in step 1.

[0144]

[0145] Table 2 shows that, on the evaluation dataset obtained in step 1, the code search algorithm proposed in this invention improves performance by 100%, 150%, 90%, and 30% over DeepCS in terms of MRR, R@1, R@5, and R@10 evaluation metrics, and by 63%, 92%, 43%, and 16% over MMAN, respectively.

[0146] The foregoing has provided a detailed description of one embodiment of the present invention. Clearly, the above embodiment is merely a part of the embodiments of the present invention, and not all of them; the above embodiment is only used to explain the present invention and does not constitute a limitation on the scope of protection of the present invention. Based on the above embodiment, all other embodiments obtained by those skilled in the art without inventive effort, that is, all modifications, equivalent substitutions, and improvements made within the spirit and principle of this application, fall within the scope of protection claimed by the present invention.< / pad>

Claims

1. A code search method based on graph serialization, implemented using a graph serialization-based code search system, the system comprising: The data acquisition module is used to collect multiple raw code snippets to form a set of raw code snippets; The code snippet preprocessing module is used to extract relevant features for each code snippet in the original code snippet set, including the method name sequence of the code snippet, the token sequence of the code snippet, and the program dependency graph (PDG) of the code snippet; Graph Sequence Converter G2SC: Used to learn the control dependencies and data dependencies of code snippets based on the program dependency graph PDG extracted by the code snippet preprocessing module, generate a program dependency graph sequence rich in code graph structure information, and send the program dependency graph sequence to the graph feature extraction module; The method name feature extraction module is used to input the method name sequence extracted by the code snippet preprocessing module into the bidirectional long short-term memory network Bi-LSTM to extract the features contained in the method name sequence, obtain the method name feature vector, and send it to the attention embedding module. The Token feature extraction module is used to input the Token sequence extracted by the code snippet preprocessing module into the multilayer perceptron (MLP) to extract the features contained in the Token sequence, obtain the Token feature vector, and send it to the attention embedding module. The graph feature extraction module is used to input the program dependency graph sequence into the bidirectional long short-term memory network Bi-LSTM to extract the features contained in the program dependency graph sequence, obtain the program dependency graph feature vector, and send it to the attention embedding module. The attention embedding module is used to receive the method name feature vector sent by the method name feature extraction module, the token feature vector sent by the token feature extraction module, and the program dependency graph feature vector sent by the graph feature extraction module, and to assign different attention coefficients to the method name feature vector, the token feature vector and the program dependency graph feature vector to obtain the fused feature vector of the code fragment and send it to the similarity calculation module. The natural language preprocessing module is used to segment the user-input query statement using a word segmentation tool, filter out special symbols contained therein, obtain a natural language sequence, and send it to the natural language feature extraction module. The natural language feature extraction module is used to input the natural language sequence extracted by the natural language preprocessing module into the bidirectional long short-term memory network Bi-LSTM to extract the features contained in the natural language sequence, obtain the natural language feature vector, and send the natural language feature vector to the similarity calculation module. A similarity calculation module is used to receive the fused feature vector of the code snippet output by the attention embedding module and the natural language feature vector output by the natural language feature extraction module, calculate the cosine similarity between the natural language feature vector and the fused feature vector of all code snippets, and return the code snippets corresponding to the fused feature vectors with the highest cosine similarity as code search results to the user; the method is characterized by the following steps: Step 1: Collect code snippets and their corresponding natural language comments; Step 2: Use a word segmentation tool to segment the natural language annotations of each code segment and filter out special symbols in the segmentation results to obtain the natural language annotation sequence D for each code segment; Step 3: Extract the method name sequence M, token sequence T, and program dependency graph G for each code snippet. PDG ; Step 4: Create a program dependency graph G for each code snippet. PDG The input is fed into the graph sequence converter G2SC, which learns the control and data dependencies of the program dependency graph, generating a program dependency graph sequence P rich in code graph structure information. N represents the set of all words in the program dependency graph sequence. P The total number of words in the program dependency graph sequence; Step 5: For each code snippet, use the method name feature extraction module to extract the features contained in the method name sequence to obtain the method name feature vector V. M The token feature vector V is obtained by extracting the features contained in the token sequence using the token feature extraction module. T And by using the graph feature extraction module to extract features contained in the program dependency graph sequence to obtain the program dependency graph feature vector V. P And use an attention mechanism to V M V T and V P The fused feature vector V is obtained by combining the features. C ; Step 6: Extract the features of the natural language annotation sequence D for each code segment using the natural language feature extraction module to obtain the natural language annotation feature vector V. D ; Step 7: Based on the fused feature vector V of each code segment C and natural language annotation feature vector V D Calculate the similarity loss function L(θ), and optimize and update the parameters in the method name feature extraction module, the token feature extraction module, the graph feature extraction module, the attention embedding module, and the natural language feature extraction module based on the similarity loss. Step 8: Repeat steps 5-7 until the similarity loss L(θ) converges, and obtain the final fused feature vector for each code segment, as well as the method name feature extraction module with optimal parameters, the token feature extraction module with optimal parameters, the graph feature extraction module with optimal parameters, the attention embedding module with optimal parameters, and the natural language feature extraction module with optimal parameters. Step 9: When the user enters a query statement q, the query statement is segmented using a word segmentation tool and special symbols are filtered out to obtain a natural language sequence Q; Step 10: Extract the features contained in the natural language sequence through the natural language feature extraction module to obtain the natural language feature vector V. Q ; Step 11: Calculate the natural language feature vector V Q The code snippets corresponding to the cosine similarity between the final fused feature vectors of all code snippets are used as the code search results and returned to the user. The method for generating a program dependency graph sequence P rich in code graph structure information by using G2SC to learn the control dependency and data dependency relationships of the program dependency graph includes the following steps: Step 4-1: For the program dependency graph G PDG Initialize the program dependency graph sequence P = {}, set v1 as the first element p1 of the program dependency graph sequence P, and set the node corresponding to the last element of the program dependency graph sequence P as e. end ; Step 4-2: Recursively select one edge from the candidate edges according to the edge selection rules to expand until all edges have been traversed; Step 4-3: If the traversed edge is a control dependency edge, then only add the attribute information corresponding to the start and end points of the edge to the program dependency graph sequence P. If the traversed edge is a data dependency edge, then add the attribute information corresponding to the start point, data dependency edge, and end point of the data dependency edge to the program dependency graph sequence P in sequence. Step 4-4: Repeat steps 4-1 to 4-3 to traverse all directed edges and obtain the program dependency graph sequence P.

2. The code search method based on graph serialization according to claim 1, characterized in that, The candidate edge refers to the edge defined by e. end An untraversed directed edge originating from a vertex.

3. The code search method based on graph serialization according to claim 2, characterized in that, The edge selection rules are as follows: Rule a: Backtracking edges are selected before forward edges. Backtracking edges are directed edges whose endpoints have been traversed, while forward edges are directed edges whose endpoints have not been traversed. Rule b: When there are multiple backtracking edges or multiple forward edges, select either control dependency edges or data dependency edges according to the preset priority. Rule c: If the edge still cannot be determined after rule b, then select the edge with the highest priority corresponding to the endpoint node as the priority edge to be expanded. Rule d, if it does not exist with e end For directed edges that have not been traversed and originating from point e, continue traversing to the next point e. end Untraversed directed edges whose nearest ancestor node is the nearest ancestor node.

4. The code search method based on graph serialization according to claim 1, characterized in that, The method name feature extraction module extracts features contained in the method name sequence to obtain the method name feature vector V. M The method is as follows: in the method name feature extraction module, extract the sequence of method names from the code snippet. Each word in the sequence is converted into a word vector and input into a bidirectional long short-term memory network (Bi-LSTM) that can learn temporal relationships. The forward hidden layer vector of the method name sequence M at time i in the forward LSTM is obtained according to formula (1). The reverse hidden layer vector of the method name sequence M at time i in the reverse LSTM is obtained according to formula (2). Finally, according to formula (3), the forward and backward hidden layer vectors of the method name sequence M at all time points are integrated to obtain the method name feature vector V. M The specific formula is as follows; Where, m i Let N be the i-th word in the sequence of method names. M ω is the total number of words in the sequence of method names M; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors; and The forward and backward LSTM neural network models representing the sequence of method names, along with their associated parameters; and These represent the forward and backward hidden layer vectors at time i for the forward and backward LSTM neural network models of the method name sequence, respectively; CONCAT() represents the concatenation function of the two vectors; maxpooling() represents the max pooling function; V M This is the final feature vector of the method name.

5. The code search method based on graph serialization according to claim 1, characterized in that, The token feature vector V is obtained by extracting features contained in the token sequence using the token feature extraction module. T The method is as follows: in the Token feature extraction module, the token sequence of the code snippet is extracted. Each word in the sequence is converted into a word vector and input into a multilayer perceptron (MLP). The hidden layer embedding vector of each word in the token sequence T in the MLP is obtained according to formula (4). Finally, the hidden layer embedding vectors of all words in the Token sequence T are integrated according to formula (5) to obtain the Token feature vector V. T The specific formula is as follows; Among them, t j Let N be the j-th word in the token sequence T; T ω is the total number of words in the token sequence T; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors; W T These are the weight parameters of the MLP; tanh() represents the tanh function. It is t j The hidden layer embedding vector in an MLP; maxpooling() represents the max pooling function.

6. The code search method based on graph serialization according to claim 1, characterized in that, The graph feature extraction module extracts features contained in the program dependency graph sequence to obtain the program dependency graph feature vector V. P The method is as follows: in the graph feature extraction module, the program dependency graph sequence of the code snippet is extracted. Each word in the sequence is converted into a word vector and input into a bidirectional long short-term memory network (Bi-LSTM) that can learn temporal relationships. The forward hidden layer vector of the program dependency graph sequence P at time k in the forward LSTM is obtained according to formula (6). The reverse hidden layer vector of the program dependency graph sequence P at time k in the reverse LSTM is obtained according to formula (7). Finally, the forward and backward hidden layer vectors of the program dependency graph sequence P at all time points are integrated according to formula (8) to obtain the program dependency graph feature vector V. P The specific formula is as follows; Where, p k N is the k-th word of the program dependency graph sequence P; P ω is the total number of words in the program dependency graph sequence P; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors; and The forward and backward LSTM neural network models and related parameters represent the program-dependent graph sequence P; and Let V represent the forward and backward hidden layer vectors of the forward and backward LSTM neural network models of the program-dependent graph sequence P at time k, respectively; CONCAT() represents the concatenation function of the two vectors; maxpooling() represents the max pooling function; V P It is the feature vector of the final program dependency graph.

7. The code search method based on graph serialization according to claim 1, characterized in that, The natural language feature extraction module extracts features contained in the natural language sequence to obtain a natural language feature vector V. Q The method is as follows: in the natural language feature extraction module, the natural language annotation sequence of the code snippet is extracted. Each word in the sequence is converted into a word vector and input into a bidirectional long short-term memory network (Bi-LSTM) that can learn temporal relationships. The forward hidden layer vector of the natural language annotation sequence D at time t in the forward LSTM is obtained according to formula (10). The inverse hidden layer vector of the natural language annotation sequence D at time t in the inverse LSTM is obtained according to formula (11). Finally, the forward and backward hidden layer vectors of the natural language annotation sequence D at all time points are integrated according to formula (12) to obtain the natural language annotation feature vector V. D The specific formula is as follows; Where, d r Let N be the r-th word in the natural language annotation sequence; D ω is the total number of words in the natural language annotation sequence D; ω() is a function that uses the Skip-Gram method to convert words in the text into word vectors. and The forward and backward LSTM neural network models representing the natural language annotation sequence D, and their related parameters; and These represent the forward and backward hidden layer vectors of the forward and backward LSTM neural network models at time r, respectively; CONCAT() represents the concatenation function of the two vectors; and maxpooling() represents the max pooling function.

Images