A code retrieval method based on code classification representation in a code repository scenario

Abstract

The application relates to the technical field of code retrieval, and discloses a code retrieval method based on code classification representation in a code repository scenario, which comprises the following steps: labeling semantics of each code segment as a user query corresponding to the code segment, and constructing a training set and a verification set; extracting auxiliary information and internal structure information of each code segment in the training set; respectively calculating a first correlation matrix of the auxiliary information and the user query and a second correlation matrix of the internal structure information and the user query; splicing the first correlation matrix and the second correlation matrix, and outputting a final correlation score through a neural network; and training a code retrieval model to convergence based on a sorting loss function. The application solves the problems that code information in an existing code retrieval model is single and the effect is difficult to improve, realizes all-round utilization of code information, and effectively improves the effect of code retrieval.

Description

Technical Field

[0001] This invention relates to the field of code retrieval technology, and more specifically to a code retrieval method based on code classification representation in a code repository scenario. Background Technology

[0002] Code retrieval is a widely used technology that can find the most matching code snippets from a codebase based on a developer's natural language query, thereby greatly improving the development efficiency of programmers.

[0003] For a long time, code retrieval has primarily relied on two methods: information retrieval and deep learning. Information retrieval treats code as ordinary text, performing keyword matching between query text and code text. Much work has been done to expand or reconstruct the matching content. However, information retrieval methods have a significant drawback: they are limited to textual information and struggle to bridge the gap between programming languages ​​and natural languages. When deep learning is applied to code retrieval, both the query and the code are fed into an encoder model to obtain embedding vectors. The similarity between the query and the code is then measured by the closeness of these vectors. Common code representation methods include abstract syntax trees and data flow graphs. However, due to the complexity of high-level programming languages, achieving truly satisfactory results in code representation and semantic learning remains challenging.

[0004] The information contained in code can be summarized into two aspects. The first aspect is explicit auxiliary information, such as method names, input parameters, and variables. Method names are often designed by the programmer and are composed of several words that are highly related to the semantics of the code and are easy to extract. The second aspect is implicit structural information, such as the code execution logic. Without relevant knowledge, it is difficult to understand the semantic information of high-level programming languages, so this type of information often requires meticulous code representation processing.

[0005] However, existing work does not effectively distinguish between these two types of code information, focusing more on the information of the entire code segment, while often failing to effectively utilize auxiliary information. Therefore, it is crucial to consider how to effectively utilize these two types of information. Furthermore, existing methods often consider applying code retrieval to individual code datasets, neglecting more practical scenarios, such as code repository scenarios, where more detailed code processing is required. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a code retrieval method based on code classification representation in a code repository scenario. First, the code segments in the code repository are processed by cutting long segments. Then, two methods are used to mine auxiliary information and internal structural information within the code. This solves the problems of limited code information and difficulty in improving performance in existing code retrieval models, achieving comprehensive utilization of code information and effectively improving code retrieval results.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] A code retrieval method based on code classification representation in a code repository scenario, the construction and training process of the code retrieval model includes:

[0009] The code repository is parsed to generate a set of code segments. Based on the minimum edge cutting principle of the control flow graph, code segments with lengths exceeding the threshold are split until all code segments meet the length threshold.

[0010] Each code segment is semantically annotated to serve as the corresponding user query for that code segment, and a training set and a validation set are constructed.

[0011] Extract auxiliary information and internal structure information of each code segment in the training set;

[0012] The first relevance matrix between auxiliary information and user query and the second relevance matrix between internal structure information and user query are calculated separately: the first relevance matrix is ​​obtained by encoding identifiers and user query through a long short-term memory network and fusing the scores of the three types of identifiers of auxiliary information through an attention mechanism; the second relevance matrix is ​​obtained by compiling internal structure information into an LLVM intermediate representation, performing data dependency encoding and control dependency encoding in stages based on graph neural networks, and weighting the basic block relevance.

[0013] The first and second correlation matrices are concatenated, and the final correlation score is output via a neural network.

[0014] The code retrieval model was trained to convergence based on the ranking loss function.

[0015] In one embodiment, the minimum edge cutting principle based on the control flow graph is used to segment code segments whose length exceeds a threshold, specifically including:

[0016] Construct line-level control flow graphs for each code segment. ,in, This represents the set of nodes corresponding to m lines of code. This represents the node corresponding to the i-th line of code in the code segment; Denotes the set of edges, edges Connecting nodes and , , Indicates the number of lines of code in the code segment; iterates through all possible split points in the code segment. In each position The code segment is divided into an upper and lower half. Edges between nodes in the upper half and nodes in the lower half are removed, and the loss after the segmentation is calculated. ,in, In position The set of edges to be cut during segmentation; choosing to use loss. Minimum split position The corresponding split code segments are added to the code segment collection.

[0017] In one embodiment, the extraction of auxiliary information and internal structure information of each code segment in the training set specifically includes:

[0018] Use lexical or syntax analysis tools to extract all identifiers from the code segment, including method names. Parameter name and variable names Auxiliary information ;

[0019] Replace all variable names in the code segment with uniform placeholders to obtain internal structure information. .

[0020] In one embodiment, the step of obtaining the first relevance matrix by encoding identifiers using a long short-term memory network and user queries, and then fusing auxiliary information through an attention mechanism to obtain the scores of three types of identifiers, specifically includes:

[0021] Stemming, word segmentation, and lowercase conversion are performed on auxiliary information and user queries to obtain word lists for auxiliary information and user queries. A word mapping mechanism is then used to process each word list to obtain word vector sequences for auxiliary information and user queries.

[0022] The word vector sequences of auxiliary information and the word vector sequence of the user query are respectively input into the Long Short-Term Memory (LSTM) network. The hidden state of the LTM network at the last time step is the semantic representation vector of the auxiliary information. semantic representation vector of user query ;

[0023] based on and Calculate the attention score matrix between auxiliary information and user queries. Then, normalization is performed to obtain the normalized attention score matrix. ;

[0024] Auxiliary information includes three types of identifiers: method name, etc. Parameter name and variable names Normalized attention score matrix Including the normalized attention score matrix corresponding to the method name Normalized attention score matrix corresponding to parameter name Normalized attention score matrix corresponding to variable name ;

[0025] Will , and By fusing elements with the same weights, the first correlation matrix is ​​obtained. :

[0026] .

[0027] In one embodiment, the based and Calculate the attention score matrix between auxiliary information and user queries. Then, normalization is performed to obtain the normalized attention score matrix. Specifically, it includes:

[0028] Will and Each representation is converted to an LLVM intermediate representation through a fully connected layer.

[0029] ;

[0030] ;

[0031] in, For matrix multiplication, This is a matrix transpose operation. and This is the weight matrix. and Let be the bias vector, and tanh be the activation function. They are respectively and The corresponding intermediate variables;

[0032] calculate and Attention score matrix between :

[0033] ;

[0034] in, It is attention weight. It is a bias vector, and the attention score matrix is ​​adjusted using the softmax function. Normalization is performed to obtain the normalized attention score matrix. .

[0035] In one embodiment, the step of compiling the internal structure information into an LLVM intermediate representation, performing data dependency encoding and control dependency encoding in stages based on a graph neural network, and weighting the basic block correlations to obtain a second correlation matrix specifically includes:

[0036] The compiler is used to compile the preprocessed internal structure information into an LLVM intermediate representation;

[0037] By analyzing the intermediate representation of LLVM, data dependencies and control dependencies in the code segment are extracted to obtain data dependency graphs and control dependency graphs.

[0038] Optimize LLVM intermediate representation, data dependency graph, and control dependency graph:

[0039] (1) Instruction merging: If two instructions in the LLVM intermediate representation have data dependencies, and the data output by the first instruction is only used by the second instruction, and the second instruction only uses the data output by the first instruction, then instruction merging is achieved by operand substitution;

[0040] (2) Eliminate unreachable instructions: By traversing the control flow graph and removing unreachable basic blocks, the unreachable instructions in the LLVM intermediate representation are removed;

[0041] (3) Eliminate useless code: If no other code uses the result of the operation of the current code, then the current code will be deleted as useless code;

[0042] After the above three optimization operations, we obtain the optimized LLVM intermediate representation and data dependency graph. and control dependency graph ;

[0043] Data dependency encoding: Each node in the algorithm represents an instruction, and each edge represents a data dependency, with the edge direction from the data producer to the data user. An initial embedding vector is assigned to the instruction corresponding to each node according to a unified word mapping mechanism. Message passing and node embedding updates are performed according to the direction of the edges in the algorithm to obtain... The embedding vector of the instruction corresponding to each node in the process;

[0044] Control dependency encoding: Each node in the control dependency graph is a basic block, and the edges are the control dependencies between basic blocks; based on the embedding vector of the instructions in the basic block that have undergone data dependency encoding, the initial embedding of the basic block is determined, and then control dependency encoding message passing and basic block embedding update are performed.

[0045] The user query is input into the second long short-term memory network to obtain the encoding result; the correlation score between the encoding result and each basic block embedding is calculated, and the obtained multiple correlation scores are normalized into correlation weights to obtain the second correlation matrix.

[0046] In one embodiment, the following is... Message passing and node embedding updates are performed according to the direction of the edges in the algorithm to obtain... The embedding vector of the instruction corresponding to each node in the code includes:

[0047] ;

[0048] ;

[0049] in, express The message passing weight matrix of the edges; It is a node The corresponding instruction is in the The embedding vector at the next iteration yields the node. Need to be passed to the node News Then, use a gated loop unit. To update the embedding vector of the instruction corresponding to the node, where, Represents a node exist The set of neighboring nodes in;

[0050] The initial embedding of the basic block is determined based on the embedding vector of the data-dependent encoded instructions in the basic block. Specifically, it includes:

[0051] ;

[0052] in, It is the set of all instructions within a basic block; It is an instruction The final embedding vector in the data dependency encoding stage;

[0053] The subsequent control-dependent encoding of message passing and basic block embedding updates specifically includes:

[0054] ;

[0055] ;

[0056] in, Indicates the first Basic blocks To the Basic blocks The process of transmitting messages, yes The message passing weight matrix of the edges, using gated cyclic units. To update the embedding vector of the basic block. express In the control dependency graph, the set of neighboring basic blocks For the first Basic blocks The embedding vector at time t; the first vector after the update. Basic blocks The embedding vector is denoted as ;

[0057] The calculation of the correlation score between the encoding result and each basic block embedding, and the normalization of the obtained multiple correlation scores into correlation weights to obtain a second correlation matrix, specifically includes:

[0058] ;

[0059] in, For the encoding result With the Embedding vectors of basic blocks The correlation score between them It is a weight matrix. It is the bias term, and tanh is the activation function. This is a matrix transpose operation; the softmax function is used to calculate the correlation scores. Normalized to correlation weights , Indicates the number of lines of code in the code segment;

[0060] The final correlation matrix .

[0061] In one embodiment, the step of extracting data dependencies and control dependencies from the code segment by analyzing the LLVM intermediate representation to obtain a data dependency graph and a control dependency graph specifically includes:

[0062] LLVM intermediate representation , for The total number of instructions in the system; for The i-th instruction In the LLVM intermediate representation, instructions are grouped into basic blocks, each of which is a sequence of instructions without internal branches or jumps, and ends with a control flow instruction.

[0063] Data dependencies can be represented by a directed graph, denoted as a data dependency graph. ,in, It is the set of nodes corresponding to the instructions. yes The set of edges in the array, edges express The output was Used as input; control dependencies are represented by a directed graph, denoted as the control dependency graph. ,in, It is the set of nodes corresponding to the basic blocks. yes The set of edges in the array, edges Represents the j-th basic block The execution will be transferred to the k-th basic block. Furthermore, this transition depends on the outcome of a certain conditional judgment.

[0064] In one embodiment, the process of concatenating the first and second correlation matrices and outputting a final correlation score via a neural network specifically includes:

[0065] ;

[0066] ;

[0067] in, This is the weight matrix. For bias terms, It is an activation function. For the sigmoid function, This is the final relevance score. As an intermediate variable, This indicates a splicing operation.

[0068] In one embodiment, training the code retrieval model to convergence based on the ranking loss function specifically includes:

[0069] Each user query The corresponding code segment label serves as a positive example. A random sample is selected from unrelated code segments as a negative example. The training objective is to ensure that the final relevance score predicted by the code retrieval model meets the following requirements. Sorting loss function for:

[0070] ;

[0071] in, It is the number of training samples. It is a pre-defined positive number; This represents the j-th parameter of the code retrieval model, where j is the corresponding index; It is a regularization parameter; For the i-th user query, for The corresponding code segment tag, for Corresponding negative examples;

[0072] Stochastic gradient descent is used to minimize the loss function. This continues until the code retrieval model no longer improves performance on the validation set or reaches the preset maximum number of iterations.

[0073] Compared with the prior art, the beneficial technical effects of the present invention are:

[0074] The technology used in this invention can be directly applied to code repository scenarios, enabling programmers to quickly search for the required code in their local projects and improving program development efficiency. Furthermore, this invention employs a code retrieval method based on code information classification and representation in code analysis, performing highly detailed processing on the two types of information in code segments. Different models are used to handle auxiliary information and internal structure information, significantly improving the accuracy of code retrieval. Therefore, by adopting the above-mentioned technical means, the semantic loss caused by the homogenization of code information in existing technologies and the difficulty in improving retrieval results can be solved, achieving comprehensive and effective utilization of code information. Attached Figure Description

[0075] Figure 1 This is a flowchart of the method in an embodiment of the present invention. Detailed Implementation

[0076] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0077] This invention proposes a code retrieval method based on code classification representation in a code repository scenario. It processes the code segments in the code repository and designs a code retrieval model to effectively classify and process code information, thereby improving the code retrieval effect.

[0078] like Figure 1 As shown, the present invention proposes a code retrieval method based on code classification representation in a code repository scenario, which includes the following steps.

[0079] S1: Organize the code in the code repository into code segments, set a certain threshold to limit the length of the code segments, and cut the code segments that exceed the threshold until all code segments meet the length limit. This includes steps S11, S12, and S13.

[0080] S11, Code Repository Parsing: First, the parser analyzes all the code in the code repository. Initial parsing is performed. Mature parsers such as ANTLR and Pygments can be used, as they can accurately identify the syntax structure of different programming languages. Through the parser's processing, the code in the code repository is divided into multiple code segments, forming a collection of code segments. ,in, This represents a code segment, which consists of several consecutive lines of code.

[0081] S12, Code segment length judgment and threshold setting: For sets Each code segment in It determines whether the number of tokens it contains exceeds a preset threshold. This threshold The threshold is set after comprehensively considering the efficiency and effectiveness of code retrieval. In common code repositories, the threshold can be set accordingly. Set it to 300. This value can be flexibly adjusted according to the characteristics of the actual code repository and retrieval needs. However, it is generally within a reasonable range, such as between 100 and 500, to balance the integrity of the code segment and processing efficiency.

[0082] S13, Splitting of Extremely Long Code Segments (when needed): If a certain code segment The number of tokens exceeded the preset threshold. If the code segment meets the length limit, it needs to be segmented until the segmented code segment meets the length limit requirement. The single segmentation process is shown in S131, S132 and S133 below.

[0083] S131, Control Flow Graph Construction: Using static analysis techniques, construct the control flow graph at the line-of-code level. Add each line of code as a node In this study, control dependencies (such as conditional statements and loop structures) and data dependencies (such as variable assignments and function calls) between lines of code are analyzed. For lines of code with dependencies, such as line j and line k, an edge is added to the edge set. Connecting nodes and These dependencies can be accurately identified using data flow analysis algorithms or control flow analysis algorithms.

[0084] S132, Segmentation Location Traversal and Loss Calculation: Traversing Segments All possible split positions In each position The paragraph is divided into an upper and lower part, all dependencies between the two parts are removed, and the loss after segmentation is calculated. , that is, the number of edges that were cut, where, In position The set of edges that are cut off during the partitioning process.

[0085] S133, Optimal Segmentation Location Selection and Segmentation Operation Execution: Selection Loss Minimum split position Click here to view the code segment Divided into and The split code segments are then added back to the set of code segments to be processed. middle.

[0086] S2: For a collection of code segments Each code segment Perform semantic annotation to generate corresponding user queries. , constitute the dataset Specifically, it includes steps S21 and S22.

[0087] S21: Organize professional personnel to conduct manual analysis and, based on the code's function, algorithm logic, or implementation purpose, write concise natural language descriptions for user queries. Alternatively, a general-purpose large model can be used for annotation, followed by manual review and correction to improve annotation efficiency and accuracy, resulting in a well-annotated dataset. .

[0088] S22: The labeled dataset The Fisher-Yates shuffle algorithm is used to randomly shuffle the dataset, ensuring the randomness and uniformity of the data distribution and avoiding the influence of data order on model training performance. The dataset is divided into training, testing, and validation sets according to a preset ratio, typically set to 70% for training, 15% for validation, and 15% for testing.

[0089] S3: Process each code segment in the dataset to extract two different types of information: auxiliary information. Internal structure information Specifically, it includes steps S31 and S32.

[0090] S31: For each code segment in the training set, use lexical or syntactic analysis tools (such as ANTLR, Pygments, etc.) to accurately extract all identifiers, including method names, parameter names, and variable names. Organize these extracted identifiers into an ordered list or set, denoted as S31. This is auxiliary information. For example, by using the tool's interface functions or custom scripts, each statement in the code segment can be scanned to identify and collect identifiers, ensuring that no key information is missed and providing semantic clues for subsequent retrieval.

[0091] S32: First, copy the original code segment to ensure that subsequent operations do not affect the original code. Next, using regular expressions or a code parsing tool, replace all variable names in the copied code with uniform placeholders, such as para_1, para_2, etc. The replacement process must adhere to variable scope and syntax rules to avoid affecting the code's logical structure. After replacement, the code segment retains only operational information such as loops, conditional statements, and function calls, while removing the semantics of specific variable names. This processed code constitutes the internal structure information. This highlights the logical architecture of the code.

[0092] S4: For auxiliary information respectively Internal structural information Perform the processing to obtain both results and the user query. The correlation matrices are denoted as the first correlation matrix. Second correlation matrix Specifically, it includes steps S41, S42, S43, and S44.

[0093] S41, Establish a unified word mapping mechanism:

[0094] From all code segments in the training set (including auxiliary information) and internal structure information Extract all unique tokens from the database. These tokens cover identifiers (e.g., variable names, function names), keywords (e.g., if, for, while), and operators (e.g., +, -, ...). The extracted tokens include punctuation marks (e.g., semicolons, curly braces, '}') and literals (e.g., numbers, strings). These tokens are organized into a global vocabulary, and each token is assigned a unique ID. Furthermore, a special UNK (unknown) tag is added for out-of-vocabulary words (i.e., tokens not present in the training set). When an out-of-vocabulary word is encountered, it is mapped to the ID corresponding to the UNK tag, thus ensuring that all words can be represented in a unified vector space.

[0095] S42: Auxiliary information Preprocessing involves three aspects: stemming, word segmentation, and unified lowercase conversion.

[0096] Stemming: Since English words have multiple forms, such as third-person singular and past tense, to ensure that different forms of the same word can be matched, it is necessary to uniformly replace the word with its stem. Stemming tools in natural language processing libraries such as NLTK or spaCy are used to process each word in the auxiliary information to obtain its stem.

[0097] Word segmentation: Keywords or method names in the code often use camelCase (e.g., "camelCase") or underscore notation (e.g., "under_score"), concatenating multiple words together. To extract individual words, regular expressions or specialized word segmentation tools are used to split these compound words. For example, "getUserName" will be split into "get", "User", and "Name", and "user_name" will be split into "user" and "name".

[0098] Unified lowercase conversion: Words in the code may appear in uppercase due to their position at the beginning of a sentence or following specific naming conventions. To ensure consistency in word matching, all words are uniformly converted to lowercase. After the above processing, a word list for each type of auxiliary information is obtained: At the same time, the same processing is performed on the query to obtain a list of words searched by the user: Finally, using the previously established word mapping mechanism, each word in the word list is mapped to a corresponding vector, thus obtaining a word vector sequence, which prepares for subsequent encoding and relevance calculation.

[0099] S43, for preprocessed auxiliary information and user query Encoding using a Long Short-Term Memory (LSTM) network:

[0100] Each LSTM model calculates the hidden state sequence according to the following formula for each word vector sequence. LSTM at time step Hidden state The calculation is as follows:

[0101] ;

[0102] ;

[0103] ;

[0104] ;

[0105] ;

[0106] ;

[0107] In the formula, These are the activation values ​​for the input gate, forget gate, candidate memory unit, and output gate, respectively. It refers to the state of a memory unit; and These are model parameters; The sigmoid function compresses the input value into the (0,1) interval, controlling the degree of information transmission. The tanh activation function maps the input value to the (-1,1) interval, generating the state of candidate memory units. The calculation at each time step depends on the current input. and the hidden state of the previous moment This enables the dynamic capture and transmission of sequence information. After calculating the entire sequence, the hidden state of the last time step is taken. semantic representation vector as auxiliary information Specifically, The word vector sequences are fed into their respective Long Short-Term Memory (LSTM) networks to obtain semantic representation vectors. .

[0108] S44, Calculate auxiliary information User query Similarity matrix :

[0109] Will semantic representation vector semantic representation vector of user query Each representation is converted to an intermediate form through a separate fully connected layer. The specific calculation formula is as follows:

[0110] ;

[0111] .

[0112] in, and This is the weight matrix, used to learn different combinations of features from the input vector. and The bias vector is used to adjust the flexibility of the model; tanh, as the activation function, can limit the output value to the range of (-1, 1), providing non-linearity for subsequent calculations. Then, the attention score matrix is ​​calculated, and the intermediate representation obtained above is used. and To perform fusion and calculation and Attention score matrix between The calculation formula is:

[0113] .

[0114] here, It is an attention weight matrix, used to learn how to combine... and Features are used to highlight important semantic connections; This is the bias vector, used to fine-tune the score. The softmax function is used to... After normalization, the normalized attention score matrix is ​​denoted as... The calculation formula is: Using the same calculation method, calculate respectively , and Attention score matrix between and .

[0115] The three normalized attention score matrices obtained above and The first correlation matrix is ​​obtained by fusion. The fusion method involves averaging the three attention score matrices with equal weights, calculated using the following formula:

[0116] .

[0117] The above steps completed the auxiliary information. User query First correlation matrix The calculation provides a crucial semantic similarity metric for subsequent code retrieval and ranking.

[0118] S45, information on internal structure Preprocessing is performed to obtain Data dependency graph Instruction dependency graph :

[0119] Information on internal structure Preprocessing removes non-code elements such as comments, redundant spaces, and line breaks, retaining only the pure code logic structure. This step aims to simplify the code, making it easier for subsequent analysis. A compiler (such as Clang) then compiles the preprocessed code into an LLVM Intermediate Representation (LLVM IR) form. LLVM IR is a low-level, hardware-independent intermediate code form that better reflects the control and data flow characteristics of the code. During compilation, all code segments are ensured to use the same compilation options and optimization levels to maintain consistency.

[0120] The obtained LLVM IR is represented as Each instruction Includes the following parts: Opcodes representing instructions, such as add, br, etc. The operands representing instructions can be registers, immediate values, or the results of other instructions. The output of the instruction (if any) is typically represented in a register. LLVM IR is also organized into basic blocks, represented as... Each basic block is a sequence of instructions without internal branches or jumps, and ends with a control flow instruction.

[0121] Iterate through each instruction in LLVM IR and analyze the data dependencies between instructions. When an instruction... Operands contain instructions When considering the results, it is believed that and There is a data dependency between them, that is The output was Used as input. Constructing a data dependency graph. ,in It is the set of nodes corresponding to the instructions. yes The set of edges (denoted as data-dependent edges) in a given set of instruction pairs that have data dependencies. ,exist Add a line from point to edge .

[0122] Analyze the control flow transfer relationships between basic blocks, when basic blocks Execution depends on basic blocks When dealing with control flow instructions (such as conditional branch instructions), it is considered that... and There is a control dependency. Construct a control dependency graph. ,in, It is the set of nodes corresponding to the basic blocks. yes The set of edges (denoted as control-dependent edges) in a primitive block pair, for which there is a control dependency. ,exist Add a line from point to edge .

[0123] By following the steps above, we have obtained the internal structure information. LLVM IR and its corresponding data dependency graph and control dependency graph .

[0124] S46: Will , , The data is fed into the optimization module to obtain the optimized LLVM intermediate representation. Data dependency graph and control dependency graph :

[0125] Perform the following three operations to reduce redundant instructions, improve model performance, and ensure semantic integrity.

[0126] Instruction merging: Traverse the instruction set and find instruction pairs that meet the following conditions. and : At the same time for all And for all This indicates The output is only by Use, and Only used The output of . At this time, will Operands replaced with The operands of two instructions are combined to form a new instruction. : The opcode is opcode, The operands are operands and Operands (excluding those for) The union of the references to the results, i.e. , The output result is The output result.

[0127] Eliminate unreachable instructions: from the program's basic entry point block Starting from the beginning, perform a breadth-first traversal of the control dependency graph. Mark all reachable basic blocks, denoted as set. Unreachable basic block set for Remove all unreachable basic blocks. This means that unreachable instructions are eliminated.

[0128] Eliminate unnecessary code: Iterate through each instruction ,like Output results And for all instructions , That is, no other instructions are used. The output result is then deleted. .

[0129] After the above optimization operations, a new... , , This provides an optimized representation for subsequent encoding and correlation calculations.

[0130] S47: Will , , The data dependency encoding and control dependency encoding are then performed in the phased dependency graph encoding module.

[0131] Data dependency encoding: First, based on the unified word mapping mechanism, each node (instruction) in the data dependency graph is encoded. Set the initial embedding vector Message passing and node embedding updates are performed according to the direction of data dependency edges. For nodes... Need to be passed to the node News The calculation method is as follows ,in It is the message passing weight matrix of data dependency edges. It is a node In the The embedding vector at the next iteration is updated using a gated recurrent unit (GRU). Embedded vector, ,in Represents a node After multiple iterations, the embedding vector of each instruction is obtained from the set of neighbor nodes in the data dependency graph.

[0132] The optimized control dependency graph The initial embedding vector of the basic block is input into the encoding module. It is calculated from the instruction embedding vector obtained in the data dependency encoding stage, specifically as follows: ,in It is the set of all instructions within a basic block. This is the final embedding vector of the instruction during the data dependency encoding stage. Next, message passing and basic block embedding updates are performed according to the direction of the control dependency edges. To basic blocks The message being delivered The calculation method is as follows ,in It is a message passing weight matrix that controls dependent edges, and uses GRU to update basic blocks. Embedded vector, ,in, Representing basic blocks The set of neighboring basic blocks in the control dependency graph. After the update, the embedding vector of the basic block is denoted as... .

[0133] S48: Calculate internal structure information and correlation matrix :

[0134] User queries The encoded result is obtained by feeding it into an independently trained LSTM model. Calculate the query encoding result. With each basic block embedding vector Correlation score between The formula is ,in, It is a weight matrix. is the bias term, tanh is the activation function, and the softmax function is used to normalize the relevance scores into weights. The final second correlation matrix is ​​denoted as .

[0135] S5: The first correlation matrix obtained based on the first two steps Second correlation matrix The final correlation score is calculated. :

[0136] matrix and Concatenate the columns to form a combined matrix As input to the neural network, a weight matrix is ​​used. and bias terms Perform a linear transformation on the input, i.e., calculate Multiply by the input matrix And add a bias term Then, the ReLU activation function is applied to perform a nonlinear transformation on the result of the linear transformation to obtain the output of the hidden layer. Its formula is: The ReLU activation function can introduce non-linear properties, enhancing the model's ability to fit complex relationships. The output of the hidden layer... As input to the second layer, the weight matrix is ​​used and bias terms right Perform another linear transformation, i.e., calculate Multiply And add a bias term Next, the sigmoid activation function is applied to compress the linearly transformed result to the range [0, 1], yielding the final correlation score. The formula is: The sigmoid function can restrict the output value to the interval [0, 1], making... The value can intuitively represent the query. and code The degree of similarity between them. The final relevance score. The larger the value, the better the query. and code The more similar they are.

[0137] S6: Calculate the loss function and train the code retrieval model described above. This includes steps S61 and S62.

[0138] S61, Calculate the loss function:

[0139] In the training samples, for each query Identify the corresponding positive example code segment. (i.e., the ground truth code segment) and the negative example code segment (Randomly drawn from unrelated code). A ranking-based loss function is constructed to ensure that the relevance scores predicted by the model meet certain requirements. The loss function is as follows:

[0140] ;

[0141] in, It is the number of training samples. It is a pre-defined positive number representing the minimum score difference between positive and negative samples. Represents the parameters of the model. It is a regularization parameter used to prevent overfitting.

[0142] S62, uses stochastic gradient descent to minimize the loss function:

[0143] Before starting optimization, all parameters of the model... Initialization typically uses random initialization, assigning small random values. In each iteration step, a training sample is randomly selected, and the loss function value and gradient of the model parameters corresponding to that sample are calculated. The model parameters are updated based on the gradient information, and the update formula is as follows:

[0144] ;

[0145] Where η is the learning rate, which controls the step size of parameter updates and determines the magnitude of each parameter update.

[0146] Repeat the above iterative steps until a stopping condition is met. The stopping condition is typically that the model's performance on the validation set no longer improves, or that the preset maximum number of iterations has been reached. During training, periodically evaluate the model's performance on the validation set to monitor convergence and generalization ability, preventing overfitting. Stop training when validation performance no longer improves. Use the trained model to evaluate it on the test set to verify its accuracy and efficiency in the code retrieval task. Calculate metrics such as inverse average rank and recall to comprehensively evaluate the model's practical application effectiveness.

[0147] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0148] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0149] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0150] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0151] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A code retrieval method based on code classification representation in a code repository scenario, characterized in that, The construction and training process of the code retrieval model includes: The code repository is parsed to generate a set of code segments. Based on the minimum edge cutting principle of the control flow graph, code segments with lengths exceeding the threshold are split until all code segments meet the length threshold. Each code segment is semantically annotated to serve as the corresponding user query for that code segment, and a training set and a validation set are constructed. Extract auxiliary information and internal structure information of each code segment in the training set; The first relevance matrix between auxiliary information and user query and the second relevance matrix between internal structure information and user query are calculated separately: the first relevance matrix is ​​obtained by encoding identifiers and user query through a long short-term memory network and fusing the scores of the three types of identifiers of auxiliary information through an attention mechanism; the second relevance matrix is ​​obtained by compiling internal structure information into an LLVM intermediate representation, performing data dependency encoding and control dependency encoding in stages based on graph neural networks, and weighting the basic block relevance. Data dependencies and control dependencies in code segments are extracted by analyzing the LLVM intermediate representation, resulting in a data dependency graph and a control dependency graph. The LLVM intermediate representation, data dependency graph, and control dependency graph are then optimized to obtain the optimized LLVM intermediate representation and data dependency graph. and control dependency graph ; Data dependency encoding: Each node in the data dependency graph represents an instruction, and each edge represents a data dependency relationship. The direction of the edge is from the data producer to the data user. An initial embedding vector is set for the instruction corresponding to each node according to a unified word mapping mechanism. Message passing and node embedding are updated according to the direction of the edge in the data dependency graph to obtain the embedding vector of the instruction corresponding to each node in the data dependency graph. Control dependency encoding: Each node in the control dependency graph is a basic block, and the edges are the control dependencies between basic blocks; based on the embedding vector of the instructions in the basic block that have undergone data dependency encoding, the initial embedding of the basic block is determined, and then control dependency encoding message passing and basic block embedding update are performed. According to Message passing and node embedding updates are performed according to the direction of the edges in the algorithm to obtain... The embedding vector of the instruction corresponding to each node in the code includes: ； ； in, express The message passing weight matrix of the edges; It is a node The corresponding instruction is in the The embedding vector at the next iteration yields the node. Need to be passed to the node News Then, use a gated loop unit. To update the embedding vector of the instruction corresponding to the node, where, Represents a node exist The set of neighboring nodes in; The initial embedding of the basic block is determined based on the embedding vector of the data-dependent encoded instructions in the basic block. Specifically, it includes: ； in, It is the set of all instructions within a basic block; It is an instruction The final embedding vector in the data dependency encoding stage; The subsequent control-dependent encoding of message passing and basic block embedding updates specifically includes: ； ； in, Indicates the first Basic blocks To the Basic blocks The process of transmitting messages, yes The message passing weight matrix of the edges, using gated cyclic units. To update the embedding vector of the basic block. express In the control dependency graph, the set of neighboring basic blocks For the first Basic blocks The embedding vector at time t; the first vector after the update. Basic blocks The embedding vector is denoted as ; The first and second correlation matrices are concatenated, and the final correlation score is output via a neural network. The code retrieval model was trained to convergence based on the ranking loss function.

2. The code retrieval method based on code classification representation in a code repository scenario according to claim 1, characterized in that, The minimum edge cutting principle based on the control flow graph is used to segment code segments whose length exceeds a threshold, specifically including: Construct line-level control flow graphs for each code segment. ,in, This represents the set of nodes corresponding to m lines of code. This represents the node corresponding to the i-th line of code in the code segment; Denotes the set of edges, edges Connecting nodes and , , Indicates the number of lines of code in the code segment; iterates through all possible split points in the code segment. In each position The code segment is divided into an upper and lower half. Edges between nodes in the upper half and nodes in the lower half are removed, and the loss after the segmentation is calculated. ,in, In position The set of edges to be cut during segmentation; choosing to use loss. Minimum split position The corresponding split code segments are added to the code segment collection.

3. The code retrieval method based on code classification representation in a code repository scenario according to claim 1, characterized in that, The extraction of auxiliary information and internal structure information for each code segment in the training set specifically includes: Use lexical or syntax analysis tools to extract all identifiers from the code segment, including method names. Parameter name and variable names Auxiliary information ; Replace all variable names in the code segment with uniform placeholders to obtain internal structure information. .

4. The code retrieval method based on code classification representation in a code repository scenario according to claim 1, characterized in that, The process of obtaining the first relevance matrix by encoding identifiers through a long short-term memory network and user queries, and then fusing auxiliary information through an attention mechanism, specifically includes: Stemming, word segmentation, and lowercase conversion are performed on auxiliary information and user queries to obtain word lists for auxiliary information and user queries. A word mapping mechanism is then used to process each word list to obtain word vector sequences for auxiliary information and user queries. The word vector sequences of auxiliary information and the word vector sequence of the user query are respectively input into the Long Short-Term Memory (LSTM) network. The hidden state of the LTM network at the last time step is the semantic representation vector of the auxiliary information. semantic representation vector of user query ; based on and Calculate the attention score matrix between auxiliary information and user queries. Then, normalization is performed to obtain the normalized attention score matrix. ; Auxiliary information includes three types of identifiers: method name, etc. Parameter name and variable names Normalized attention score matrix Including the normalized attention score matrix corresponding to the method name Normalized attention score matrix corresponding to parameter name Normalized attention score matrix corresponding to variable name ; Will , and By fusing elements with the same weights, the first correlation matrix is ​​obtained. : 。 5. The code retrieval method based on code classification representation in a code repository scenario according to claim 4, characterized in that, The basis and Calculate the attention score matrix between auxiliary information and user queries. Then, normalization is performed to obtain the normalized attention score matrix. Specifically, it includes: Will and Each representation is converted to an LLVM intermediate representation through a fully connected layer. ； ； in, For matrix multiplication, This is a matrix transpose operation. and This is the weight matrix. and Let be the bias vector, and tanh be the activation function. They are respectively and The corresponding intermediate variables; calculate and Attention score matrix between : ； in, It is attention weight. It is a bias vector, and the attention score matrix is ​​adjusted using the softmax function. Normalization is performed to obtain the normalized attention score matrix. .

6. The code retrieval method based on code classification representation in a code repository scenario according to claim 1, characterized in that, The process involves compiling the internal structure information into an LLVM intermediate representation, performing data dependency encoding and control dependency encoding in stages based on a graph neural network, weighting the basic block correlations, and obtaining a second correlation matrix. Specifically, this includes: The compiler is used to compile the preprocessed internal structure information into an LLVM intermediate representation; By analyzing the intermediate representation of LLVM, data dependencies and control dependencies in the code segment are extracted to obtain data dependency graphs and control dependency graphs. Optimize LLVM intermediate representation, data dependency graph, and control dependency graph: (1) Instruction merging: If two instructions in the LLVM intermediate representation have data dependencies, and the data output by the first instruction is only used by the second instruction, and the second instruction only uses the data output by the first instruction, then instruction merging is achieved by operand substitution; (2) Eliminate unreachable instructions: By traversing the control flow graph and removing unreachable basic blocks, the unreachable instructions in the LLVM intermediate representation are removed; (3) Eliminate useless code: If no other code uses the result of the operation of the current code, then the current code will be deleted as useless code; After the above three optimization operations, we obtain the optimized LLVM intermediate representation and data dependency graph. and control dependency graph ; The user query is input into the second long short-term memory network to obtain the encoding result; the correlation score between the encoding result and each basic block embedding is calculated, and the obtained multiple correlation scores are normalized into correlation weights to obtain the second correlation matrix.

7. The code retrieval method based on code classification representation in a code repository scenario according to claim 6, characterized in that, The calculation of the correlation score between the encoding result and each basic block embedding, and the normalization of the obtained multiple correlation scores into correlation weights to obtain a second correlation matrix, specifically includes: ； in, For the encoding result With the Embedding vectors of basic blocks The correlation score between them It is a weight matrix. It is the bias term, and tanh is the activation function. This is a matrix transpose operation; the softmax function is used to calculate the correlation scores. Normalized to correlation weights , Indicates the number of lines of code in the code segment; The final correlation matrix .

8. A code retrieval method based on code classification representation in a code repository scenario according to claim 6, characterized in that, The step of extracting data dependencies and control dependencies from code segments by analyzing LLVM intermediate representations to obtain data dependency graphs and control dependency graphs specifically includes: LLVM intermediate representation , for The total number of instructions in the system; for The i-th instruction In the LLVM intermediate representation, instructions are grouped into basic blocks, each of which is a sequence of instructions without internal branches or jumps, and ends with a control flow instruction. Data dependencies can be represented by a directed graph, denoted as a data dependency graph. ,in, It is the set of nodes corresponding to the instructions. yes The set of edges in the array, edges express The output was Used as input; control dependencies are represented by a directed graph, denoted as the control dependency graph. ,in, It is the set of nodes corresponding to the basic blocks. yes The set of edges in the array, edges Represents the j-th basic block The execution will be transferred to the k-th basic block. Furthermore, this transition depends on the outcome of a certain conditional judgment.

9. A code retrieval method based on code classification representation in a code repository scenario according to claim 1, characterized in that, The concatenation of the first and second correlation matrices, followed by a neural network output of the final correlation score, specifically includes: ； ； in, This is the weight matrix. For bias terms, It is an activation function. For the sigmoid function, This is the final relevance score. As an intermediate variable, This indicates a splicing operation.

10. A code retrieval method based on code classification representation in a code repository scenario according to claim 1, characterized in that, The process of training the code retrieval model to convergence based on the ranking loss function specifically includes: Each user query The corresponding code segment label serves as a positive example. A random sample is selected from unrelated code segments as a negative example. The training objective is to ensure that the final relevance score predicted by the code retrieval model meets the following requirements. Sorting loss function for: ； in, It is the number of training samples. It is a pre-defined positive number; This represents the j-th parameter of the code retrieval model, where j is the corresponding index; It is a regularization parameter; For the i-th user query, for The corresponding code segment tag, for Corresponding negative examples; Stochastic gradient descent is used to minimize the loss function. This continues until the code retrieval model no longer improves performance on the validation set or reaches the preset maximum number of iterations.

Images