Code retrieval method based on multi-view contrastive code representation learning independent of naming style

Abstract

The code retrieval method based on the code retrieval method of the multi-view contrast code representation learning of the naming style is irrelevant to the technical field of code retrieval. It aims to provide a scheme for utilizing AST to process different naming conventions in code search tasks through multi-view contrast learning of the abstract syntax tree AST of the code. NACS removes the information of the bound variable name from the abstract syntax tree AST and focuses on capturing only the intrinsic properties from the AST structure. Realistic data is prepared using semantic and syntactic enhancement techniques, and a graph view modeling component is designed in NACS using contrast learning to enhance the understanding of the code snippet. Further, the AST is modeled in the path view, and the graph view modeling component is strengthened through multi-view contrast learning. NACS provides superior code search performance, improves the accuracy of the code retrieval model, and helps existing code search methods to overcome the influence of different naming conventions.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of code retrieval, and particularly relates to a code retrieval method based on multi-view contrast code representation learning independent of naming style, which can improve the accuracy of a code retrieval model and improve the model's recognition ability for different naming conventions. BACKGROUND

[0002] Code Search takes a text query that shows the intention of a software developer as input and returns the code snippet that the developer needs. To improve the productivity and quality of software development, programmers need to use code search tools to find high-quality code snippets in existing projects when debugging, writing code, looking for code to reuse, or learning API usage (Chao Liu, Xin Xia, David Lo, Cuiyun Gao, Xiaohu Yang, and John C. Grundy. 2022. Opportunities and Challenges in Code Search Tools. ACM Comput. Surv. 54, 9 (2022), 196:1-196:40). Existing research shows that about one-fifth of the development time is spent searching for code examples in the software development process (Weisong Sun, Chunrong Fang, Yuchen Chen, Guanhong Tao, Tingxu Han, and Quanjun Zhang. 2022. Code Search based on Context-aware Code Translation. In ICSE. 388-400), which shows that code search has become an important part of software development.

[0003] Due to the critical role of code search in software development, people have been working on improving the quality of code search. Early work mainly used keyword matching between code snippets and search queries written in natural language. These methods usually use traditional information retrieval algorithms (Sushil Krishna Bajracharya, Joel Ossher, and Cristina Videira Lopes. 2010. Leveraging usage similarity for effective retrieval of examples in code repositories. In SIGSOFT FSE. 157-166) to measure the relevance between queries and candidate code snippets. Recently, the success of deep learning techniques greatly facilitates the development of code search methods (Yanming Yang, Xin Xia, David Lo, and John C. Grundy. 2021. A Survey on Deep Learning for Software Engineering. ACM Comput. Surv. 2021). Various deep learning techniques have been introduced to improve code search, including but not limited to deep neural networks (Xiaodong Gu, Hongyu Zhang, and Sunghun Kim. 2018. Deep code search. In ICSE. 933-944), meta-learning (Yitian Chai, Hongyu Zhang, Beijun Shen, and Xiaodong Gu. 2022. Cross-Domain Deep Code Search with Few-Shot Meta Learning. arXiv Preprint (2022). https: / / arxiv.org / abs / 2201.00150), and pre-training (Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In EMNLP (Findings) (Findings of ACL, Vol. EMNLP 2020). 1536-1547).They encode queries and code snippets into low-dimensional vector spaces and measure their relevance by representing the similarity between vectors (e.g., cosine similarity).

[0004] However, existing works do not consider the impact of different naming styles (referred to as the naming problem for simplicity in the following). Software developers can not follow the same naming convention, leading to different names for the same variable (Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin B. Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021. GraphCodeBERT: Pre-training Code Representations with Data Flow. In ICLR.). The naming problem constitutes an obstacle to effectively deploying deep learning techniques in code search, for the following reasons: (1) when the same variable has different variable names, it is difficult for developers to find the corresponding code snippets, (2) many deep learning techniques rely on entity correspondence (i.e., the same object is always represented by the same vector). There are very few studies in the literature that address the naming problem. Figure 1 The left side of Figure 1 provides an example of two python code snippets. The bottom code snippet is the result of changing the variable names in the top code snippet. Figure 1 The right side of Figure 1 shows the AST of the two code snippets. The blue blocks correspond to the original variable names, which are replaced by green blocks after changing the variable names. Current code representation learning techniques have difficulty understanding that the corresponding variables in the two code snippets (e.g., lst and l) are the same, because their names are different. It is necessary for researchers to study how to avoid the impact of the naming problem when modeling the abstract syntax tree (AST).

[0005] Abstract Syntax Tree (AST): Any programming language has an explicit context-free grammar, which can be used to parse the source code into an AST, which represents the abstract grammatical structure of the source code (Robert Harper. 2016. Practical Foundations for Programming Languages ​​(2nd. Ed.). Cambridge University Press). An AST is a tree where each non-leaf node corresponds to a non-terminal node in the context-free grammar that specifies structural information (e.g., for statements and while statements). Each leaf node corresponds to a node in the context-free grammar encoding the program text (e.g., variable names and operators). An AST can be easily converted back to source code. From Figure 1 In the AST shown, we can see that each non-leaf node contains a type attribute (e.g., Arguments), and each leaf node contains a type attribute and a value attribute (e.g., Name:ix means the type is Name and the value is ix). AST has been widely used in the design of software engineering tools (Jian Zhang, Xu Wang, Hongyu Zhang, Hailong Sun, Kaixuan Wang, and Xudong Liu. 2019. A novel neural source code representation based on abstract syntax tree. In ICSE. 783-794.). On the one hand, compared with ordinary source code, AST is abstract and does not include all details and separators such as punctuation. On the other hand, AST can describe the lexical and grammatical structure information of the code fragment and provide a structural view of the source code, which is crucial in helping NACS achieve naming-independent code search.

[0006] Software development is a repetitive task, as developers often reuse or draw inspiration from existing code. Code search, which involves retrieving relevant code snippets from a codebase based on the developer's intent expressed as a query, has become increasingly important in the software development process. In recent years, a large number of deep learning-based code search methods have emerged and achieved promising results. However, developers often do not follow the same naming conventions, resulting in the same variable having different names in different implementations. This poses a challenge to deep learning-based code search methods, as these methods often rely on explicit variable correspondences to understand the source code. Summary of the Invention

[0007] The present application aims at the above-mentioned problems existing in the prior art, and provides a code retrieval method based on naming-agnostic multi-view contrast code representation learning (NACS), which adopts the ideas of contrast learning and multi-view learning to process the influence of different naming conventions. The method is used for retrieving relevant code snippets from a code library for the intention of a query. The purpose is to improve the accuracy of the code retrieval model and the recognition ability of the model to different naming conventions by performing multi-view contrast learning on the abstract syntax tree (AST) of the code.

[0008] The present application comprises three parts:

[0009] 1) Data augmentation: deleting the information of bound variable names from the abstract syntax tree (AST) and focusing on capturing only the intrinsic properties from the AST structure; using semantic and syntactic augmentation techniques to prepare realistic and reasonable data; using comparative study to design a graph view modeling component in NACS to enhance the understanding of the code snippets;

[0010] 2) Multi-view contrast code representation learning pre-training: modeling the AST in the path view and strengthening the graph view modeling component through multi-view contrast learning;

[0011] 3) Code search model enhancement: given the code and query representation from the training data, using the pre-trained NACS to enhance the code search.

[0012] The specific steps of the present application are as follows:

[0013] Step 1: randomly select a batch of N code snippets and their AST graphs from the code library; apply semantic and syntactic data augmentation strategies to each code snippet and generate three positive samples for each code snippet

[0014] Step 2: for each node n in the AST graph g, encode the AST node type and degree information to obtain and perform eigenvalue decomposition on the normalized Laplacian determinant of the AST graph to construct the graph Laplacian eigenvector of the node Splice the three to obtain the node feature representation of the AST graph, and use GIN to calculate the graph representation r g of all ASTs.

[0015] Step 3, extract each Path from root node to leaf node z c Represents the total number of paths, encodes each path using Bi-LSTM and uses mean pooling to get an average vector representation of all paths, which is used as the path vector representation w of the AST c ;

[0016] Step 4, encode the query using Bi-LSTM to get the vector representation s of the query i ;

[0017] Step 5, calculate the Loss of the model by the following formula:

[0018]

[0019] L=L g +L p +L q

[0020] Where N represents the size of the batch, τ is the temperature coefficient, sim(r1,r2) is the cosine similarity between r1 and r2, Ι [v≠c] Indicates that the function is 0 when v=c, otherwise 1;

[0021] Step 6, loop steps 1-5 to get the pre-trained model of NACS;

[0022] Step 7, concatenate the pre-trained model of NACS to the CodeBERT model, use NACS and CodeBERT to calculate the vector representation of the query and code respectively, and use the following formula to calculate the similarity between the query and the code:

[0023]

[0024] score=score1(q,c)+λscore2(q,c)

[0025] Where score1 represents the matching score from CodeBERT, score2 represents the matching score from the NACS pre-trained model, and λ is a predefined parameter used to balance the two parts.

[0026] Given a batch of query code pairs with size N, use the following loss function to fine-tune the CodeBERT and NACS pre-trained models:

[0027]

[0028] After training, the final code retrieval model is obtained.

[0029] After adopting the above scheme, the beneficial effects of the present invention are:

[0030] 1) Provide a solution on how to use AST to handle different naming conventions in code search tasks.

[0031] 2) To reduce the reliance on explicit correspondence between identical variables in different code snippets in code representation learning, we remove variable name information from the input AST and focus on capturing intrinsic properties solely from the AST structure. We then design various semantic and syntactic enhancement techniques to prepare realistic and plausible data, and employ contrastive learning to design a graph view modeling component in NACS to enhance understanding of code snippets.

[0032] 3) To avoid information loss caused by ignoring explicit variable correspondences, another path view modeling component is designed to model AST paths. Path view modeling is to enhance the graph view modeling component through multi-view learning.

[0033] 4) The NACS of the present invention provides superior code search performance, and NACS can help existing code search methods overcome the influence of different naming conventions. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Here are two examples of code snippets and their corresponding ASTs.

[0035] Figure 2 Pre-trained model framework for NACS.

[0036] Figure 3 An overall framework enhancement for code search models. DETAILED DESCRIPTION

[0037] The following, along with accompanying figures, details NACS, a novel code search method based on name-independent contrastive multi-view learning. NACS consists of three main components: data augmentation, pre-training for multi-view contrastive code representation learning, and code search model enhancement. Figure 2 Describes data augmentation and an overview of pre-trained models for learning multi-view contrastive code representations for NACS. Figure 3 Describes the overall framework for code search model enhancement.

[0038] 1. Data Augmentation

[0039] Data augmentation is an important step in contrastive learning. It creates realistic and plausible data by applying certain transformation strategies that do not significantly affect the original data information. Data augmentation can enrich the supervisory signal without the need for manual labeling.

[0040] Different program transformation (PT) strategies and structure transformation (ST) strategies are adopted to generate augmented data for code snippets in NACS. They are applied to the abstract syntax tree (AST) of code snippets and help NACS better capture the intrinsic code characteristics and distinguish different code snippets without more manual labels.

[0041] PT strategies are semantic-level augmentations, while ST strategies are syntax-level augmentations. The following PT and ST strategies are designed for data augmentation in NACS:

[0042] 1.1 PT strategies

[0043] Three PT strategies are used to generate augmented versions of code snippets that do not change the original semantics, which can help NACS better capture program semantics:

[0044] Inserting useless code statements: three useless code statements are given, and a code statement that does not affect the functionality of the code snippet is randomly inserted into a random position in each code snippet. It is equivalent to adding a subtree to the corresponding AST.

[0045] Swapping independent statements: by analyzing the dependencies between variables, NACS swaps two statements that are independent of each other in each code snippet.

[0046] Changing loop statements: by replacing the corresponding nodes in the AST, NACS changes the for loop to a while loop, and vice versa. If there are multiple iteration structures in a code snippet, one of them will be randomly selected and changed.

[0047] 1.2 ST strategies

[0048] Two graph transformation methods are designed to generate structural augmentation data for the AST of each code snippet, enhancing the structural-level understanding of NACS:

[0049] Subtree deletion: the existing node deletion or edge deletion method for augmented graph data (Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph Contrastive Learning with Augmentations. In NeurIPS) randomly deletes some vertices and / or edges in the graph. However, doing so will turn an AST into a disconnected graph and significantly change the structural information of the AST. A subtree in the AST is randomly discarded, and the probability of discarding a subtree is inversely proportional to the number of nodes in the subtree.

[0050] Feature transformation: randomly exchange the features of nodes in an AST. The motivation of this strategy is that the transformed graph is still similar to the original AST part (i.e., structurally similar, but with different node features). The difference between the transformed graph and the original AST graph can help NACS better understand the intrinsic properties of the AST.

[0051] For a piece of code c, its AST is denoted as Two of the three PT strategies are used to construct its positive samples for code c Then use two ST strategies to generate positive samples and Since the PT strategy needs to traverse and analyze the AST, it is very costly, and two of the three PT strategies are randomly used for each code snippet. The difference is that the overhead of the ST strategy is relatively small. Subtree deletion involves AST traversal, but no expensive analysis. Therefore, the two ST strategies are used on each code snippet, but they are deployed independently to avoid causing large deviations from the original code snippet.

[0052] II. Pre-trained model of multi-view contrastive code representation learning

[0053] To overcome the difficulty of aligning variables in different code snippets with the same meaning, a naming-independent contrastive multi-view code representation learning component is designed in NACS. The core idea is to remove the information bound to the variable name from the input AST and focus on capturing the intrinsic properties only from its structure. To avoid the information loss caused by decoupling, in addition to the common modeling method of modeling AST as a graph, the AST is modeled by multi-view learning of the graph view and the path view of the AST, which helps to enhance the understanding of NACS on the AST structure. In addition, by using the enhanced data introduced in the last section, graph contrastive learning is used to amplify the supervision signal and alleviate the lack of manual labels. Figure 2 An overview of pre-trained NACS through naming style-independent contrastive multi-view code representation learning is provided.

[0054] The following describes the graph view modeling component and the path view modeling component in NACS, and how multi-view contrastive learning is used in each part.

[0055] 2.1 Graph view model

[0056] First, NACS captures the syntax level (structural level) information by modeling AST as a graph. This is the core modeling component in NACS. Each AST is treated as a graph, and a graph topology encoder (GT-Encoder) is pre-trained to capture the topological information of AST (i.e., the syntax information of the code snippet). The core of the graph topology encoder is a graph neural network (GNN) that utilizes the graph isomorphism network (GIN) model (Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks. In ICLR), but other GNN architectures can also be adopted. The AST graph identification task is adopted as the pre-training task for graph view modeling. By contrasting samples, NACS is trained to move the AST graph in the representation space close to its positive samples but far from its negative samples, helping NACS to distinguish AST graphs. The specific steps are as follows:

[0057] (1) Randomly select a batch of N code snippets (and their ASTs) from the code base. Apply the PT and ST strategies on each code snippet to generate its three positive samples. In this way, there are 3N AST graph samples after transformation.

[0058] (2) For each node s in the AST graph g, encode its AST node type and degree information as part of the node features. In addition, use the initialization method used in existing GNN pre-training methods (Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020. GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training. In KDD. 1150-1160.) to perform feature decomposition on the normalized graph Laplacian vector of each AST graph, s.t. I-D -1 / 2 AD -1 / 2 = UΛU T Then add the top feature vector of U to construct the node feature of s:

[0059]

[0060] where b s is the node feature of s, is a feature vector of U, The connection operation is represented, and then each AST input GIN encoder generates a node representation:

[0061]

[0062] where MLP (k) denotes the k-th layer of fully connected neural network, the k-th layer output representation of node s, N(s) is the neighbor set of node s, and ε is a learnable parameter, which is set to Then, the average pooling operation is performed on all node representations in the AST graph g output using GIN. The result is input to the MLP to obtain the graph representation of g from the i-th layer The last graph representation r g of the AST graph g is the sum of where K is the number of layers:

[0063] M g denotes the number of nodes in the AST graph g.

[0064] (3) Calculate three contrastive losses according to the AST representation of each code snippet c in the batch and the representation of each positive sample and The contrastive loss value is defined as follows:

[0065]

[0066] where N represents the size of the batch, τ is the temperature coefficient, sim(r1, r2) is the cosine similarity between r1 and r2, and [v≠c] indicates that the function is 0 when v = c, and 1 otherwise.

[0067] 2.3 Path View Model

[0068] In addition to the graph view, NACS also models the AST path view, providing another view of the structural information in the AST. The path view modeling is an auxiliary modeling component in NACS, which helps to strengthen the model's understanding of the AST.

[0069] Extract each path from the root node to the leaf node z c denotes the total number of paths, and since the number of paths in different ASTs is different, z c is not fixed, y c,i = {n c,i,1 , n c,i,2 ,..., n c,i,li}, l i denotes the length of the AST path i, and nc,i,j Indicates the type of the jth node in path i. Use Bi-LSTM to encode each path, which can extract the forward and reverse information of the path. In path i, at each time step s, Bi-LSTM reads the embedding of the first s-1 nodes in path i and then calculates the hidden state h c,i,s :

[0070]

[0071] where emb(n) is the embedding of AST node type n. Note that the path view modeling component and the graph view modeling component do not share the AST node type representation, i.e., emb(n) is the same as in Equation 1. The types are different.

[0072] Concatenate the last hidden states in the forward and backward directions to represent the AST path i:

[0073]

[0074] Then, yes Representation of all AST paths in Perform average pooling to obtain The final path representation w c :

[0075]

[0076] Three positive samples ( and ) a positive sample For example, in path view modeling, the positive sample pair The contrast loss is defined as follows:

[0077]

[0078] Formula (8) is similar to Formula (4). The difference is that in Formula (9), NACS compares representations from both the graph view and the path view, i.e., cross-view comparison. In comparison, Formula (4) only compares graph view representations.

[0079] The overall path view loss function is defined as follows:

[0080]

[0081] Given a query b i ={w1,...,w t}, which contains a natural language word sequence of length t. A Bi-LSTM is used to embed the query sequence into its representation si The encoding process is similar to equation (5), equation (6) and equation (7).

[0082] With InfoNCE loss function, a contrastive loss function for self-supervised learning, is used to model the query:

[0083]

[0084] where r c is the AST graph representation of the corresponding code snippet of query u, r c and r v are extracted according to equation 3. For query encoding, the contrastive query representation and AST graph representation. AST graph modeling is the main modeling component in NACS. The path representation (from path view) is not merged in equation (10) to reduce the overhead when encoding the query, because AST path encoding involves multiple AST paths for each AST traversal.

[0085] The overall objective of optimizing the contrastive multi-view learning component in NACS is defined as:

[0086] L = L g + L p + L q (11)

[0087] III. Enhancing code search using NACS

[0088] The following describes how to use the pre-trained NACS to enhance code search.

[0089] 3.1 Generating code representation

[0090] For each code snippet c in the code base, its code token sequence is input into the pre-trained CodeBERT (Zhangyin Feng, et al. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In EMNLP (Findings) (Findings of ACL, Vol. EMNLP 2020). 1536-1547.) to obtain its code token representation:

[0091]

[0092] CodeBERT is a dual-peak model for natural language and programming language, which can derive high-quality text and code embeddings.

[0093] Then, the AST graph r The graph encoder (Eq. 3) in the contrastive multi-view component of NACS is inputted with the code c, generating an AST representation of the code snippet:

[0094]

[0095] Similar to the query encoding objective defined in Eq. (11), only the main modeling method in NACS (i.e., the graph view modeling) is used at search time to avoid the cost of traversing multiple AST paths, making the search phase lightweight. In multi-view learning, the graph encoder is trained together with the path encoder, with the ability to understand ASTs in multiple views.

[0096] In summary, for each code snippet c, there is a code representation generated by the pre-trained CodeBert and an AST representation generated by the pre-trained NACS

[0097] 3.2 Generating code representations

[0098] Each query q is inputted into the query encoder of the pre-trained CodeBERT and the pre-trained NACS, computing two query representations respectively:

[0099]

[0100] The two matching scores of query q and each code snippet c are computed:

[0101]

[0102] where score1 represents the matching score from CodeBERT, and score2 represents the matching score from the NACS pre-trained model.

[0103] 3.3 NACS fine-tuning

[0104] Given the code and query representations from the training data, CodeBert and NACS are fine-tuned to enhance code search using multi-view representations.

[0105] Three examples of useless code statements are given as follows:

[0106]

[0107] score1(q,c) and score2(q,c) are fused to represent the matching score between query q and code snippet c:

[0108] score = score1(q,c) + λscore2(q,c) (16)

[0109] where λ is a predefined parameter used to balance the two parts.

[0110] During the optimization, given a batch size of N query-code training pairs, the whole framework is trained to minimize the following loss:

[0111]

[0112] 3.4 Code Retrieval

[0113] During the search, given an input query q, the relevance score between the query q and each code snippet c in the codebase is computed as Equation 16. The relevance scores between the query q and all candidate code snippets are sorted, and the top-k code snippets with the highest relevance scores are returned as the search results for the query q.

[0114] This step is lightweight: the representations of all code snippets in the codebase can be pre-computed, and the two query representations can be directly obtained from the fine-tuned CodeBert and NACS (Equation 14).

[0115] IV. Training Details

[0116] The batch size is set to 48. The dimensions of AST path node embedding, AST path embedding, and query embedding are 128, respectively. The dimensions of AST graph node embedding and AST graph embedding are 128. The Adam optimizer is used with an initial learning rate of 0.001. NACS and baselines use the same settings as much as possible. Otherwise, the default settings of the baselines will be adopted to achieve a fair comparison. A grid search is performed to find the best settings of λ and τ for NACS. It is found that λ = 0.0001 and τ = 0.07 bring the best results, and they are used as the default settings.

[0117] The experiments are run on a machine with two Intel(R) Xeon(R) CPU E5-2678 v3 @ 2.50GHz, 256GB main memory, and 8 GeForce RTX 2080Ti graphics cards, each with 11GB of memory. During the running process, each program will monopolize one graphics card, even if it does not need the full 11GB of memory.

[0118] The MRR of various methods on three datasets is shown in Table 1.

[0119] Table 1

[0120] Model CodeSearchNet CoSQA CoSQA-Var OCoR 0.165 0.352 0.293 CSRS 0.231 0.373 0.231 DeepCS 0.164 0.472 0.433 CodeBERT 0.665 0.652 0.558 GraphCodeBERT 0.694 0.648 0.456 CoCLR 0.637 0.647 0.515 NACS 0.701 0.708 0.704

[0121] The ranking results of the modified variable name before and after the real code snippet under different methods are shown in Table 2.

[0122] Table 2

[0123]

[0124] NACS enhances the performance of existing code search methods, as shown in Table 3.

[0125] Table 3

[0126] Method CoSQA CoSQA-Var Decline Percentage DeepCS 0.472 0.433 ↓8.26％ DeepCS * ]] 0.488 0.473 ↓3.07％ GraphCodeBERT 0.648 0.456 ↓42.11％ GraphCodeBERT * ]]> 0.691 0.675 ↓2.32％ CoCLR 0.647 0.515 ↓20.40％ CoCLR * ]] 0.677 0.657 ↓2.95％

[0127] Five, application field

[0128] It can be applied in various code retrieval fields of enterprises to improve the development efficiency and quality of developers. Meanwhile, the pre-trained model of AST can also be applied to other code-related downstream tasks, such as anomaly detection and code generation.

[0129] The present application provides a naming-agnostic code search method (NACS) based on comparative multi-view code representation learning. NACS removes the information of bound variable names from an abstract syntax tree (AST) and focuses on capturing only the intrinsic properties from the AST structure. A semantic level and a syntax level enhancement technique (Data Augmentation) is used to prepare realistic and reasonable data, and a comparative study (Comparative Study) is adopted to design a graph view modeling component in NACS to enhance the understanding of code snippets. Further, the AST is modeled in a path view (Path View), and the graph view modeling component is strengthened through multi-view comparative learning. The NACS of the present application provides superior code search performance, and NACS can help existing code search methods to overcome the influence of different naming conventions.

Claims

1. A code retrieval method based on multi-view comparative code representation learning independent of naming style, characterized by include: 1) Data augmentation: Remove the bound variable name information from the abstract syntax tree (AST) and capture intrinsic properties from the AST structure; Prepare realistic and reasonable data using semantic and syntactic enhancement techniques; We use contrastive learning to design a graph view modeling component in NACS to enhance the understanding of code snippets. 2) Multi-view comparative code representation learning pre-training: Modeling the AST in the path view and strengthening the graph view modeling component through multi-view comparative learning; 3) Code search model enhancement: Given code and query representations from training data, code search is enhanced using pre-trained NACS; The specific steps are as follows: Step 1: Randomly select a batch of N code snippets and their AST graphs from the code base; in each code snippet Apply semantic and grammatical data enhancement strategies to generate three positive samples A total of 3N converted codes are obtained; Step 2: For each node n in the AST graph g, encode its AST node type and degree information to obtain and Perform eigendecomposition on the normalized Laplace determinant of the AST graph to construct the graph Laplace eigenvector of the node The three are spliced ​​together to obtain the node feature representation of the AST graph, and GIN is used to calculate the graph representation r of all ASTs. g ; Step 3, extract each AST The path from the root node to the leaf node z c Represents the total number of paths, uses Bi-LSTM to encode each path and uses mean pooling to obtain an average vector representation of all paths, which is used as the path vector representation w of AST c ; Step 4: Use Bi-LSTM to encode the query to obtain the query vector representation s i ; Step 5: Calculate the model's loss using the following formula: L=L g +L p +L q Where N is the batch size, τ is the temperature coefficient, sim(r1,r2) is the cosine similarity between r1 and r2, and [v≠c] It means that the function is 0 when v=c, and 1 otherwise; Step 6: Repeat steps 1 to 5 to obtain the NACS pre-training model. Step 7: Splice the NACS pre-trained model onto the CodeBERT model, use NACS and CodeBERT to calculate the vector representation of the query and code respectively, and calculate the similarity between the query and code using the following formula: score=score1(q,c)+λscore2(q,c) Among them, score1 represents the matching score from CodeBERT, score2 represents the matching score from the NACS pre-trained model, and λ is a predefined parameter used to balance the two parts; Given a batch size N of query-code pairs, the following loss function is used to fine-tune the CodeBERT and NACS pre-trained models: After the training is completed, the final code retrieval model is obtained.

Images