A program understanding method and system based on abstract syntax tree

Through the program understanding method based on abstract syntax tree, the syntax structure and semantic information of the program code are extracted, combined with masking strategy and pre-training model, the problem of excessive dependence on labeled data by neural network models is solved, and the efficiency and accuracy of program understanding are improved. , realizes efficient training on super-large code datasets.

CN112835585BActive Publication Date: 2024-12-03SHANDONG NORMAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110096711.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-25
Publication Date
2024-12-03
Estimated Expiration
2041-01-25

AI Technical Summary

Technical Problem

Existing neural network models rely on a large number of labeled data sets in program understanding, resulting in overfitting problems and limiting the efficiency and accuracy of program understanding.

Method used

Using the program understanding method based on abstract syntax tree, the syntax tree is generated by obtaining program code, the syntax tree path and word vector sequence are extracted, and the path representation vector and word vector sequence are formed in combination with the masking strategy. The program understanding model is input for pre-training, and a hybrid is proposed. Pre-train target path mask language modeling and node sequence prediction, jointly train the encoder-decoder to improve the performance of the generation task.

Benefits of technology

Pre-trained models trained on super-large code data sets can reduce dependence on labeled data, improve program understanding accuracy and training speed, and solve the problem that neural network methods are limited by data scale.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112835585B_ABST
    Figure CN112835585B_ABST
Patent Text Reader

Abstract

The present invention discloses a program understanding method and system based on an abstract syntax tree, including: obtaining program code; generating a syntax tree from the program code; extracting the path from the root node to the terminal node of each syntax tree; forming a path representation vector after masking the nodes in the path according to a masking strategy; forming a word vector sequence after masking some nodes in the program code according to a masking strategy; inputting the path representation vector set and the word vector sequence into a program understanding model to complete the pre-training task, and obtaining a trained program understanding model for program understanding. The grammatical structure information and semantic information in the program code are fully extracted, the problem of over-reliance on labeled data in deep neural network training is alleviated, and the efficiency and accuracy of program understanding are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of program comprehension, and in particular to a program comprehension method and system based on an abstract syntax tree. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Program comprehension is a key activity in software engineering and has always been a research hotspot in the field of software engineering. It plays an important role in tasks such as software development, reuse, testing and maintenance.

[0004] The complexity of program structure is a challenge faced by program understanding. Structurality is an important feature of programs, and the semantics of programs are also reflected by the combination of program statements and expressions with different grammatical structures. For example, the grammatical structure of a program can be decomposed into a combination of three basic grammatical structures: sequence, branch, and loop. Its functions are also reflected by the combination of these basic grammatical structures. Therefore, to complete the task of program understanding, it is necessary to fully extract and abstract the structural information of the program.

[0005] Although neural network models have been successful in program understanding, the inventors believe that existing neural models for program decomposition rely on labeled data sets during training. However, the scale of labeled data sets for program understanding is relatively small, and deep neural network models usually have a large number of parameters. Smaller training data sets will lead to overfitting, which limits the better performance of program understanding. Summary of the invention

[0006] In order to solve the above problems, the present disclosure proposes a program understanding method and system based on an abstract syntax tree, which fully extracts the grammatical structure information and semantic information in the program code, alleviates the problem of excessive reliance on labeled data in deep neural network training, and improves the efficiency and accuracy of program understanding.

[0007] To achieve the above objectives, the present disclosure adopts the following technical solutions:

[0008] First, a program understanding method based on an abstract syntax tree is proposed, including:

[0009] Get the program code;

[0010] Generate a syntax tree from program code;

[0011] Extract the path from the root node to the terminal node of each syntax tree;

[0012] After masking some nodes in the path according to the masking strategy, a path representation vector is formed;

[0013] After masking some nodes in the program code according to the masking strategy, a word vector sequence is formed;

[0014] The path representation vector and the word vector sequence are input into the program comprehension model to complete the pre-training task, and a trained program comprehension model for program comprehension is obtained.

[0015] Secondly, a program comprehension system based on abstract syntax tree is proposed, including:

[0016] A program code acquisition module, used to acquire program codes;

[0017] A syntax tree generation module is used to generate a syntax tree from program code;

[0018] Path extraction module, used to extract the path from the root node to the terminal node of each syntax tree;

[0019] A path representation vector generation module is used to form a path representation vector after masking some nodes in the path according to a masking strategy;

[0020] A word vector sequence generation module is used to form a word vector sequence after masking some nodes in the program code according to the masking strategy;

[0021] The model training module is used to input the path representation vector and the word vector sequence into the program understanding model to complete the pre-training task and obtain the trained program understanding model for program understanding.

[0022] In a third aspect, an electronic device is proposed, comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the steps described in a program understanding method based on an abstract syntax tree are completed.

[0023] In a fourth aspect, a computer-readable storage medium is proposed for storing computer instructions. When the computer instructions are executed by a processor, the steps of a program understanding method based on an abstract syntax tree are completed.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. This paper proposes a hybrid pre-training target path masked language modeling (PMLM) and node order prediction (NOP) for AST. PMLM designs a new masking strategy for AST paths, so that the encoder-decoder can be jointly trained to improve the performance of generation tasks. NOP makes the model pay attention to the node constraint relationship on the AST path so that more code structure features can be extracted.

[0026] 2. The present invention is trained on a very large code dataset and provides a pre-trained model based on an encoder-decoder structure, which can be fine-tuned and directly used for downstream tasks aimed at program understanding to improve its accuracy.

[0027] 3. The present disclosure solves the shortcoming that the neural network method is easily limited by the data scale when performing program understanding, which can reduce the amount of labeled data required for training and greatly improve the training speed of downstream tasks of program understanding.

[0028] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0030] Figure 1 This is a schematic diagram of the code representation pre-training task flow based on the abstract syntax tree disclosed in Example 1 of the present disclosure, wherein dark gray indicates that the node (or word) is blocked, and the v5 and v6 nodes are swapped;

[0031] Figure 2 This is a schematic diagram of the Transformer structure disclosed in Example 1 of the present disclosure;

[0032] Figure 3 Generate an AST schematic diagram for the code snippet described in Example 1 of the present disclosure;

[0033] Figure 4 This is a schematic diagram of the code snippet processing described in Example 1 of the present disclosure. DETAILED DESCRIPTION

[0034] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0035] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0036] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0037] In the present disclosure, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom" and the like indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present disclosure, and do not specifically refer to any part or element in the present disclosure and should not be understood as limitations on the present disclosure.

[0038] In the present disclosure, terms such as "fixed connection", "connected", "connection", etc. should be understood in a broad sense, indicating that it can be fixedly connected, integrally connected or detachably connected; it can be directly connected or indirectly connected through an intermediate medium. For relevant scientific research or technical personnel in this field, the specific meanings of the above terms in the present disclosure can be determined according to specific circumstances, and they cannot be understood as limitations on the present disclosure.

[0039] Example 1

[0040] In this embodiment, a program understanding method based on an abstract syntax tree is disclosed, comprising:

[0041] Get the program code;

[0042] Generate a syntax tree from program code;

[0043] Extract the path from the root node to the terminal node of each syntax tree;

[0044] After masking some nodes in the path according to the masking strategy, a path representation vector is formed;

[0045] After masking some nodes in the program code according to the masking strategy, a word vector sequence is formed;

[0046] The path representation vector and the word vector sequence are input into the program comprehension model to complete the pre-training task, and a trained program comprehension model for program comprehension is obtained.

[0047] Furthermore, the program comprehension model adopts the encoder-decoder structure of Transformer, removes the positional encoding on the decoder side, and adds a binary classifier for prediction, where the encoder reads in a set of AST paths and generates a representation of the AST, and the decoder uses the representation to infer the code snippet corresponding to the AST.

[0048] Furthermore, the output of the decoder is processed as follows: "INDENT" and "DEDENT" are used to indicate the beginning and end of the program code; "NEWLINE" is used to indicate a line break in the program code; "_" is used to indicate a space in the program code; and comments in the program code are deleted.

[0049] Furthermore, when the program comprehension model is trained, the training loss is the sum of the binary cross entropy loss and the multi-classification cross entropy loss.

[0050] Furthermore, each path is a sequence of nodes.

[0051] Furthermore, some nodes in the path are shielded according to the shielding strategy, and each node in the path is represented as a vector. After the vectors are concatenated, a path representation vector is formed.

[0052] Furthermore, the specific process of forming a word vector sequence after masking some nodes in the program code according to the masking strategy is as follows:

[0053] Keep the nodes in the program code that correspond to the obscured nodes in the path;

[0054] Mask out the remaining nodes in the program code;

[0055] Get the word vector of each node in the program code to form a word vector sequence.

[0056] Combination Figure 1-4 The training process of the model in the program understanding method based on the abstract syntax tree disclosed in this embodiment is described in detail.

[0057] like Figure 1 As shown, the training process of the model disclosed in this embodiment includes:

[0058] A large-scale, unlabeled code dataset is used to convert each code snippet into an Abstract Syntax Tree (AST). The path from each AST root node to the terminal node is extracted, and the value nodes in the AST and all the words in the code snippet are divided into sub-words through byte pair encoding.

[0059] According to the requirements of the two pre-training tasks of Path Masked Language Modeling (PMLM) and Node Order Prediction (NOP), AST and code snippets are processed and represented as a set of path vectors and corresponding path representation vectors.

[0060] The code snippets are processed, and "INDENT" and "DEDENT" are used instead of indentation to indicate the start and end of the code snippet; "NEWLINE" is used to represent the line break of the code snippet; spaces in the code snippets are replaced with "_"; at the same time, the comments in the code snippets are deleted from the training data, and the processed code snippets corresponding to the AST are used as the output of the decoder in the program comprehension model.

[0061] The program comprehension model uses a codec structure similar to Transformer, but removes the decoder-side position encoding and adds a binary classifier to complete the node order prediction (NOP) task. The vectors in the path vector set are merged and then represented as a fixed-dimensional vector through a fully connected layer as the input of the model encoder, and the corresponding code snippet vector is used as the input of the model decoder;

[0062] The model is trained according to the two pre-training tasks of PMLM and NOP to obtain a trained program comprehension model for program comprehension.

[0063] Each code snippet in the code dataset is converted into an AST, and the path from each AST root node to the terminal node is extracted. The value nodes in the AST and all the words in the code snippet are divided into subwords through byte pair encoding. Specifically:

[0064] (1) Figure 2 As shown in the figure, the code snippet is converted into AST. The nodes of the AST tree are divided into two categories. One is the type node, such as "argument", "Eptr", represented by the symbol v, and the other is the value node, that is, the word in the code, such as "sys_path", "join_path", which are almost all terminal nodes, represented by the symbol x. We use the path from the root node to the terminal node to represent AST, A = {p1, p2, ..., p n}, where n is the number of paths contained in an AST. Each path is a sequence of nodes. Where m is the length of the path. In addition, each code snippet is segmented into word sequences x1, x2, …, x L , where L is the length of the code snippet and all nodes are represented as real-valued vectors.

[0065] (2) Use byte pair encoding to learn the subwords with the highest frequency among all words in the value nodes in the AST and the code snippets from the corpus, and segment the words. For example, 'third_pirty' may be segmented into 'third', '_', and 'pirty'. Then, the subwords with the highest frequency are collected to form a dictionary and then used for model training.

[0066] We represent each word by adding up the vectors of all the subwords that make up the word:

[0067]

[0068] Where E s is the vector representation of the subword.

[0069] According to the PMLM and NOP pre-training tasks, the AST and its corresponding code fragments are processed in a certain way. The specific processing steps are as follows:

[0070] (1) According to the PMLM rules, mask the value nodes of the AST and some words of the code snippet. The steps are as follows:

[0071] (1-1) Shield some value nodes in the AST path according to the probability {q i} i=1…N The nodes on the path are sampled using a distribution where:

[0072]

[0073] Where i represents the level of the current node, N is the maximum node level on the path, and N is subtracted to prevent numerical overflow.

[0074] Select the node set m with a higher probability according to a certain ratio A , by masking these nodes in the path, we can obtain

[0075]

[0076] (1-2) At the decoder side, the words in the code snippet are masked, and we retain m A , while masking other nodes in the code snippet. In this way, when predicting the next word, PMLM can force the decoder to rely on the feature representation of the AST instead of the previous words in the code snippet.

[0077] C masked =REPLACE(C, m C , [mask])

[0078]

[0079] (2) According to the NOP rule, decide whether to randomly swap A with a probability of 0.5. masked The positions of some nodes in the path are obtained to obtain the path set A exc h ange , which is used as the input of the encoder, and then the model is trained to distinguish whether the order of path nodes in the AST is correct, such as Figure 1 As shown, we swapped the positions of the x5 node and the x6 node.

[0080] Before entering the encoder of the program understanding model, the following operations need to be performed:

[0081] To represent a path, we merge the vectors of each node on the path and then apply a fully connected layer to compress the dimension of the vector:

[0082]

[0083] Where W path is the weight. After the path representation vector passes through the fully connected layer, the application layer is normalized to obtain the path representation vector set X AST .

[0084] The code snippets are processed, using "INDENT" and "DEDENT" instead of indentation to indicate the start and end of the code snippet; "NEWLINE" is used to represent line breaks; spaces in the code snippets are replaced with "_"; and comments in the code snippets are deleted from the training data. The processing results are as follows: Figure 4 As shown, the processed code snippet corresponding to the AST is used as the output of the decoder in the program comprehension model.

[0085] like Figure 2 As shown in the figure, the program understanding model adopts the encoder-decoder structure of Transformer, but removes the decoder-side position encoding and adds a binary classifier to complete the node order prediction task.

[0086] The encoder includes a multi-head attention mechanism, residual connection, layer normalization, and a feedforward network to transform X AST As the input of the encoder, the operation process specifically includes:

[0087] First, through the multi-head attention mechanism, the multi-head attention has three hidden layers, namely the query matrix Q, the key matrix K and the value matrix V, whose values ​​are equal to X AST ; Calculate each head:

[0088]

[0089]

[0090] in, are all learnable parameters; d k K R Dimensions;

[0091] Head i Splice and get the output of the enhancer, namely:

[0092]

[0093] Among them, the parameter matrix

[0094] Residual connection and layer normalization are performed in sequence, specifically:

[0095] Residual Connection:

[0096]

[0097] Layer Normalization:

[0098]

[0099]

[0100]

[0101] Among them, α and β are learnable stretch parameters and learnable offset parameters respectively; ∈ is to prevent division by zero.

[0102] The output of the encoder can be obtained through the last layer of the feedforward network of the encoder, which consists of the GeLU activation function and the linear transformations on both sides of it.

[0103] The first layer of the decoder is multi-head attention, where K E and V E All come from the encoder, Q is the output of the previous time step, and the output value Z will be input to the second layer; the second layer has the same structure as the first layer, but K R and V R The vector sequence from the encoder, i.e. the code snippet, is calculated with the intermediate vector through the attention mechanism and finally input into the feedforward network to obtain the output of the current time step. The next word of the target code is then used as the input of the decoder to obtain the output of the next time step. This process is repeated until the end mark is generated or the maximum length of the output sequence is reached.

[0104] The program comprehension model is trained through PMLM and NOP pre-training objectives, including:

[0105] The output of the encoder is passed through the fully connected layer, and the sigmoid loss function is used to convert the output into a probability distribution between 0 and 1. Finally, the binary cross entropy loss is calculated according to the following formula:

[0106] loss1=-(y t *log(y p )+(1-y t )log(1-y p ))

[0107] where y t is the true label, y p is the output of the encoder.

[0108] The output of each time step of the decoder is passed through the fully connected layer and the Softmax function in turn to obtain the target code word, and finally the multi-classification cross entropy loss is calculated according to the following formula:

[0109]

[0110] Where k represents a total of k label values, and the probability that the i-th sample is predicted to be the k-th label is p i, , N represents the total number of samples.

[0111] The final total loss is the sum of the above two loss values, that is, loss1+loss2.

[0112] After the program comprehension model training is completed, the trained program comprehension model is used for program comprehension of program codes. In this embodiment, the programming languages ​​targeted are Java and Python, and can also be expanded to other languages.

[0113] The present invention performs representation learning by carrying out hypothetical tasks on existing data itself, fully extracts grammatical structure information and semantic information in the programming language, and alleviates the problem of over-reliance on labeled data in deep neural network training, thereby requiring less labeled data for training and accelerating training.

[0114] This paper proposes a hybrid pre-training target path masked language modeling (PMLM) and node order prediction (NOP) for AST. PMLM designs a new masking strategy for AST path, so that the encoder-decoder can be jointly trained to improve the performance of generation tasks. NOP makes the model pay attention to the node constraint relationship on the AST path so that more code structure features can be extracted.

[0115] The present invention is trained on a very large code dataset and provides a pre-trained model based on an encoder-decoder structure, which can be fine-tuned and widely used in downstream tasks for program understanding to improve its accuracy.

[0116] The present disclosure solves the shortcoming that neural network methods are easily limited by data scale when performing program understanding, which can reduce the amount of labeled data required for training and greatly improve the training speed of downstream tasks of program understanding.

[0117] Example 2

[0118] In this embodiment, a program understanding system based on an abstract syntax tree is disclosed, comprising:

[0119] A program code acquisition module, used to acquire program codes;

[0120] A syntax tree generation module is used to generate a syntax tree from program code;

[0121] Path extraction module, used to extract the path from the root node to the terminal node of each syntax tree;

[0122] A path representation vector generation module is used to form a path representation vector after masking some nodes in the path according to a masking strategy;

[0123] A word vector sequence generation module, used to form a word vector sequence after masking some nodes in the program code according to the masking strategy;

[0124] The model training module is used to input the path representation vector and the word vector sequence into the program understanding model for pre-training, and obtain the trained program understanding model for program understanding.

[0125] Example 3

[0126] In this embodiment, an electronic device is disclosed, including a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps described in the program understanding method based on an abstract syntax tree disclosed in Embodiment 1 are completed.

[0127] Example 4

[0128] In this embodiment, a computer-readable storage medium is disclosed for storing computer instructions. When the computer instructions are executed by a processor, the steps of a program understanding method based on an abstract syntax tree disclosed in Embodiment 1 are completed.

[0129] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A program understanding method based on an abstract syntax tree, characterized in that: include: Get the program code; Generate a syntax tree AST from program code; Extract the path from the root node to the terminal node of each syntax tree; After masking some nodes in the path according to the masking strategy, a path representation vector is formed; After masking some nodes in the program code according to the masking strategy, a word vector sequence is formed; Input the path representation vector and the word vector sequence into the program comprehension model to complete the pre-training task, and obtain the trained program comprehension model for program comprehension; Among them, byte pair encoding is used to learn the subwords with the highest frequency among all words in the value nodes and code snippets in the corpus, the words are segmented, and the subwords with the highest frequency are collected to form a dictionary and then used for model training; Shield some value nodes in the AST path according to the probability The nodes on the path are sampled according to the distribution, and the node set with higher probability is selected. , by masking these nodes in the path, we can obtain ; in, Indicates the level of the current node. is the maximum node level on the path; On the decoder side, the words in the code snippet are masked and retained. The decoder can be forced to rely on the feature representation of the AST rather than the previous words in the code snippet when predicting the next word. According to the rules of node order prediction, decide whether to randomly swap based on probability The positions of some nodes in the path are obtained to obtain the path set , use it as the input of the encoder of the program understanding model, and then train the model to distinguish whether the order of path nodes in the AST is correct; Before entering the encoder of the program understanding model, the vectors of each node on the path are merged, and then the fully connected layer is applied to compress the dimension of the vector. After the representation vector of the path passes through the fully connected layer, layer normalization is applied to obtain a set of path representation vectors; The program comprehension model adopts the encoder-decoder structure of Transformer.

2. A program understanding method based on an abstract syntax tree as claimed in claim 1, characterized in that: The encoder-decoder structure removes the positional encoding on the decoder side and adds a binary classifier for prediction, where the encoder reads a set of AST paths and generates a representation of the AST, and the decoder uses the representation to infer the code fragment corresponding to the AST.

3. A program understanding method based on an abstract syntax tree as claimed in claim 2, characterized in that: The output processing of the decoder is as follows: use "INDENT" and "DEDENT" to indicate the beginning and end of the program code; use "NEWLINE" to represent a line break in the program code; use "_" to represent a space in the program code; delete comments in the program code.

4. A program understanding method based on an abstract syntax tree as claimed in claim 2, characterized in that: When training the program comprehension model, the training loss is the sum of the binary cross entropy loss and the multi-classification cross entropy loss.

5. A program understanding method based on an abstract syntax tree as claimed in claim 1, characterized in that: Each path is a sequence of nodes.

6. A program understanding method based on an abstract syntax tree as claimed in claim 1, characterized in that: According to the shielding strategy, some nodes in the path are shielded, and each node in the path is represented as a vector. After splicing the vectors, a path representation vector is formed.

7. A program understanding method based on an abstract syntax tree as claimed in claim 6, characterized in that: The specific process of forming a word vector sequence after masking some nodes in the program code according to the masking strategy is as follows: Keep the nodes in the program code that correspond to the obscured nodes in the path; Mask out the remaining nodes in the program code; Get the word vector of each node in the program code to form a word vector sequence.

8. A program comprehension system based on an abstract syntax tree, characterized in that: include: A program code acquisition module, used to acquire program codes; Syntax tree generation module, used to generate syntax tree AST from program code; Path extraction module, used to extract the path from the root node to the terminal node of each syntax tree; A path representation vector generation module is used to form a path representation vector after masking some nodes in the path according to a masking strategy; A word vector sequence generation module is used to form a word vector sequence after masking some nodes in the program code according to the masking strategy; A model training module is used to input the path representation vector and the word vector sequence into the program comprehension model for pre-training, and obtain a trained program comprehension model for program comprehension; Among them, byte pair encoding is used to learn the subwords with the highest frequency among all words in the value nodes and code snippets in the corpus, the words are segmented, and the subwords with the highest frequency are collected to form a dictionary and then used for model training; Shield some value nodes in the AST path according to the probability The nodes on the path are sampled according to the distribution, and the node set with higher probability is selected. , by masking these nodes in the path, we can obtain ; in, Indicates the level of the current node. is the maximum node level on the path; On the decoder side, the words in the code snippet are masked and retained. The decoder can be forced to rely on the feature representation of the AST rather than the previous words in the code snippet when predicting the next word. According to the rules of node order prediction, decide whether to randomly swap based on probability The positions of some nodes in the path are obtained to obtain the path set , use it as the input of the encoder of the program understanding model, and then train the model to distinguish whether the order of path nodes in the AST is correct; Before entering the encoder of the program understanding model, the vectors of each node on the path are merged, and then the fully connected layer is applied to compress the dimension of the vector. After the representation vector of the path passes through the fully connected layer, layer normalization is applied to obtain a set of path representation vectors; The program comprehension model adopts the encoder-decoder structure of Transformer.

9. An electronic device, characterized in that: The invention comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps of a program understanding method based on an abstract syntax tree as described in any one of claims 1 to 7 are completed.

10. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of a program understanding method based on an abstract syntax tree as described in any one of claims 1 to 7.