A cross-language code program classification, classification model training method and device

Abstract

The present invention discloses a cross-language code program classification, classification model training method and device, the training method comprises the following steps: obtaining a code snippet pair set, the code snippet pair set comprising a first code snippet pair and a second code snippet pair; parsing all code snippets in the code snippet pair set into an abstract syntax tree structure and obtaining a graph structure feature vector of the code snippet according to the abstract syntax tree structure; converting all code snippets in the code snippet pair set into a token feature vector; fusing the token feature vector of the code snippet and its corresponding graph structure feature vector to obtain a fused feature vector; inputting the fused feature vector corresponding to the first code snippet pair and the second code snippet pair into a classification model for training. It can improve the accuracy of cross-language code similarity classification.

Description

Technical Field

[0001] The present invention belongs to the technical field of cross-language code program classification, and specifically relates to a cross-language code program classification, classification model training method and device. Background Art

[0002] With the rapid development of information technology today, software development has become increasingly complex and diverse. In modern software engineering, source code reuse and cross-language code collaboration have become common phenomena. Different programming languages ​​have their own unique syntax, structure, and functional advantages, so mixing multiple programming languages ​​in a project can greatly improve development efficiency and flexibility. However, this also brings great challenges to cross-language source code similarity detection. Identifying and detecting code clones between different programming languages ​​is an important task to improve software quality, optimize code reuse, and prevent code redundancy.

[0003] Traditional code program classification methods are mostly targeted at a single programming language and are difficult to deal with code similarity issues in a multi-language environment. As the scale of software projects continues to expand, the complexity and diversity of codes are also increasing. Single-language program classification tools are unable to cope with multi-language code bases. Cross-language code program classification technology has emerged to solve the code similarity problem between different programming languages ​​and ensure the consistency and reusability of codes in different language environments.

[0004] At present, in the field of cross-language source code similarity detection, although many studies have made certain progress, such as the "Cross-programming language migration method and system for code similarity detection" with application number CN202311418645.2, which realizes code similarity detection through clustering, its similarity classification accuracy needs to be improved due to the complexity of different programming languages ​​and the difficulty of code semantic understanding. Summary of the invention

[0005] In order to solve the problem that the existing cross-language code classification has low similarity classification accuracy, the present invention proposes a cross-language code program classification, classification model training method and device, which can improve the accuracy of cross-language code similarity classification.

[0006] The objective of the present invention is achieved through the following technical solutions:

[0007] The first aspect of the present invention discloses a cross-language code program classification model training method, comprising the following steps:

[0008] Acquire a set of code snippet pairs, the set of code snippet pairs comprising a first code snippet pair and a second code snippet pair, the first code snippet pair comprising two code snippets in different programming languages ​​but with similar functions, the second code snippet pair comprising two code snippets in different programming languages ​​and with dissimilar functions;

[0009] Parsing all code snippets in the code snippet pair set into an abstract syntax tree structure and obtaining a graph structure feature vector of the code snippet according to the abstract syntax tree structure;

[0010] Convert all code snippets in the code snippet pair set into token feature vectors;

[0011] The token feature vector of the code snippet and its corresponding graph structure feature vector are fused to obtain a fused feature vector;

[0012] The fused feature vectors corresponding to the first code snippet pair and the second code snippet pair are input into a classification model for training.

[0013] The second aspect of the present invention discloses a cross-language code program classification model training device, comprising a first memory and a first controller that are communicatively connected in sequence, wherein a first computer program is stored in the first memory, and the first controller is used to read the first computer program and execute a cross-language code program classification model training method described in the first aspect.

[0014] A third aspect of the present invention discloses a cross-language code program classification method, characterized in that it comprises the following steps:

[0015] Obtaining a pair of code snippets to be classified, wherein the pair of code snippets to be classified includes code snippets in two programming languages;

[0016] Parsing two code snippets in the pair of code snippets to be classified into an abstract syntax tree structure and obtaining a graph structure feature vector of the code snippet according to the abstract syntax tree structure;

[0017] Converting two code snippets in the pair of code snippets to be classified into token feature vectors;

[0018] The token feature vector and its corresponding graph structure feature vector are fused to obtain the fused feature vector of the code snippet pair to be classified;

[0019] The fused feature vector of the code fragment pair to be classified is input into the classification model trained by the method described in the first aspect to obtain the classification result of the code fragment pair to be classified, and the classification result is functional similarity or functional dissimilarity.

[0020] The fourth aspect of the present invention discloses a cross-language code program classification device, comprising a second memory and a second controller which are communicatively connected in sequence, wherein a second computer program is stored in the second memory, and the second controller is used to read the second computer program and execute a cross-language code program classification method described in the third aspect.

[0021] The beneficial effects of the present invention are:

[0022] The present invention converts code snippets of different programming languages ​​into token feature vectors and graph structure feature vectors, and then realizes code classification and recognition based on the fused feature vectors after fusing the token feature vectors and the graph structure feature vectors. While representing the source codes of different programming languages ​​as a unified high-dimensional vector, the syntax and structure information of the source codes are maintained, thereby improving the accuracy of cross-language code similarity classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 The present invention is a flowchart of the cross-language code program classification model training method. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0029] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use, or the positions or positional relationships commonly understood by those skilled in the art, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0030] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0031] The first aspect of the present invention discloses a cross-language code program classification model training method, such as Figure 1 As shown, the method includes steps S1 to S5. It should be noted that the step identifiers in this solution are only for the convenience of explaining the method and do not constitute a limitation on the order of precedence. The order of each step is based on its language description and the order of each signal.

[0032] Step S1, obtaining a set of code snippet pairs, the set of code snippet pairs comprising a first code snippet pair and a second code snippet pair, the first code snippet pair comprising two code snippets in different programming languages ​​but with similar functions, the second code snippet pair comprising two code snippets in different programming languages ​​and with dissimilar functions.

[0033] The code snippet pair set contains code snippets of different programming languages. After the code snippet pair set is obtained, in order to improve the effectiveness of the code, the code snippets need to be filtered and combined.

[0034] After the code snippets are collected, the code snippet machine may be preprocessed, and the preprocessing includes steps S11 to S14.

[0035] Step S11, formatting the code snippets in different programming languages ​​to make the code snippets consistent in terms of lines, indents, and spaces; formatting helps to reduce the diversity of code representations, making similarity detection more accurate.

[0036] Step S12, delete the comment information and blank lines in the code snippet. Comment information includes single-line comments, multi-line comments, document comments, etc. There are many considerations for adopting this step. First, comments are usually intended to help human developers understand the code, but when the computer analyzes the code snippet, these comments may be regarded as redundant information and do not directly contribute to extracting the core semantics of the code. Secondly, eliminating comments helps reduce the complexity of the code snippet and reduces the length of the input sequence, thereby providing more concise and efficient data for subsequent processing and analysis. Finally, by eliminating comments, the noise that may exist in the data can be reduced, making the extraction of the semantics of the code snippet more focused and accurate.

[0037] Step S13, delete code snippets whose number of lines of code is less than a threshold. Because some code snippets are too short, some code snippets may only contain a few lines of code. This situation may cause the code snippets to lack sufficient context and completeness, making it difficult for subsequent models to accurately capture the core logic and functions of the code snippets. The threshold setting in this step can effectively filter out those code snippets that do not have sufficient complexity and completeness, that is, filter out code snippets that can reflect the core logic and functional units of the code snippets. Preferably, the threshold is greater than or equal to 6. Longer code snippets are more likely to contain complete algorithm implementations, function definitions, or important control structures. This step can ensure that the selected code snippets have sufficient complexity and completeness, better reflect the core logic and functional units of the source code, and provide better quality data for model training.

[0038] Step S14: Standardize the identifiers in the code snippets. Specifically, unified naming, hash replacement, or mask processing can be used to ensure the logical consistency of the code without being affected by the naming of specific identifiers.

[0039] By processing the code snippet using the above steps, a more effective code representation can be obtained.

[0040] Step S2: Parse all code snippets in the code snippet pair set into an abstract syntax tree structure and obtain a graph structure feature vector of the code snippet according to the abstract syntax tree structure.

[0041] Specifically, this step first converts the code snippet into an Abstract Syntax Tree (AST), which represents the nodes of the grammatical structure; then constructs an adjacency matrix for representing the association relationship between the nodes of the abstract syntax tree; then converts the node into an index in a public vocabulary according to the feature vector of the node, and finally inputs the abstract syntax tree and the adjacency matrix into a graph convolutional neural network GCN to obtain the graph structure feature vector of the code snippet.

[0042] The abstract syntax tree is a graph data, including nodes and edges, wherein the nodes represent code elements and the edges represent the relationship between nodes. Each node of the graph data represents a construct that appears in the source code, which records the grammatical structure in the code snippet; the edges represent the relationship between nodes, such as parent-child node relationships, data dependencies, and function call relationships, thereby reflecting the structure and logical depth of the source code. There are some common grammatical structures and semantic features between different programming languages, such as variable declarations, assignments, conditional judgments, loops, etc. After converting them into AST using the above steps, the nodes between ASTs of different languages ​​also have the same properties, but the specific representations are different.

[0043] The connection relationship between nodes in the abstract syntax tree is clearly expressed in the form of an adjacency matrix, where the elements of the matrix indicate whether there is a direct connection between the nodes.

[0044] After the code snippet is converted into an abstract syntax tree, the node in the abstract syntax tree is converted into a unique serial number by querying the public vocabulary. This serial number represents the index of the node's feature vector in the public vocabulary. In this way, the text attributes of the node are converted into numerical features, so that the machine learning model can process and learn this information. This feature representation method not only improves the richness and expressiveness of node features, but also enhances the model's ability to handle unknown or rare nodes with the help of the versatility of the pre-trained model.

[0045] Specifically, the nodes representing translation units, programs, and modules are unified into the serial numbers of unit nodes; the nodes representing compound statements and state blocks are unified into the serial numbers of block nodes; the nodes representing integers and floating-point types are unified into the serial numbers of primitive type nodes; the nodes representing function definition names and function declaration names are unified into the serial numbers of method definition name nodes; the nodes representing function declarators are unified into the serial numbers of method declaration nodes; the nodes representing formal parameters and parameter lists are unified into the serial numbers of parameter nodes; as shown in Table 1 below.

[0046] Table 1

[0047]

[0048] The above method uses a unified code representation method in a high-dimensional vector space to map code snippets in different languages ​​into a unified feature space. Through this method, code snippets in different languages ​​can be compared for similarity in the same space, thereby achieving unified processing of cross-language codes.

[0049] The graph convolutional neural network in this solution consists of the first graph convolutional layer, the second graph convolutional layer and a fully connected layer. Using the powerful capabilities of the graph convolutional network, the model can capture the structural and semantic features of the code and convert the elements in the source code and their relationships into high-dimensional vector representations.

[0050] Specifically, the first graph convolution layer is used to convert nodes and their corresponding adjacency matrices into hidden features; at the same time, the nonlinear activation function ReLU is introduced to enhance the expressiveness of the model. After the first layer of processing, the dropout layer is used to reduce the risk of overfitting.

[0051] The second graph convolution layer is used to convert the output of the first graph convolution layer into a score corresponding to a category of each node and determine the overall feature of the abstract syntax tree. The overall feature of the abstract syntax tree can be determined by using an average method.

[0052] The fully connected layer is used to map the overall features of the abstract syntax tree to the classification space to obtain the graph structure feature vector of the code snippet. That is, the fully connected layer maps the overall features from the high-dimensional space to the final classification space.

[0053] Step S3: convert the code in the code pair into a token feature vector.

[0054] This step is mainly implemented by batch processing. The existing batch processing method uses a large amount of data sets to be evenly divided into small batches so that they can be processed batch by batch during the training process of the machine learning model. However, in the cross-language code program classification scenario, this uniform division may not capture the differences and diversity between different programming languages. In this regard, this solution is implemented by steps S31-S34.

[0055] Step S31: Use a word segmenter to segment the code snippet to obtain a word list of the code snippet, wherein the word list includes multiple sub-words.

[0056] The code snippet is processed by a tokenizer, which splits the source code snippet into subwords, thereby converting the entire source code into a word list. These subwords in the word list can include identifiers, keywords, operators, etc. in the source code.

[0057] Step S32: Mark the beginning of the word list with a first tag and mark the end with a second tag, wherein the first tag represents comprehensive information of the source code fragment, and the second tag represents different parts in the separated input sequence.

[0058] Step S33: Aggregate the first label, the word list, and the second label of each code snippet to obtain an aggregated sequence of the code snippet.

[0059] Step S34: input the aggregated sequence into a code pre-training model to obtain a token feature vector of the code fragment.

[0060] Exemplary:

[0061] The first label CLS (Classification) and the second label SEP (Separator) are introduced.

[0062] The [CLS] tag is located at the beginning of the sequence and is used to capture the comprehensive information of the entire source code fragment and provide the overall context for the model. for:

[0063]

[0064] In the formula, is the attention weight of the ith position, is the embedding representation of the corresponding position, and N is the length of the aggregate sequence. This summation operation is calculated through the self-attention mechanism, and the attention weight determines the contribution of each position to [CLS].

[0065] [SEP] tags are often used to separate different parts of the input sequence to help the model understand the structure and boundaries of the sequence. The tokenized source code is represented in the form of a token sequence. for:

[0066]

[0067] In the formula, ω represents subword, n is the total number of subwords, CLS represents Classification tag, and SEP represents Separator tag.

[0068] This process helps to reveal the abstract hierarchy of code snippets and transform them into smaller semantic units for subsequent representation and learning.

[0069] Through this process, code snippets are represented as sequences while retaining key semantic and syntactic information, providing richer and more useful input for subsequent model learning. It better meets the needs of cross-language code program classification and fully considers the grammatical and structural differences between different programming languages. Through the above batch processing method, samples of different tasks and different programming languages ​​can be fully considered in small batches, providing the model with more diverse and specific training data, improving effectiveness and generalization.

[0070] The length of the aggregation sequence is less than or equal to 512.

[0071] The determination of token feature vectors is based on the code pre-training model. The code pre-training model is a natural language processing technology based on deep learning. It learns the language structure and programming habits of the code by training a large amount of code data, thereby realizing intelligent understanding and generation of the code.

[0072] Among them, the code pre-training model can use UniXCoder, which includes a data processing module, a sequence embedding module and a multi-layer Transformer module. The data processing module is mainly responsible for processing and tokenizing the input data, the sequence embedding module is used to convert the tokenized input sequence into an embedded vector representation, and the multi-layer Transformer module is the core component of the model, which is used to learn the representation of the input sequence.

[0073] UniXcoder utilizes the contextual information of a large-scale code base, learns the semantic features and structural relationships of the code through deep neural networks, and converts code sequences into high-dimensional vector representations. These vectors capture key information in the code, including dependencies between variables, the hierarchy of function calls, and other grammatical and semantic features, thereby providing rich information for subsequent code similarity analysis tasks. In this process, UniXcoder's powerful representation capabilities can effectively bridge the semantic differences between natural language and programming language, and improve the accuracy and efficiency of the model in programming language processing tasks. In this way, the preprocessed code sequence is converted into a vector space representation with rich semantics, laying a solid foundation for in-depth code understanding and analysis.

[0074] Step S4: fuse the token feature vector of the code and its corresponding graph structure feature vector to obtain a fused feature vector set including the fused feature vector.

[0075] This step can realize the fusion of token feature vectors and their corresponding graph structure feature vectors based on the multi-layer perceptron MLP, that is, combining two different types of features from the source code. In the cross-language source code program classification scenario, the source code, i.e., the code snippet, has a variety of feature representations. For example, the token feature vector captures the lexical and grammatical information of the code, while the graph structure feature vector reflects the structural information of the code. By using MLP for fusion, these two different types of feature vectors can be organically combined to make full use of the rich information of the source code and improve the model's ability to understand the source code and classification performance. MLP can integrate different types of features through multi-level nonlinear transformations and learning, so that the model can more accurately capture the semantic and structural information of the source code.

[0076] Step S5: input the fused feature vectors corresponding to the first code snippet pair and the second code snippet pair into a classification model for training.

[0077] The code snippets in the first code snippet pair are in different programming languages ​​but have similar functions, while the code snippets in the second code snippet are in different programming languages ​​and have different functions. The functional similarity here refers to syntactic similarity and semantic similarity, for example, both implement sorting algorithms. The programming language can be Java, Python, C language, etc.

[0078] Specifically, this step can use the classification loss calculation method and the triplet loss calculation method to perform feedback training on the classification model. Specifically, first use the classification loss calculation method and the triplet loss calculation method to obtain the triplet loss and the classification loss, and then adjust the proportion of the triplet loss and the classification loss to obtain the final mixed loss to perform feedback training on the classification model.

[0079] The classification loss calculation method adopted is: using the cosine-margin triplet loss function to calculate the similarity of the fused feature vectors corresponding to the two code fragments in the first code fragment pair and the fused feature vectors corresponding to the two code fragments in the second code fragment pair.

[0080] The triplet loss calculation method is: use the cross entropy algorithm and label type to calculate the loss.

[0081] There are significant grammatical and structural differences between different programming languages, including differences in naming rules, statement structure, data types, and control flow. These differences make it extremely complicated to detect code similarity between languages. For example, Java and Python are fundamentally different in grammatical structure and code style, so how to effectively establish a similarity mapping between the two becomes a difficult problem. The method of this scheme is adopted, by converting code snippets of different programming languages ​​into token feature vectors and graph structure feature vectors, and then realizing code classification and recognition based on the fused feature vector after fusing the token feature vector and the graph structure feature vector. While representing the source code of different programming languages ​​as a unified high-dimensional vector, the grammatical and structural information of the source code is maintained, thereby improving the accuracy of cross-language code similarity classification.

[0082] By fusing token feature vectors and graph structure feature vectors, richer code features are obtained. This hybrid code representation method not only improves the accuracy and recall of cross-language code program classification, but also achieves unified code representation between different languages, providing an efficient and reliable solution to the problem of cross-language code program classification.

[0083] The method of this scheme realizes the feedback training of the classification model through the hybrid loss of triple loss and classification loss. The classification model can distinguish between code snippets with similar and dissimilar functions; the model performance is further enhanced by combining the classification loss. The classification loss classifies the code snippets to assist the model in learning more refined feature representations. By optimizing the hybrid loss function, the accuracy and recall rate of cross-language code program classification are improved.

[0084] The second aspect of the present invention discloses a cross-language code program classification model training device, including a first memory and a first controller connected in sequence, wherein a first computer program is stored in the first memory, and the first controller is used to read the first computer program and execute a cross-language code program classification model training method described in the first aspect. Specifically, the first memory may include, but is not limited to, a random access memory (Random-Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a first-input first-out memory (First Input First Output, FIFO) and or a first-input last-out memory (First Input Last Output, FILO), etc.; the first controller may not be limited to a microcontroller of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power supply unit, a display screen and other necessary components.

[0085] The operating principle of the device disclosed in the second aspect of the present invention is detailed in the first aspect and will not be elaborated here.

[0086] A third aspect of the present invention discloses a cross-language code program classification method, comprising the following steps S21 to S25.

[0087] Step S21: Obtain a pair of code snippets to be classified, wherein the pair of code snippets to be classified includes code snippets in two programming languages.

[0088] Step S22: parsing the two code snippets in the pair of code snippets to be classified into an abstract syntax tree structure and obtaining a graph structure feature vector of the code snippet according to the abstract syntax tree structure.

[0089] Step S23: convert the two code snippets in the pair of code snippets to be classified into token feature vectors.

[0090] Step S24: fuse the token feature vector and its corresponding graph structure feature vector to obtain a fused feature vector of the code snippet pair to be classified.

[0091] Step S25: input the fused feature vector of the code fragment pair to be classified into the classification model trained by the method described in the first aspect to obtain the classification result of the code fragment pair to be classified, wherein the classification result is functional similarity or functional dissimilarity.

[0092] Specifically, the specific methods of steps S22 to S24 in this method can refer to steps S12 to S14, which will not be described in detail here.

[0093] The fourth aspect of the present invention discloses a cross-language code program classification device, including a second memory and a second controller connected in sequence, wherein a second computer program is stored on the second memory, and the second controller is used to read the second computer program and execute a cross-language code program classification method described in the third aspect. Specifically, the second memory may include, but is not limited to, a random access memory (Random-Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a first-in-first-out memory (First Input First Output, FIFO) and or a first-in-last-out memory (First Input Last Output, FILO), etc.; the second controller may not be limited to a microcontroller of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power supply unit, a display screen and other necessary components.

[0094] The operating principle of the device disclosed in the fourth aspect of the present invention is detailed in the third aspect and will not be elaborated here.

[0095] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A cross-language code program classification model training method, It is characterized in that The following steps are involved: Acquire a set of code snippet pairs, the set of code snippet pairs comprising a first code snippet pair and a second code snippet pair, the first code snippet pair comprising two code snippets in different programming languages ​​but with similar functions, the second code snippet pair comprising two code snippets in different programming languages ​​and with dissimilar functions; Parsing all code snippets in the code snippet pair set into an abstract syntax tree structure and obtaining a graph structure feature vector of the code snippet according to the abstract syntax tree structure; Convert all code snippets in the code snippet pair set into token feature vectors; The token feature vector of the code snippet and its corresponding graph structure feature vector are fused to obtain a fused feature vector; Inputting the fused feature vectors corresponding to the first code snippet pair and the second code snippet pair into a classification model for training; The step of parsing all the code snippets in the code snippet pair set into an abstract syntax tree structure and obtaining a graph structure feature vector of the code snippet according to the abstract syntax tree structure comprises: Converting a code snippet into an abstract syntax tree, wherein the abstract syntax tree represents nodes of a grammatical structure, Construct an adjacency matrix to represent the relationship between nodes in the abstract syntax tree. converting a node into an index in a common vocabulary based on the feature vector of said node, Inputting the abstract syntax tree and the adjacency matrix into a graph convolutional neural network to obtain a graph structure feature vector of the code snippet; The graph convolutional neural network comprises: A first graph convolutional layer, which is used to convert nodes and their corresponding adjacency matrices into hidden features, A second graph convolution layer, the second graph convolution layer is used to convert the output of the first graph convolution layer into a score corresponding to the category of each node and determine the overall characteristics of the abstract syntax tree, The fully connected layer is used to map the overall features of the abstract syntax tree to the classification space to obtain the graph structure feature vector of the code snippet.

2. A cross-language code program classification model training method according to claim 1, It is characterized in that The step of obtaining the code snippet pair set further includes: Format code snippets to make them consistent in terms of lines, indentation, and spaces; Delete comments and blank lines in code snippets; Delete code snippets whose number of code lines is less than a threshold; Standardize identifiers in code snippets.

3. A cross-language code program classification model training method according to claim 1, It is characterized in that The converting all code snippets in the code snippet pair set into token feature vectors comprises: Using a word segmenter to segment the code snippet to obtain a word list of the code snippet, wherein the word list includes a plurality of subwords; Marking the beginning of the word list with a first tag and marking the end with a second tag, wherein the first tag represents comprehensive information of the source code fragment, and the second tag represents different parts in the separated input sequence; Aggregating the first label, the word list, and the second label of each code snippet to obtain an aggregated sequence of the code snippet; The aggregate sequence is input into a code pre-training model to obtain a token feature vector of the code snippet.

4. A cross-language code program classification model training method according to claim 1, It is characterized in that The converting of the code snippet into an abstract syntax tree comprises: Convert the code snippet into an abstract syntax tree, wherein the abstract syntax tree includes nodes and edges, wherein the nodes represent code elements, and the edges represent relationships between nodes; The nodes in the abstract syntax tree are converted into a unique serial number by querying a common vocabulary.

5. A cross-language code program classification model training method according to claim 4, It is characterized in that The step of converting the node in the abstract syntax tree into a unique serial number by querying a common vocabulary table comprises: Unify the nodes representing translation units, programs, and modules into unit node numbers; Unify the nodes representing compound statements and state blocks into the serial numbers of block nodes; Unify the nodes representing integer and floating point types into the serial numbers of the original type nodes; Unify the nodes representing function definition names and function declaration names into the sequence numbers of method definition name nodes; Unify the nodes representing function declarators into the serial numbers of method declaration nodes; The nodes representing formal parameters and parameter lists are unified into the serial numbers of parameter nodes.

6. A cross-language code program classification model training device, comprising a first memory and a first controller which are communicatively connected in sequence, wherein a first computer program is stored in the first memory, Features: The first controller is used to read the first computer program and execute the cross-language code program classification model training method described in any one of claims 1-5.

7. A cross-language code program classification method, It is characterized in that The following steps are involved: Obtaining a pair of code snippets to be classified, wherein the pair of code snippets to be classified includes code snippets in two programming languages; Parsing two code snippets in the pair of code snippets to be classified into an abstract syntax tree structure and obtaining a graph structure feature vector of the code snippet according to the abstract syntax tree structure; Converting two code snippets in the pair of code snippets to be classified into token feature vectors; The token feature vector and its corresponding graph structure feature vector are fused to obtain the fused feature vector of the code snippet pair to be classified; The fused feature vector of the code fragment pair to be classified is input into the classification model trained by any one of the methods described in claims 1 to 5 to obtain the classification result of the code fragment pair to be classified, wherein the classification result is functional similarity or functional dissimilarity.

8. A cross-language code program classification device, comprising a second memory and a second controller which are communicatively connected in sequence, wherein a second computer program is stored in the second memory, Features: The second controller is used to read the second computer program and execute the cross-language code program classification method described in claim 7.

Images