A cross-language code similarity detection method based on semantic analysis

Abstract

This invention discloses a cross-language code similarity detection method based on semantic analysis. The method comprises the following steps: Step 1: First, obtain the first source code of the first programming language and the second source code of the second programming language to be detected; Step 2: Further parse the first source code and the second source code in Step 1 respectively; Step 3: Further extract the code statements contained in the nodes of the first code attribute graph and the second code attribute graph in Step 2; Step 4: Further input the first initial attribute graph and the second initial attribute graph in Step 3 into a pre-trained graph neural network model; Step 5: Finally, calculate the cosine similarity between the first graph-level feature vector and the second graph-level feature vector in Step 4. Through deep fusion processing logic, cross-language semantic noise is eliminated from the underlying computational mechanism, improving the feature extraction accuracy and detection efficiency of the computer system when processing heterogeneous code tracing tasks.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of software engineering and artificial intelligence, specifically a cross-language code similarity detection method based on semantic analysis. Background Technology

[0002] With the booming development of the open source software community, the reuse and porting of code across programming languages ​​has become increasingly common. As a result, problems such as cross-language code plagiarism, violations of open source agreements, and tracing the source of malicious code variants urgently need to be solved. Code similarity detection technology is the core means to deal with the above problems.

[0003] However, existing code similarity detection methods mainly focus on code comparison within a single programming language. The usual approach is to parse the source code, extract its abstract syntax tree (AST), or further extract the control flow graph and data flow graph, and then use graph matching algorithms or graph neural networks to compare the similarity of the above graph structures.

[0004] However, in cross-language detection scenarios (such as comparing C language code and Python language code), existing graph structure-based comparison schemes face serious limitations. The underlying syntax design, keyword system and standard library calling methods of different programming languages ​​are completely different. Even if the code snippets implement the same underlying logic, the graph structure generated after parsing still has huge heterogeneity in node attributes (i.e. the text and syntactic features contained in the nodes).

[0005] For example, the control nodes of traditional for loops in C and the list comprehension nodes in Python are completely unrelated in terms of character representation. When existing methods directly input these control flow graphs or data flow graphs containing heterogeneous text features into the comparison model, the computing system cannot effectively align the semantics of nodes in the heterogeneous feature space across languages. This leads to severe semantic noise accumulation in graph neural networks when aggregating features of neighboring nodes, ultimately causing deep technical pain points such as high false positive rate (structural similarity but different logic) and low recall rate (same logic but different grammatical expressions) when checking for duplicate code across languages. Summary of the Invention

[0006] The purpose of this invention is to provide a cross-language code similarity detection method based on semantic analysis, in order to solve the problem that when existing methods mentioned in the background directly input control flow graphs or data flow graphs containing heterogeneous text features into the comparison model, the computing system cannot effectively align the semantics of nodes in the heterogeneous feature space of cross-language languages. This leads to severe semantic noise accumulation in graph neural networks when aggregating the features of neighboring nodes, ultimately causing the deep technical pain point of high false positive rate and low recall rate in cross-language code deduplication.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A cross-language code similarity detection method based on semantic analysis is proposed, and its steps are as follows: Step 1: First, obtain the first source code of the first programming language to be tested and the second source code of the second programming language; Step 2: Further parse the first source code and the second source code from Step 1, extract the first control flow graph and the first data flow graph corresponding to the first source code, and extract the second control flow graph and the second data flow graph corresponding to the second source code; merge the first control flow graph and the first data flow graph to construct the first code attribute graph, and merge the second control flow graph and the second data flow graph to construct the second code attribute graph; Step 3: Further extract the code statements contained in the nodes of the first code attribute map and the second code attribute map in Step 2; call the pre-trained cross-language semantic mapping model to process the code statements respectively, and generate the node initial feature vector of each code statement in the unified semantic space; concatenate the node initial feature vectors to the corresponding nodes of the first code attribute map and the second code attribute map respectively to obtain the first initial attribute map and the second initial attribute map. Step 4: Further input the first initial attribute map and the second initial attribute map from Step 3 into the pre-trained graph neural network model; aggregate the initial feature vectors of adjacent nodes based on the edge relationships between nodes through the graph convolutional layer in the graph neural network model to obtain the updated node structure feature vector; perform dimensionality reduction and aggregation on the node structure feature vector through the global pooling layer in the graph neural network model to output the first graph-level feature vector representing the first source code and the second graph-level feature vector representing the second source code, respectively. Step 5: Finally, calculate the cosine similarity between the first graph-level feature vector and the second graph-level feature vector in Step 4; if the cosine similarity is greater than the preset judgment threshold, output the detection result that the first source code and the second source code have code similarity.

[0008] As a further aspect of the present invention: in step two, the first control flow graph and the first data flow graph are merged to construct a first code attribute graph, and the specific steps are as follows: S1: First, extract the first set of nodes and the first set of control flow edges in the first control flow graph, and the second set of nodes and the first set of data flow edges in the first data flow graph; S2: Further, based on the line numbers of the source code and the node identifiers of the abstract syntax tree, align the first node set and the second node set, and merge them to obtain a unified node set; S3: Further assign a first type label to the edges in the first control flow edge set, and assign a second type label to the edges in the first data flow edge set; S4: Further concatenate the first type of label, the second type of label, and the unified node set into a first code attribute graph with heterogeneous edge types.

[0009] As a further aspect of the present invention: in step three, a pre-trained cross-language semantic mapping model is invoked to process the code statements respectively, generating an initial feature vector for each code statement in a unified semantic space. The specific steps are as follows: S1: First, perform lexical analysis on the code statements, extract the operator identifier sequence and operand identifier sequence in the code statements, and identify the memory address operator and object reference operator in the operator identifier sequence; S2: Further input the operator identifier sequence and operand identifier sequence into the encoder layer of the cross-language semantic mapping model; S3: Further, based on the underlying physical mapping relationship between memory address operators and object reference operators, construct an attention bias matrix for a specific language, and calculate the local attention weights of the operator identifier sequence and operand identifier sequence by combining the attention bias matrix with the self-attention mechanism in the encoder layer. S4: Further, the hidden state tensors of the encoder layer are weighted and summed according to the local attention weights to output the initial feature vectors of the nodes with fixed dimensions.

[0010] As a further aspect of the present invention: in step four, the graph convolutional layer in the graph neural network model aggregates the initial feature vectors of adjacent nodes based on the edge relationships between nodes to obtain the updated node structure feature vector. The specific steps are as follows: S1: First, extract the first node initial feature vector of the central node in the first initialization attribute graph, and the second node initial feature vector of the neighboring nodes connected to the central node, and identify the original syntax type label of the connection edge between the central node and the neighboring nodes. S2: Further construct a dual-channel convolutional computation path. In the first convolutional channel, use a local topological weight matrix that matches the original syntax type label to perform a linear transformation on the initial feature vector of the second node. In the second convolutional channel, use a global cross-language semantic weight matrix to perform a linear transformation on the initial feature vector of the second node. S3: Further calculate and generate physically controllable gating activation vectors based on the initial feature vectors of the first node and the initial feature vectors of the second node; S4: Further utilize the gated activation vector to perform dynamic feature weighting and fusion on the outputs of the first and second convolutional channels to obtain the node structure feature vector of the central node.

[0011] As a further aspect of the present invention: in step four, the node structure feature vector is reduced in dimension and aggregated by the global pooling layer in the graph neural network model to output the first graph-level feature vector representing the first source code. The specific steps are as follows: S1: First, extract the feature matrix from the output of the graph neural network model, which contains the feature vectors of the node structure; S2: Further perform global average pooling and global max pooling operations on the feature matrix respectively, and output the average eigenvector and the maximum eigenvector; S3: Further, the average feature vector and the maximum feature vector are concatenated and nonlinearly mapped by a multilayer perceptron to output a first graph-level feature vector of fixed length.

[0012] As a further aspect of the present invention: before the cosine similarity in step five exceeds a preset threshold, the specific steps are as follows: S1: First, count the total number of nodes in the first code attribute graph and the second code attribute graph; S2: Further calculate the average of the total number of nodes as the baseline value for code complexity; S3: Further, based on the code complexity baseline value, call the preset penalty coefficient function to calculate the dynamic compensation coefficient; S4: Further multiply the basic comparison threshold with the dynamic compensation coefficient to generate the judgment threshold specific to the current detection task.

[0013] As a further aspect of the present invention: the first control flow graph and the first data flow graph corresponding to the first source code are extracted in step two, and the specific steps are as follows: S1: First, the syntax parser is invoked to traverse and parse the first source code; S2: If a syntax error is detected in the first source code that causes the parsing to be interrupted, record the line number of the interruption location; S3: Further extract the code snippets before the line number markers to construct a local control flow graph and a local data flow graph, and add unclosed abnormal connection edges to the broken tail nodes of the local control flow graph; S4: Further, when the graph neural network model performs message passing for unclosed abnormal connection edges, a preset abnormal mask is invoked to block the transmission of the original structural features on the edge, and the global graph structure compensation vector of the previous iteration of the graph neural network model is routed to the broken tail node.

[0014] As a further aspect of the present invention: in step two, the first control flow graph and the first data flow graph are merged to construct the first code attribute graph. The subsequent operation steps are as follows: S1: First, traverse the subgraph structures in the first code attribute graph and the second code attribute graph; S2: Further identify control flow subgraphs with cyclic characteristics; S3: Further extract the initialization condition nodes, boundary decision nodes, and step size update nodes from the control flow subgraph; S4: Use a preset structural paradigm template to perform topological reorganization on the initialization condition nodes, boundary judgment nodes, and step size update nodes to generate a unified cyclic structure subgraph and replace the original control flow subgraph.

[0015] As a further aspect of the present invention, the training steps of the graph neural network model in step four are as follows: S1: First, obtain positive sample pairs containing semantically equivalent cross-language code snippets, and negative sample pairs containing semantically unrelated cross-language code snippets; S2: Further transform the positive sample pairs and the negative sample pairs into corresponding sample code attribute maps and input them into the initial network model to output sample map-level feature vectors; S3: Further utilize the contrastive loss function to calculate the positive distance between the two sample graph-level feature vectors in the positive sample pair, and the negative distance between the two sample graph-level feature vectors in the negative sample pair; S4: Based on the optimization objective of minimizing the positive distance and maximizing the negative distance, calculate the gradient and update the weight parameters of the initial network model through backpropagation until the loss value converges to obtain the trained graph neural network model.

[0016] As a further aspect of the present invention, the training steps of the cross-language semantic mapping model in step three are as follows: S1: First, obtain a parallel code corpus of multiple programming languages; S2: Further perform random identifier masking on the code fragments in the parallel code corpus to generate masked code sequences; S3: Further input the mask code sequence into the initial semantic encoder and output the predicted identifier probability distribution; S4: Further calculate the cross-entropy loss between the predicted identifier probability distribution and the true identifier; S5: Further update the attention mechanism parameters of the initial semantic encoder based on cross-entropy loss to generate a cross-language semantic mapping model.

[0017] Compared with the prior art, the beneficial effects of the present invention are: This invention, based on extracting control flow graphs and data flow graphs from different languages ​​and constructing code attribute graphs, completely abandons the conventional path of directly relying on surface text or single syntax tags for node initialization. Instead, it introduces a cross-language semantic mapping model pre-trained on multilingual parallel corpora. This forces code statements from different programming languages ​​to be mapped to a unified-dimensional absolute semantic space before being input into the graph neural network. As a result, when the graph neural network model performs feature aggregation and message passing, it no longer faces heterogeneous features with language barriers, but rather aligned, pure semantic vectors. By deeply integrating the processing logic of "low-level graph structure" and "high-dimensional unified semantics," cross-language semantic noise is eliminated from the underlying computational mechanism, significantly improving the feature extraction accuracy and detection efficiency of computer systems when processing heterogeneous code tracing tasks. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a cross-language code similarity detection method based on semantic analysis.

[0019] Figure 2 This is a schematic diagram illustrating the code attribute graph construction process in a semantic analysis-based cross-language code similarity detection method.

[0020] Figure 3 This is a schematic diagram illustrating the process of generating initial feature vectors for nodes in a cross-language code similarity detection method based on semantic analysis.

[0021] Figure 4 This is a schematic diagram illustrating the subsequent operation process of fusing the first control flow graph and the first data flow graph to construct the first code attribute graph in a semantic analysis-based cross-language code similarity detection method.

[0022] Figure 5 This diagram illustrates the training method of the cross-language semantic mapping model in a cross-language code similarity detection method based on semantic analysis. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Please see Figure 1-5 In this embodiment of the invention, a cross-language code similarity detection method based on semantic analysis is provided. The method and steps are as follows: Step 1: First, obtain the first source code of the first programming language to be tested and the second source code of the second programming language; Step 2: Further parse the first source code and the second source code from Step 1, extract the first control flow graph and the first data flow graph corresponding to the first source code, and extract the second control flow graph and the second data flow graph corresponding to the second source code; merge the first control flow graph and the first data flow graph to construct the first code attribute graph, and merge the second control flow graph and the second data flow graph to construct the second code attribute graph; Step 3: Further extract the code statements contained in the nodes of the first code attribute map and the second code attribute map in Step 2; call the pre-trained cross-language semantic mapping model to process the code statements respectively, and generate the node initial feature vector of each code statement in the unified semantic space; concatenate the node initial feature vectors to the corresponding nodes of the first code attribute map and the second code attribute map respectively to obtain the first initial attribute map and the second initial attribute map. Step 4: Further input the first initial attribute map and the second initial attribute map from Step 3 into the pre-trained graph neural network model; aggregate the initial feature vectors of adjacent nodes based on the edge relationships between nodes through the graph convolutional layer in the graph neural network model to obtain the updated node structure feature vector; perform dimensionality reduction and aggregation on the node structure feature vector through the global pooling layer in the graph neural network model to output the first graph-level feature vector representing the first source code and the second graph-level feature vector representing the second source code, respectively. Step 5: Finally, calculate the cosine similarity between the first graph-level feature vector and the second graph-level feature vector in Step 4; if the cosine similarity is greater than the preset judgment threshold, output the detection result that the first source code and the second source code have code similarity.

[0025] In step two, the first control flow graph and the first data flow graph are merged to construct the first code attribute graph. The specific steps are as follows: S1: First, extract the first set of nodes and the first set of control flow edges in the first control flow graph, and the second set of nodes and the first set of data flow edges in the first data flow graph; S2: Further, based on the line numbers of the source code and the node identifiers of the abstract syntax tree, align the first node set and the second node set, and merge them to obtain a unified node set; S3: Further assign a first type label to the edges in the first control flow edge set, and assign a second type label to the edges in the first data flow edge set; S4: Further concatenate the first type of label, the second type of label, and the unified node set into a first code attribute graph with heterogeneous edge types; In step three, a pre-trained cross-language semantic mapping model is invoked to process the code statements, generating an initial feature vector for each code statement in a unified semantic space. The specific steps are as follows: S1: First, perform lexical analysis on the code statements, extract the operator identifier sequence and operand identifier sequence in the code statements, and identify the memory address operator and object reference operator in the operator identifier sequence; S2: Further input the operator identifier sequence and operand identifier sequence into the encoder layer of the cross-language semantic mapping model; S3: Further, based on the underlying physical mapping relationship between memory address operators and object reference operators, construct an attention bias matrix for a specific language, and calculate the local attention weights of the operator identifier sequence and operand identifier sequence by combining the attention bias matrix with the self-attention mechanism in the encoder layer. S4: Further, the hidden state tensors of the encoder layer are weighted and summed according to the local attention weights to output the initial feature vectors of the nodes with fixed dimensions.

[0026] In step four, the graph convolutional layer in the graph neural network model aggregates the initial feature vectors of adjacent nodes based on the edge relationships between nodes to obtain the updated node structure feature vector. The specific steps are as follows: S1: First, extract the first node initial feature vector of the central node in the first initialization attribute graph, and the second node initial feature vector of the neighboring nodes connected to the central node, and identify the original syntax type label of the connection edge between the central node and the neighboring nodes. S2: Further construct a dual-channel convolutional computation path. In the first convolutional channel, use a local topological weight matrix that matches the original syntax type label to perform a linear transformation on the initial feature vector of the second node. In the second convolutional channel, use a global cross-language semantic weight matrix to perform a linear transformation on the initial feature vector of the second node. S3: Further calculate and generate physically controllable gating activation vectors based on the initial feature vectors of the first node and the initial feature vectors of the second node; S4: Further utilize the gated activation vector to perform dynamic feature weighting and fusion on the outputs of the first and second convolutional channels to obtain the node structure feature vector of the central node.

[0027] In step four, the node structure feature vector is reduced in dimension and aggregated using a global pooling layer in the graph neural network model to output a first graph-level feature vector representing the first source code. The specific steps are as follows: S1: First, extract the feature matrix from the output of the graph neural network model, which contains the feature vectors of the node structure; S2: Further perform global average pooling and global max pooling operations on the feature matrix respectively, and output the average eigenvector and the maximum eigenvector; S3: Further, the average feature vector and the maximum feature vector are concatenated and nonlinearly mapped by a multilayer perceptron to output a first graph-level feature vector of fixed length.

[0028] Before the cosine similarity in step five exceeds the preset threshold, the specific steps are as follows: S1: First, count the total number of nodes in the first code attribute graph and the second code attribute graph; S2: Further calculate the average of the total number of nodes as the baseline value for code complexity; S3: Further, based on the code complexity baseline value, call the preset penalty coefficient function to calculate the dynamic compensation coefficient; S4: Further multiply the basic comparison threshold with the dynamic compensation coefficient to generate the judgment threshold specific to the current detection task.

[0029] The specific steps for extracting the first control flow graph and the first data flow graph corresponding to the first source code in step two are as follows: S1: First, the syntax parser is invoked to traverse and parse the first source code; S2: If a syntax error is detected in the first source code that causes the parsing to be interrupted, record the line number of the interruption location; S3: Further extract the code snippets before the line number markers to construct a local control flow graph and a local data flow graph, and add unclosed abnormal connection edges to the broken tail nodes of the local control flow graph; S4: Further, when the graph neural network model performs message passing for unclosed abnormal connection edges, a preset abnormal mask is invoked to block the transmission of the original structural features on the edge, and the global graph structure compensation vector of the previous iteration of the graph neural network model is routed to the broken tail node.

[0030] In step two, the first control flow graph and the first data flow graph are merged to construct the first code attribute graph. Subsequent steps are as follows: S1: First, traverse the subgraph structures in the first code attribute graph and the second code attribute graph; S2: Further identify control flow subgraphs with cyclic characteristics; S3: Further extract the initialization condition nodes, boundary decision nodes, and step size update nodes from the control flow subgraph; S4: Use a preset structural paradigm template to perform topological reorganization on the initialization condition nodes, boundary judgment nodes, and step size update nodes to generate a unified cyclic structure subgraph and replace the original control flow subgraph.

[0031] The training steps for the graph neural network model in step four are as follows: S1: First, obtain positive sample pairs containing semantically equivalent cross-language code snippets, and negative sample pairs containing semantically unrelated cross-language code snippets; S2: Further transform the positive sample pairs and the negative sample pairs into corresponding sample code attribute maps and input them into the initial network model to output sample map-level feature vectors; S3: Further utilize the contrastive loss function to calculate the positive distance between the two sample graph-level feature vectors in the positive sample pair, and the negative distance between the two sample graph-level feature vectors in the negative sample pair; S4: Based on the optimization objective of minimizing the positive distance and maximizing the negative distance, calculate the gradient and update the weight parameters of the initial network model through backpropagation until the loss value converges to obtain the trained graph neural network model.

[0032] The training steps for the cross-language semantic mapping model in step three are as follows: S1: First, obtain a parallel code corpus of multiple programming languages; S2: Further perform random identifier masking on the code fragments in the parallel code corpus to generate masked code sequences; S3: Further input the mask code sequence into the initial semantic encoder and output the predicted identifier probability distribution; S4: Further calculate the cross-entropy loss between the predicted identifier probability distribution and the true identifier; S5: Further update the attention mechanism parameters of the initial semantic encoder based on cross-entropy loss to generate a cross-language semantic mapping model.

[0033] In step one, the first source code of the first programming language to be detected and the second source code of the second programming language are obtained. The first and second source codes cover the target files for which similarity comparison or deduplication needs to be performed. The first and second programming languages ​​are high-level programming languages ​​belonging to different syntax systems; for example, the first programming language is C, a procedural language with low-level memory operations, while the second programming language is Python, an interpreted and dynamically typed language. The acquisition is achieved by calling a low-level file reading interface or pulling a data stream from the version control system's code repository via a network protocol.

[0034] In step two, the first source code and the second source code are parsed separately to extract the first control flow graph and the first data flow graph corresponding to the first source code, as well as the second control flow graph and the second data flow graph corresponding to the second source code. The first control flow graph and the first data flow graph are merged to construct the first code attribute graph, and the second control flow graph and the second data flow graph are merged to construct the second code attribute graph. Specifically, a compiler front-end parsing tool such as Tree-sitter or Joern is used to perform lexical and syntactic analysis on the text-based source code to generate an abstract syntax tree. A control flow graph is constructed based on the execution timing dependencies of nodes in the abstract syntax tree. A data flow graph is constructed based on the propagation trajectory of variable definitions, assignments, and reference states along the execution path. The control flow graph represents the execution logic skeleton of the code, and the data flow graph represents the read and write state transitions of memory variables. By superimposing and fusing the control flow and data flow, the generated code attribute graph possesses a multi-dimensional topological structure that simultaneously describes the program's "control timing" and "data state."

[0035] Step three involves extracting the code statements contained in the nodes of the first and second code attribute graphs; calling a pre-trained cross-language semantic mapping model to process the code statements, generating initial feature vectors for each code statement in a unified semantic space; and concatenating these initial feature vectors to the corresponding nodes in the first and second code attribute graphs to obtain the first and second initialized attribute graphs. The cross-language semantic mapping model is a deep learning network architecture with a bidirectional attention mechanism; the code statements are the actual lines of code or expression text to which each node in the code attribute graph is attached. The model maps discrete strings to continuous floating-point vectors; the unified semantic space represents the pure logical feature dimension after eliminating language grammatical differences. The extracted vectors are then reinjected into the original graph topology nodes, ensuring that the graph structure retains its original execution context while possessing a numerical initial state that can be readily computed by subsequent networks.

[0036] In step four, the first and second initial attribute maps are input into a pre-trained graph neural network model. The graph convolutional layers in the graph neural network model aggregate the initial feature vectors of adjacent nodes based on edge relationships to obtain updated node structure feature vectors. The global pooling layers in the graph neural network model perform dimensionality reduction and aggregation on the node structure feature vectors, outputting a first graph-level feature vector representing the first source code and a second graph-level feature vector representing the second source code. The graph convolutional layers execute a message passing mechanism; during message passing, the network model extracts feature data from the local receptive field of each node based on the graph topology and performs weighted fusion. As the number of graph convolutional layers increases, the node structure feature vectors can capture a wider range of code execution context. The global pooling layers compress and map the node feature tensors of varying numbers in the graph into single-dimensional fixed-length vectors, enabling two code segments of different structural sizes to be converted into numerical matrices of the same dimension, thus supporting the underlying distance calculation.

[0037] In step five, the cosine similarity between the first graph-level feature vector and the second graph-level feature vector is calculated. If the cosine similarity is greater than a preset judgment threshold, the detection result that the first source code and the second source code are similar is output. Cosine similarity is used to measure the angle between the directions of two vectors in high-dimensional space; the smaller the angle, the closer the similarity is to 1. The judgment threshold is a system-preset floating-point numerical limit, for example, set to 0.85. By outputting floating-point judgment results through underlying tensor dot product operations, the semantic misalignment pain point caused by direct comparison of different languages ​​is resolved, and the execution accuracy of the system for large-scale cross-language open-source code tracing is improved.

[0038] Example 1 Extract the first set of nodes and the first set of control flow edges from the first control flow graph, and the second set of nodes and the first set of data flow edges from the first data flow graph. Based on the line numbers of the source code and the node identifiers of the abstract syntax tree, align the first set of nodes and the second set of nodes and merge them to obtain a unified set of nodes. Assign a first type label to the edges in the first control flow edge set and a second type label to the edges in the first data flow edge set. Concatenate the first type labels, the second type labels, and the unified set of nodes to form a first code attribute graph with heterogeneous edge types. Specifically, the first type labels and the second type labels are used to distinguish the physical attributes of connections in the graph data structure. By assigning one-hot encoding category labels to different dependencies in memory, a single topological graph can carry multi-dimensional dependency information, preventing feature aliasing between control dependencies and data dependencies in subsequent convolutional computations.

[0039] Example 2 Lexical analysis is performed on the code statements to extract operator and operand identifier sequences, and memory address operators and object reference operators are identified within the operator identifier sequences. These sequences are then input into the encoder layer of a cross-language semantic mapping model. Based on the underlying physical mapping relationship between memory address and object reference operators, a language-specific attention bias matrix is ​​constructed. The local attention weights of the operator and operand identifier sequences are calculated using a self-attention mechanism in the encoder layer combined with the attention bias matrix. The hidden state tensors of the encoder layer are then weighted and summed based on these local attention weights to output fixed-dimensional initial feature vectors for the nodes. Specifically, the input character stream is deconstructed at the physical mapping level. Memory address operators represent direct memory access instructions to the underlying hardware, such as pointer dereferencing and address-of operators in C; object reference operators represent memory lifecycle management behaviors within high-level languages, such as object reference counting in Python. For the two operational features with significant differences in physical memory management, instead of using equal-weighted black-box mapping, a specific attention bias matrix is ​​constructed to apply physical intervention; the specific feature derivation process of self-attention computation satisfies the mathematical formula: in: This is a query matrix that represents the active addressing characteristics of the current code node; The key matrix represents the passive syntax matching features of the addressed nodes; The value matrix represents the tensor of the actual physical semantic content of the node; This is the feature dimension scaling factor, used to prevent the gradient from vanishing during backpropagation due to an excessively large dot product result. For the attention bias matrix constructed for a specific language, when a pointer dereference operator for C language is identified, the matrix element at the corresponding operator position is given a positive value with a significant gain, forcing the network to give an absolute attention weight response when processing the underlying memory allocation instructions. By injecting this mathematical constraint into the dot product calculation process, the semantic misalignment caused by the inconsistency between pointer and reference mechanisms across languages ​​is accurately overcome, and the physical fidelity when mapping to a unified semantic space is significantly improved.

[0040] Example 3 The process involves extracting the first node initial feature vector of the central node in the first initialization attribute graph, and the second node initial feature vector of the neighboring nodes connected to the central node, identifying the original grammar type labels of the connecting edges between the central node and its neighbors; constructing a dual-channel convolutional computation path; linearly transforming the second node initial feature vector using a local topological weight matrix matching the original grammar type labels in the first convolutional channel, and linearly transforming the second node initial feature vector using a global cross-language semantic weight matrix in the second convolutional channel; calculating and generating physically controllable gated activation vectors based on the first and second node initial feature vectors; and dynamically weighting and fusing the outputs of the first and second convolutional channels using the gated activation vectors to obtain the node structure feature vector of the central node. Specifically, this resolves the underlying contradiction between preserving heterogeneous grammar topology and achieving unified semantic mapping. The first convolutional channel is specifically responsible for capturing the language-specific abstract syntax tree hierarchical dependencies; the second convolutional channel is specifically used to aggregate cross-language global logical semantics in a unified space; the specific generation process of the physically controllable gated activation vectors satisfies the mathematical formula: in: Xtopo is the local topological feature matrix extracted by the first convolutional channel, whose matrix elements contain the parent-child node hierarchical dependency information unique to the abstract syntax tree; Xsemantic is the global logical feature matrix extracted by the second convolutional channel, whose matrix elements represent the pure logical semantic state vector that crosses language barriers; W1 is the learnable linear mapping weight matrix corresponding to the local topological channel, used to dynamically adjust the retention ratio of language-specific syntactic structural features during model training; W2 is the learnable linear mapping weight matrix corresponding to the global semantic channel, used to dynamically adjust the introduction ratio of unified logical features; b is the basic bias term of the network layer, used to provide the baseline feature activation threshold; sigma is the Sigmoid activation function, which, through this non-linear mapping function, strictly compresses the unbounded feature tensor calculated across channels to a floating-point probability range between 0 and 1, generating an activation vector that controls the gating switch. By using this vector to dynamically control the information flow of two independent convolution channels during feature fusion, global semantic constraints are introduced while preserving the language-specific local branch structure. This prevents the loss of key logical features caused by single feature aggregation and significantly enhances the robustness of the network structure for comparison of complex logic across languages.

[0041] Example 4 The system extracts a feature matrix containing node structure feature vectors from the output of the graph neural network model. Global average pooling and global max pooling operations are then performed on the feature matrix to output the average and maximum feature vectors. A multilayer perceptron is used to concatenate and non-linearly map the average and maximum feature vectors, outputting a fixed-length first-level graph feature vector. Global average pooling captures the global contextual semantics of the entire code graph; global max pooling captures the core exception logic or key algorithm nodes that play a decisive role in the graph. Through dual-channel routing concatenation, the system can preserve multi-level topological features to the maximum extent during dimensionality reduction.

[0042] Example 5 When determining the similarity threshold, considering the objective impact of the physical length of the code segment on the distribution of similarity values, dynamic threshold adjustment logic is executed. The total number of nodes in the first and second code attribute graphs is counted; the average of the total number of nodes is calculated as the code complexity baseline; based on the code complexity baseline, a preset penalty coefficient function is called to calculate the dynamic compensation coefficient, and the base comparison threshold is multiplied by the dynamic compensation coefficient to generate the judgment threshold specific to the current detection task; the specific derivation process of the above threshold generation satisfies the mathematical model: Thresholdnew is the decision threshold specific to the current detection task, while Thresholdbase is the basic comparison threshold. When comparing very short code snippets of NavgNbase, a negative penalty term that grows logarithmically is introduced to raise the final decision threshold, strictly preventing the extremely high probability of false positives caused by accidental structural similarity. When comparing very long code snippets, logarithmic compensation is introduced to appropriately relax the decision threshold, thereby effectively reducing engineering defects such as missed detections.

[0043] Example 6 The parser iterates through and parses the first source code. If a syntax error is detected in the first source code, causing the parsing to stop, the line number of the interruption is recorded. The code segment before the line number is extracted to construct a local control flow graph and a local data flow graph. Unclosed anomalous connection edges are added to the broken tail nodes of the local control flow graph. When the graph neural network model passes messages for unclosed anomalous connection edges in subsequent iterations, a preset anomaly mask is invoked to block the transmission of the original structural features on the edge, and the global graph structure compensation vector of the previous iteration of the graph neural network model is routed to the broken tail node. Specifically, the parser defines the anomaly handling routing rules under extreme boundary conditions. Unclosed anomalous connection edges represent dangling graph topology connections caused by syntax truncation; the anomaly mask is a binary control matrix composed of zero elements. When the graph neural network traverses to the fracture region during the message passing phase, it directly calls the anomaly mask multiplied by the transfer matrix of that path, completely blocking the spread of noise features generated by the incomplete graph structure at the physical computation level. Simultaneously, it activates the directed information compensation mechanism, forcibly writing the cached global graph structure compensation vector into the hidden state of the fracture tail node. Through this low-level communication isolation and compensation route reconstruction, the system avoids the collapse of local graph convolution calculations when encountering syntactically incomplete code, ensuring the safe comparison process of the remaining code logic.

[0044] Example 7 To further eliminate friction between language structures, the system traverses the subgraph structures in the first and second code attribute graphs; identifies control flow subgraphs with loop characteristics; extracts initialization condition nodes, boundary decision nodes, and step update nodes from the control flow subgraphs; and uses a preset structural paradigm template to perform topological recombination on the initialization condition nodes, boundary decision nodes, and step update nodes, generating a unified loop structure subgraph and replacing the original control flow subgraph. Specifically, addressing the differences in the underlying implementation of loop logic in different languages, the system performs normalization mapping at the graph structure level, forcibly stripping heterogeneous loops of their language shells and uniformly transforming them into standard initialization, decision, and update triplet topological connections, significantly reducing the fitting difficulty of the graph neural network.

[0045] Example 8 The process involves obtaining positive sample pairs containing semantically equivalent cross-language code snippets and negative sample pairs containing semantically unrelated cross-language code snippets. These positive and negative sample pairs are then transformed into corresponding sample code attribute maps and input into the initial network model, outputting sample map-level feature vectors. A contrastive loss function is used to calculate the positive distance between two sample map-level feature vectors in a positive sample pair and the negative distance between two sample map-level feature vectors in a negative sample pair. Based on the optimization objective of minimizing the positive distance and maximizing the negative distance, gradients are calculated and the weight parameters of the initial network model are updated via backpropagation until the loss value converges, resulting in a trained graph neural network model. The contrastive loss function encourages logically equivalent vectors in high-dimensional space to cluster together, while logically unrelated vectors repel each other. Through the chain rule in the backpropagation algorithm, the loss signal is passed layer by layer back to the relational weight matrix of the graph convolutional layer, achieving a physical reshaping of the internal structural weights.

[0046] Example 9 This paper acquires parallel code corpora from multiple programming languages, performs random identifier masking on code snippets in these corpora, and generates masked code sequences. These masked code sequences are then input into an initial semantic encoder, which outputs a predicted identifier probability distribution. The cross-entropy loss between the predicted identifier probability distribution and the true identifiers is calculated. Based on this cross-entropy loss, the attention mechanism parameters of the initial semantic encoder are updated to generate a cross-language semantic mapping model. The masked code sequences are input tensors that intentionally obscure some key operators or variable names. By forcing the encoder to predict the masked identifiers based on context, the model spontaneously learns deep semantic association rules across different programming languages ​​without manual labeling, enabling the output feature vectors to possess universal representational capabilities across language barriers.

[0047] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cross-language code similarity detection method based on semantic analysis, characterized in that: The method and steps are as follows: Step 1: First, obtain the first source code of the first programming language to be tested and the second source code of the second programming language; Step 2: Further parse the first source code and the second source code in Step 1 respectively, extract the first control flow graph and the first data flow graph corresponding to the first source code, and extract the second control flow graph and the second data flow graph corresponding to the second source code; The first control flow graph and the first data flow graph are merged to construct the first code attribute graph, and the second control flow graph and the second data flow graph are merged to construct the second code attribute graph; Step 3: Further extract the code statements contained in the nodes of the first and second code attribute graphs in Step 2; call the pre-trained cross-language semantic mapping model to process the code statements respectively, and generate the initial feature vector of each code statement in the unified semantic space. The initial feature vectors of the nodes are concatenated to the nodes of the corresponding first code attribute map and second code attribute map, respectively, to obtain the first initial attribute map and the second initial attribute map; Step 4: Further input the first initial attribute map and the second initial attribute map from Step 3 into the pre-trained graph neural network model; aggregate the initial feature vectors of adjacent nodes based on the edge relationships between nodes through the graph convolutional layer in the graph neural network model to obtain the updated node structure feature vector; perform dimensionality reduction and aggregation on the node structure feature vector through the global pooling layer in the graph neural network model to output the first graph-level feature vector representing the first source code and the second graph-level feature vector representing the second source code, respectively. Step 5: Finally, calculate the cosine similarity between the first graph-level feature vector and the second graph-level feature vector in Step 4; if the cosine similarity is greater than the preset judgment threshold, output the detection result that the first source code and the second source code have code similarity.

2. The cross-language code similarity detection method based on semantic analysis according to claim 1, characterized in that: In step two, the first control flow graph and the first data flow graph are merged to construct the first code attribute graph. The specific steps are as follows: S1: First, extract the first set of nodes and the first set of control flow edges in the first control flow graph, and the second set of nodes and the first set of data flow edges in the first data flow graph; S2: Further, based on the line numbers of the source code and the node identifiers of the abstract syntax tree, align the first node set and the second node set, and merge them to obtain a unified node set; S3: Further assign a first type label to the edges in the first control flow edge set, and assign a second type label to the edges in the first data flow edge set; S4: Further concatenate the first type of label, the second type of label, and the unified node set into a first code attribute graph with heterogeneous edge types.

3. The cross-language code similarity detection method based on semantic analysis according to claim 1, characterized in that: In step three, a pre-trained cross-language semantic mapping model is invoked to process the code statements, generating an initial feature vector for each code statement in a unified semantic space. The specific steps are as follows: S1: First, perform lexical analysis on the code statements, extract the operator identifier sequence and operand identifier sequence in the code statements, and identify the memory address operator and object reference operator in the operator identifier sequence; S2: Further input the operator identifier sequence and operand identifier sequence into the encoder layer of the cross-language semantic mapping model; S3: Further, based on the underlying physical mapping relationship between memory address operators and object reference operators, construct an attention bias matrix for a specific language, and calculate the local attention weights of the operator identifier sequence and operand identifier sequence by combining the attention bias matrix with the self-attention mechanism in the encoder layer. S4: Further, the hidden state tensors of the encoder layer are weighted and summed according to the local attention weights to output the initial feature vectors of the nodes with fixed dimensions.

4. The cross-language code similarity detection method based on semantic analysis according to claim 1, characterized in that: In step four, the graph convolutional layer in the graph neural network model aggregates the initial feature vectors of adjacent nodes based on the edge relationships between nodes to obtain the updated node structure feature vector. The specific steps are as follows: S1: First, extract the first node initial feature vector of the central node in the first initialization attribute graph, and the second node initial feature vector of the neighboring nodes connected to the central node, and identify the original syntax type label of the connection edge between the central node and the neighboring nodes. S2: Further construct a dual-channel convolutional computation path. In the first convolutional channel, use a local topological weight matrix that matches the original syntax type label to perform a linear transformation on the initial feature vector of the second node. In the second convolutional channel, use a global cross-language semantic weight matrix to perform a linear transformation on the initial feature vector of the second node. S3: Further calculate and generate physically controllable gating activation vectors based on the initial feature vectors of the first node and the initial feature vectors of the second node; S4: Further utilize the gated activation vector to perform dynamic feature weighting and fusion on the outputs of the first and second convolutional channels to obtain the node structure feature vector of the central node.

5. The cross-language code similarity detection method based on semantic analysis according to claim 1, characterized in that: In step four, the node structure feature vector is reduced in dimension and aggregated using a global pooling layer in the graph neural network model to output a first graph-level feature vector representing the first source code. The specific steps are as follows: S1: First, extract the feature matrix from the output of the graph neural network model, which contains the feature vectors of the node structure; S2: Further perform global average pooling and global max pooling operations on the feature matrix respectively, and output the average eigenvector and the maximum eigenvector; S3: Further, the average feature vector and the maximum feature vector are concatenated and nonlinearly mapped by a multilayer perceptron to output a first graph-level feature vector of fixed length.

6. The cross-language code similarity detection method based on semantic analysis according to claim 1, characterized in that: Before the cosine similarity in step five exceeds the preset threshold, the specific steps are as follows: S1: First, count the total number of nodes in the first code attribute graph and the second code attribute graph; S2: Further calculate the average of the total number of nodes as the baseline value for code complexity; S3: Further, based on the code complexity baseline value, call the preset penalty coefficient function to calculate the dynamic compensation coefficient; S4: Further multiply the basic comparison threshold with the dynamic compensation coefficient to generate the judgment threshold specific to the current detection task.

7. The cross-language code similarity detection method based on semantic analysis according to claim 1, characterized in that: The specific steps for extracting the first control flow graph and the first data flow graph corresponding to the first source code in step two are as follows: S1: First, the syntax parser is invoked to traverse and parse the first source code; S2: If a syntax error is detected in the first source code that causes the parsing to be interrupted, record the line number of the interruption location; S3: Further extract the code snippets before the line number markers to construct a local control flow graph and a local data flow graph, and add unclosed abnormal connection edges to the broken tail nodes of the local control flow graph; S4: Further, when the graph neural network model performs message passing for unclosed abnormal connection edges, a preset abnormal mask is invoked to block the transmission of the original structural features on the edge, and the global graph structure compensation vector of the previous iteration of the graph neural network model is routed to the broken tail node.

8. The cross-language code similarity detection method based on semantic analysis according to claim 1, characterized in that: In step two, the first control flow graph and the first data flow graph are merged to construct the first code attribute graph. Subsequent steps are as follows: S1: First, traverse the subgraph structures in the first code attribute graph and the second code attribute graph; S2: Further identify control flow subgraphs with cyclic characteristics; S3: Further extract the initialization condition nodes, boundary decision nodes, and step size update nodes from the control flow subgraph; S4: Use a preset structural paradigm template to perform topological reorganization on the initialization condition nodes, boundary judgment nodes, and step size update nodes to generate a unified cyclic structure subgraph and replace the original control flow subgraph.

9. The cross-language code similarity detection method based on semantic analysis according to claim 1, characterized in that: The training steps for the graph neural network model in step four are as follows: S1: First, obtain positive sample pairs containing semantically equivalent cross-language code snippets, and negative sample pairs containing semantically unrelated cross-language code snippets; S2: Further transform the positive sample pairs and the negative sample pairs into corresponding sample code attribute maps and input them into the initial network model to output sample map-level feature vectors; S3: Further utilize the contrastive loss function to calculate the positive distance between the two sample graph-level feature vectors in the positive sample pair, and the negative distance between the two sample graph-level feature vectors in the negative sample pair; S4: Based on the optimization objective of minimizing the positive distance and maximizing the negative distance, calculate the gradient and update the weight parameters of the initial network model through backpropagation until the loss value converges to obtain the trained graph neural network model.

10. The cross-language code similarity detection method based on semantic analysis according to claim 1, characterized in that: The training steps for the cross-language semantic mapping model in step three are as follows: S1: First, obtain a parallel code corpus of multiple programming languages; S2: Further perform random identifier masking on the code fragments in the parallel code corpus to generate masked code sequences; S3: Further input the mask code sequence into the initial semantic encoder and output the predicted identifier probability distribution; S4: Further calculate the cross-entropy loss between the predicted identifier probability distribution and the true identifier; S5: Further update the attention mechanism parameters of the initial semantic encoder based on cross-entropy loss to generate a cross-language semantic mapping model.

Images