Abnormal code detection method and device, medium and program product
By integrating the attribute, structural, and semantic feature vectors of multilingual code text, abnormal code can be detected automatically, solving the problems of low detection accuracy and efficiency in traditional methods and achieving efficient multilingual code text detection.
Patent Information
- Application Number
- CN202511489791.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-17
- Publication Date
- 2026-01-23
AI Technical Summary
Traditional machine learning methods have poor compatibility in anomaly detection of multilingual code text, requiring manual writing of grammar rules, resulting in low detection accuracy and efficiency.
By acquiring the attribute, structural, and semantic feature vectors of multilingual code text, and performing fusion processing, features are automatically extracted, replacing manual feature design.
It enables automated detection of multilingual code text, improves the accuracy and efficiency of abnormal code detection, and reduces the need for multilingual adaptation.
Smart Images
Figure CN121387697A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of code detection technology, and in particular to a method, device, medium, and program product for detecting abnormal code. Background Technology
[0002] Abnormal code in a software program may cause the program to malfunction; even if the program can run normally, the abnormal code it contains can be easily exploited by attackers to harm the program provider or user.
[0003] Traditional machine learning methods rely on manually designed features for anomaly detection. They are suitable for anomaly detection in code text of a single programming language, but have poor compatibility with code text of multiple programming languages. Technicians need to pre-write syntax rules to adapt to multiple programming languages, which leads to low accuracy and low efficiency in anomaly detection of multilingual code text. Summary of the Invention
[0004] This invention provides a method, device, medium, and program product for detecting abnormal code, so as to improve the accuracy and efficiency of anomaly detection for multilingual code text.
[0005] In a first aspect, embodiments of the present invention provide a method for detecting abnormal codes, including:
[0006] Obtain code text and attribute data of the code text; the code text includes code snippets in at least two programming languages; the attribute data of the code text includes metadata for each of the code snippets;
[0007] Feature extraction is performed on the attribute data of the code text to obtain the attribute feature vector of the code text;
[0008] Extract the structural feature vector of the code text, and extract the semantic feature vector of the code text;
[0009] The structural feature vector, the semantic feature vector, and the attribute feature vector are fused to obtain a fused feature vector.
[0010] Based on the fused feature vector, determine whether there is abnormal code in the code text.
[0011] Secondly, embodiments of the present invention also provide an abnormal code detection device, comprising:
[0012] An acquisition module is used to acquire code text and attribute data of the code text; the code text includes code fragments in at least two programming languages; the attribute data of the code text includes metadata of each code fragment;
[0013] The first extraction module is used to extract features from the attribute data of the code text to obtain the attribute features of the code text;
[0014] The second extraction module is used to extract the structural features of the code text and the semantic features of the code text;
[0015] The fusion module is used to fuse the structural features, the semantic features, and the attribute features to obtain fused features;
[0016] The determination module is used to determine whether there is abnormal code in the code text based on the fusion features.
[0017] Thirdly, embodiments of the present invention also provide an electronic device, comprising:
[0018] At least one processor; and
[0019] A memory that is communicatively connected to at least one processor; wherein
[0020] The memory stores instructions that can be executed by at least one processor, which enables the at least one processor to perform the abnormal code detection method provided in any embodiment of the present invention.
[0021] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method for detecting abnormal code according to any embodiment of the present invention.
[0022] Fifthly, embodiments of the present invention also provide a computer program product, characterized in that the computer program product includes a computer program, which, when executed by a processor, implements the method for detecting abnormal code in any embodiment of the present invention.
[0023] This invention achieves automated detection of multilingual code text by acquiring code text and its attribute data; the code text includes code fragments in at least two programming languages; the attribute data of the code text includes metadata for each code fragment; feature extraction is performed on the attribute data of the code text to obtain attribute feature vectors; structural feature vectors and semantic feature vectors of the code text are extracted; the structural feature vectors, semantic feature vectors, and attribute feature vectors are fused to obtain a fused feature vector; and the presence of abnormal code in the code text is determined based on the fused feature vector. This enables the parsing and feature extraction of multilingual code text, thereby achieving automated detection of multilingual code text. Compared to existing technologies, it eliminates the need for technicians to write and adapt syntax rules for multiple programming languages; the automated feature extraction process replaces the manual feature design in existing technologies, improving the accuracy and efficiency of abnormal code detection in multilingual code text.
[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0026] Figure 1 This is a flowchart of an abnormal code detection method provided according to Embodiment 1 of the present invention;
[0027] Figure 2 This is a flowchart of an abnormal code detection method provided according to Embodiment 2 of the present invention;
[0028] Figure 3 This is a schematic diagram of the structure of an abnormal code detection device provided in Embodiment 3 of the present invention;
[0029] Figure 4 This is a structural diagram of an electronic device that implements an abnormal code detection method according to an embodiment of the present invention. Detailed Implementation
[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0031] It should be noted that the terms "first" and "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0032] In the technical solutions of the embodiments of the present invention, the acquisition, storage, and application of code text and attribute data of code text all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0033] Example 1
[0034] Figure 1 This is a flowchart of an abnormal code detection method provided in Embodiment 1 of the present invention. This embodiment is applicable to the detection of abnormal codes in multi-programming language code text. The method can be executed by an abnormal code detection device, which can be implemented in hardware and / or software and specifically configured in an electronic device, such as a mobile device.
[0035] See Figure 1 The method for detecting the exception code shown includes:
[0036] S101. Obtain code text and attribute data of the code text; the code text includes code fragments in at least two programming languages; the attribute data of the code text includes metadata of each code fragment.
[0037] S102. Extract features from the attribute data of the code text to obtain the attribute feature vector of the code text.
[0038] S103. Extract the structural feature vector of the code text and extract the semantic feature vector of the code text.
[0039] S104. The structural feature vector, the semantic feature vector, and the attribute feature vector are fused to obtain a fused feature vector.
[0040] S105. Based on the fused feature vector, determine whether there is abnormal code in the code text.
[0041] In this embodiment, the code text includes at least two code snippets; each code snippet is composed of a programming language; the code text contains at least two code snippets, and the two code snippets are programmed in different languages; for example, the code text may include four code snippets: code snippet A, code snippet B, code snippet C, and code snippet D. The programming language of code snippet A may be Rust; the programming language of code snippet B may be TypeScript; the programming language of code snippet C may be Python; the programming language of code snippet D may be Python. Metadata may be parameters of a single code snippet; metadata may include, but is not limited to, at least one of the following: programming language type, type of each variable, and number of parameters in each function.
[0042] Exceptional code can include, but is not limited to, at least one of the following: vulnerable code, low-performance code, and error code; low-performance code can be code with low execution efficiency; error code can be code with syntax errors.
[0043] Attribute feature vectors can be used to represent the metadata features of code text; structural feature vectors can be used to represent the syntactic structure features of code text; and semantic feature vectors can be used to represent the semantic features of code text. The fused feature vector is the result of fusing the structural feature vector, the semantic feature vector, and the attribute feature vector.
[0044] Specifically, the process involves: acquiring code text and its attribute data; normalizing the code parameters of each code segment for each type of code parameter in the code text attribute data; concatenating the normalized code parameters of each code segment according to a preset parameter order to obtain the parameter concatenation result of the code segment; concatenating the parameter concatenation results of each code segment according to their order in the code text to obtain an attribute feature vector; extracting features from the attribute data of the code text to obtain the attribute feature vector of the code text; extracting the structural feature vector of the code text, and extracting... The semantic feature vector of the code text; the structural feature vector, the semantic feature vector, and the attribute feature vector are fused to obtain a fused feature vector; an anomaly probability of the code text is output based on the fused feature vector using a pre-trained classifier; if the anomaly probability of the code text is greater than a preset threshold, it is determined that there is abnormal code in the code text; otherwise, it is determined that there is no abnormal code in the code text; the classifier can be pre-trained using the fused feature vector and labeled labels of at least one sample code text; the labeled labels are manually labeled and used to characterize whether there is abnormal code in the sample code text.
[0045] Optionally, fusing the structural feature vector, the semantic feature vector, and the attribute feature vector to obtain a fused feature vector includes: normalizing the structural feature vector, the semantic feature vector, and the attribute feature vector respectively to obtain normalized structural feature vectors, normalized semantic feature vectors, and normalized attribute feature vectors; mapping the normalized structural feature vectors, the normalized semantic feature vectors, and the normalized attribute feature vectors respectively to obtain mapped structural feature vectors, mapped semantic feature vectors, and mapped attribute feature vectors; the mapped structural feature vectors, the mapped semantic feature vectors, and the mapped attribute feature vectors have the same number of dimensions; performing weighted fusion on the mapped structural feature vectors, the mapped semantic feature vectors, and the mapped attribute feature vectors to obtain a weighted feature vector; and performing residual processing on the weighted feature vectors to obtain the fused feature vector.
[0046] Specifically, L2 normalization is performed on the structural feature vector, semantic feature vector, and attribute feature vector respectively to eliminate the dimensional differences among them, ensuring that their magnitudes are all equal to 1. Feature dimension mapping is then performed on the normalized structural feature vector, normalized semantic feature vector, and normalized attribute feature vector respectively to obtain mapped structural feature vectors, mapped semantic feature vectors, and mapped attribute feature vectors with the same number of dimensions. In one specific implementation, two fully connected layers and a LeakyReLU activation function can be used to map the structural feature vector to obtain a mapped structural feature vector. A linear transformation layer and an LN layer (Layer Normalization) are used to map the semantic feature vector to obtain a mapped semantic feature vector, preserving the semantic distribution characteristics of the semantic features. Finally, a fully connected layer and a GELU (Gaussian Error Linear Unit) activation function are used to map the attribute feature vector to obtain a mapped attribute feature vector, enhancing the nonlinear expression of the attribute features.
[0047] The mapping structure feature vector, mapping semantic feature vector, and mapping attribute feature vector are concatenated along their dimensional directions to obtain a concatenated feature vector. This concatenated feature vector is then transformed using three different learnable parameter matrices to obtain a query vector, a key vector, and a value vector. Based on the query vector and key vector, a 3x3 attention score matrix is calculated. The three elements in the 0th row of the attention score matrix represent, respectively, the degree of association between the mapping structure feature vector and itself, the strength of association between the mapping structure feature vector and the mapping semantic feature vector, and the strength of association between the mapping structure feature vector and the mapping attribute feature vector. The three elements in the 1st row of the attention score matrix represent, respectively, the degree of association between the mapping semantic feature vector and the mapping structure feature vector, the strength of association between the mapping semantic feature vector and itself, and the strength of association between the mapping semantic feature vector and the mapping attribute feature vector. The three elements in the 2nd row of the attention score matrix represent, respectively, the degree of association between the mapping attribute feature vector and the mapping structure feature vector, the strength of association between the mapping attribute feature vector and the mapping semantic feature vector, and the strength of association between the mapping attribute feature vector and itself.
[0048] The concatenated feature vectors are processed using a gating mechanism to generate a first gating coefficient corresponding to the mapping structure feature vector, a second gating coefficient corresponding to the mapping semantic feature vector, and a third gating coefficient corresponding to the mapping attribute feature vector. The cumulative values of each row element in the attention score matrix are calculated. The cumulative value of the element in row 0 is determined as the total association strength of the mapping structure feature vector; the cumulative value of the element in row 1 is determined as the total association strength of the mapping semantic feature vector; the cumulative value of the element in row 2 is determined as the total association strength of the mapping attribute feature vector; the first gating coefficient is multiplied by the total association strength of the mapping structure feature vector to obtain the weight of the mapping structure feature vector; the second gating coefficient is multiplied by the total association strength of the mapping semantic feature vector to obtain the weight of the mapping semantic feature vector; and the third gating coefficient is multiplied by the total association strength of the mapping attribute feature vector to obtain the weight of the mapping attribute feature vector.
[0049] Based on the weights of the mapping structure feature vector, the mapping semantic feature vector, and the mapping junction attribute feature vector, the mapping structure feature vector, the mapping semantic feature vector, and the mapping junction attribute feature vector are weighted and fused to obtain a weighted feature vector.
[0050] For example, the weighted feature vector can be determined using the following formula:
[0051] ;
[0052] in, Represents the weighted eigenvector; The weights represent the feature vectors of the mapping structure; Represents the feature vector of the mapping structure; The weights represent the feature vectors of the mapping structure; Represents the feature vector of the mapping structure; Represents the feature vector of the mapping node attribute; This represents the feature vector of the mapping node attribute.
[0053] The weighted feature vector is processed across layers using a gated attention unit to obtain the first residual processing result; for example, the first residual processing result can be determined using the following formula:
[0054] ;
[0055] in, Represents the weighted eigenvector; This represents the processing result of the gated attention unit on the weighted feature vector; This indicates the result of the first residual processing;
[0056] The result of the first residual processing is added to the residuals of the mapping structure feature vector, the mapping semantic feature vector, and the mapping attribute feature vector to obtain the fused feature vector; for example, the fused feature vector can be obtained using the following formula:
[0057] ;
[0058] in, Represents the fused feature vector; This represents the residual coefficient.
[0059] Understandably, by employing the above technical solution, the weights of each feature vector are obtained by calculating the fractional attention matrix and gating coefficients, thus avoiding the problem of poor accuracy in the weighted feature representation caused by fixed weights. This allows the weighted feature vectors to accurately reflect the core characteristics of different types of code. By performing residual processing on the weighted feature vectors, the vanishing gradient phenomenon during training is alleviated, improving training efficiency. By adding the result of the first residual processing to the residuals of the mapped structural feature vector, the mapped semantic feature vector, and the mapped attribute feature vector, a fused feature vector is obtained. This fused feature vector can simultaneously represent the correlation features between structure, semantics, and attributes in the code text, as well as the syntactic and statistical characteristics of the code structure and attributes in the code text. This avoids feature bias caused by information loss and improves the feature representation accuracy of the final fused feature vector.
[0060] This invention achieves automated detection of multilingual code text by acquiring code text and its attribute data; the code text includes code fragments in at least two programming languages; the attribute data of the code text includes metadata for each code fragment; feature extraction is performed on the attribute data of the code text to obtain attribute feature vectors; structural feature vectors and semantic feature vectors of the code text are extracted; the structural feature vectors, semantic feature vectors, and attribute feature vectors are fused to obtain a fused feature vector; and the presence of abnormal code in the code text is determined based on the fused feature vector. This enables the parsing and feature extraction of multilingual code text, thereby achieving automated detection of multilingual code text. Compared to existing technologies, it eliminates the need for technicians to write and adapt syntax rules for multiple programming languages; the automated feature extraction process replaces the manual feature design in existing technologies, improving the accuracy and efficiency of abnormal code detection in multilingual code text.
[0061] Example 2
[0062] Figure 2This is a flowchart of an abnormal code detection method provided in Embodiment 2 of the present invention. Based on the technical solution of the above embodiments, the present invention optimizes and improves the determination operation of structural feature vector and semantic feature vector.
[0063] Furthermore, the process of "extracting the structural feature vector of the code text and extracting the semantic feature vector of the code text" is refined to "converting the code text into a token sequence; generating a first abstract syntax tree of the code text; identifying structural boundary syntax units in the abstract syntax tree and determining the boundary type of each structural boundary syntax unit; the structural boundary syntax unit includes at least one of function definition words, class declaration words, and control syntax words; the boundary type includes start boundary and end boundary; for each structural boundary syntax unit, determining the structural label of the structural boundary syntax unit according to the boundary type of the structural boundary syntax unit; inserting the structural labels of each structural boundary syntax unit into the token sequence to obtain a structural token sequence; extracting the structural feature vector of the code text according to the structural token sequence and the code text; and extracting the semantic feature vector of the code text according to the token sequence," thus improving the determination operations of the structural feature vector and the semantic feature vector.
[0064] It should be noted that for any parts not described in detail in the embodiments of the present invention, please refer to the description in the foregoing embodiments.
[0065] See Figure 2 The methods for detecting the exception codes shown include:
[0066] S201. Obtain code text and attribute data of the code text; the code text includes code fragments in at least two programming languages; the attribute data of the code text includes metadata of each code fragment.
[0067] S202. Extract features from the attribute data of the code text to obtain the attribute feature vector of the code text.
[0068] S203. Convert the code text into a word sequence.
[0069] S204. Generate the first abstract syntax tree of the code text.
[0070] S205. Identify structural boundary grammar units in the abstract syntax tree, and determine the boundary type of each structural boundary grammar unit; the structural boundary grammar unit includes at least one of function definition words, class declaration words, and control grammar words; the boundary type includes start boundary and end boundary.
[0071] S206. For each structural boundary syntax unit, determine the structural label of the structural boundary syntax unit according to the boundary type of the structural boundary syntax unit.
[0072] S207. Insert the structural labels of each of the structural boundary grammar units into the lexical sequence to obtain the structural lexical sequence.
[0073] S208. Extract the structural feature vector of the code text based on the structural lexical sequence and the code text.
[0074] S209. Extract the semantic feature vector of the code text based on the word sequence.
[0075] S210. The structural feature vector, the semantic feature vector, and the attribute feature vector are fused to obtain a fused feature vector.
[0076] S211. Based on the fused feature vector, determine whether there is abnormal code in the code text.
[0077] In this embodiment, the lexical sequence includes at least one lexical unit; the first abstract syntax tree can be an abstract syntax tree (AST) generated by processing the code text; the abstract syntax tree can be generated using a code parsing tool. Structural boundary syntax units can be code syntax words used to identify the boundaries between different logical code blocks or different functional code blocks in the code text; structural boundary syntax units can include, but are not limited to, at least one of function definition words, class declaration words, and control syntax words; function definition words can be syntax words used in the code text to define functions; class declaration words can be syntax words used in the code text to declare classes; control syntax words can be code words used in control structures, such as if, else, and while. The start boundary can be a structural boundary syntax unit representing the start position of a code block, and the end boundary can be a structural boundary syntax unit representing the end position of a code block.
[0078] Specifically, the code text is traversed character by character. During this process, specific code characters are identified, and the text portions not identified as these specific code characters are designated as segments to be segmented. These specific code characters can be defined by technical personnel. The Byte Pair Encoding (BPE) algorithm is used to segment the text segments, including: splitting the text segments into multiple individual characters according to the character splitting method; merging adjacent characters based on a pre-trained sub-word frequency table to generate sub-words; querying the lexical units corresponding to each sub-word and the specific code character; and combining the lexical units corresponding to each sub-word and the specific code character according to their order in the code text to obtain a lexical unit sequence.
[0079] Using a code parsing tool, structural boundary grammar units in the abstract syntax tree are identified, and the boundary type of each structural boundary grammar unit is determined. For each structural boundary grammar unit, the first tag corresponding to the structural boundary grammar unit is queried; the second tag corresponding to the boundary type of the structural boundary grammar unit is queried; the first tag and the second tag are combined to obtain the structural tag of the structural boundary grammar unit; for each structural boundary grammar unit, the lexical unit matching the structural boundary grammar unit is determined; the structural tag of each structural boundary grammar unit is inserted into the lexical sequence after the matching lexical unit to obtain the structural lexical sequence.
[0080] In one optional implementation, a pre-trained structural feature vector extraction module is used to extract features from the structural lexical sequence and the code text to obtain a structural feature vector of the code text; a pre-trained semantic feature extraction module is used to extract features from the lexical sequence to obtain a semantic feature vector of the code text.
[0081] Optionally, the step of extracting the structural feature vector of the code text based on the structural lexical sequence and the code text includes: performing multi-head attention-based feature extraction on the structural lexical sequence using a pre-trained feature extraction module to obtain a first feature vector; performing autoregressive decoding on the first feature vector to obtain a second abstract syntax tree; performing feature extraction on the second abstract syntax tree to obtain a first structural feature vector; converting the second abstract syntax tree into a directed grammar graph; the nodes in the directed grammar graph are grammatical units in the second abstract syntax tree; the edges in the directed grammar graph represent the association relationship between two connected nodes; performing feature extraction on the directed grammar graph to obtain a second structural feature vector; and fusing the first structural feature vector and the second structural feature vector to obtain the structural feature vector of the code text.
[0082] The pre-trained feature extraction module can be a pre-trained module used for multi-head attention feature extraction of structural lexical sequences. The second abstract syntax tree is the abstract syntax tree obtained by autoregressive decoding of the first feature vector.
[0083] Specifically, the pre-trained feature extraction module performs multi-head attention-based feature extraction on the structural lexical sequence to obtain a first feature vector. Optionally, during the multi-head attention-based feature extraction process on the structural lexical sequence, the attention score matrix calculated by multiplying the learnable parameter matrix with the attention score matrix of each head can be corrected. While ensuring that the probability distribution characteristics are met, additional suppression is applied to the attention scores across boundaries (such as the attention of lexical terms to class declarations within a function) to reduce cross-structural interference.
[0084] Autoregressive decoding is performed on the first feature vector to obtain the second abstract syntax tree; features are extracted from the second abstract syntax tree using a tree-structured long short-term memory network model to obtain the first structural feature vector.
[0085] Using the grammatical units in the second abstract syntax tree as graph nodes, edges are constructed between the graph nodes according to the association relationships between the grammatical units to obtain a directed grammar graph; wherein, the association relationships may include at least one of the main interaction relationships such as call relationship, inheritance relationship, param relationship, return relationship and contain relationship.
[0086] Feature extraction is performed on the directed graph of the syntax using Graph Convolutional Networks (GCN) to obtain a second structural feature vector; the first structural feature vector and the second structural feature vector are then fused to obtain the structural feature vector of the code text.
[0087] It is understandable that by adopting the above technical solution, the pre-trained feature extraction module performs multi-head attention-based feature extraction on the structural lexical sequence to obtain a first feature vector; the first feature vector is then subjected to autoregressive decoding to obtain a second abstract syntax tree; features are extracted from the second abstract syntax tree to obtain a first structural feature vector; the second abstract syntax tree is converted into a directed grammar graph; features are extracted from the directed grammar graph to obtain a second structural feature vector; and the first and second structural feature vectors are fused to obtain the structural feature vector of the code text, thereby improving the richness and accuracy of the feature information of the structural feature vector.
[0088] Optionally, the pre-trained feature extraction module is pre-trained as follows: acquiring sample code text; the sample code text includes sample code fragments in at least two programming languages; acquiring annotation information for each sample code fragment; the annotation information describes the function of the corresponding sample code fragment; constructing at least one positive sample pair and at least one negative sample pair based on the annotation information of each sample code fragment; the positive sample pair includes two sample code fragments with the same annotation information but different programming languages; the negative sample pair includes two sample code fragments with different annotation information but the same programming language; using an initial feature extraction module, performing feature extraction on the sample code fragments in each positive sample pair and each negative sample pair, respectively, to obtain feature extraction results for the sample code fragments in each positive sample pair and each negative sample pair; training the initial feature extraction module with the objective of maximizing the similarity between the feature extraction results of the two sample code fragments in each positive sample pair and minimizing the similarity between the feature extraction results of the two sample code fragments in each negative sample pair, to obtain a pre-trained feature extraction module.
[0089] The initial feature extraction module can be an initialized, untrained feature extraction module. It is understood that by employing the above technical solution, the feature extraction module can be pre-trained using sample code text from multiple languages, optimizing the feature vector space alignment of the code text from multiple programming languages.
[0090] Optionally, a LoRA adapter is inserted after the Q matrix (query matrix) and K matrix (key matrix) of all multi-head attention modules in the feature extraction module, but not after the V matrix (value matrix) to reduce the number of parameters. The LoRA adapter adopts the structure of input dimension → low-rank matrix → activation function → low-rank matrix B. The initial feature extraction module is trained by freezing 90% of the parameters and retaining only the adapter, layer normalization and bias terms to reduce the total number of parameters and improve the pre-training speed.
[0091] Optionally, the step of extracting the semantic feature vector of the code text based on the lexical sequence includes: identifying semantic lexical units in the lexical sequence; the semantic lexical units include at least one of variable lexical units, function lexical units, and control semantic lexical units; determining the semantic type label of each semantic lexical unit for each semantic lexical unit; inserting the semantic type labels of each semantic lexical unit into the lexical sequence to obtain a semantic lexical unit sequence; and performing feature extraction on the semantic lexical unit sequence through a pre-trained semantic feature extraction module to obtain the semantic feature vector of the code text.
[0092] The control semantic units can be, for example, units in control statements such as if, else, and while. Specifically, lexical analysis is performed on the code text to identify semantic units in the unit sequence; for each semantic unit, a semantic type label is determined based on its type; for example, the text of the semantic unit and the type text of the semantic unit are concatenated to form the semantic type label of the semantic unit; the semantic type labels of each semantic unit are inserted into the unit sequence, after the corresponding semantic unit, to obtain the semantic unit sequence; through a pre-trained semantic feature extraction module, features are extracted from the semantic unit sequence to obtain the semantic feature vector of the code text.
[0093] Optionally, the semantic feature extraction module can adopt an encoder-decoder architecture; wherein the encoder includes a 6-layer Transformer encoder with shared weights; the decoder includes a 6-layer code generation decoder and a 6-layer annotation generation decoder; the semantic feature extraction module can be trained as follows: the encoder in the initial semantic feature extraction module extracts features from the semantic word sequence to obtain the feature extraction result; the annotation generation decoder in the decoder processes the feature extraction result and outputs the natural language annotations corresponding to the semantic word sequence;
[0094] The encoder in the initial semantic feature extraction module extracts features from the code comment text to obtain the comment extraction results. The code generation decoder in the decoder processes the comment extraction results and outputs grammatically correct code snippets, thereby constructing a semantic space shared by code and comments and optimizing semantic feature representation.
[0095] The loss functions used to train the initial semantic feature extraction model include code generation loss (cross-entropy) with a weight of 0.25, annotation generation loss (cross-entropy) with a weight of 0.25, and consistency loss with a weight of 0.3. The consistency loss includes the edit distance loss between the reconstructed code and the original code, the BLEU (Bilingual Evaluation Understudy) loss between the reconstructed annotation and the original annotation, and the contrast loss with a weight of 0.2.
[0096] Optionally, in addition to the regular temporal mask, a variable dependency mask can be added to the attention layer of the encoder in the initial semantic feature extraction module to ensure that variables can only be referenced after they are defined, and a control flow mask can be added to ensure that the terms in the conditional branches only focus on the branch conditions; alternatively, the definition of variables can be identified through static analysis, and a variable dependency graph can be constructed using chains (such as the dependency of x on y in x=1→y=x+2) to generate a dependency adjacency matrix, which can then be converted into attention bias terms and added to the self-attention calculation process.
[0097] In one alternative embodiment, knowledge distillation technology can be used to train a large model and transfer the semantic understanding capabilities of the trained large model to the initial semantic feature extraction module, thereby improving the training and inference efficiency of the pre-trained semantic feature extraction module.
[0098] This invention improves the richness and accuracy of structural relationship representations in the extracted structural feature vectors by converting the code text into a lexical sequence; generating a first abstract syntax tree for the code text; identifying structural boundary grammatical units in the abstract syntax tree and determining the boundary type of each structural boundary grammatical unit; determining the structural label of each structural boundary grammatical unit based on its boundary type for each structural boundary grammatical unit; inserting the structural labels of each structural boundary grammatical unit into the lexical sequence to obtain a structural lexical sequence; extracting the structural feature vector of the code text based on the structural lexical sequence; and extracting the semantic feature vector of the code text based on the lexical sequence.
[0099] Example 3
[0100] Figure 3 This is a schematic diagram of an abnormal code detection device provided in Embodiment 3 of the present invention. This embodiment of the present invention is applicable to detecting abnormal codes in multi-programming language code text. The device can execute an abnormal code detection method and can be implemented in hardware and / or software. The device can be configured in an electronic device.
[0101] See Figure 3 The abnormal code detection device shown includes an acquisition module 301, a first extraction module 302, a second extraction module 303, a fusion module 304, and a determination module 305, wherein...
[0102] The acquisition module 301 is used to acquire code text and attribute data of the code text; the code text includes code fragments in at least two programming languages; the attribute data of the code text includes metadata of each code fragment;
[0103] The first extraction module 302 is used to extract features from the attribute data of the code text to obtain the attribute features of the code text;
[0104] The second extraction module 303 is used to extract the structural features of the code text and the semantic features of the code text;
[0105] The fusion module 304 is used to fuse the structural features, the semantic features, and the attribute features to obtain fused features;
[0106] The determination module 305 is used to determine whether there is abnormal code in the code text based on the fusion features.
[0107] In this embodiment of the invention, an acquisition module acquires code text and its attribute data. The code text includes code fragments from at least two programming languages. The attribute data of the code text includes metadata for each code fragment. A first extraction module extracts features from the attribute data of the code text to obtain attribute features of the code text. A second extraction module extracts structural features and semantic features of the code text. A fusion module fuses the structural features, semantic features, and attribute features to obtain fused features. A determination module determines whether abnormal code exists in the code text based on the fused features.
[0108] Optionally, the second feature extraction module includes:
[0109] A conversion unit is used to convert the code text into a sequence of tokens;
[0110] A generation unit is used to generate a first abstract syntax tree for the code text;
[0111] The identification unit is used to identify structural boundary syntax units in the abstract syntax tree and to determine the boundary type of each structural boundary syntax unit; the structural boundary syntax unit includes at least one of function definition words, class declaration words, and control syntax words; the boundary type includes start boundary and end boundary;
[0112] The first determining unit is used to determine the structural label of each structural boundary syntax unit according to the boundary type of the structural boundary syntax unit.
[0113] An insertion unit is used to insert the structural tags of each of the structural boundary syntax units into the lexical sequence to obtain a structural lexical sequence;
[0114] The first extraction unit is used to extract the structural feature vector of the code text based on the structural word sequence and the code text;
[0115] The second extraction unit is used to extract the semantic feature vector of the code text based on the word sequence.
[0116] Optionally, the first extraction unit includes:
[0117] The first extraction subunit is used to perform multi-head attention-based feature extraction on the structured word sequence through a pre-trained feature extraction module to obtain a first feature vector;
[0118] The decoding subunit is used to perform autoregressive decoding on the first feature vector to obtain the second abstract syntax tree;
[0119] The second extraction subunit is used to extract features from the second abstract syntax tree to obtain the first structural feature vector;
[0120] A transformation subunit is used to transform the second abstract syntax tree into a directed syntax graph; the nodes in the directed syntax graph are syntax units in the second abstract syntax tree; the edges in the directed syntax graph represent the association relationship between two connected nodes;
[0121] The third extraction subunit is used to extract features from the directed graph to obtain the second structural feature vector;
[0122] The fusion subunit is used to fuse the first structural feature vector and the second structural feature vector to obtain the structural feature vector of the code text.
[0123] Optionally, the device may also include:
[0124] A sample acquisition module is used to acquire sample code text; the sample code text includes sample code fragments in at least two programming languages;
[0125] The annotation acquisition module is used to acquire annotation information for each sample code segment; the annotation information is used to describe the function of the corresponding sample code segment.
[0126] A sample pair construction module is used to construct at least one positive sample pair and at least one negative sample pair based on the annotation information of each sample code segment; the positive sample pair includes two sample code segments with the same annotation information but different programming languages; the negative sample pair includes two sample code segments with different annotation information but the same programming language.
[0127] The third extraction module is used to extract features from the sample code segments in each positive sample pair and the sample code segments in each negative sample pair through the initial feature extraction module, so as to obtain the feature extraction results of the sample code segments in each positive sample pair and the feature extraction results of the sample code segments in each negative sample pair.
[0128] The pre-training module is used to train the initial feature extraction module with the goal of maximizing the similarity between the feature extraction results of two sample code segments in each positive sample pair and minimizing the similarity between the feature extraction results of two sample code segments in each negative sample pair, so as to obtain a pre-trained feature extraction module.
[0129] Optionally, the second feature extraction unit is specifically used for:
[0130] Identify semantic lexical units in the lexical sequence; the semantic lexical units include at least one of variable lexical units, function lexical units, and control semantic lexical units.
[0131] For each semantic lexical unit, determine the semantic type label for that semantic lexical unit;
[0132] Insert the semantic type label of each semantic word into the word sequence to obtain the semantic word sequence;
[0133] The semantic feature extraction module, which is pre-trained, extracts features from the semantic word sequence to obtain the semantic feature vector of the code text.
[0134] Optionally, the fusion module is specifically used for:
[0135] The structural feature vector, the semantic feature vector, and the attribute feature vector are normalized respectively to obtain normalized structural feature vector, normalized semantic feature vector, and normalized attribute feature vector;
[0136] The normalized structural feature vector, the normalized semantic feature vector, and the normalized attribute feature vector are respectively mapped by feature dimensions to obtain mapped structural feature vector, mapped semantic feature vector, and mapped attribute feature vector; the number of dimensions among the mapped structural feature vector, the mapped semantic feature vector, and the mapped attribute feature vector is the same.
[0137] The mapping structure feature vector, the mapping semantic feature vector, and the mapping attribute feature vector are weighted and fused to obtain a weighted feature vector.
[0138] The weighted feature vector is subjected to residual processing to obtain the fused feature vector.
[0139] The abnormal code detection device provided in this embodiment of the invention can execute the abnormal code detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the abnormal code detection method.
[0140] Example 4
[0141] Figure 4A schematic diagram of an anomaly code detection device 410, which can be used to implement embodiments of the present invention, is shown. The anomaly code detection device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The anomaly code detection device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.
[0142] like Figure 4 As shown, the abnormal code detection device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the abnormal code detection device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.
[0143] Multiple components in the anomaly code detection device 410 are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, optical disk, etc.; and a communication unit 419, such as a network card, modem, wireless transceiver, etc. The communication unit 419 allows the anomaly code detection device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0144] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as methods for detecting abnormal codes.
[0145] In some embodiments, the abnormal code detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed onto the abnormal code detection device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the abnormal code detection method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured to execute the abnormal code detection method by any other suitable means (e.g., by means of firmware).
[0146] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0147] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable exception code detection device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.
[0148] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0149] To provide user interaction, the systems and techniques described herein can be implemented on an anomaly code detection device, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the anomaly code detection device. Other types of devices can also be used to provide user interaction; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0150] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0151] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.
[0152] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0153] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A method of detecting abnormal code, characterized by, The method comprises: acquiring code text and attribute data of the code text; the code text comprises code segments of at least two programming languages; the attribute data of the code text comprises metadata of each code segment; performing feature extraction on the attribute data of the code text to obtain an attribute feature vector of the code text; extracting a structural feature vector of the code text and a semantic feature vector of the code text; fusing the structural feature vector, the semantic feature vector and the attribute feature vector to obtain a fused feature vector; determining whether there is abnormal code in the code text according to the fused feature vector.
2. The method of claim 1, wherein, The extraction of the structural feature vector of the code text and the semantic feature vector of the code text comprises: converting the code text into a token sequence; generating a first abstract syntax tree of the code text; identifying structural boundary syntax units in the abstract syntax tree and determining boundary types of each structural boundary syntax unit; the structural boundary syntax unit comprises at least one of a function definition word, a class declaration word and a control syntax word; the boundary type comprises a start boundary and an end boundary; for each structural boundary syntax unit, determining a structural label of the structural boundary syntax unit according to the boundary type of the structural boundary syntax unit; inserting the structural label of each structural boundary syntax unit into the token sequence to obtain a structural token sequence; extracting a structural feature vector of the code text according to the structural token sequence and the code text; extracting a semantic feature vector of the code text according to the token sequence.
3. The method of claim 2, wherein, The extraction of the structural feature vector of the code text according to the structural token sequence and the code text comprises: performing multi-head attention-based feature extraction on the structural token sequence by a pre-trained feature extraction module to obtain a first feature vector; performing autoregressive decoding on the first feature vector to obtain a second abstract syntax tree; performing feature extraction on the second abstract syntax tree to obtain a first structural feature vector; converting the second abstract syntax tree into a syntax directed graph; a node in the syntax directed graph is a syntax unit in the second abstract syntax tree; an edge in the syntax directed graph represents an association relationship between two connected nodes; performing feature extraction on the syntax directed graph to obtain a second structural feature vector; fusing the first structural feature vector and the second structural feature vector to obtain the structural feature vector of the code text.
4. The method of claim 3, wherein, The pre-trained feature extraction module is obtained by pre-training in the following manner: acquiring sample code text; the sample code text comprises sample code segments of at least two programming languages; acquiring annotation information of each sample code segment; the annotation information is used to describe the function of the corresponding sample code segment; construct at least one positive sample pair and at least one negative sample pair according to annotation information of each of the sample code snippets; the positive sample pair includes two sample code snippets with the same annotation information and different programming languages; the negative sample pair includes two sample code snippets with different annotation information and the same programming language; train the initial feature extraction module according to each of the positive sample pairs and each of the negative sample pairs to obtain the pre-trained feature extraction module.
5. The method of claim 4, wherein, The training of the initial feature extraction module according to each of the positive sample pairs and each of the negative sample pairs to obtain the pre-trained feature extraction module includes: extract features of the sample code snippets in each of the positive sample pairs and the sample code snippets in each of the negative sample pairs by the initial feature extraction module to obtain feature extraction results of the sample code snippets in each of the positive sample pairs and feature extraction results of the sample code snippets in each of the negative sample pairs; train the initial feature extraction module to obtain the pre-trained feature extraction module, with the maximum similarity between the feature extraction results of the two sample code snippets in each of the positive sample pairs and the minimum similarity between the feature extraction results of the two sample code snippets in each of the negative sample pairs as the target.
6. The method of claim 2, wherein, The extracting of the semantic feature vector of the code text according to the word sequence includes: identify semantic words in the word sequence; the semantic words include at least one of variable words, function words and control semantic words; determine a semantic type label of each semantic word; insert the semantic type labels of the semantic words into the word sequence to obtain a semantic word sequence; extract features of the semantic word sequence by a pre-trained semantic feature extraction module to obtain the semantic feature vector of the code text.
7. The method of claim 1, wherein, The fusing of the structural feature vector, the semantic feature vector and the attribute feature vector to obtain a fused feature vector includes: perform normalization processing on the structural feature vector, the semantic feature vector and the attribute feature vector respectively to obtain a normalized structural feature vector, a normalized semantic feature vector and a normalized attribute feature vector; perform feature dimension mapping on the normalized structural feature vector, the normalized semantic feature vector and the normalized attribute feature vector respectively to obtain a mapped structural feature vector, a mapped semantic feature vector and a mapped attribute feature vector; the mapped structural feature vector, the mapped semantic feature vector and the mapped attribute feature vector have the same number of dimensions; perform weighted fusion on the mapped structural feature vector, the mapped semantic feature vector and the mapped attribute feature vector to obtain a weighted feature vector; perform residual processing on the weighted feature vector to obtain the fused feature vector.
8. An electronic device, comprising: The electronic device includes: at least one processor; and a memory connected in communication with the at least one processor; and The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method for detecting abnormal code according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to perform the method for detecting abnormal code according to any one of claims 1-6 when executed.
10. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program implements the method for detecting abnormal code according to any one of claims 1-6 when executed by the processor.