An unannotated code vulnerability detection method and device based on distillation learning

Abstract

This disclosure presents embodiments of a method, apparatus, electronic device, and computer-readable medium for detecting vulnerabilities in uncommented code based on distillation learning. One specific implementation of the method includes: constructing a teacher model and an initial student model; inputting target code-related information into the teacher model, which includes an embedding layer, a multi-layer encoder, a selector, and an interaction module, to obtain teacher feature representation information; determining joint loss information using the teacher feature representation information, the initial student model, and the ground truth labels; training the initial student model to obtain a distilled student model; inputting the target code information into the distilled student model to obtain detection results; generating warning information and sending the warning information to a target terminal for display. This implementation improves the model's ability to detect vulnerabilities in uncommented code, reduces reliance on comment information, and enhances the model's knowledge transfer and generalization capabilities.

Description

Technical Field

[0001] The embodiments of this disclosure relate to the field of computer technology, and more specifically to a method, apparatus, electronic device, and computer-readable medium for detecting vulnerabilities in uncommented code based on distillation learning. Background Technology

[0002] Vulnerability detection is a crucial means of ensuring software system security. With increasing software complexity, automated vulnerability detection technology is becoming increasingly critical in maintaining system stability and mitigating security risks. Existing vulnerability detection technologies are mainly divided into two categories: program analysis-based methods and deep learning-based methods. Program analysis-based methods rely on predefined rules to detect vulnerabilities in code, but they have significant limitations. First, they have poor scalability, proving ineffective when handling complex code logic or cross-file calls, as the rules struggle to cover all vulnerability types, especially those requiring a deep understanding of code semantics. Second, static analysis struggles to handle dynamic behavior, such as complex control flow, data flow, and multi-threaded concurrency issues; its effectiveness gradually diminishes with increasing code complexity. Furthermore, static analysis relies on mechanical rules when processing uncommented code, failing to infer code intent and easily overlooking vulnerabilities requiring contextual understanding. Large Language Models (LLMs) have made significant progress in code generation, completion, and vulnerability detection, especially performing exceptionally well with well-commented code.

[0003] However, these models still suffer from the following technical problems: over-reliance on context: Large language models can understand the semantics and intent of code through annotations in high-quality code, generating more accurate vulnerability detection results. However, when faced with unannotated code, the lack of contextual semantic support limits their reasoning ability, making it difficult for the model to accurately understand complex logic and thus hindering effective vulnerability detection.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure propose a method, apparatus, electronic device, and computer-readable medium for detecting uncommented code vulnerabilities based on distillation learning, in order to solve one or more of the technical problems mentioned in the background section above.

[0007] Firstly, some embodiments of this disclosure provide a method for detecting unannotated code vulnerabilities based on distillation learning, comprising: constructing a teacher model and an initial student model, wherein the input to the teacher model is code-related information, including code, code comment information, and a corresponding abstract syntax tree, and the output is a teacher soft label; the teacher model includes an embedding layer, a multi-layer encoder, a selector, an interaction module, and a classifier; the multi-layer encoder includes multiple intermediate feature extraction layers, wherein the intermediate feature extraction layer is used to extract features from the code, code comment information, and the corresponding abstract syntax tree; and inputting the target code-related information into the embedding layer, multi-layer encoder, and selector included in the teacher model. The system and its interaction module obtain teacher feature representation information, wherein the target code related information includes: a first target code, first target code annotation information, and a corresponding first target abstract syntax tree; using the teacher feature representation information, the initial student model, and the real labels, joint loss information is determined; the initial student model is trained using the joint loss information to obtain a distilled student model; the target code information is input into the distilled student model to obtain a detection result, wherein the target code information includes: a second target code and a corresponding second target abstract syntax tree; in response to the detection result indicating a vulnerability, a warning message is generated and sent to the target terminal for display.

[0008] Secondly, some embodiments of this disclosure provide an annotation-free code vulnerability detection device based on distillation learning, comprising: a construction unit configured to construct a teacher model and an initial student model, wherein the input of the teacher model is code-related information, including code, code comment information, and a corresponding abstract syntax tree, and the output is a teacher soft label; the teacher model includes: an embedding layer, a multi-layer encoder, a selector, an interaction module, and a classifier; the multi-layer encoder includes: multiple intermediate feature extraction layers, wherein the intermediate feature extraction layers are used to extract features from the code, code comment information, and the corresponding abstract syntax tree; and an input unit configured to input target code-related information into the embedding layer, multi-layer encoder, selector, and interaction module included in the teacher model, to obtain... The system includes teacher feature representation information, wherein the target code related information includes: a first target code, first target code annotation information, and a corresponding first target abstract syntax tree; a determination unit configured to determine joint loss information using the teacher feature representation information, the initial student model, and the real labels; a training unit configured to perform distillation training on the initial student model using the joint loss information to obtain a distilled student model; a detection unit configured to input the target code information into the distilled student model to obtain a detection result, wherein the target code information includes: a second target code and a corresponding second target abstract syntax tree; and a display unit configured to generate warning information in response to the detection result indicating a vulnerability, and to send the warning information to the target terminal for display.

[0009] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0010] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0011] The above embodiments of this disclosure have the following beneficial effects: the annotation-free code vulnerability detection method obtained by the distillation learning-based method of some embodiments of this disclosure improves the detection capability of annotation-free code vulnerabilities. Specifically, the reason for the insufficient annotation-free code vulnerability detection capability is that the prior art relies on annotation information to understand the code context. Based on this, the annotation-free code vulnerability detection method based on distillation learning of some embodiments of this disclosure first constructs a teacher model and an initial student model. The input of the teacher model is code-related information, including code, code annotation information, and the corresponding abstract syntax tree. The output is a teacher soft label. The teacher model includes an embedding layer, a multi-layer encoder, a selector, an interaction module, and a classifier. The multi-layer encoder includes multiple intermediate feature extraction layers. The intermediate feature extraction layers are used to extract features from the code, code annotation information, and the corresponding abstract syntax tree. Multimodal input (code, code annotation information, and the corresponding abstract syntax tree) can comprehensively capture the semantic and structural information of the code, avoiding the loss of information from single data. The hierarchical intermediate feature extraction layers abstract from lexical, syntactic, to semantic layers, and can accurately extract multi-granular features. The use of selectors allows for focusing on key features, while the interaction module integrates multi-source information, enhancing the model's ability to represent complex code patterns. Outputting soft labels (confidence probability distributions) instead of hard labels conveys "uncertainty" information, guiding the student model to learn more detailed vulnerability association patterns and improving knowledge transfer efficiency. Then, target code-related information is input into the teacher model's embedding layer, multi-layer encoder, selector, and interaction module to obtain teacher feature representation information. This target code-related information includes: the first target code, first target code annotation information, and the corresponding first target abstract syntax tree. The generated teacher feature representation information is more comprehensive and accurate, effectively conveying the semantic knowledge of complex code and providing high-quality supervision signals for the student model. Next, using the aforementioned teacher feature representation information, the initial student model, and the ground truth labels, joint loss information is determined. The joint loss integrates task supervision (ground truth labels) and knowledge transfer (teacher feature alignment), both constraining the student model to learn real patterns and encouraging it to inherit the teacher's generalization ability, improving the accuracy and generalization of vulnerability detection for unannotated code. Finally, the initial student model is distilled using the aforementioned joint loss information to obtain the distilled student model. Next, the target code information is input into the distilled student model to obtain the detection results. The target code information includes the second target code and its corresponding second target abstract syntax tree. Joint loss integrates task supervision and knowledge transfer. Distillation training allows the student model to inherit the teacher's generalization ability while learning real-world patterns, effectively improving the accuracy and generalization of unannotated code vulnerability detection.Finally, in response to the vulnerability detected above, an early warning message is generated and sent to the target terminal for display. Generating and pushing early warning messages to the terminal immediately upon vulnerability detection ensures users receive timely risk alerts, shortens vulnerability response time, and improves the timeliness and effectiveness of security protection. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a flowchart of some embodiments of the uncommented code vulnerability detection method based on distillation learning according to the present disclosure;

[0014] Figure 2 This is a schematic diagram of the structure of some embodiments of the uncommented code vulnerability detection device based on distillation learning according to the present disclosure;

[0015] Figure 3 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0016] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0017] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0018] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0019] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0020] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0021] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] refer to Figure 1 The diagram illustrates a flow 100 of some embodiments of a method for detecting uncommented code vulnerabilities based on distillation learning according to the present disclosure. This method for detecting uncommented code vulnerabilities based on distillation learning includes the following steps:

[0023] Step 101: Construct the teacher model and the initial student model.

[0024] In some embodiments, the entity performing the classification of recycled materials described above can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster of multiple servers or terminal devices, or as a single server or terminal device. When the computing device is software, it can be installed in the hardware devices listed above. It can be implemented as multiple software programs or software modules to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0025] In other embodiments, the aforementioned execution entity can construct a teacher model and an initial student model. The input to the teacher model is code-related information, including code, code comment information, and the corresponding abstract syntax tree. The output is a teacher soft label. The teacher model includes an embedding layer, a multi-layer encoder, a selector, an interaction module, and a classifier. The multi-layer encoder includes multiple intermediate feature extraction layers, which extract features from the code, code comment information, and the corresponding abstract syntax tree. In distillation learning, the teacher model is typically a pre-trained high-performance model used to guide the training of the student model. Here, the input to the teacher model is code-related information, and the output is a teacher soft label for knowledge distillation. In practice, the teacher model can use the UniXcoder framework. The initial student model can be a lightweight model, including an embedding layer, a multi-layer encoder, and a classifier. It uses the same framework as the teacher model (UniXcoder framework). The multi-layer encoder can be multiple cascaded Transformer encoders. The code can be specific program code. The code comments mentioned above can be descriptions of code functionality, logic, or potential risks in natural language, typically located next to or above lines of code, and serve as important clues to aid in understanding code semantics. The abstract syntax tree mentioned above can be a syntactic structure tree representation of the code, with each node corresponding to a syntactic element (e.g., function definition, variable, and operator), representing a formal representation of code syntax analysis. The teacher soft labels mentioned above can be the confidence-based class probability distribution output by the teacher model. The interaction module mentioned above includes: a multi-head attention layer, a first normalization layer, a forward propagation layer, and a second normalization layer. The interaction module can be a layer in the teacher model that handles the interaction of multimodal features (code, code comments, and the corresponding abstract syntax tree). The classifier mentioned above can be the output layer of the teacher model, mapping the fused features to predefined vulnerability categories (e.g., "buffer overflow," "null pointer reference") and outputting soft labels.

[0026] Step 102: Input the target code information into the embedding layer, multi-layer encoder, selector and interaction module of the teacher model to obtain the teacher feature representation information.

[0027] In some embodiments, the aforementioned execution entity can input target code-related information into the embedding layer, multi-layer encoder, selector, and interaction module of the teacher model to obtain teacher feature representation information. The target code-related information includes: a first target code, first target code comment information, and a corresponding first target abstract syntax tree. In practice, the first target code can be specific program text (e.g., C language code, C++ code, Python code). The first target code comment information can be a natural language description of the code's function. For example, " / / Simply copy data without checking input length, which may lead to overflow." In practice, the first target abstract syntax tree can be a syntax tree structure of the first target code. For example, the abstract syntax tree of the code may include nodes such as FunctionDef (function definition), VariableDef (variable buf), and CallExpr (strcpy call). The teacher feature representation information can be the final feature generated after processing by the embedding layer, multi-layer encoder, selector, and interaction module. It is a comprehensive representation of the code semantics and vulnerability patterns by the teacher model, used to guide student model training.

[0028] In some optional implementations of certain embodiments, the aforementioned execution entity can input target code-related information into the embedding layer, multi-layer encoder, selector, and interaction module included in the teacher model to obtain teacher feature representation information, which may include the following steps:

[0029] The first step is to extract key information from the target code information through the multi-layer encoder of the teacher model, and obtain the intermediate layer feature representation information set of the first target code annotation information, the intermediate layer feature representation information of the first target code, and the intermediate layer feature representation information of the first target abstract syntax tree. The key information can be the core semantic or structural features that play a decisive role in identifying vulnerabilities in the code vulnerability detection task. In practice, it can be information related to dangerous function calls (such as strcpy, malloc), or information related to the relationship between buffer variables and input variables (such as buf

[10] and src). The intermediate layer feature representation information set can be a multi-granularity, hierarchical vector set generated by the teacher model after extracting features from the first target code annotation information (natural language text) through the intermediate encoder layer. Each vector corresponds to key information (such as lexical, syntactic, semantic) at a certain level in the annotation. The intermediate layer representation information of the first target code can be a multi-granularity, hierarchical vector representation generated by the teacher model after extracting features from the text (such as Python code) corresponding to the first target code through the multi-layer encoder. The intermediate layer feature representation information of the first target abstract syntax tree mentioned above can be a structured hierarchical feature representation generated by the teacher model after extracting features from the first target abstract syntax tree through a multi-layer encoder.

[0030] As an example, the comment information for the first target code above is: / / The user input size parameter is not validated, which may lead to heap overflow. Intermediate features may include: Lexical layer: embedding vector of ["unvalidated", "input", "size parameter"]; Syntactic layer: vector of the modification relationship between "unvalidated" and "size parameter"; Semantic layer: association vector of "unvalidated input -> heap overflow risk".

[0031] The second step involves fusing the intermediate-layer feature representation information of the first target code and the intermediate-layer feature representation information of the first target abstract semantic tree to obtain the fused code feature representation information. This fused code feature representation information can be a multi-dimensional, high-level comprehensive feature representation generated by fusing the intermediate-layer feature representation information of the first target code (such as lexical, syntactic, and semantic features) with the intermediate-layer feature representation information of the first target abstract syntax tree (such as node type, structural relationships, and semantic associations).

[0032] In practice, the feature representation information of the fused code can be obtained using the following formula:

[0033]

[0034] The intermediate layer feature representation information for the first target code output by the Nth (last) layer encoder. This represents the intermediate layer feature representation information of the first target abstract semantic tree output by the Nth (last) layer encoder. The above γ1∈[0,1] represents... The weight. Information for the feature representation of the fused code

[0035] The third step involves performing a nonlinear projection on the fused code feature representation information to obtain the projected code feature representation information. Specifically, the Gelu activation function can be used to perform a nonlinear projection on the fused code feature representation information.

[0036] In practice, the projected feature representation information can be obtained using the following formula:

[0037]

[0038] in, This represents the feature representation information of the code after projection. Gule() represents the nonlinear projection function. W1 and b1 are preset parameters.

[0039] The fourth step involves inputting the intermediate-layer feature representation information set of the aforementioned code comment information into the selector of the teacher model to obtain the selected code comment representation information. This selector can be the feature filtering module within the teacher model, whose core function is to filter out key features strongly relevant to the vulnerability detection task from the intermediate-layer feature representation information set of the code comment information, while filtering out redundant or irrelevant information. The selected code comment representation information can be the aligned representation information (matched to the dimension of the comment feature) after a linear transformation of the intermediate-layer feature representation information of the first target code.

[0040] The fifth step involves using the selected code annotation representation and the projected code feature representation to obtain teacher feature representation information through the teacher model's interaction module. This interaction module includes a multi-head attention layer, a first normalization layer, a forward propagation layer, and a second normalization layer. The multi-head attention layer captures the cross-modal correlation between the selected code annotation representation and the projected code feature representation. The first and second normalization layers can be layers that normalize layer output information. The forward propagation layer can be a layer that performs non-linear transformations on the features. In practice, the selected code annotation representation and the projected code feature representation are input to the interaction module. First, the correlation is captured by multi-head attention. Then, normalization is performed by the first normalization layer. Next, feature extraction is performed through forward propagation. Finally, the teacher feature representation information is generated through the second normalization layer.

[0041] Step 102 and its related content, as an inventive point of this disclosure, solve the technical problem of "difficulty in aligning and fusing the semantics of multimodal code-related information (code, code comment information, and corresponding abstract syntax trees)". However, when performing vulnerability detection on complex code, the following technical problem is often encountered: it reduces the model's adaptability to complex scenarios. The reasons for the above technical problem are often as follows: low-confidence abstract syntax trees (such as missing key nodes) or low-confidence comment information (such as error descriptions) will transmit incorrect associations (such as incorrectly binding "meaningless comments" with "vulnerability"). Therefore, the following solution can be adopted:

[0042] Optionally, before the aforementioned executing entity extracts key information from the first target code related information through the multi-layer encoder of the aforementioned teacher model to obtain the intermediate layer feature representation information set of the first target code annotation information, the intermediate layer feature representation information of the first target code, and the intermediate layer feature representation information of the first target abstract syntax tree, it may include the following steps:

[0043] The first step is to perform subgraph sampling on the first target abstract syntax tree (AST), obtaining a subset of AST subgraphs. These subgraphs can be core substructures extracted from the first target AST, retaining key nodes such as function definitions, variable bindings, and critical function calls, while filtering out redundant control flow or expression nodes. In practice, firstly, the first target AST can be filtered by node type using preset filtering rules, retaining key nodes strongly related to code functionality (e.g., FunctionDef, CallExpr, VariableDef) and filtering out non-core nodes (e.g., IfStmt, ForStmt). Then, based on the filtered key nodes, their child nodes are recursively traversed using depth-first search (DFS), generating multiple local subgraphs (e.g., function definition subgraph, buffer variable definition subgraph, dangerous function call subgraph). Finally, the generated subgraphs are deduplicated and merged to obtain a subset of the AST.

[0044] The second step is to encode the aforementioned abstract syntax tree subgraph set to obtain the graph embedding set of the abstract syntax tree. In practice, a graph neural network can be used to encode the aforementioned abstract syntax tree subgraph set to obtain the graph embedding set of the abstract syntax tree. The graph embedding set of the aforementioned abstract syntax tree can be a 768-dimensional vector.

[0045] The third step involves extracting key phrases from the first target code annotation information to obtain a text summary embedding set. In practice, firstly, a pre-trained model (such as SBERT or RoBERTa) can be used to perform semantic role annotation on the first target code annotation information to identify key actions (such as "copy" and "write") and core entities (such as "buf") and locate the core information in the annotations, thus obtaining the annotation results. Then, the TextRank algorithm is used to extract high-frequency and important keyword phrases (such as "copy to buffer buf") from the annotation results to filter redundant stop words and retain semantically condensed phrases. Finally, the extracted key phrases are input into a pre-trained language model (such as CodeBERT) to generate a text summary embedding set. The dimension of the above text summary embedding is the same as the dimension of the graph embedding of the above abstract syntax tree, which can be 768 dimensions.

[0046] The fourth step involves cross-modal alignment of the graph embedding set and the text summary embedding set to obtain aligned graph embedding sets and aligned text summary embedding sets. In practice, firstly, the cosine similarity between each graph embedding and its corresponding text summary embedding in the graph embedding set can be determined to obtain a cosine similarity set. Then, positive sample pairs (abstract syntax trees and corresponding annotation information of the same code segment) and negative sample pairs (abstract syntax trees and annotation information of different code segments, for example, the abstract syntax tree of code A and the annotation of code B) are constructed using the cosine similarity set. The similarity difference of positive sample pairs is minimized, and the similarity difference of negative sample pairs is maximized, forcing them to align in the semantic space. Finally, the aligned graph embedding set and the aligned text summary embedding set are obtained.

[0047] The fifth step involves evaluating the credibility of the aligned graph embeddings and the aligned text summary embeddings to obtain credibility scores for the graph embeddings and the text summary embeddings. In practice, a lightweight classifier can be used to evaluate the credibility of the aligned graph embeddings and the aligned text summary embeddings to obtain credibility scores for the graph embeddings and the text summary embeddings.

[0048] Step 6: Based on the credibility scores of the graph embedding and the text summary embedding, the relevant information of the first target code is filtered to obtain the filtered first code-related information. In practice, the relevant information of the first target code can be filtered using a preset threshold decision rule based on the credibility scores of the graph embedding and the text summary embedding.

[0049] As an example, the above threshold preset rule could be:

[0050] Set threshold τ g (Graph embedding confidence threshold) = 0.6 and τ t(Text summary credibility threshold) = 0.5, if S g ≥τ g And S t ≥τ t : Retain the complete first-target abstract syntax tree, first-target code comments, and first-target code. If S g <τ g : Only retain the key nodes in the AST (e.g., CallExpr(strcpy)). If S t <τ t : Only retain keywords from comments (e.g., "buf"). Where S g S is the credibility score for graph embeddings. t The credibility score embedded in the above abstract.

[0051] Step 7: Determine the first code-related information after the above filtering as the first code-related information.

[0052] The aforementioned optional steps and related content, as an inventive point of this disclosure, solve the aforementioned technical problem of "model adaptability to complex scenarios." Factors leading to this technical problem often include: low-reliability abstract syntax trees or low-reliability annotation information can transmit incorrect associations. Solving these factors ensures the model's adaptability to complex scenarios. This invention significantly improves the model's ability to understand complex code semantics and enhances its adaptability to complex scenarios by optimizing input quality and enhancing cross-modal alignment.

[0053] Step 103: Determine the joint loss information using teacher feature representation information, the initial student model, and the real labels.

[0054] In some embodiments, the executing entity can utilize the aforementioned teacher feature representation information, the aforementioned initial student model, and the ground truth labels to determine joint loss information. The ground truth labels are the manually annotated results of "whether the code has vulnerabilities" or "vulnerability types" in the aforementioned target code-related information, serving as a supervision signal for model training. The joint loss information can be a combination of multi-dimensional losses, integrating feature alignment loss (first loss information), teacher supervision loss (second loss information), and student self-supervision loss (third loss information).

[0055] In some optional implementations of certain embodiments, the executing entity may utilize the aforementioned teacher feature representation information, the aforementioned initial student model, and the true labels to determine joint loss information, which may include the following steps:

[0056] The first step involves inputting the aforementioned first target code and the aforementioned first target abstract syntax tree into the embedding layer and multi-layer encoder of the initial student model to obtain student feature representation information. The aforementioned student feature representation information can be a multi-dimensional, multi-level comprehensive semantic vector generated by the initial student model after processing the first target code and the first target abstract syntax tree through the embedding layer and multi-layer encoder.

[0057] The second step involves inputting the aforementioned teacher feature representation information into the classifier of the teacher model to generate teacher soft labels. These teacher soft labels can be the vulnerability category probability distribution output by the teacher model after processing the teacher feature representation information through the classifier.

[0058] The third step involves inputting the aforementioned student feature representation information into the classifier of the initial student model to generate soft labels for the students. These soft labels can be the vulnerability category probability distribution output by the initial student model after processing the student feature representation information through the classifier.

[0059] The fourth step involves using the aforementioned teacher and student feature representations to obtain the first loss information through a first loss function. This first loss information can be a loss value that quantifies the alignment between the teacher and student feature representations in the semantic space. Specifically, the first loss function can be the KL divergence loss function. In practice, the first loss information can be obtained using the following formula:

[0060]

[0061] Among them, the above P teacher (i) represents the probability of the i-th class in the teacher's soft label, P student (i) represents the probability of the i-th class in the student soft label. KL This is the first piece of information indicating a loss.

[0062] Fifth, using the aforementioned teacher soft labels and real labels, a second loss information is obtained through a second loss function. This second loss information can be a quantitative indicator of the difference between the teacher soft labels and real labels. Specifically, the second loss function can be the cross-entropy loss function.

[0063] Step 6: Using the aforementioned student soft labels and real labels, obtain the third loss information through the third loss function. This third loss information can be a quantitative indicator of the difference between the student soft labels and real labels. Specifically, the third loss function can be the cross-entropy loss function.

[0064] Step 7: Determine the aforementioned first loss information, second loss information, and third loss information as joint loss information.

[0065] In practice, the joint loss function can be obtained using the following formula:

[0066] L total =α·L KL +β·L RD +γ·L COE ,

[0067] Among them, L total For combined loss information. L KL This is the first piece of information indicating a loss. (L) RD This is the second loss information. COE This is the third loss information. α, β, and γ are the first loss information, the first loss information, and the weight hyperparameters corresponding to the first loss information, respectively (e.g., α = 0.5, β = 0.3, γ = 0.2), which can be adjusted according to requirements.

[0068] The above-described step 103 and its related content, as an inventive point of this disclosure, solve the technical problems of "feature misalignment, soft label bias, and multi-objective conflict in knowledge distillation." However, when performing vulnerability detection on code requiring deep domain knowledge (e.g., industrial-grade code and drug development-related code) and cross-sample task code (e.g., memory management detection migrated from C to Rust), the following technical problems are often encountered: insufficient detection capability for domain-specific code and complex code across sample tasks. The reasons for the above technical problems are often as follows: lack of explicit guidance from domain knowledge, making it difficult for the model to capture deep semantic relationships across samples. Therefore, the following solution can be adopted:

[0069] Optionally, after determining the first loss information, the second loss information, and the third loss information as joint loss information, the executing entity may include the following steps:

[0070] The first step is to extract structured knowledge from the target codebase to construct a code knowledge graph. This target codebase can be a collection of structured or semi-structured code data gathered for a specific task (e.g., code vulnerability detection, intelligent code review, code recommendation). For example, it could be a domain-specific code dataset or an internal enterprise codebase. In practice, firstly, an AST parser (e.g., Clang AST, Python's ast module) can be used to extract key entities (e.g., functionDef, variableDef, API callCallExpr) from the code. Then, NLP tools (e.g., spaCy) extract entities from code comment information. Secondly, based on code structure (e.g., call relationships, inheritance relationships, variable binding) and semantic rules (e.g., "CallExpr(func=strcpy)" associated with "BufferOverflow vulnerability"), relationships between entities are obtained. Finally, a graph database (e.g., Neo4j) or RDF triples are used to store entities (nodes) and relationships (edges), forming a structured code knowledge graph.

[0071] The second step is to map the entities and relations in the aforementioned code knowledge graph to a low-dimensional vector space, obtaining the knowledge embedding matrix. In practice, the knowledge embedding matrix can be obtained through graph neural network encoding. For example, the aforementioned knowledge embedding matrix can be represented as E∈R V×d (V represents the total number of entities, and d represents the embedding dimension).

[0072] The third step is to determine the first similarity score between the teacher feature representation information and the knowledge embedding matrix.

[0073] In practice, the first similarity score can be obtained using the following formula:

[0074]

[0075] Among them, s t This represents the first similarity score. E′ R This represents information indicating teacher characteristics. v Let v represent the v-th entity in the knowledge embedding matrix, where V represents the number of entities.

[0076] The fourth step is to determine the second similarity score between the student feature representation information and the knowledge embedding matrix.

[0077] In practice, the second similarity score can be obtained using the following formula:

[0078]

[0079] Among them, S s This represents the second similarity score. This represents information about student characteristics. v Let V represent the v-th entity in the knowledge embedding matrix, where V represents the number of entities.

[0080] The fifth step is to determine the weights corresponding to the fourth loss information guided by the knowledge graph based on the first similarity score and the second similarity score mentioned above.

[0081] In practice, the weights corresponding to the fourth loss information can be obtained using the following formula:

[0082] λ = max(0, s) t -s s ),

[0083] Where λ represents the weight corresponding to the fourth loss information. t This represents the first similarity score. S This represents the second similarity score. `max()` represents the maximum value function.

[0084] The sixth step is to identify the aforementioned joint loss information and the aforementioned fourth loss information as enhanced joint loss information.

[0085] In practice, the enhanced joint loss information can be obtained using the following formula:

[0086] L total =α·L KL +β·L RD +γ·L COE +λ·L KG ,

[0087] Among them, L total To enhance joint loss information. L KL This is the first piece of information indicating a loss. (L) RD This is the second loss information. COE This is the third type of loss information. L KG This is the fourth loss information, where L KG =λ·(s) t -s s ) 2 α, β, and γ are the first loss information, the first loss information, and the weight hyperparameters corresponding to the first loss information, respectively (e.g., α = 0.5, β = 0.3, γ = 0.2), which can be adjusted according to requirements.

[0088] Step 7: Determine the enhanced joint loss information mentioned above as joint loss information.

[0089] The aforementioned optional steps and related content, as an inventive point of this disclosure, address the aforementioned technical problem of "insufficient vulnerability detection capability for code in a specialized domain and complex code across sample tasks." Factors leading to this technical problem often include: a lack of explicit guidance from domain knowledge, making it difficult for the model to capture deep semantic relationships across samples. Solving these factors ensures improved vulnerability detection capability for code in a specialized domain and complex code across sample tasks. This invention improves the model's ability to detect code in a specialized domain and complex code across sample tasks by structuring domain knowledge and constraining the alignment of model features with knowledge embeddings, thereby forcing the learning of general patterns across samples.

[0090] Step 104: Use the joint loss information to distill the initial student model to obtain the distilled student model.

[0091] In some embodiments, the aforementioned execution entity can utilize the aforementioned joint loss information to perform distillation training on the initial student model to obtain a distilled student model. The distilled student model is a lightweight model optimized through knowledge distillation training. In practice, gradient descent can be chosen as the optimization algorithm for training. Backpropagation is used to calculate the gradient of the joint loss with respect to the parameters of each layer of the student model. The parameters are then adjusted based on the gradient and learning rate to complete the optimization update and obtain the distilled student model.

[0092] Step 105: Input the target code information into the distilled student model to obtain the detection results.

[0093] In some embodiments, the execution entity can input the target code information into the distilled student model to obtain a detection result. The target code information includes a second target code and a corresponding second target abstract syntax tree. The target code information can be specific code data to be detected by the model, including the second target code and the corresponding second target abstract syntax tree. The detection result can be the output after analyzing the input target code information. In practice, the experimental result can be information indicating whether the code has vulnerabilities and the type of vulnerabilities (e.g., "buffer overflow" or "no vulnerabilities").

[0094] In some optional implementations of certain embodiments, the execution entity can input the target code information into the distilled student model to obtain a detection result. The target code information includes a second target code and a corresponding second target abstract syntax tree, and may include the following steps:

[0095] The first step involves inputting the target code information into the embedding layer and multi-layer encoder of the distilled student model to obtain initial feature representations of the target and the initial feature representation of the target syntax tree. The embedding layer can be the input processing layer of the distilled student model, converting the original multiple inputs into low-dimensional continuous vector representations. For example, [0.5(buf), 0.4(strcpy), 0.3(FunctionDef)]. The initial feature representations of the target can be preliminary feature vectors generated after processing the target code information through the embedding layer. For example, [0.6(CallExpr), 0.5(VariableDef(buf)), 0.4(FunctionDef(copy_data))]. The initial feature representations of the target syntax tree can be syntactic structure feature vectors generated after processing the abstract syntax tree of the target code through the embedding layer.

[0096] The second step involves fusing the initial feature representation information of the target and the initial feature representation information of the target syntax tree to obtain the target fused feature representation information. This target fused feature representation information can be a multi-dimensional comprehensive feature vector generated by fusing the initial feature representation information of the target and the initial feature representation information of the target syntax tree. For specific generation steps, please refer to step 102 for the generation of the fused code feature representation information, which will not be repeated here.

[0097] The third step involves inputting the aforementioned target fusion feature information into the classifier of the distilled student model to obtain the detection result. The classifier of the distilled student model can be...

[0098] The output layer of the student model after distillation typically consists of a fully connected layer and a Softmax function. It is responsible for converting the fused feature vector into a vulnerability category probability distribution. After processing the input target fused feature information, the classifier outputs the probability that the code belongs to each vulnerability category. For example, the vulnerability category probability distribution can be [buffer overflow: 0.85, no vulnerability: 0.15], which ultimately indicates whether the code has a vulnerability and its specific type (such as "there is a risk of buffer overflow").

[0099] Step 106: In response to the detection result indicating a vulnerability, generate an early warning message and send the early warning message to the target terminal for display.

[0100] In some embodiments, the aforementioned execution entity may generate early warning information in response to the existence of vulnerabilities in the aforementioned detection results, and send the aforementioned early warning information to the target terminal for display.

[0101] In some optional implementations of certain embodiments, the execution entity may generate a warning message in response to the detection result indicating a vulnerability, and send the warning message to the target terminal for display, which may include the following steps:

[0102] The first step is to extract basic vulnerability information from the above detection results to obtain structured basic information, which includes: vulnerability location information, vulnerability type information, and risk level information. The vulnerability type can be segmented using a threshold (e.g., probability > 0.8) or a classifier threshold to extract the vulnerability type with the highest probability (e.g., "buffer overflow"). The vulnerability location information can be located precisely at the location of the vulnerability using a second-target abstract syntax tree and line number mapping (e.g., line 5 of the function copy_data, strcpy(buf, src)). The risk level information can be determined by predefined risk rules (e.g., "probability > 0.8 is high risk, 0.5-0.8 is medium risk") to classify the risk level (e.g., "high risk").

[0103] The second step involves matching the vulnerability type information with a vulnerability impact analysis template and extracting appropriate suggestions from a remediation suggestion library to generate standardized warning text. In practice, firstly, based on the vulnerability type (e.g., "buffer overflow"), a corresponding template can be called from a predefined template library (e.g., "may lead to stack overflow attacks, overwriting return addresses or critical data"). Then, appropriate suggestions for the vulnerability type are matched from the remediation suggestion library (e.g., "use strncpy instead of strcpy, limit input length to the buffer size"). Finally, the impact analysis and remediation suggestions are integrated into standardized text (e.g., "Vulnerability type: buffer overflow; Impact: may lead to stack overflow attacks; Remediation suggestion: use strncpy(src, buf, 10) instead of strcpy(buf, src)").

[0104] The third step involves supplementing the standardized warning text with contextual information to obtain a rich text warning package. In practice, this is first done by obtaining code snippets before and after the vulnerability location (e.g., the three lines before and after the vulnerability line) from a code repository (e.g., an open-source code knowledge base), and supplementing the function call chain (e.g., main() → copy_data()). Then, vulnerability-related variables (e.g., buf size is 10) and input sources (e.g., src is user input parameters) are marked. Finally, the supplemented contextual information is combined with the standardized text to generate rich text (e.g., Markdown or HTML format) containing hyperlinks and lists.

[0105] The fourth step involves populating the rich text warning package into a preset visualization template and highlighting the code snippets corresponding to the vulnerabilities to obtain visualized warning information. This visualized warning information can be the final output of the code vulnerability detection process. Through structured template rendering and interactive design, abstract vulnerability data (e.g., risk level, code location, impact analysis) is transformed into intuitive and actionable visual content. In practice, rich text content can be populated into a preset visualization template using Jinja2 templates or front-end components to generate a structured page containing a title, risk level color blocks (e.g., red for high risk), an impact analysis module, and a remediation suggestion module. In practice, code highlighting tools (e.g., Prism.js or IDE built-in highlighting) are used to add syntax highlighting (e.g., red background for key functions) to the code snippets corresponding to the vulnerabilities (e.g., strcpy(buf, src)) and add line number markers.

[0106] The fifth step involves matching notification channels according to the aforementioned risk levels and preset notification rules to obtain a notification task queue. In practice, the preset notification rules can be determined by querying the notification rule library based on the risk level to establish the priority of notification channels (e.g., high risk: email (within 10 minutes), SMS (within 30 minutes), WeChat Work (within 1 hour)). In practice, task entries can be generated for each notification channel (e.g., email tasks include recipient, subject, and content; SMS tasks include phone number and content), and task status can be marked (e.g., "pending sending"). The task entries are then stored in a message queue (e.g., RabbitMQ or Redis queue) according to priority and channel type to obtain the notification task queue (structured task list).

[0107] Step 6: Based on the notification task queue described above, send the visualized alert information to the target terminal through the corresponding interface. In practice, the corresponding interface can be called to send the visualized alert information to the target terminal according to the channel type in the notification task queue. Specifically, for email: call the SMTP interface to send an email containing a visualized page link or attachment; for SMS: call the SMS gateway interface to send a short link (e.g., https: / / alert.example.com / 123); for WeChat Work: call the WeChat Work robot API to send a graphic message containing visualized content.

[0108] The annotation-free code vulnerability detection method based on distillation learning, as described in some embodiments of this disclosure, improves the detection capability of annotation-free code vulnerabilities. Specifically, the reason for the insufficient detection capability of annotation-free code vulnerabilities lies in the fact that existing technologies rely on annotation information to understand the code context. Based on this, the annotation-free code vulnerability detection method based on distillation learning, as described in some embodiments of this disclosure, firstly constructs a teacher model and an initial student model. The input of the teacher model is code-related information, including code, code annotation information, and the corresponding abstract syntax tree. The output is a teacher soft label. The teacher model includes an embedding layer, a multi-layer encoder, a selector, an interaction module, and a classifier. The multi-layer encoder includes multiple intermediate feature extraction layers, which are used to extract features from the code, code annotation information, and the corresponding abstract syntax tree. Multimodal input (code, code annotation information, and the corresponding abstract syntax tree) can comprehensively capture the semantic and structural information of the code, avoiding the loss of information from single data. The hierarchical intermediate feature extraction layers abstract from lexical, syntactic, to semantic layers, enabling accurate extraction of multi-granular features. The use of selectors allows for focusing on key features, while the interaction module integrates multi-source information, enhancing the model's ability to represent complex code patterns. Outputting soft labels (confidence probability distributions) instead of hard labels conveys "uncertainty" information, guiding the student model to learn more detailed vulnerability association patterns and improving knowledge transfer efficiency. Then, target code-related information is input into the teacher model's embedding layer, multi-layer encoder, selector, and interaction module to obtain teacher feature representation information. This information includes the first target code, its annotations, and the corresponding first target abstract syntax tree. The generated teacher feature representation information is more comprehensive and accurate, effectively conveying the semantic knowledge of complex code and providing high-quality supervision signals for the student model. Next, using the aforementioned teacher feature representation information, the initial student model, and the ground truth labels, joint loss information is determined. This joint loss integrates task supervision (ground truth labels) and knowledge transfer (teacher feature alignment), both constraining the student model to learn real patterns and encouraging it to inherit the teacher's generalization ability, improving the accuracy and generalization of vulnerability detection for unannotated code. Finally, the initial student model is distilled using the joint loss information to obtain the distilled student model. Next, the target code information is input into the distilled student model to obtain the detection results. The target code information includes the second target code and its corresponding second target abstract syntax tree. Joint loss integrates task supervision and knowledge transfer. Distillation training allows the student model to inherit the teacher's generalization ability while learning real-world patterns, effectively improving the accuracy and generalization of unannotated code vulnerability detection.Finally, in response to the vulnerability detected above, an early warning message is generated and sent to the target terminal for display. Generating and pushing early warning messages to the terminal immediately upon vulnerability detection ensures users receive timely risk alerts, shortens vulnerability response time, and improves the timeliness and effectiveness of security protection.

[0109] Further reference Figure 2 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of an annotation-free code vulnerability detection device based on distillation learning. These device embodiments are similar to... Figure 1 Corresponding to the method embodiments shown, this distillation learning-based uncommented code vulnerability detection device can be specifically applied to various electronic devices.

[0110] like Figure 2 As shown, a code vulnerability detection device 200 based on distillation learning includes: a construction unit 201, an input unit 202, a determination unit 203, a training unit 204, a detection unit 205, and a display unit 206. The construction unit 201 is configured to construct a teacher model and an initial student model. The input to the teacher model is code-related information, including code, code comment information, and a corresponding abstract syntax tree. The output is a teacher soft label. The teacher model includes an embedding layer, a multi-layer encoder, a selector, an interaction module, and a classifier. The multi-layer encoder includes multiple intermediate feature extraction layers, where the intermediate feature extraction layers extract features from the code and code comment information. The input unit 202 is configured to input target code-related information into the embedding layer, multi-layer encoder, selector, and interaction module of the teacher model to obtain teacher feature representation information. The target code-related information includes a first target code, first target code comment information, and a corresponding first target abstract syntax tree. The determination unit 203 is configured to determine joint loss information using the aforementioned teacher feature representation information, the aforementioned initial student model, and the real labels. The training unit 204 is configured to perform distillation training on the aforementioned initial student model using the aforementioned joint loss information to obtain a distilled student model. The detection unit 205 is configured to input target code information into the distilled student model to obtain a detection result, wherein the aforementioned target code information includes: a second target code and a corresponding second target abstract syntax tree. The display unit 206 is configured to generate a warning message in response to the detection result indicating a vulnerability, and to send the warning message to the target terminal for display.

[0111] It is understandable that the units described in the distillation-based learning-based uncommented code vulnerability detection device 200 are related to the reference... Figure 1The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the uncommented code vulnerability detection device 200 based on distillation learning and the units contained therein, and will not be repeated here.

[0112] The following is for reference. Figure 3 It shows a schematic diagram of the structure of an electronic device (e.g., an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0113] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0114] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0115] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0116] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0117] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0118] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: construct a teacher model and an initial student model, wherein the input to the teacher model is code-related information, including: code, code comment information, and the corresponding abstract syntax tree; and the output is a teacher soft label. The teacher model includes: an embedding layer, a multi-layer encoder, a selector, an interaction module, and a classifier. The multi-layer encoder includes: multiple intermediate feature extraction layers, wherein the intermediate feature extraction layers are used to extract features from the code, code comment information, and the corresponding abstract syntax tree; and input the target code-related information into the embedding layer, multi-layer encoder, and other components of the teacher model. The system comprises a coder, a selector, and an interaction module to obtain teacher feature representation information. The target code information includes: a first target code, first target code annotation information, and a corresponding first target abstract syntax tree. Using the teacher feature representation information, the initial student model, and the ground truth labels, joint loss information is determined. The initial student model is trained using the joint loss information to obtain a distilled student model. The target code information is input into the distilled student model to obtain a detection result. The target code information includes: a second target code and a corresponding second target abstract syntax tree. In response to the detection result indicating a vulnerability, a warning message is generated and sent to the target terminal for display.

[0119] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0120] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0121] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a construction unit, an input unit, a determination unit, a training unit, a detection unit, and a display unit. The names of these units do not necessarily limit the unit itself; for example, a construction unit may also be described as "a unit configured to construct a teacher model and an initial student model."

[0122] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0123] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for detecting vulnerabilities in uncommented code based on distillation learning, comprising: A teacher model and an initial student model are constructed. The input of the teacher model is code-related information, which includes code, code comment information, and the corresponding abstract syntax tree. The output is a teacher soft label. The teacher model includes an embedding layer, a multi-layer encoder, a selector, an interaction module, and a classifier. The multi-layer encoder includes multiple intermediate feature extraction layers. The intermediate feature extraction layers are used to extract features from the code, code comment information, and the corresponding abstract syntax tree. The target code-related information is input into the teacher model, which includes the embedding layer, multi-layer encoder, selector, and interaction module, to obtain teacher feature representation information. The target code-related information includes: first target code, first target code annotation information, and corresponding first target abstract syntax tree. The joint loss information is determined using the teacher feature representation information, the initial student model, and the real labels; The initial student model is trained by distillation using the joint loss information to obtain the distilled student model. The target code information is input into the distilled student model to obtain the detection result, wherein the target code information includes: a second target code and a corresponding second target abstract syntax tree; In response to the detection results indicating a vulnerability, an early warning message is generated and sent to the target terminal for display.

2. The method of claim 1, wherein, The process involves inputting target code-related information into the teacher model, which includes an embedding layer, a multi-layer encoder, a selector, and an interaction module, to obtain teacher feature representation information. The target code-related information includes: a first target code, first target code annotation information, and a corresponding first target abstract syntax tree, comprising: Through the multi-layer encoder of the teacher model, key information is extracted from the relevant information of the first target code to obtain the intermediate layer feature representation information set of the first target code annotation information, the intermediate layer feature representation information of the first target code, and the intermediate layer feature representation information of the first target abstract syntax tree. The intermediate layer feature representation information of the first target code and the intermediate layer feature representation information of the first target abstract semantic tree are fused to obtain the fused code feature representation information. The parameters of the fused code feature representation information are nonlinearly projected to obtain the projected code feature representation information. The intermediate layer feature representation information set of the code annotation information is input into the selector of the teacher model to obtain the selected code annotation representation information; Teacher feature representation information is obtained by using the selected code annotation representation information and the projected code feature representation information through the interaction module of the teacher model. The interaction module includes: a multi-head attention layer, a first normalization layer, a forward propagation layer, and a second normalization layer.

3. The method of claim 1, wherein, The step of determining joint loss information using the teacher feature representation information, the initial student model, and the real labels includes: The first target code and the first target abstract semantic tree are input into the embedding layer and multi-layer encoder of the initial student model to obtain student feature representation information; The teacher feature representation is input into the classifier of the teacher model to generate teacher soft labels; The student feature representation information is input into the classifier of the initial student model to generate student soft labels; Using the teacher feature representation information and the student feature representation information, a first loss information is obtained through a first loss function; Using the aforementioned teacher soft labels and real labels, a second loss information is obtained through a second loss function; Using the student soft labels and the real labels, a third loss information is obtained through a third loss function; The first loss information, the second loss information, and the third loss information are determined as joint loss information.

4. The method of claim 1, wherein, The step involves inputting the target code information into the distilled student model to obtain the detection result. The target code information includes: a second target code and a corresponding second target abstract syntax tree, comprising: The target code information is input into the embedding layer and multi-layer encoder of the distilled student model to obtain the target initial feature representation information and the target syntax tree initial feature representation information; The target initial feature representation information and the target syntax tree initial feature representation information are fused to obtain the target fused feature representation information; The target fusion feature information is input into the classifier of the student model to obtain the detection result.

5. The method according to claim 1, wherein, The response to the detection result indicating a vulnerability, generating an early warning message, and sending the early warning message to the target terminal for display includes: The detection results are subjected to vulnerability basic information extraction to obtain structured basic information, wherein the structured basic information includes: vulnerability location information, vulnerability type information and risk level information; The vulnerability type information is used to match the vulnerability impact analysis template, and the vulnerability type information is used to extract adaptation suggestions from the remediation suggestion library to generate standardized warning text; The standardized warning text is supplemented with contextual information to obtain a rich text warning package; The rich text warning package is filled into a preset visualization template, and the code snippet corresponding to the vulnerability is highlighted to obtain visual warning information; Based on the risk level and preset notification rules, a notification task queue is obtained by matching notification channels. According to the notification task queue, the visualized early warning information is sent to the target terminal through the corresponding interface.

6. A device for detecting vulnerabilities in uncommented code based on distillation learning, comprising: The building unit is configured to build a teacher model and an initial student model. The input of the teacher model is code-related information, which includes code, code comment information and the corresponding abstract syntax tree. The output is a teacher soft label. The teacher model includes an embedding layer, a multi-layer encoder, a selector, an interaction module and a classifier. The multi-layer encoder includes multiple intermediate feature extraction layers. The intermediate feature extraction layer is used to extract features from the code and code comment information. The input unit is configured to input target code-related information into the teacher model, which includes an embedding layer, a multi-layer encoder, a selector, and an interaction module, to obtain teacher feature representation information. The target code-related information includes: a first target code, first target code annotation information, and a corresponding first target abstract syntax tree. The determining unit is configured to determine joint loss information using the teacher feature representation information, the initial student model, and the real labels; The training unit is configured to perform distillation training on the initial student model using the joint loss information to obtain a distilled student model. The detection unit is configured to input target code information into the distilled student model to obtain a detection result, wherein the target code information includes: a second target code and a corresponding second target abstract syntax tree; The display unit is configured to generate a warning message in response to the detection result indicating a vulnerability, and to send the warning message to a target terminal for display.

7. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

8. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images