A method for automatically discovering code vulnerability patterns based on conditional variational autoencoder

Abstract

This invention discloses an automatic code vulnerability pattern discovery method based on conditional variational autoencoders. The method includes: constructing a multilingual vulnerability code dataset; training a variational autoencoder with programming language labels as conditions; jointly optimizing the dataset using variational lower bound loss and contrastive learning auxiliary loss to allow similar vulnerabilities to naturally cluster in a continuous latent space; defining a hazard function based on Mahalanobis distance; constructing vulnerability distribution terrain and delineating boundary exploration zones and gap exploration zones; generating candidate latent vectors within the exploration zones through Gaussian perturbation sampling and interpolation sampling via an active learning strategy; generating candidate vulnerability codes via a decoder; automatically confirming exploitability via a hierarchical validator; and feeding newly discovered vulnerability patterns back to the training set to trigger model updates, continuously expanding the vulnerability knowledge base. This method supports multilingual vulnerability modeling and unknown vulnerability pattern discovery, and is suitable for exploring unknown patterns near the boundaries of known vulnerability types.

Description

Technical Field

[0001] This invention relates to the intersection of software security and artificial intelligence, and in particular to an automatic method for discovering code vulnerability patterns based on a conditional variational autoencoder. Background Technology

[0002] Software vulnerabilities are a core source of threat in the field of cybersecurity. With the widespread application of large-scale language model-driven code agents in software development, the security issues of the code generated by these agents are becoming increasingly prominent, necessitating methods that can automatically discover new vulnerability patterns to support the construction of code security assessment systems.

[0003] Existing vulnerability discovery methods can be mainly divided into the following three categories.

[0004] The first category is fuzzing methods. Fuzzing triggers anomalous behavior by injecting a large amount of random or mutated input into a program. Representative works include AFL (American Fuzzy Lop) developed by Zalewski et al. (Zalewski M. American fuzzy lop. URL: http: / / lcamtuf.coredump.cx / afl, 2014) and libFuzzer proposed by Serebryany et al. (Serebryany K. libFuzzer: A library forcoverage-guided fuzz testing. LLVM Project, 2015).

[0005] However, fuzzing relies on random exploration of the program input space, making it difficult to systematically discover vulnerability patterns related to the semantic depth of the code, and it is less adaptable to multi-language code scenarios.

[0006] The second category is program analysis methods. Symbolic execution solves for program branches by addressing path constraints. KLEE proposed by Cadar et al. (Cadar C, Dunbar D, Engler D R. KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs. OSDI, 2008) is a representative of this type of method. Static analysis tools such as the QL query language and its derivative system CodeQL proposed by Avgustinov et al. (Avgustinov P, et al. QL: Object-oriented Queries on Relational Data. ECOOP, 2016) and Semgrep perform vulnerability detection based on predefined rules.

[0007] However, the above methods have inherent blind spots for vulnerability types not included in the rule base, and their cross-language transfer capabilities are weak, making it difficult to discover new vulnerabilities that are at the semantic boundaries of known vulnerability types.

[0008] The third category consists of deep learning-based vulnerability detection methods. VulDeePecker (Li Z, et al. VulDeePecker: A Deep Learning-Based System for VulnerabilityDetection. NDSS, 2018) proposed by Li et al. and Devign (Zhou Y, et al. Devign: Effective Vulnerability Identification by Learning Comprehensive ProgramSemantics via Graph Neural Networks. NeurIPS, 2019) proposed by Zhou et al. combine code representation learning with vulnerability classification, achieving good results in detecting known vulnerability types.

[0009] However, such methods are essentially classifiers based on existing labeled data, which cannot proactively discover new vulnerability patterns, and the exploitability verification of candidate vulnerabilities still heavily relies on manual review, making it difficult to support large-scale automated vulnerability discovery processes.

[0010] In summary, existing methods still face key limitations in three dimensions: continuous modeling of vulnerability semantics, proactive discovery of novel vulnerability patterns, and automation of exploitability verification. These shortcomings collectively constitute the core challenges in the field of automated code vulnerability discovery, urgently requiring a method capable of continuously modeling the distribution of vulnerability semantics and proactively discovering novel vulnerabilities through potential space exploration. Summary of the Invention

[0011] To address the shortcomings of existing technologies, this invention provides an automatic code vulnerability pattern discovery method based on conditional variational autoencoders. This method is achieved by constructing a vulnerability semantic latent space and an iterative exploration and expansion framework. This framework includes four key steps: vulnerability distribution modeling, latent space exploration and sampling, hierarchical exploitability verification, and iterative knowledge base expansion.

[0012] This invention is achieved through the following technical solution: An automatic code vulnerability pattern discovery method based on conditional variational autoencoder includes the following steps: (1) Collect known vulnerability code in multiple programming languages ​​from public vulnerability databases and open source code repositories, and perform normalization preprocessing on the code samples to construct a multi-language vulnerability code dataset with programming language tags and vulnerability type tags; (2) The conditional variational autoencoder with programming language labels is trained and the variational lower bound loss and contrastive learning auxiliary loss are jointly optimized to make the vulnerability codes with similar semantics cluster into an identifiable cluster structure in the latent space. (3) Extract the potential mean vector of all vulnerability codes in the training set, perform hierarchical clustering to identify vulnerability cluster sets, define the hazard function based on Mahalanobis distance, construct the vulnerability distribution terrain and delineate the boundary exploration area and gap exploration area; (4) Based on the active learning strategy, potential vectors with uncertain risk levels are selected first within the defined exploration area. Candidate potential vectors are obtained through Gaussian perturbation sampling and interpolation sampling. Candidate vulnerability codes are generated by decoding in the specified target language. (5) The candidate vulnerability code is verified for exploitability through the static analysis layer and the dynamic sandbox verification layer in sequence. The static analysis layer outputs a comprehensive risk score and pre-filters samples whose comprehensive static risk score is lower than the preset filtering threshold. The dynamic sandbox layer injects targeted test cases to confirm the exploitability flag. (6) Add the candidate vulnerability code that has been verified as usable to the training set, retrain the variational autoencoder and update the vulnerability distribution terrain, and repeat steps (3) to (5) until the number of newly discovered vulnerability patterns is lower than the preset threshold, the size of the vulnerability knowledge base reaches the preset upper limit or the preset maximum iteration round is reached. (7) Output the multilingual vulnerability code dataset with exploitability verification and the vulnerability distribution terrain with a risk rating interface.

[0013] Specifically, the various programming languages ​​used in step (1) include, but are not limited to, C / C++, Python, Java, and JavaScript.

[0014] Further, step (1) performs normalization preprocessing on the code samples, specifically: unifying indentation, removing comments, and replacing variable names and function names with normalized placeholders; constructing a corresponding benign version for each vulnerability sample, forming a quadruple of vulnerability code, benign code, language tag, and vulnerability type tag, constituting the initial training set. ,in The initial total number of samples, This is a sample of vulnerable code. To correspond with the benign version, For language tags, The vulnerability type is tagged; the vulnerability types include, but are not limited to, buffer overflow, integer overflow, SQL injection, command injection, path traversal, and use-and-release.

[0015] Furthermore, in step (2), training a conditional variational autoencoder based on programming language labels specifically involves: first, the encoder of the variational autoencoder receives the embedded representations of the code and its language labels, and outputs the mean of the latent distribution. With log variance Then, latent vectors are sampled using reparameterization techniques. ,in Finally, the latent vector is obtained through the decoder. and target language tags Generate code sequences for conditional autoregression It supports decoding the same latent vector into equivalent implementations in different target programming languages.

[0016] Furthermore, in step (2), the variational lower bound loss and contrastive learning auxiliary loss are jointly optimized. The joint training is specifically as follows: the AdamW optimizer is used for batch stochastic gradient descent, and the learning rate adopts a cosine annealing scheduling strategy; during training, the KL divergence weights are annealed, and the weights are gradually increased from 0 to 1 to avoid the KL terms being too strong in the early stage of training, which would lead to posterior collapse; training stops after the validation set maximizes the variational lower bound ELBO convergence; the joint training objective expression is as follows: ; The first term is the reconstruction term, which measures the model's ability to reconstruct the original code; the second term is the KL divergence regularization term, which constrains the posterior distribution to approximate the standard Gaussian prior. To contrast the learning aid loss, a minus sign is used to introduce it, maximizing... This is equivalent to minimizing the loss; by using similar vulnerable code as positive samples and different vulnerable code and their corresponding benign versions as negative samples, similar vulnerabilities are encouraged to cluster in the potential space, and vulnerable code and its benign version are separated from each other. Set its weight hyperparameter.

[0017] Furthermore, step (3) includes the following sub-steps: (3.1) Vulnerability Clustering and Identification: After CVAE training is completed, the latent mean vector is extracted from all vulnerability code samples in the training set by the encoder. A point cloud is formed in the latent space; and a hierarchical clustering algorithm is performed on the point cloud to automatically identify vulnerability cluster sets. Each cluster Corresponding to a class of semantically similar vulnerability patterns; then the number of clusters. The silhouette coefficient is automatically determined without prior setting; for each cluster... Calculate the centroid vector and covariance matrix This is used for subsequent hazard calculations and exploratory sampling; (3.2) Risk Function: Based on the identified vulnerability clustering, define any point in the potential space Risk function : ; Danger level For point The Mahalanobis distance to the nearest vulnerability cluster indicates that the code corresponding to the potential vector is closer to the known vulnerability pattern; a larger distance indicates that the region has not yet been covered by known vulnerabilities. When the number of clustering samples is insufficient to guarantee... When the covariance matrix is ​​full rank, regularization is applied: ( (for small positive numbers), ensure exist; (3.3) Exploration Area Delineation: Two types of exploration areas are delineated based on the terrain of vulnerability distribution, with thresholds meeting certain conditions. Border Exploration Zone To meet The region, located near known cluster boundaries, corresponds to novel variants that may be related to known vulnerabilities; gap exploration area. To meet And simultaneously located within the influence range of at least two different clusters, meaning the difference in risk between the two most recent clusters is less than a threshold. This is to correspond to the new composite vulnerability semantics that integrate the characteristics of multiple types of vulnerabilities.

[0018] Furthermore, step (4) specifically includes the following sub-steps: (4.1) Gaussian perturbation sampling: using the centroid of the vulnerability cluster. Centered on, when Iterate through all satisfied... Cluster pairs, along the direction of clustering unit direction vector When a directional disturbance is applied, It degenerates into isotropic pure Gaussian sampling. Take a random unit vector, the sampling formula is as follows: ,in , Control the offset amplitude. Control the scope of exploration; (4.2) Interpolation Sampling: Linear interpolation is performed between the centroids of two different types of vulnerability clusters. The sampling formula is as follows: ,in Explore the semantics of complex vulnerabilities in gap regions; (4.3) Generate each candidate latent vector by decoding using a beam search strategy. There are 10 candidate codes, and any target programming language can be specified.

[0019] Furthermore, the static analysis layer in step (5) includes pattern matching, taint analysis, and control flow analysis. First, a comprehensive static risk score is calculated: , in For pattern matching scoring based on abstract syntax trees, To track the taint propagation chain from external inputs to hazardous operations, taint analysis scoring is used. To detect unreachable release paths, race windows, and integer arithmetic overflow points in the control flow graph, the control flow analysis score is used. , , Let be the weight coefficients for each analysis score and satisfy the following conditions: ;right Candidate code below the threshold is directly filtered out and does not proceed to dynamic verification; the dynamic sandbox verification layer automatically generates a set of targeted test cases for candidate code that passes static screening, based on tainted paths and dangerous operations identified by static analysis, using constraint solvers or large language models. Compile, execute, and monitor vulnerability triggering behavior within an isolated sandbox, and output exploitability flags. The vulnerability triggering behaviors include, but are not limited to, memory access anomalies, unexpected system calls, and abnormal program outputs; wherein, each test input in the test case set... ( This is used to cover boundary values, abnormal inputs, or malicious payload scenarios. This represents the total number of test cases; when the exploitability flag is 1, it indicates that at least one test input triggers the preset vulnerability behavior, and when it is 0, it indicates that it is not triggered.

[0020] Furthermore, the candidate vulnerability code that has been verified as usable is evaluated using the hazard function defined in step (3) to assess the distance between its potential vector and the known vulnerability cluster. If the hazard function exceeds a preset threshold, it is marked as a new vulnerability pattern and assigned a new type label; otherwise, it is marked as a new variant of a known vulnerability.

[0021] Specifically, the multilingual vulnerability code dataset in step (7) contains all multilingual vulnerability code samples that have been verified for exploitability, along with their language labels and vulnerability type labels; it is directly used for vulnerability detection model training and code security benchmark evaluation; the vulnerability distribution terrain is used to provide a standardized calling interface that can accept the potential vector of any code sample as input and return a risk score for external systems to call for code intelligence security evaluation.

[0022] The beneficial effects of this invention are as follows: This invention achieves continuous modeling of vulnerability semantics through a conditional variational autoencoder, and uses a hazard function... This invention employs proactive learning-guided sampling as an uncertain proxy indicator, enabling it to systematically explore and discover novel vulnerability patterns within the semantic boundaries and cluster gaps of known vulnerability types, unconstrained by the scope of a predefined vulnerability rule base. The condition generation mechanism supports cross-language vulnerability pattern transfer, allowing the same potential vector to be decoded into equivalent vulnerability implementations in different target programming languages. The hierarchical verifier of this invention achieves fully automated assessment of candidate vulnerability exploitability through a two-stage process of static analysis layer pre-screening and dynamic sandbox layer confirmation, avoiding reliance on manual review. The iterative expansion mechanism of this invention feeds newly discovered vulnerability patterns back to the training set and triggers model updates, continuously expanding the vulnerability knowledge base with each iteration. Attached Figure Description

[0023] Figure 1 This is the overall flowchart of the automatic code vulnerability pattern discovery method based on conditional variational autoencoder of the present invention. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0025] (1) Overall architecture The method of this invention takes a multilingual vulnerability code dataset as input, trains a conditional variational autoencoder to establish a vulnerability semantic latent space, constructs vulnerability distribution terrain in this space, generates candidate vulnerability codes through guided latent space exploration, confirms exploitability through a hierarchical validator, and feeds new vulnerabilities back to the training set to trigger iterative updates, finally outputting a vulnerability code dataset. With vulnerability distribution terrain The overall process is as follows: Figure 1 As shown, each module is executed sequentially and forms a closed-loop iteration.

[0026] (2) Construction of a multilingual vulnerability code dataset We collected known vulnerability code samples from public vulnerability databases such as NVD, CVE, and CWE, as well as open-source code repositories and security research reports. These samples cover mainstream programming languages ​​such as C / C++, Python, Java, and JavaScript, and typical vulnerability types such as buffer overflow, integer overflow, SQL injection, command injection, path traversal, and use-and-release.

[0027] For each code sample, perform normalization preprocessing: standardize indentation format, remove comments, and replace variable and function names with normalized placeholders to reduce the interference of surface form differences on semantic learning. For each vulnerability sample, generate a corresponding benign version through manual repair or automatic patching, forming a four-tuple (vulnerable code, benign code, language tag, vulnerability type tag) to constitute the initial training set. in The initial total number of samples, This is a sample of vulnerable code. To correspond with the benign version, For programming language tags, This is a vulnerability type label.

[0028] (3) Conditional variational autoencoder training (3.1) Model Structure The conditional variational autoencoder consists of three parts: an encoder, a reparameterization module, and a decoder.

[0029] encoder Receive code and its language tags The embedding representation outputs the mean of the latent distribution. With log variance Code Context-aware vector representations and language labels are obtained through pre-trained code representation models (such as CodeBERT or UniXcoder). The learnable embedding matrix is ​​mapped to a dense vector and then concatenated with the code representation as input to the encoder. The reparameterization module samples latent vectors from the encoding distribution using reparameterization techniques, avoiding the sampling operation from blocking gradient propagation. decoder With latent vectors and target language tags Given the condition, the autoregressive code sequence is generated: By using language tag conditionalization, the decoder supports decoding the same latent vector into equivalent implementations in different programming languages.

[0030] (3.2) Training Objectives Training the CVAE (Conditional Variational Autoencoder) is achieved by maximizing the variational lower bound (ELBO). To further improve the representation quality of vulnerability semantics, a contrastive learning auxiliary loss is introduced, and the joint training objective is defined as: ; The first term is the reconstruction term, which uses log-likelihood to measure the decoder's ability to reconstruct the original code sequence; the second term is the KL divergence regularization term (introduced with a minus sign), which constrains the posterior distribution. The code sample is mapped by the encoder to the mean vector or sampled latent vector of the latent distribution, and the semantic similarity of the vulnerability is characterized by the distance and clustering relationship in this space.

[0031] Contrast learning aid loss (Introduced by a minus sign, in maximizing) When the loss is minimized, it is defined as: ; in The cosine similarity function is used. To and Positive sample latent vectors of the same type of vulnerability, Traverse all samples within the same batch (including (It consists of itself, positive samples, and negative samples). Negative samples contain potential vectors of different types of vulnerabilities and potential vectors of corresponding benign code. , For temperature coefficient, The weight coefficients of the contrastive learning auxiliary loss are used to compare the learning process. The contrastive loss promotes the clustering of similar vulnerabilities in the latent space and the separation of vulnerable code from its benign version, so that the latent space forms a clustering structure organized according to the vulnerability semantics. (3.3) Training process The AdamW optimizer is used for batch stochastic gradient descent, and the learning rate is scheduled using a cosine annealing strategy. During training, the KL divergence weights are annealed, gradually increasing from 0 to 1 to avoid excessive strength of the KL term in the early stages of training, which could lead to posterior collapse. Training stops when the validation set ELBO converges.

[0032] (4) Construction of vulnerability distribution terrain (4.1) Vulnerability clustering identification After CVAE training is completed, the latent mean vector is extracted from all vulnerable code samples in the training set using the encoder. A point cloud is formed in the latent space. A hierarchical clustering algorithm is then applied to this point cloud to automatically identify vulnerability cluster sets. Each cluster This corresponds to a class of vulnerability patterns with similar semantics. Number of clusters. The silhouette coefficient is automatically determined, without the need for pre-setting. For each cluster... Calculate the centroid vector and covariance matrix This is used for subsequent hazard calculations and exploratory sampling.

[0033] (4.2) Hazard function Based on the clustering of identified vulnerabilities, any point in the potential space is defined. Risk function : Danger level For point The smaller the Mahalanobis distance to the nearest vulnerability cluster, the closer the code corresponding to the potential vector is to a known vulnerability pattern; the larger the distance, the less likely the region is to be covered by known vulnerabilities. When the number of cluster samples is insufficient to guarantee... When the covariance matrix is ​​full rank, regularization is applied: ( (for small positive numbers), ensure exist.

[0034] (4.3) Exploration of the delineation of the area Two types of exploration areas were delineated based on the vulnerability distribution terrain (thresholds met). ): Border Exploration Zone To meet The region, located near known cluster boundaries, corresponds to novel variants that may be related to known vulnerabilities; gap exploration area. To meet And simultaneously located within the influence range of at least two different clusters (i.e., the difference in risk between the two most recent clusters is less than a threshold). This is to correspond to the new composite vulnerability semantics that integrate the characteristics of multiple types of vulnerabilities.

[0035] The boundary exploration zone is a region located at a moderate distance from the cluster center of known vulnerabilities and near the cluster boundary, used to prioritize the exploration of new variants of known vulnerability types; the gap exploration zone is a region far from a single cluster but simultaneously affected by at least two clusters, used to prioritize the exploration of composite vulnerability patterns that integrate the characteristics of multiple vulnerability types.

[0036] (5) Potential space exploration and candidate vulnerability code generation (5.1) Active learning-guided latent space sampling In the exploration area Internally, an active learning strategy is used to guide sampling: based on the risk function value. As a proxy indicator of uncertainty in this location exploration, the uncertainty is ranked separately in the two types of exploration areas—for the boundary exploration area, in order of meeting the following conditions... Among the candidate locations, the location furthest from existing sampling points is selected first; for the gap exploration area, the location must meet the following conditions. Among the candidate locations, priority is given to the location with the smallest difference in the risk of the two nearest clusters—concentrating exploration resources on the potential spatial regions with the greatest discovery value. Based on this, two complementary strategies are used to complete the sampling.

[0037] Gaussian perturbation sampling with known cluster centroids Centered on, when Time-pointing adjacent clustering unit direction vector ( Iterate through all satisfied The clustering index is subjected to a guided perturbation when Time degenerates into isotropic pure Gaussian sampling: in Control the offset amplitude. Controlling the exploration range of isotropic random perturbations; by examining all cluster pairs ( The sampling process is traversed to systematically explore the surrounding areas of each cluster in the potential space.

[0038] Interpolation sampling performs linear interpolation between the centroids of two different types of vulnerability clusters: Interpolation vectors correspond to semantic positions between the two types of vulnerabilities, exploring novel composite vulnerabilities that may combine features of both types; when If the time-slot exploration region is empty, skip interpolation sampling and only perform Gaussian perturbation sampling.

[0039] (5.2) Generation of candidate vulnerability code The sampled candidate latent vector set ( (The total number of candidate vectors sampled in this round), and then processed by the decoder using a beam search strategy for each potential vector. In the specified target language generate Candidate vulnerability code: in Indicates the first step of beam search decoding The outputs corresponding to each candidate path form a candidate set. Target language It allows for flexible specification and supports decoding the same potential vector into equivalent vulnerability implementations in multiple programming languages.

[0040] (6) Hierarchical validator (6.1) Static Analysis Layer The static analysis layer performs a rapid pre-screening of candidate code, rejecting samples that clearly lack vulnerability characteristics, and outputs a comprehensive static risk score: ; in Scoring of pattern matching based on Abstract Syntax Tree (AST) to match the syntactic features of known vulnerabilities; A taint analysis score is used to track the propagation chain of unpurified taints from external sources to hazardous sinks; Control flow analysis scoring is used to detect features in the control flow graph such as unreachable release paths, conditional race windows, and integer arithmetic overflow points; among which , , Let be the weight coefficients for each analysis score and satisfy the following conditions: .

[0041] The static analysis layer outputs a comprehensive risk score to pre-filter low-risk samples (candidate codes with a comprehensive static risk score below a preset filtering threshold), i.e., it pre-filters the comprehensive static risk score. Candidate code is filtered directly. Set a preset filtering threshold; only high-risk candidates ( The data is sent to the dynamic verification layer, so that the dynamic verification layer only processes high-risk candidates.

[0042] (6.2) Dynamic Sandbox Validation Layer For candidate code filtered by the static analysis layer, the dynamic sandbox verification layer automatically generates targeted test cases and executes them in an isolated environment to actually verify whether vulnerabilities can be triggered and exploited. Using tainted paths and dangerous operations identified by the static analysis layer as clues, a targeted test input set is automatically generated through a constraint solver (such as Z3) or a large language model. It covers typical vulnerability triggering scenarios such as boundary values, abnormal inputs, and malicious payloads. Candidate code is compiled and executed in an isolated sandbox environment (Docker container), test cases are injected, and vulnerability triggering behaviors such as abnormal memory access, unexpected system calls, and abnormal program output are monitored, outputting exploitability indicators. ; For available ( Candidate codes of ) if the risk level of their corresponding potential vectors is Exceeding the preset threshold If it is a new vulnerability pattern, it is marked as such and assigned a new type label; otherwise, it is marked as a new variant of a known vulnerability.

[0043] The vulnerability triggering behaviors include, but are not limited to, memory access anomalies, unexpected system calls, and abnormal program outputs; wherein, each test input in the test case set... ( This is used to cover boundary values, abnormal inputs, or malicious payload scenarios. This represents the total number of test cases; an exploitability flag of 1 indicates that at least one test input triggers the preset vulnerability behavior, while a flag of 0 indicates that it is not triggered. Specifically, pattern matching score is used to measure the vulnerability syntax features hit by the abstract syntax tree or rule template; taint analysis score is used to measure the integrity and severity of the unpurified propagation chain from external input to dangerous operation; control flow analysis score is used to measure abnormal control flow features such as unreachable release paths, conditional race windows, and integer arithmetic overflow points. The static analysis layer weights the above three scores to obtain a comprehensive static risk score and filters out candidate code below a preset threshold.

[0044] (7) Iterative updates and expansion of the vulnerability knowledge base Add the candidate vulnerability code that has been confirmed as exploitable in the current round of verification to the training set, and set the benign version field of the newly added sample to empty. The language tag is the target language specified during decoding. Vulnerability type tags The results from Section 6.2 are assigned values ​​(new labels are assigned to new vulnerability patterns, and new variants of known vulnerabilities inherit the labels from the most recent cluster type), and stored in a unified four-tuple structure: ; by Retrain CVAE on the training set according to the procedure described in Section 3 (or perform incremental fine-tuning). Newly added vulnerability samples cause the corresponding potential spatial regions to form new cluster cores, and the existing cluster boundaries are updated accordingly. After retraining, recalculate the vulnerability clusters according to Section 4. and risk function Update the vulnerability distribution terrain, redefine the exploration area, and enter the next exploration cycle.

[0045] The iteration terminates when any of the following conditions are met: (a) continuity wheel( The number of newly discovered vulnerability patterns during the iteration (which is a preset threshold for consecutive stagnation rounds) is all below the threshold. (b) The vulnerability knowledge base has reached its preset upper limit. (c) Reaching the preset maximum number of iterations .

[0046] The final output is a dataset of vulnerable codes. It includes all exploitable, multilingual vulnerability code and corresponding annotations, which can be directly used for vulnerability detection model training and code security benchmarking; as well as vulnerability distribution topology. It provides a standardized calling interface that accepts the potential vector of any code sample as input and returns a danger score, which can be called by external systems such as code intelligence agents for safety assessment.

[0047] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0048] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for automatically discovering code vulnerability patterns based on a conditional variational autoencoder, characterized in that, Includes the following steps: (1) Collect known vulnerability code in multiple programming languages ​​from public vulnerability databases and open source code repositories, and perform normalization preprocessing on the code samples to construct a multi-language vulnerability code dataset with programming language tags and vulnerability type tags; (2) The conditional variational autoencoder with programming language labels is trained and the variational lower bound loss and contrastive learning auxiliary loss are jointly optimized to make the vulnerability codes with similar semantics cluster into an identifiable cluster structure in the latent space. (3) Extract the potential mean vector of all vulnerability codes in the training set, perform hierarchical clustering to identify vulnerability cluster sets, define the hazard function based on Mahalanobis distance, construct the vulnerability distribution terrain and delineate the boundary exploration area and gap exploration area; (4) Based on the active learning strategy, potential vectors with uncertain risk levels are selected first within the defined exploration area. Candidate potential vectors are obtained through Gaussian perturbation sampling and interpolation sampling. Candidate vulnerability codes are generated by decoding in the specified target language. (5) The candidate vulnerability code is verified for exploitability through the static analysis layer and the dynamic sandbox verification layer in sequence. The static analysis layer outputs a comprehensive risk score and pre-filters samples whose comprehensive static risk score is lower than the preset filtering threshold. The dynamic sandbox layer injects targeted test cases to confirm the exploitability flag. (6) Add the candidate vulnerability code that has been verified as usable to the training set, retrain the variational autoencoder and update the vulnerability distribution terrain, and repeat steps (3) to (5) until the number of newly discovered vulnerability patterns is lower than the preset threshold, the size of the vulnerability knowledge base reaches the preset upper limit or the preset maximum iteration round is reached. (7) Output the multilingual vulnerability code dataset with exploitability verification and the vulnerability distribution terrain with a risk rating interface.

2. The method according to claim 1, characterized in that, The various programming languages ​​used in step (1) include, but are not limited to, C / C++, Python, Java, and JavaScript.

3. The method according to claim 1, characterized in that, Step (1) involves preprocessing the code samples to standardize them, specifically by unifying indentation, removing comments, and replacing variable and function names with standardized placeholders. A benign version is constructed for each vulnerability sample, forming a quadruple of vulnerability code, benign code, language tag, and vulnerability type tag, which constitutes the initial training set. ,in The initial total number of samples, This is a sample of vulnerable code. To correspond with the benign version, For language tags, The vulnerability type is tagged; the vulnerability types include, but are not limited to, buffer overflow, integer overflow, SQL injection, command injection, path traversal, and use-and-release.

4. The method according to claim 1, characterized in that, In step (2), training a conditional variational autoencoder based on programming language labels involves the following steps: First, the encoder of the variational autoencoder receives the embedded representations of the code and its language labels, and outputs the mean of the latent distribution. With log variance Then, latent vectors are sampled using reparameterization techniques. ,in Finally, the latent vector is obtained through the decoder. and target language tags Generate code sequences for conditional autoregression It supports decoding the same latent vector into equivalent implementations in different target programming languages.

5. The method according to claim 1, characterized in that, The joint optimization in step (2) using variational lower bound loss and contrastive learning auxiliary loss is specifically as follows: batch stochastic gradient descent is performed using the AdamW optimizer, and the learning rate is scheduled using a cosine annealing strategy; during training, an annealing strategy is applied to the KL divergence weights, with the weights gradually increasing from 0 to 1 to avoid excessive strength of the KL term in the early stages of training, which could lead to posterior collapse; training stops after the validation set maximizes the variational lower bound ELBO and converges; the joint training objective expression is as follows: ； The first term is the reconstruction term, which measures the model's ability to reconstruct the original code; the second term is the KL divergence regularization term, which constrains the posterior distribution to approximate the standard Gaussian prior. To contrast the learning aid loss, a minus sign is used to introduce it, maximizing... This is equivalent to minimizing the loss; by using similar vulnerable code as positive samples and different vulnerable code and their corresponding benign versions as negative samples, similar vulnerabilities are encouraged to cluster in the potential space, and vulnerable code and its benign version are separated from each other. Set its weight hyperparameter.

6. The method according to claim 1, characterized in that, Step (3) includes the following sub-steps: (3.1) Vulnerability Clustering and Identification: After CVAE training is completed, the latent mean vector is extracted from all vulnerability code samples in the training set by the encoder. A point cloud is formed in the latent space; and a hierarchical clustering algorithm is performed on the point cloud to automatically identify vulnerability cluster sets. Each cluster Corresponding to a class of semantically similar vulnerability patterns; then the number of clusters. The silhouette coefficient is automatically determined without prior setting; for each cluster... Calculate the centroid vector and covariance matrix This is used for subsequent hazard calculations and exploratory sampling; (3.2) Risk Function: Based on the identified vulnerability clustering, define any point in the potential space Risk function : ； Danger level For point The Mahalanobis distance to the nearest vulnerability cluster indicates that the code corresponding to the potential vector is closer to the known vulnerability pattern; a larger distance indicates that the region has not yet been covered by known vulnerabilities. When the number of clustering samples is insufficient to guarantee... When the covariance matrix is ​​full rank, regularization is applied: ( (for small positive numbers), ensure exist; (3.3) Exploration Area Delineation: Two types of exploration areas are delineated based on the terrain of vulnerability distribution, with thresholds meeting certain conditions. Border Exploration Zone To meet The region, located near known cluster boundaries, corresponds to novel variants that may be related to known vulnerabilities; gap exploration area. To meet And simultaneously located within the influence range of at least two different clusters, meaning the difference in risk between the two most recent clusters is less than a threshold. This is to correspond to the new composite vulnerability semantics that integrate the characteristics of multiple types of vulnerabilities.

7. The method according to claim 1, characterized in that, Step (4) specifically includes the following sub-steps: (4.1) Gaussian perturbation sampling: using the centroid of the vulnerability cluster. Centered on, when Iterate through all satisfied... Cluster pairs, along the direction of clustering unit direction vector When a directional disturbance is applied, It degenerates into isotropic pure Gaussian sampling. Take a random unit vector, the sampling formula is as follows: ,in , Control the offset amplitude. Control the scope of exploration; (4.2) Interpolation Sampling: Linear interpolation is performed between the centroids of two different types of vulnerability clusters. The sampling formula is as follows: ,in Explore the semantics of complex vulnerabilities in gap regions; (4.3) Generate each candidate latent vector by decoding using a beam search strategy. There are 10 candidate codes, and any target programming language can be specified.

8. The method according to claim 1, characterized in that, The static analysis layer in step (5) includes pattern matching, taint analysis, and control flow analysis. First, a comprehensive static risk score is calculated: ， in For pattern matching scoring based on abstract syntax trees, To track the taint propagation chain from external inputs to hazardous operations, taint analysis scoring is used. To detect unreachable release paths, race windows, and integer arithmetic overflow points in the control flow graph, the control flow analysis score is used. , , Let be the weight coefficients for each analysis score and satisfy the following conditions: ;right Candidate code below the threshold is directly filtered out and does not proceed to dynamic verification; the dynamic sandbox verification layer automatically generates a set of targeted test cases for candidate code that passes static screening, based on tainted paths and dangerous operations identified by static analysis, using constraint solvers or large language models. Compile, execute, and monitor vulnerability triggering behavior within an isolated sandbox, and output exploitability flags. The vulnerability triggering behaviors include, but are not limited to, memory access anomalies, unexpected system calls, and abnormal program outputs; wherein, each test input in the test case set... ( This is used to cover boundary values, abnormal inputs, or malicious payload scenarios. This represents the total number of test cases; when the exploitability flag is 1, it indicates that at least one test input triggers the preset vulnerability behavior, and when it is 0, it indicates that it is not triggered.

9. The method according to claim 1, characterized in that, The candidate vulnerability code that has been verified as usable is evaluated using the hazard function defined in step (3) to assess the distance between its potential vector and the known vulnerability cluster. If the hazard function exceeds a preset threshold, it is marked as a new vulnerability pattern and assigned a new type label; otherwise, it is marked as a new variant of a known vulnerability.

10. The method according to claim 1, characterized in that, The multilingual vulnerability code dataset in step (7) contains all exploitable multilingual vulnerability code samples and their language tags and vulnerability type tags; it is directly used for vulnerability detection model training and code security benchmark evaluation; the vulnerability distribution terrain is used to provide a standardized calling interface that can accept the potential vector of any code sample as input and return a risk score for external systems to call for code intelligence security evaluation.

Images