Vulnerability remediation code detection method and related device

Abstract

The present application provides a vulnerability remediation code identification method, comprising: acquiring commit information of open-source software, the commit information comprising a committed code snippet and text information associated with the code snippet; inputting the commit information into a first language model and performing identification to obtain function change information and test case information; inputting the function change information into a vulnerability remediation identification model to obtain a vulnerability remediation code snippet for a target vulnerability, and inputting the test case information into a vulnerability test case identification model to obtain a test case code snippet for the target vulnerability; and inputting the vulnerability remediation code snippet for the target vulnerability and the test case code snippet for the target vulnerability into a second language model and performing integration and identification to obtain an identification result. On the basis of notable characteristics in a test case for a vulnerability, the identification rate of vulnerability remediation code can be effectively increased. In this way, a user can discover potential vulnerabilities in advance, thus reducing security risks.

Description

A vulnerability repair code detection method and related equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on October 26, 2023, with application number 202311402518.3, and with the invention name “A Vulnerability Repair Code Detection Method and Related Equipment”, and claims priority to the Chinese patent application filed with the State Intellectual Property Office on January 30, 2024, with application number 202410135488.2, and with the invention name “A Vulnerability Repair Code Detection Method and Related Equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of software development and testing technology, and in particular to a vulnerability repair code identification method, system, computing device cluster, computer-readable storage medium, and computer program product. Background Art

[0003] With the continuous development of the computer industry and the continued expansion of the open source community, the use of open source software has become a significant trend in the field. The open nature of open source software allows anyone to view, modify, and distribute its source code, significantly improving development efficiency. However, this also introduces potential risks. These risks can arise from security vulnerabilities, also known as vulnerabilities. These are flaws in computer system security that threaten the confidentiality, integrity, availability, and access control of the system or its application data.

[0004] The widespread interdependence between open source software means that once unpatched vulnerabilities are exposed, the impact and damage will be enormous. Therefore, researchers and security experts can audit and analyze the code of open source software to identify potential vulnerabilities. Once a vulnerability is discovered, they can report it to the software maintenance team or vulnerability disclosure platforms to encourage them to fix it.

[0005] Considering the openness of the source code, attackers can also exploit this vulnerability to attack software that widely uses the open source software, bringing unknown security risks. For this reason, most open source contributors adopt a collaborative vulnerability disclosure model, allowing the open source responsible party enough time to repair and remedy the vulnerability, and then disclose the vulnerability to the public after a period of time.

[0006] However, the coordinated disclosure model results in a certain time lag between vulnerability repair and vulnerability disclosure. Detailed information about most vulnerabilities is disclosed a week or even longer after they are fixed, making it difficult for users to discover potential vulnerabilities in advance and posing security risks.

[0007] Summary of the Invention

[0008] This application provides a vulnerability repair code identification method. This method constructs recognition models for vulnerability repair code submissions and test case submissions, such as vulnerability repair identification models and vulnerability test case identification models, and can effectively improve the recognition rate of vulnerability repair code based on significant features in test cases targeting vulnerabilities. In this way, users can discover potential vulnerabilities in advance and reduce security risks. This application also provides a vulnerability repair code identification system, computing device cluster, computer-readable storage medium, and computer program product corresponding to the above method.

[0009] In a first aspect, the present application provides a vulnerability repair code identification method. Among them, the vulnerability repair code can also be referred to as a patch code, or simply a patch. The method can be executed by a vulnerability repair code identification system. The vulnerability repair code identification system can also be referred to as a vulnerability repair code mining tool and service, a vulnerability patch mining tool and service, a vulnerability identification tool and service, including but not limited to known vulnerability mining tools and services, open source vulnerability identification tools and services. Among them, the vulnerability repair code identification system can be a software system, which can be an independent software system or integrated into other software, for example, in the form of a plug-in integrated into a software testing platform or an integrated development environment. The above-mentioned software system can be deployed in a computing device cluster, and the computing device cluster executes the program code of the software system, thereby executing the vulnerability repair code identification method of the present application. In some examples, the vulnerability repair code identification system can also be a hardware system, and when the hardware system is running, it executes the vulnerability repair code device method of the present application.

[0010] Specifically, the vulnerability fixing code identification system obtains submission information of the open source software, which includes the submitted code snippet and text information associated with the code snippet. The vulnerability fixing code identification system then inputs the submission information into the first language model for recognition to obtain functional change information and test case information.

[0011] The function change information includes function change code snippets and function change text, and the test case information includes test case code snippets and test case text. The vulnerability repair code identification system can input the function change information into the vulnerability repair identification model to obtain vulnerability repair code snippets for the target vulnerability, and input the test case information into the vulnerability test case identification model to obtain test case code snippets for the target vulnerability. The vulnerability repair code identification system can input the vulnerability repair code snippets for the target vulnerability and the test case code snippets for the target vulnerability into the second language model for integration and recognition to obtain recognition results.

[0012] This method takes into account that when submitting a vulnerability fix, not only the vulnerability fix code is submitted, but also the test case code is usually submitted. Therefore, recognition models for vulnerability fix code submission and test case submission are designed, such as vulnerability fix recognition models and vulnerability test case recognition models. Based on the significant features in the test cases for the vulnerability, the recognition rate of the vulnerability fix code can be effectively improved. This can improve the recognition of undisclosed vulnerabilities and supplement the vulnerability identification capabilities. In addition, this method takes into account that in a real development environment, vulnerability fixes and normal feature submissions may be integrated, which will bring greater noise to the identification of vulnerability fix code. Therefore, before performing vulnerability fix code identification, the correlation between a single code submission and the vulnerability fix is ​​first identified based on the first language model and the function-level code snippet is accurately extracted. Then, through task decomposition, the precise function-level code snippet is input into the downstream model to complete precise identification, further improving the accuracy of recognition.

[0013] In some possible implementations, the vulnerability repair identification model includes a vulnerability repair code semantic recognition model and a vulnerability repair text semantic recognition model, and the vulnerability test case identification model includes a vulnerability test case code semantic recognition model and a vulnerability test case text semantic recognition model.

[0014] The vulnerability repair code identification system inputs the function change code snippet into the vulnerability repair code semantic recognition model, and inputs the function change text into the vulnerability repair text semantic recognition model, and obtains the vulnerability repair code snippet for the target vulnerability based on the semantic recognition results of the function change code snippet and the semantic recognition results of the function change text.

[0015] The vulnerability repair code identification system inputs the test case code snippet into the vulnerability test case code semantic recognition model, and inputs the test case text into the vulnerability test case text semantic recognition model, and obtains the test case code snippet for the target vulnerability based on the semantic recognition results of the test case code snippet and the semantic recognition results of the test case text.

[0016] This method not only identifies vulnerability repair codes from the dimensions of functional changes and test cases, but also performs code semantic recognition and text semantic recognition for functional changes and test cases respectively. This can further improve the recognition rate and accuracy of vulnerability repair codes, and effectively complete vulnerability patch mining and undisclosed vulnerability identification.

[0017] In some possible implementations, the vulnerability-fixing code identification system can obtain code data from a vulnerability-fixing dataset or a vulnerability-testing case dataset, preprocess the code data to obtain full code differences and function-level code differences with vulnerability labels, and then input the full code differences into a pre-trained language model for fine-tuning to obtain a fine-tuned model. The vulnerability-fixing code identification system also inputs the function-level code differences with vulnerability labels into the fine-tuned model to obtain a representation of the function-level code differences. Based on the representation of the function-level code differences, a submission representation is obtained. The submission representation is then input into a classifier for training to obtain a code semantic recognition model.

[0018] This method first fine-tunes the pre-trained language model using the full code differences, and then uses the fine-tuned model to extract features from function-level code differences, so that the representation of function-level code differences is input into the classifier for training. This method can train a code semantic recognition model without requiring a large amount of function-level patch data, solving the problem of lack of function-level patch data in open source vulnerability patch data.

[0019] In some possible implementations, the vulnerability repair code identification system can also obtain text data from a vulnerability repair dataset or a vulnerability test case dataset, and then search the vulnerability description knowledge base based on the text data to obtain the vulnerability description knowledge corresponding to the text data, and search the Common Weakness Enumeration (CWE) database based on the text data to obtain the CWE knowledge corresponding to the text data. The vulnerability repair code identification system can splice the text data, vulnerability description knowledge and CWE knowledge into prompt information, input the prompt information into a general language model for training, and obtain a text semantic recognition model. Among them, the general language model can be a language model used for question and answer and conversation in different fields. In some examples, the prompt information can be prompt information for vulnerability patch identification, and the general language model can output a category based on the prompt information to determine whether it is a vulnerability patch. For example, the general language model can output category 0 or category 1, category 0 is used to identify non-patches, and category 1 is used to identify patches.

[0020] This method constructs prompt information by combining vulnerability description knowledge and CWE knowledge, and uses the prompt information as input for model training. On the one hand, it can reduce the amount of required training data, and on the other hand, it can improve training efficiency.

[0021] In some possible implementations, the dataset is constructed as follows:

[0022] Establish the association between vulnerability repair code samples and CWE information;

[0023] Inputting the vulnerability repair code sample into a vulnerability repair code analysis engine to obtain the function change code sample and the test case code sample;

[0024] According to the function change code sample, using prompt information and language model, obtain a vulnerability repair code sample for the vulnerability and a corresponding function description; according to the test case code sample, using prompt information and language model, obtain a vulnerability test case code sample for the vulnerability and a corresponding function description;

[0025] The metadata of the vulnerability, the CWE information, the vulnerability repair code sample and the corresponding functional description are stored in the vulnerability repair dataset, and / or the metadata of the vulnerability, the CWE information, the vulnerability test case code sample and the corresponding functional description are stored in the vulnerability test case dataset.

[0026] This allows the construction of function-level vulnerability fix datasets and vulnerability test case datasets, addressing the current lack of function-level patch data in open-source vulnerability patch data. Furthermore, these datasets can be used to construct vulnerability fix and test case corpora. These corpora can then be used to train a first language model for task decomposition, helping to improve patch recognition rates.

[0027] In some possible implementations, the first language model can be trained as follows:

[0028] Obtain vulnerability repair corpus pairs and vulnerability test case corpus pairs. The vulnerability repair corpus pairs include vulnerability repair code samples and associated CWE information. The vulnerability test case corpus pairs include vulnerability test case samples and associated CWE information. Then, input the vulnerability repair corpus pairs and vulnerability test case corpus pairs into the pre-trained model for supervised fine-tuning to obtain the first language model.

[0029] This method obtains the first language model for task decomposition by fine-tuning the pre-trained model using vulnerability repair corpus pairs and vulnerability test case corpus. The first language model is then able to extract function-level data and split the task into vulnerability repair identification and vulnerability test case identification. Fine-grained task decomposition can improve the recognition rate of the decomposed tasks, thereby improving the overall recognition rate.

[0030] In some possible implementations, a vulnerability fix code identification system can obtain vulnerability metadata from a vulnerability database and then, based on this metadata, retrieve commit information for open source software. This allows for the mining of vulnerability fixes (patches) for known vulnerabilities, improving the vulnerability database and enabling version iterations, such as on user branches, to mitigate security risks.

[0031] In some possible implementations, a vulnerability fix code identification system can monitor open source repositories to obtain submission information for open source software. This allows the system to identify whether a code submission is a vulnerability patch, thereby identifying undisclosed vulnerabilities and improving vulnerability identification capabilities.

[0032] In some possible implementations, the first language model or the second language model is a large language model (LLM). This method uses a large language model for data processing, which can solve the problem that traditional recognition methods based on code semantics and submission text information are limited by the lack of patch data and CWE mapping.

[0033] In a second aspect, the present application provides a vulnerability repair code identification system. The system includes:

[0034] An acquisition module, configured to acquire submission information of the open source software, wherein the submission information includes a submitted code snippet and text information associated with the code snippet;

[0035] a disassembly module, configured to input the submitted information into a first language model for recognition to obtain function change information and test case information, wherein the function change information includes a function change code snippet and a function change text, and the test case information includes a test case code snippet and a test case text;

[0036] an identification module, configured to input the function change information into a vulnerability repair identification model to obtain a vulnerability repair code snippet for a target vulnerability, and input the test case information into a vulnerability test case identification model to obtain a test case code snippet for the target vulnerability;

[0037] The integration module is used to input the vulnerability repair code snippet for the target vulnerability and the test case code snippet for the target vulnerability into the second language model for integration and recognition to obtain a recognition result.

[0038] In some possible implementations, the vulnerability repair identification model includes a vulnerability repair code semantic identification model and a vulnerability repair text semantic identification model, and the vulnerability test case identification model includes a vulnerability test case code semantic identification model and a vulnerability test case text semantic identification model;

[0039] The identification module is specifically used for:

[0040] Inputting the function-changing code snippet into the vulnerability-fixing code semantic recognition model, and inputting the function-changing text into the vulnerability-fixing text semantic recognition model, and obtaining a vulnerability-fixing code snippet for the target vulnerability based on the semantic recognition results of the function-changing code snippet and the semantic recognition results of the function-changing text;

[0041] The test case code snippet is input into the vulnerability test case code semantic recognition model, and the test case text is input into the vulnerability test case text semantic recognition model, and the test case code snippet for the target vulnerability is obtained according to the semantic recognition result of the test case code snippet and the semantic recognition result of the test case text.

[0042] In some possible implementations, the system further includes:

[0043] A training module is configured to obtain code data from a vulnerability repair dataset or a vulnerability test case dataset, preprocess the code data to obtain full code differences and function-level code differences with vulnerability labels, and input the full code differences into a pre-trained language model for fine-tuning to obtain a fine-tuned model;

[0044] The training module is also used to input the function-level code differences with vulnerability labels into the fine-tuning model to obtain the representation of the function-level code differences, obtain the submission representation based on the representation of the function-level code differences, input the submission representation into the classifier for training, and obtain the code semantic recognition model.

[0045] In some possible implementations, the system further includes:

[0046] A training module is configured to obtain text data from a vulnerability repair dataset or a vulnerability test case dataset, search a vulnerability description knowledge base based on the text data to obtain vulnerability description knowledge corresponding to the text data, and search a common vulnerability enumeration (CWE) database based on the text data to obtain CWE knowledge corresponding to the text data;

[0047] The training module is further used to splice the text data, the vulnerability description knowledge and the CWE knowledge into prompt information, input the prompt information into a common language model for training, and obtain the text semantic recognition model.

[0048] In some possible implementations, the system further includes:

[0049] A data set construction module is used to establish an association between vulnerability repair code samples and CWE information, input the vulnerability repair code samples into the vulnerability repair code analysis engine, obtain the function change code samples and test case code samples, and obtain vulnerability repair code samples and corresponding functional descriptions for the vulnerabilities based on the function change code samples using prompt information and a language model; and obtain vulnerability test case code samples and corresponding functional descriptions for the vulnerabilities based on the test case code samples using prompt information and a language model;

[0050] The dataset construction module is also used to store the metadata of the vulnerability, the CWE information, the vulnerability repair code sample and the corresponding functional description into the vulnerability repair dataset, and / or store the metadata of the vulnerability, the CWE information, the vulnerability test case code sample and the corresponding functional description into the vulnerability test case dataset.

[0051] In some possible implementations, the system further includes:

[0052] A training module is used to obtain a vulnerability repair corpus pair and a vulnerability test case corpus pair, wherein the vulnerability repair corpus pair includes a vulnerability repair code sample and associated CWE information, and the vulnerability test case corpus pair includes a vulnerability test case sample and associated CWE information, and input the vulnerability repair corpus pair and the vulnerability test case corpus pair into a pre-trained model for supervised fine-tuning SFT to obtain the first language model.

[0053] In some possible implementations, the acquisition module is specifically configured to:

[0054] Get vulnerability metadata from the vulnerability database;

[0055] Obtain submission information of the open source software based on the vulnerability metadata.

[0056] In some possible implementations, the acquisition module is specifically configured to:

[0057] Monitor open source repositories and obtain submission information of open source software.

[0058] In some possible implementations, the first language model or the second language model is a large language model (LLM).

[0059] In a third aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, wherein the at least one computing device includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is configured to execute instructions stored in the at least one memory, so that the computing device or computing device cluster performs the vulnerability repair code identification method described in the first aspect or any implementation of the first aspect.

[0060] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, wherein the instructions instruct a computing device or a computing device cluster to execute the vulnerability repair code identification method described in the first aspect or any implementation of the first aspect.

[0061] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when run on a computing device or a computing device cluster, enables the computing device or computing device cluster to execute the vulnerability repair code identification method described in the first aspect or any implementation of the first aspect.

[0062] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical methods of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments.

[0064] FIG1 is a schematic diagram of an application scenario of a vulnerability repair code identification method provided by the present application;

[0065] FIG2 is a schematic diagram of the logic flow of a vulnerability patch identification service provided by this application;

[0066] FIG3 is a flow chart of a vulnerability repair code identification method provided by the present application;

[0067] FIG4 is a schematic diagram of the construction process of a vulnerability repair dataset and a vulnerability test case dataset provided by this application;

[0068] FIG5 is a schematic diagram of the construction process of a large language model fine-tuning corpus dataset provided by this application;

[0069] FIG6 is a schematic diagram of the construction process of a vulnerability repair identification model and a vulnerability test case identification model provided by this application;

[0070] FIG7 is a schematic diagram of the process of constructing a large language model for vulnerability repair and identification provided by this application;

[0071] FIG8 is a logic flow chart of a known vulnerability patch mining service provided by this application;

[0072] FIG9 is a logic flow chart of an open source vulnerability identification method provided by this application;

[0073] FIG10 is a schematic diagram of the structure of a vulnerability repair code identification system provided by the present application;

[0074] FIG11 is a schematic diagram of the structure of a computing device provided by the present application;

[0075] FIG12 is a schematic diagram of the structure of a computing device cluster provided by this application;

[0076] FIG13 is a schematic diagram of the structure of another computing device cluster provided by the present application;

[0077] FIG14 is a schematic diagram of the structure of another computing device cluster provided in this application. DETAILED DESCRIPTION

[0078] The terms "first" and "second" in the embodiments of this application are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0079] First, some technical terms involved in the embodiments of this application are introduced.

[0080] Open source software refers to software whose source code can be publicly viewed, used and modified, and the community can freely access, distribute and improve the software.

[0081] Software vulnerabilities, also known as vulnerabilities or simply vulnerabilities, are security flaws found in software, such as open source software. Vulnerabilities can arise from a variety of sources, including design flaws or coding errors in software or operating systems. Alternatively, they can arise from design flaws or logical inconsistencies in business interaction processes. When exploited, these vulnerabilities can threaten the confidentiality, integrity, availability, and access control of the system or its application data.

[0082] Vulnerability remediation is the process of improving or patching existing vulnerabilities in software or systems to enhance their security and stability. Vulnerability remediation code, also known as a patch or simply a patch, is a fix or code snippet developed by developers to address discovered vulnerabilities. It is used to repair security holes in software or systems.

[0083] Code review is a systematic inspection and evaluation process of software code, aimed at discovering potential errors, vulnerabilities, inefficiencies or violations of coding standards, and making improvement suggestions to ensure software quality and security.

[0084] Open source software submission refers to software contributors in the open source community submitting their own developed code, functions, fixes or improvement suggestions to open source projects to promote the development and sharing of open source software.

[0085] When developers submit open source software, they usually submit vulnerability fix code in the form of functional changes, making it difficult to identify whether the open source software submission is vulnerability fix code or vulnerability patches. This makes it difficult for users to discover potential vulnerabilities in advance, posing a security risk.

[0086] Currently, the industry has proposed a solution for automated analysis of open source project patches. This solution collects patch code for all open source project vulnerabilities, categorizes and analyzes these patches by fix type, and effectively reduces the effort required to manually check for open source project vulnerabilities to identify and categorize bug fixes. The analysis process is achieved by constructing an abstract syntax tree (AST) of the patch code and comparing it with the AST of the original open source project code.

[0087] However, the error categories supported by this solution are limited to six custom vulnerability categories, which is not comprehensive enough to encompass most of the hot issues in the open source community. Furthermore, the solution is also limited by the efficiency of constructing an abstract syntax tree, making it difficult to use in large-scale open source software scenarios.

[0088] In view of this, the present application provides a vulnerability repair code identification method. The method can be performed by a vulnerability repair code identification system. The vulnerability repair code identification system can also be referred to as a vulnerability repair code mining tool and service, a vulnerability patch mining tool and service, a vulnerability identification tool and service, including but not limited to known vulnerability mining tools and services, open source vulnerability identification tools and services. Among them, the vulnerability repair code identification system can be a software system, which can be an independent software system or integrated into other software, for example, integrated into a software testing platform or an integrated development environment (IDE) in the form of a plug-in. The above-mentioned software system can be deployed in a computing device cluster, and the computing device cluster executes the program code of the software system, thereby executing the vulnerability repair code identification method of the present application. In other possible implementations, the vulnerability repair code identification system can also be a hardware system, and when the hardware system is running, it executes the vulnerability repair code device method of the present application.

[0089] Specifically, the vulnerability repair code identification system obtains the submission information (such as commit information) of the open source software, which includes the submitted code snippet and the text information associated with the code snippet. The submission information is then input into the first large language model (LLM) for recognition to obtain function change information and test case information, wherein the function change information includes the function change code snippet and the function change text, and the test case information includes the test case code snippet and the test case text. The vulnerability repair code identification system inputs the function change information into the vulnerability repair identification model to obtain the vulnerability repair code snippet for the target vulnerability, and inputs the test case information into the vulnerability test case identification model to obtain the test case code snippet for the target vulnerability. Then, the vulnerability repair code identification system inputs the vulnerability repair code snippet for the target vulnerability and the test case code snippet for the target vulnerability into the second LLM for integration and recognition to obtain the recognition result. It should be noted that LLM is a language model with a large parameter scale, which can be composed of an artificial neural network with many parameters (usually billions of weights or more), and is trained on a large amount of unlabeled text using self-supervised learning or semi-supervised learning. In this application, the LLM can also be replaced by other language models (LMs). For example, the first LLM and the second LLM can be replaced by small language models with the same functions or capabilities as the first LLM and the second LLM, such as an n-gram language model. For ease of description, the following examples use LLMs as language models.

[0090] When submitting a vulnerability fix, not only the vulnerability fix code is submitted, but usually the test case code is also submitted. This method constructs recognition models for vulnerability fix code submission and test case submission, such as a vulnerability fix recognition model and a vulnerability test case recognition model. Based on the significant features in the test cases for the vulnerability, the recognition rate of the vulnerability fix code can be effectively improved.

[0091] Furthermore, the method also supports the construction of a large language model for patch recognition that has vulnerability patch recognition knowledge and completes the task decomposition of code submissions, such as the first LLM mentioned above. In most patch recognition solutions, the input code of the model is at the commit level or file level. In a real development environment, contributors often merge vulnerability fixes with normal feature submissions, which brings a lot of noise to the identification of vulnerability repair code. The solution of the present application constructs a large language model for patch recognition, identifies the correlation between a single code submission and a vulnerability fix, and accurately extracts function-level code snippets. Through task decomposition, the precise function-level code snippets are input into the downstream model to complete precise recognition, further improving the recognition accuracy. Moreover, the traditional recognition method based on code semantics and submission text is subject to problems such as small patch data and missing mapping, and the recognition rate is low. The present application processes the submission information through a large language model to achieve the decomposition and analysis of the vulnerability repair recognition task, which can greatly improve the recognition rate of vulnerability repairs.

[0092] Furthermore, this solution enables end-to-end vulnerability patch code identification. Specifically, the vulnerability patch code identification system, by integrating a large language model with vulnerability patch knowledge, such as the first LLM, decomposes the identification of single submission information into tasks. Combined with the vulnerability patch identification model and the vulnerability test case identification model, it effectively completes the mining of vulnerability patch code or the identification of undisclosed vulnerabilities without manual intervention.

[0093] On the one hand, it can be used to improve open source vulnerability data. Specifically, when a disclosed vulnerability is confirmed to have a vulnerability fix version but the vulnerability fix code (patch code and other patch information) cannot be associated, the vulnerability patch can be found by identifying and screening code submissions within the repair range. On the other hand, it can identify vulnerabilities before open source vulnerabilities are disclosed to ensure security. Specifically, by monitoring the submission information of open source software and determining whether the submission information includes vulnerability fix code (patches), it can identify vulnerabilities that have not yet been disclosed.

[0094] The scenarios in which the vulnerability repair code identification method of the present application can be applied are shown in Figure 1. The first application scenario can be that the vulnerability metadata of the unassociated vulnerability repair code (patch) is read from the vulnerability library through the vulnerability patch detection engine to obtain the submission information. The submission information can be a commit interval, and the vulnerability repair code to be confirmed (patch to be confirmed) can be obtained through the vulnerability patch identification service (for example, a vulnerability repair code identification system). The second application scenario can be reading the open source repository and obtaining the submission information of the open source software through the open source monitoring engine, recorded as commits, and the vulnerability to be confirmed can be obtained after the obtained commits are input into the vulnerability patch identification service. For the vulnerability repair code to be confirmed and the vulnerability to be confirmed, the corresponding metadata to be annotated is obtained through the annotation service, and after completing the manual annotation and security review through the annotation engine, it can be stored in the vulnerability library (or vulnerability storage library).

[0095] Figure 2 also shows the logical flow of the vulnerability patch identification service. The commit information to be identified (such as commit information) can obtain the commit code information and text information through commit preprocessing, and the code information and text information can be input into the vulnerability patch identification engine. Specifically, the vulnerability patch identification engine can include a first LLM, a vulnerability repair identification model and a vulnerability test case identification model, and a second LLM. The first LLM is a patch identification large language model (also called a patch LLM or a patch large model), which can identify and distribute commit information. The second LLM can be a large model for integrating and judging to obtain the final identification result.

[0096] Among them, by identifying and distributing the commit information through the patch big model, function change information and test case information can be obtained, wherein the function change information includes function change code snippets and function change text related to vulnerability repair, and the test case information includes test case code snippets and test case text (such as description text). The above function change information can be input into the vulnerability repair identification model to obtain vulnerability repair code identification results, such as vulnerability repair code snippets for the target vulnerability, and the above test case change information can be input into the vulnerability test case identification model to obtain test case identification results, such as test case code snippets for the target vulnerability. The pre-results (such as vulnerability repair code identification results and vulnerability test case identification results) can be input into the second LLM (usually a general big model) including the recognition task prompt information (prompt) to complete information integration and recognition and obtain recognition results.

[0097] The vulnerability repair code identification method of the present application is described in detail below with reference to the accompanying drawings.

[0098] 3 shows a flowchart of a vulnerability repair code identification method. The method can be performed by a vulnerability repair code identification system and includes the following steps:

[0099] S302: The vulnerability repair code identification system obtains submission information of the open source software.

[0100] Submission information includes the submitted code snippet and textual information associated with the code snippet. The submitted code snippet may include a functional change snippet, which may be a new functional change snippet or a vulnerability fix snippet. It should be noted that when a vulnerability fix snippet is submitted, a test case snippet is often also submitted. Therefore, the submitted code snippet may also include a test case snippet. The textual information associated with the code snippet may include annotations, comments, or a commit message.

[0101] The vulnerability repair code identification system can be applied to different scenarios and obtain the submission information of open source software through different methods.

[0102] In some possible implementations, the vulnerability fix code identification system can obtain vulnerability metadata from a vulnerability repository and then, based on the vulnerability metadata, retrieve commit information for the open source software. Specifically, the vulnerability fix code identification system can first obtain vulnerability data for which vulnerability fix code (patches) need to be mined from the annotated vulnerability repository, input this vulnerability data into a vulnerability patch detection engine to obtain vulnerability metadata, and then input this vulnerability metadata into a commit acquisition engine to obtain a commit group to be identified.

[0103] In other possible implementations, the vulnerability fix code identification system can monitor open source repositories to obtain commit information for open source software. Specifically, the vulnerability fix code identification system can obtain the open source repositories to be monitored from the open source software library, and then input the open source repositories into the commit monitoring engine to obtain a new commit group.

[0104] S304: The vulnerability repair code identification system inputs the submitted information into the first LLM for identification to obtain function change information and test case information.

[0105] Among them, the function change information includes the function change code snippet and the function change text, and the test case information includes the test case code snippet and the test case text. The first LLM is a task decomposition model, which is used to decompose the submission information identification task into an identification subtask based on the function change information and an identification subtask based on the test case information. In some cases, the decomposed task may also include associated subtasks, such as a message associated subtask. The input of the first LLM can be submission information, and the output of the first LLM can be function change information and test case information. In this way, the problem of vulnerability repair code recognition rate can be solved.

[0106] In some possible implementations, the commit information may be at the commit level or the file level. The first LLM may decompose the commit information into function-level information. In this case, the output of the first LLM may be function-level functional change information and test case information.

[0107] S306: The vulnerability repair code identification system inputs the function change information into the vulnerability repair identification model to obtain a vulnerability repair code snippet for the target vulnerability.

[0108] S308. The vulnerability repair code identification system inputs the test case information into the vulnerability test case identification model to obtain a test case code snippet targeting the target vulnerability.

[0109] Functional change information includes functional change code snippets and functional change text. Similarly, test case information includes test case code snippets and test case text. Code and text are different types of data. When identifying the above functional change information or test case information, the vulnerability repair code identification system can perform semantic recognition on the code and text separately.

[0110] In some possible implementations, the vulnerability repair identification model includes a vulnerability repair code semantics identification model and a vulnerability repair text semantics identification model, and the vulnerability test case identification model includes a vulnerability test case code semantics identification model and a vulnerability test case text semantics identification model. Accordingly, the vulnerability repair code identification system can input a function-changing code snippet into the vulnerability repair code semantics identification model, and input a function-changing text into the vulnerability repair text semantics identification model, and obtain a vulnerability repair code snippet for the target vulnerability based on the semantics identification results of the function-changing code snippet and the function-changing text. The vulnerability repair code identification system can input a test case code snippet into the vulnerability test case code semantics identification model, and input a test case text into the vulnerability test case text semantics identification model, and obtain a test case code snippet for the target vulnerability based on the semantics identification results of the test case code snippet and the test case text.

[0111] It should be noted that the above S306 and S308 can be executed in parallel, or can be executed in sequence according to a set order. For example, S306 can be executed first and then S308. This embodiment does not limit this.

[0112] S310 , the vulnerability repair code identification system inputs the vulnerability repair code fragment for the target vulnerability and the test case code fragment for the target vulnerability into the second LLM for integration and identification to obtain an identification result.

[0113] Specifically, the second LLM can integrate, for example, by splicing together, the vulnerability fix code snippet for the target vulnerability and the test case code snippet for the target vulnerability to obtain prompt information. The second LLM then identifies the vulnerability fix based on the prompt information and obtains an identification result of the vulnerability fix code. The vulnerability fix code identification system can obtain the identification result returned by the second LLM.

[0114] Based on the above description, this application provides a vulnerability repair code identification method. This method combines a code semantic analysis model for test case code and uses test case code to determine the vulnerability repair method. By identifying vulnerability test cases, it improves the recognition of public but undisclosed vulnerabilities and supplements the vulnerability identification capability. Moreover, this method uses a large language model such as the first LLM to split the submission information recognition into subtasks such as vulnerability repair code recognition and vulnerability repair test case recognition, thereby improving the vulnerability repair code recognition rate.

[0115] The first LLM in the embodiment of Figure 3 can be obtained by performing supervised fine-tuning (SFT) on the pre-trained large model. The present application uses the example of obtaining the first LLM through training a vulnerability repair code identification system. Specifically, the vulnerability repair code identification system can use vulnerability repair corpus pairs and vulnerability test case corpus pairs, wherein the vulnerability repair corpus pairs include vulnerability repair code samples and associated vulnerability common defect enumeration (Common Weakness Enumeration, CWE) information, which can be recorded as CWE-vulnerability repair corpus pairs, and the vulnerability test case corpus pairs include vulnerability test case samples and associated CWE information, which can be recorded as CWE-vulnerability test case corpus pairs. The vulnerability repair code identification system inputs the vulnerability repair corpus pairs and the vulnerability test case corpus pairs into the pre-trained large model for SFT to obtain the first LLM.

[0116] The vulnerability fix code identification system can include a patch identification large language model construction engine. The patch identification large language model construction engine can use CWE-vulnerability fix corpus pairs and CWE-vulnerability test case corpus pairs to form training text data, and use this training text data as a dataset to train a pre-trained large model. The patch identification large language model construction engine can adjust the parameters of the pre-trained large model using the SFT method, and after multiple rounds of training, obtain a patch identification large language model, specifically the first LLM mentioned above.

[0117] SFT involves pre-training a neural network model on a source dataset, the source model, and then creating a new neural network model, the target model. The target model replicates all model design and parameters of the source model except for the output layer. These model parameters embody the knowledge learned on the source dataset, and this knowledge is also applicable to the target dataset. The output layer of the source model is closely related to the labels of the source dataset and is therefore not used in the target model. During fine-tuning, an output layer with an output size equal to the number of classes in the target dataset is added to the target model, and the model parameters of this layer are randomly initialized. When training the target model on the target dataset, the output layer is trained from scratch, and the parameters of the remaining layers are fine-tuned based on the parameters of the source model. In this embodiment, the target dataset can be the dataset formed by the aforementioned CWE-vulnerability fix corpus and CWE-vulnerability test case corpus. The source model can be a pre-trained large model, and the target model can be a model with the output layer of the source model replaced with an output layer with an output size equal to the number of classes in the target dataset. The parameters of this output layer are randomly initialized, and the target model is then trained on the target dataset to obtain the first LLM.

[0118] In the embodiment of FIG3 , the code semantic recognition models for different codes are constructed in a similar manner, and the text semantic recognition models for different texts are constructed in a similar manner. The construction of the code semantic recognition model and the construction of the text semantic recognition model are respectively explained below.

[0119] In some possible implementations, the vulnerability repair code identification system can obtain code data from a vulnerability repair dataset or a vulnerability test case dataset, preprocess the code data to obtain full code differences and function-level code differences with vulnerability labels, then input the full code differences into a pre-trained language model for fine-tuning to obtain a fine-tuned model, and input the function-level code differences with vulnerability labels into the fine-tuned model to obtain a representation of the function-level code differences. The vulnerability repair code identification system can obtain a submission representation based on the representation of the function-level code differences, for example, by integrating the representations of multiple functions in a submission message. The vulnerability repair code identification system can then input the submission representation into a classifier for training to obtain a code semantic recognition model.

[0120] In other possible implementations, the vulnerability repair code identification system can obtain text data from a vulnerability repair dataset or a vulnerability test case dataset, search a vulnerability description knowledge base based on the text data to obtain vulnerability description knowledge corresponding to the text data, and search a vulnerability common defect enumeration CWE database based on the text data to obtain CWE knowledge corresponding to the text data, and then splice the text data, vulnerability description knowledge and CWE knowledge into prompt information, input the prompt information into a general large model for training, and obtain a text semantic recognition model.

[0121] In order to make the technical solution of this application clearer, the following explanation is given from the aspects of data set construction, recognition model construction, vulnerability patch engine construction, and known vulnerability patch mining service construction.

[0122] Vulnerability fix code for open source software is difficult to obtain, and constructing a vulnerability fix dataset (also known as a patch dataset) is a long and complex process. This is primarily because open source software developers typically do not directly provide vulnerability fix code (patch code), but rather expect users to upgrade to the patched version. Therefore, vulnerability fix code data, also known as vulnerability fix code samples, can be obtained from various sources, such as official open source software websites, code repositories, security communities, and third-party vulnerability libraries. This solution uses these sources to obtain vulnerability CVE numbers and corresponding commit lists as input for constructing a vulnerability fix dataset, namely, patch data.

[0123] The process of constructing the vulnerability repair dataset and vulnerability test case dataset is shown in Figure 4. It includes the key steps of constructing the function-level vulnerability repair dataset. First, the association between the vulnerability repair code sample and CWE information is established. Then, the vulnerability repair code sample is input into the vulnerability repair code analysis engine (patch analysis engine). The engine determines which parts of the vulnerability repair code sample (patch data) are test cases and which parts are functional change code related to the vulnerability repair. This results in functional change code samples and test case code samples.

[0124] Based on the function change code sample, using prompt information and a large language model, you can obtain a vulnerability repair code sample for the vulnerability and the corresponding function description. Based on the test case code sample, using prompt information and a large language model, you can obtain a vulnerability test case code sample for the vulnerability and the corresponding function description.

[0125] Specifically, after identifying functional change code samples and test case code samples, prompts and a large language model can be used to construct a repair code identification module and a vulnerability test case identification module for analyzing the functional code. Inputting the test case code into the vulnerability test case identification module generates the test case code for the vulnerability; inputting the functional change code sample into the vulnerability repair code identification module generates the vulnerability repair code for the vulnerability.

[0126] Then, based on the prompt and the large language model, a code comprehension module is built. The function of the code comprehension module is to describe the complete functionality of the code segment. The functional description of the corresponding code segment is obtained by inputting the cleaned test case code and vulnerability fix code into the code comprehension module.

[0127] Next, the vulnerability metadata, CWE information, vulnerability repair code samples, and corresponding functional descriptions are stored in a vulnerability repair dataset, and / or the vulnerability metadata, CWE information, vulnerability test case code samples, and corresponding functional descriptions are stored in a vulnerability test case dataset. In a specific implementation, the vulnerability metadata, CWE information, and the vulnerability repair code and its description, as well as the test case code and its description, output from the previous steps, can be combined and stored in a database to form a vulnerability repair dataset and a vulnerability test case dataset.

[0128] In addition to the aforementioned datasets for training the vulnerability fix identification model and the dataset for training the vulnerability test case identification model, a large language model fine-tuning corpus dataset can also be constructed. The large language model fine-tuning corpus dataset is used to train the first LLM.

[0129] Figure 5 shows the construction of the large language model fine-tuning corpus dataset. The CWE knowledge base and the aforementioned patch dataset are input into the corpus pair construction engine. In the corpus pair construction engine, each vulnerability repair code segment, its code description, the corresponding vulnerability description, the corresponding CWE description, and the corresponding original code are pieced together to form a task-decomposition text. The purpose of this is to construct data to enable the large language model to learn which part of the code segment is the vulnerability repair code. In this solution, this is called a CWE-vulnerability repair corpus pair. Similarly, vulnerability test case code can be used to generate a CWE-vulnerability test case corpus pair.

[0130] Through the above two steps, the training data construction required in this solution is completed, and the vulnerability repair dataset, vulnerability test case dataset, CWE-vulnerability repair corpus pairs and CWE-vulnerability test case corpus pairs are obtained.

[0131] Next, vulnerability identification models, such as vulnerability repair identification models and vulnerability test case identification models, are constructed. As shown in Figure 6, after the initial construction of the vulnerability repair dataset or the update of the vulnerability repair dataset, the vulnerability repair identification model of this solution can be trained using this dataset.

[0132] For different data sets, such as vulnerability repair data sets and vulnerability test case data sets, the approaches to training code semantic recognition models and text semantic recognition models are basically the same.

[0133] For the vulnerability repair dataset, the dataset is split into code changes (referred to as code change, or functional change code snippets) and text information (such as commit messages, referred to as commit msg). The code changes are then input into the code semantic recognition model building engine to obtain a code semantic recognition model 1 for vulnerability repair, and the text information is input into the text semantic recognition model building engine to obtain a text semantic recognition model 1 for vulnerability repair.

[0134] For the vulnerability test case dataset, the dataset is split into test code (denoted as test code) and test description text (denoted as test desc); then the test code is input into the code semantic recognition model construction engine to obtain the code semantic recognition model 2 for the vulnerability test case; then the submitted test description text is input into the text semantic recognition model construction engine to obtain the text semantic recognition model 2 for the vulnerability test case.

[0135] Among them, the code semantic recognition model can be constructed by a code semantic recognition model construction engine. The code semantic recognition model construction engine can obtain code data (such as source code data, test code data) from a vulnerability repair dataset or a vulnerability test case dataset, and then preprocess the code data to obtain full code differences and function-level code differences with vulnerability labels. The full code differences are used to train the fine-tuning model, and the function-level code differences with vulnerability labels are used to train the patch classification model (for example, a code semantic recognition model for vulnerability repair). The code semantic recognition model construction engine can input the full code differences into a pre-trained language model for fine-tuning to obtain a fine-tuning model. Among them, the pre-trained language model selection includes but is not limited to CodeBert, SecureBert, etc. The code semantic recognition model construction engine inputs the function-level code differences with vulnerability labels into the fine-tuning model to obtain a representation of the function-level code differences, and then obtains a submission representation based on the representation of the function-level code differences, and inputs the submission representation into the classifier for training to obtain a code semantic recognition model. Specifically, the representation of the same submission information can be obtained through the integrator to obtain the submission representation of the submission information, and the submission representation is input into the classifier for training to obtain a code semantic recognition model for vulnerability repair (mainly used for classifying vulnerability repair code, also known as patch classification model, patch recognition model), where the choice of classifier includes but is not limited to random forest, multi-layer perceptron, XGBoost and other machine learning classification models.

[0136] The text semantic recognition model can be constructed by a text semantic recognition model construction engine. The specific process is as follows: First, the text semantic recognition model construction engine uses the original text information (or text data) input by the search engine to obtain vulnerability description knowledge corresponding to the original text information from the vulnerability description knowledge base, such as vulnerability description knowledge similar to the original text information. The text semantic recognition model construction engine then obtains CWE knowledge similar to the original text information from the CWE knowledge base. Finally, the text semantic recognition model construction engine combines the input original text information, vulnerability description knowledge, and CWE knowledge into a prompt for vulnerability repair code identification, and inputs the prompt into the general large model. The category 0 or 1 output by the large model is used to determine whether it is a vulnerability patch. Among them, the general large model includes but is not limited to question-and-answer large models provided by different manufacturers.

[0137] Figure 7 also shows the construction process of the vulnerability repair identification large language model. The vulnerability repair identification large language model, or the patch identification large language model, is specifically the first LLM mentioned above. The model can be constructed by the patch identification large language model construction engine. As shown in Figure 7, the patch identification large language model construction engine combines the CWE-vulnerability repair corpus pairs and the CWE-vulnerability test case corpus pairs into training text data. Specifically, the patch identification large language model construction engine can perform use case understanding on the CWE-vulnerability test case corpus pairs, perform repair understanding on the CWE-vulnerability repair corpus pairs, and then obtain the code-repair behavior understanding pairs. The code-repair behavior understanding pairs and the vulnerability entity-CWE entity pairs (such as the CWE-vulnerability repair corpus pairs and the CWE-vulnerability test case corpus pairs) can form training text data (train text data). The patch identification large language model construction engine uses the training text data as a data set to train the pre-trained large language model, adjusts the parameters of the pre-trained large model through the SFT method, and obtains the patch identification large language model after multiple rounds of training.

[0138] Based on the model of Figure 6, for example, code semantic recognition model 1 for vulnerability repair, text semantic recognition model 1 for vulnerability repair, code semantic recognition model 2 for vulnerability test cases, and text semantic recognition model 2 for vulnerability test cases, the known vulnerability patch mining service of this embodiment is defined, as shown in Figure 8.

[0139] The Known Vulnerability Patch Mining service can be combined with the Vulnerability Patch Identification service to identify patch code for known vulnerabilities. Identification results can be stored in the storage layer and integrated with annotation services, manual verification, and review systems. The following details the Known Vulnerability Patch Mining process.

[0140] First, we retrieve the vulnerability data for patches from the annotated vulnerability database. This data is then fed into the vulnerability patch detection engine to obtain vulnerability metadata. This metadata is then fed into the commit acquisition engine to obtain the commit group to be identified. This commit group is then fed into the vulnerability patch identification server to obtain the identification results. The patch identification service then processes the results to obtain the patches to be confirmed, which are then stored in the database.

[0141] Secondly, the patch to be confirmed and its component package metadata and vulnerability-related metadata can be used in the form of triples to form a set of patch data pairs to be annotated. After manual annotation and annotation review, the patch data pairs to be annotated that are confirmed to be patches will be marked as 1, otherwise they will be marked as 0, and finally stored in the annotated patch library.

[0142] Based on the model of Figure 6, for example, code semantic recognition model 1 for vulnerability repair, text semantic recognition model 1 for vulnerability repair, code semantic recognition model 2 for vulnerability test cases, and text semantic recognition model 2 for vulnerability test cases, the open source vulnerability identification tool service of this embodiment is defined, as shown in Figure 9.

[0143] The open source vulnerability identification service can be combined with the vulnerability patch identification service to achieve open source vulnerability identification. Identification results can be stored in the storage layer and integrated with annotation services, manual verification, and review systems. The following details the open source vulnerability identification process.

[0144] First, the open source warehouse to be monitored is obtained from the open source software library, and the open source warehouse is input into the commit monitoring engine to obtain a new commit group; the obtained commit group is input into the vulnerability patch identification server to obtain the identification result, and then the vulnerability to be confirmed is obtained through post-processing of the vulnerability identification service and stored in the database.

[0145] Secondly, the vulnerability to be confirmed and the metadata of the component package to which it belongs, as well as the vulnerability-related metadata, can be used in the form of triples to form a set of vulnerability data pairs to be labeled. After manual labeling and labeling review, the vulnerability data pairs to be labeled that are confirmed to be vulnerabilities will be labeled as 1, otherwise they will be labeled as 0, and finally stored in the labeled vulnerability library.

[0146] Based on the above description, it can be seen that the method of the present application is based on vulnerability-related corpus, vulnerability-related knowledge base, and function-level patch corpus to train a large language model to complete function-level code snippet recognition and task decomposition tasks, and split patch recognition into subtasks such as vulnerability repair code recognition, vulnerability repair test case recognition, and message association through a large language model to solve the problem of low vulnerability patch recognition rate. Among them, the method constructs a function-level vulnerability repair dataset based on a large language model to solve the problem that current open source vulnerability patch data lacks function-level patch data. The method improves the recognition of public but undisclosed vulnerabilities and supplements the vulnerability identification capability by constructing a code semantic analysis model for vulnerability test cases and using test case code to judge the vulnerability repair method.

[0147] Based on the aforementioned vulnerability repair code identification method, the present application also provides a vulnerability repair code identification system. As shown in FIG10 , the vulnerability repair code identification system 1000 includes:

[0148] An acquisition module 1002 is configured to acquire submission information of the open source software, the submission information including the submitted code snippet and text information associated with the code snippet;

[0149] a disassembly module 1004 for inputting the submitted information into a first language model for recognition to obtain function change information and test case information, wherein the function change information includes a function change code snippet and a function change text, and the test case information includes a test case code snippet and a test case text;

[0150] Identification module 1006, configured to input the function change information into a vulnerability repair identification model to obtain a vulnerability repair code snippet for a target vulnerability, and input the test case information into a vulnerability test case identification model to obtain a test case code snippet for the target vulnerability;

[0151] The integration module 1008 is configured to input the vulnerability repair code snippet for the target vulnerability and the test case code snippet for the target vulnerability into the second language model for integration and recognition to obtain a recognition result.

[0152] Exemplarily, the acquisition module 1002 , the disassembly module 1004 , the identification module 1006 , and the integration module 1008 may be implemented by hardware or software.

[0153] When implemented via software, the acquisition module 1002, disassembly module 1004, identification module 1006, or integration module 1008 can be an application running on a computing device. This application can be provided to users in the form of a virtualization service. Virtualization services can include virtual machine (VM) services, bare metal server (BMS) services, or container services. VM services can be services that use virtualization technology to create a virtual machine (VM) resource pool on multiple physical hosts to provide users with VMs on demand. BMS services are services that create a virtual BMS resource pool on multiple physical hosts to provide users with BMSs on demand. Container services are services that create a virtual container resource pool on multiple physical hosts to provide users with containers on demand. A VM is a simulated virtual computer, or logically a single computer. BMS is a scalable, high-performance computing service with computing performance comparable to that of a traditional physical machine and secure physical isolation. Containers are a kernel virtualization technology that provides lightweight virtualization to isolate user space, processes, and resources. It should be understood that the VM service, BMS service and container service in the above-mentioned virtualization services are only specific examples. In actual applications, virtualization services can also be other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0154] When implemented in hardware, the acquisition module 1002, the disassembly module 1004, the identification module 1006, or the integration module 1008 may include at least one computing device, such as a server. Alternatively, the acquisition module 1002, the disassembly module 1004, the identification module 1006, or the integration module 1008 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0155] In some possible implementations, the vulnerability repair identification model includes a vulnerability repair code semantic identification model and a vulnerability repair text semantic identification model, and the vulnerability test case identification model includes a vulnerability test case code semantic identification model and a vulnerability test case text semantic identification model;

[0156] The identification module 1006 is specifically used for:

[0157] Inputting the function-changing code snippet into the vulnerability-fixing code semantic recognition model, and inputting the function-changing text into the vulnerability-fixing text semantic recognition model, and obtaining a vulnerability-fixing code snippet for the target vulnerability based on the semantic recognition results of the function-changing code snippet and the semantic recognition results of the function-changing text;

[0158] The test case code snippet is input into the vulnerability test case code semantic recognition model, and the test case text is input into the vulnerability test case text semantic recognition model, and the test case code snippet for the target vulnerability is obtained according to the semantic recognition result of the test case code snippet and the semantic recognition result of the test case text.

[0159] In some possible implementations, the system 1000 further includes:

[0160] A training module 1005 is configured to obtain code data from a vulnerability repair dataset or a vulnerability test case dataset, preprocess the code data to obtain full code differences and function-level code differences with vulnerability labels, and input the full code differences into a pre-trained language model for fine-tuning to obtain a fine-tuned model.

[0161] The training module 1005 is also used to input the function-level code differences with vulnerability labels into the fine-tuning model to obtain the representation of the function-level code differences, obtain the submission representation based on the representation of the function-level code differences, input the submission representation into the classifier for training, and obtain the code semantic recognition model.

[0162] Similar to the aforementioned acquisition module 1002, disassembly module 1004, identification module 1006, or integration module 1008, the training module 1005 can be implemented via software or hardware. When implemented via software, the training module 1005 can be an application running on a computing device. This application can be provided to users in the form of a virtualized service. Virtualized services can include VM services, BMS services, or container services. When implemented via hardware, the training module 1005 can include at least one computing device, such as a server. Alternatively, the training module 1005 can be implemented using an ASIC or a PLD.

[0163] In some possible implementations, the system 1000 further includes:

[0164] A training module 1005 is configured to obtain text data from a vulnerability repair dataset or a vulnerability test case dataset, search a vulnerability description knowledge base based on the text data to obtain vulnerability description knowledge corresponding to the text data, and search a Common Weakness Enumeration (CWE) database based on the text data to obtain CWE knowledge corresponding to the text data;

[0165] The training module 1005 is further configured to combine the text data, the vulnerability description knowledge, and the CWE knowledge into prompt information, input the prompt information into a general language model for training, and obtain the text semantic recognition model.

[0166] In some possible implementations, the system 1000 further includes:

[0167] The data set construction module 1007 is configured to establish an association between a vulnerability repair code sample and CWE information, input the vulnerability repair code sample into the vulnerability repair code analysis engine, obtain the function change code sample and the test case code sample, and obtain a vulnerability repair code sample and a corresponding function description for the vulnerability based on the function change code sample using prompt information and a language model. Based on the test case code sample, obtain a vulnerability test case code sample and a corresponding function description for the vulnerability using prompt information and a language model.

[0168] The data set construction module 1007 is also used to store the metadata of the vulnerability, the CWE information, the vulnerability repair code sample and the corresponding functional description into the vulnerability repair data set, and / or store the metadata of the vulnerability, the CWE information, the vulnerability test case code sample and the corresponding functional description into the vulnerability test case data set.

[0169] Similar to training module 1005, dataset construction module 1007 can be implemented via software or hardware. When implemented via software, dataset construction module 1007 can be an application running on a computing device. This application can be provided to users in the form of a virtualized service. Virtualized services may include VM services, BMS services, or container services. When implemented via hardware, dataset construction module 1007 can include at least one computing device, such as a server. Alternatively, training module 1005 can be implemented using an ASIC or PLD.

[0170] In some possible implementations, the system 1000 further includes:

[0171] The training module 1005 is used to obtain a vulnerability repair corpus pair and a vulnerability test case corpus pair, wherein the vulnerability repair corpus pair includes a vulnerability repair code sample and associated CWE information, and the vulnerability test case corpus pair includes a vulnerability test case sample and associated CWE information. The vulnerability repair corpus pair and the vulnerability test case corpus pair are input into the pre-training model for supervised fine-tuning SFT to obtain the first language model.

[0172] In some possible implementations, the obtaining module 1002 is specifically configured to:

[0173] Get vulnerability metadata from the vulnerability database;

[0174] Obtain submission information of the open source software based on the vulnerability metadata.

[0175] In some possible implementations, the obtaining module 1002 is specifically configured to:

[0176] Monitor open source repositories and obtain submission information of open source software.

[0177] In some possible implementations, the first language model or the second language model is a large language model (LLM).

[0178] This application also provides a computing device 1100. As shown in Figure 11, computing device 1100 includes a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. Processor 1104, memory 1106, and communication interface 1108 communicate with each other via bus 1102. Computing device 1100 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 1100.

[0179] Bus 1102 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG11 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 1102 may include a path for transmitting information between various components of computing device 1100 (e.g., memory 1106, processor 1104, and communication interface 1108).

[0180] The processor 1104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0181] The memory 1106 may include a volatile memory, such as a random access memory (RAM). The memory 1106 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD). The memory 1106 stores executable program code, and the processor 1104 executes the executable program code to implement the aforementioned vulnerability repair code identification method. Specifically, the memory 1106 stores instructions for the vulnerability repair code identification system 1000 to execute the vulnerability repair code identification method.

[0182] The communication interface 1108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0183] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0184] 12 , the computing device cluster includes at least one computing device 1100. The memory 1106 of one or more computing devices 1100 in the computing device cluster may store the same instructions of the vulnerability repair code identification system 1000 for executing the vulnerability repair code identification method.

[0185] In some possible implementations, one or more computing devices 1100 in the computing device cluster may also be used to execute some of the instructions of the vulnerability fix code identification system 1000 for executing the vulnerability fix code identification method. In other words, the combination of one or more computing devices 1100 may jointly execute the instructions of the vulnerability fix code identification system 1000 for executing the vulnerability fix code identification method.

[0186] It should be noted that the memory 1106 in different computing devices 1100 in the computing device cluster may store different instructions for executing part of the functions of the vulnerability repair code identification system 1000 .

[0187] FIG13 illustrates a possible implementation. As shown in FIG13 , two computing devices 1100A and 1100B are connected via a communication interface 1108. The memory in computing device 1100A stores instructions for executing the functions of acquisition module 1002 and disassembly module 1004. The memory in computing device 1100B stores instructions for executing the functions of identification module 1006 and integration module 1008. In other words, the memories 1106 of computing devices 1100A and 1100B jointly store instructions for vulnerability repair code identification system 1000 to execute the vulnerability repair code identification method. Furthermore, computing device 1100A may also store instructions for executing the functions of training module 1005 and dataset construction module 1007.

[0188] The connection method between the computing device clusters shown in Figure 13 can be considered to be based on the fact that the vulnerability repair code identification method provided in this application requires a large amount of computing power to identify the functional change information and test case information obtained after disassembly. Therefore, it is considered to transfer the functions implemented by the identification module 1006 and the integration module 1008 to the computing device 1100B.

[0189] It should be understood that the functionality of the computing device 1100A shown in FIG13 may also be implemented by multiple computing devices 1100. Similarly, the functionality of the computing device 1100B may also be implemented by multiple computing devices 1100.

[0190] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. FIG14 shows a possible implementation. As shown in FIG14 , two computing devices 1100C and 1100D are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1106 in the computing device 1100C stores instructions for executing the functions of the acquisition module 1002 and the disassembly module 1004. At the same time, the memory 1106 in the computing device 1100D stores instructions for executing the functions of the identification module 1006 and the integration module 1008.

[0191] The connection method between the computing device clusters shown in Figure 14 can be that considering that the vulnerability repair code identification method provided in this application requires a large amount of computing power to identify the functional change information and test case information obtained after disassembly, it is considered to entrust the functions implemented by the identification module 1006 and the integration module 1008 to the computing device 1100D for execution.

[0192] It should be understood that the functionality of the computing device 1100C shown in FIG14 may also be accomplished by multiple computing devices 1100. Similarly, the functionality of the computing device 1100D may also be accomplished by multiple computing devices 1100.

[0193] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned vulnerability repair code identification system 1000 for executing the vulnerability repair code identification method.

[0194] The present application also provides a computer program product comprising instructions. The computer program product may be software or a program product comprising instructions that can be executed on a computing device or stored on any available medium. When the computer program product is executed on at least one computing device, the at least one computing device executes the vulnerability patch code identification method described above.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A vulnerability repair code identification method, characterized in that: The method comprises: The vulnerability repair code identification system obtains submission information of the open source software, wherein the submission information includes a submitted code snippet and text information associated with the code snippet; The vulnerability repair code identification system inputs the submission information into a first language model for identification to obtain function change information and test case information, wherein the function change information includes a function change code snippet and a function change text, and the test case information includes a test case code snippet and a test case text; The vulnerability repair code identification system inputs the function change information into a vulnerability repair identification model to obtain a vulnerability repair code snippet for a target vulnerability, and inputs the test case information into a vulnerability test case identification model to obtain a test case code snippet for the target vulnerability; The vulnerability repair code identification system inputs the vulnerability repair code fragment for the target vulnerability and the test case code fragment for the target vulnerability into the second language model for integration and identification to obtain an identification result.

2. The method according to claim 1, characterized in that The vulnerability repair identification model includes a vulnerability repair code semantic identification model and a vulnerability repair text semantic identification model, and the vulnerability test case identification model includes a vulnerability test case code semantic identification model and a vulnerability test case text semantic identification model; The method comprises: The vulnerability repair code identification system inputs the function change code snippet into the vulnerability repair code semantic identification model, and inputs the function change text into the vulnerability repair text semantic identification model, and obtains the vulnerability repair code snippet for the target vulnerability according to the semantic recognition result of the function change code snippet and the semantic recognition result of the function change text; The vulnerability repair code identification system inputs the test case code snippet into the vulnerability test case code semantic recognition model, and inputs the test case text into the vulnerability test case text semantic recognition model, and obtains the test case code snippet for the target vulnerability based on the semantic recognition result of the test case code snippet and the semantic recognition result of the test case text.

3. The method according to claim 2, characterized in that The method further comprises: Obtain code data from vulnerability fix datasets or vulnerability test case datasets; Preprocessing the code data to obtain full code differences and function-level code differences with vulnerability labels; Inputting the full code difference into a pre-trained language model for fine-tuning to obtain a fine-tuned model; Inputting the function-level code difference with the vulnerability label into the fine-tuning model to obtain a representation of the function-level code difference; A submission representation is obtained according to the representation of the function-level code difference, and the submission representation is input into a classifier for training to obtain the code semantic recognition model.

4. The method according to claim 2, characterized in that: The method further comprises: Get text data from vulnerability fix datasets or vulnerability test case datasets; Searching a vulnerability description knowledge base according to the text data to obtain vulnerability description knowledge corresponding to the text data, and searching a vulnerability common defect enumeration CWE database according to the text data to obtain CWE knowledge corresponding to the text data; Splicing the text data, the vulnerability description knowledge and the CWE knowledge into prompt information; The prompt information is input into a general language model for training to obtain the text semantic recognition model.

5. The method according to claim 3 or 4, characterized in that: The dataset is constructed in the following way: Establish the relationship between vulnerability repair code samples and CWE information; Inputting the vulnerability repair code sample into a vulnerability repair code analysis engine to obtain the function change code sample and the test case code sample; According to the function change code sample, using prompt information and a language model, a vulnerability repair code sample for the vulnerability and a corresponding function description are obtained; according to the test case code sample, using prompt information and a language model, a vulnerability test case code sample for the vulnerability and a corresponding function description are obtained; The metadata of the vulnerability, the CWE information, the vulnerability repair code sample and the corresponding functional description are stored in the vulnerability repair data set, and / or the metadata of the vulnerability, the CWE information, the vulnerability test case code sample and the corresponding functional description are stored in the vulnerability test case data set.

6. The method according to any one of claims 1 to 5, characterized in that: The first language model is trained in the following way: Obtaining a vulnerability repair corpus pair and a vulnerability test case corpus pair, wherein the vulnerability repair corpus pair includes a vulnerability repair code sample and associated CWE information, and the vulnerability test case corpus pair includes a vulnerability test case sample and associated CWE information; The vulnerability repair corpus pair and the vulnerability test case corpus pair are input into a pre-trained model for supervised fine-tuning SFT to obtain the first language model.

7. The method according to any one of claims 1 to 6, characterized in that: The vulnerability repair code identification system obtains submission information of open source software, including: The vulnerability repair code identification system obtains vulnerability metadata from the vulnerability library; The vulnerability repair code identification system obtains submission information of the open source software according to the vulnerability metadata.

8. The method according to any one of claims 1 to 6, characterized in that: The vulnerability repair code identification system obtains submission information of open source software, including: The vulnerability repair code identification system monitors the open source repository to obtain submission information of the open source software.

9. The method according to any one of claims 1 to 8, characterized in that: The first language model or the second language model is a large language model LLM.

10. A vulnerability repair code identification system, characterized in that: The system comprises: An acquisition module, used to acquire submission information of the open source software, wherein the submission information includes a submitted code snippet and text information associated with the code snippet; a disassembly module, configured to input the submitted information into a first language model for recognition to obtain function change information and test case information, wherein the function change information includes a function change code snippet and a function change text, and the test case information includes a test case code snippet and a test case text; An identification module, used for inputting the function change information into a vulnerability repair identification model to obtain a vulnerability repair code snippet for a target vulnerability, and inputting the test case information into a vulnerability test case identification model to obtain a test case code snippet for the target vulnerability; The integration module is used to input the vulnerability repair code snippet for the target vulnerability and the test case code snippet for the target vulnerability into the second language model for integration and recognition to obtain a recognition result.

11. The system according to claim 10, characterized in that The vulnerability repair identification model includes a vulnerability repair code semantic identification model and a vulnerability repair text semantic identification model, and the vulnerability test case identification model includes a vulnerability test case code semantic identification model and a vulnerability test case text semantic identification model; The identification module is specifically used for: Inputting the function change code snippet into the vulnerability repair code semantic recognition model, and inputting the function change text into the vulnerability repair text semantic recognition model, and obtaining a vulnerability repair code snippet for a target vulnerability according to a semantic recognition result of the function change code snippet and a semantic recognition result of the function change text; The test case code snippet is input into the vulnerability test case code semantic recognition model, and the test case text is input into the vulnerability test case text semantic recognition model, and the test case code snippet for the target vulnerability is obtained according to the semantic recognition result of the test case code snippet and the semantic recognition result of the test case text.

12. The system according to claim 11, characterized in that The system further comprises: A training module is used to obtain code data from a vulnerability repair dataset or a vulnerability test case dataset, preprocess the code data to obtain full code differences and function-level code differences with vulnerability labels, and input the full code differences into a pre-trained language model for fine-tuning to obtain a fine-tuned model; The training module is also used to input the function-level code differences with vulnerability labels into the fine-tuning model to obtain the representation of the function-level code differences, obtain the submission representation based on the representation of the function-level code differences, input the submission representation into the classifier for training, and obtain the code semantic recognition model.

13. The system according to claim 11, characterized in that The system further comprises: A training module is used to obtain text data from a vulnerability repair data set or a vulnerability test case data set, search a vulnerability description knowledge base according to the text data to obtain vulnerability description knowledge corresponding to the text data, and search a vulnerability common defect enumeration CWE database according to the text data to obtain CWE knowledge corresponding to the text data; The training module is also used to splice the text data, the vulnerability description knowledge and the CWE knowledge into prompt information. The prompt information is input into a general language model for training to obtain the text semantic recognition model.

14. The system according to claim 12 or 13, characterized in that The system further comprises: A data set construction module is used to establish an association relationship between a vulnerability repair code sample and CWE information, input the vulnerability repair code sample into a vulnerability repair code analysis engine, obtain the function change code sample and the test case code sample, obtain the vulnerability repair code sample for the vulnerability and the corresponding function description according to the function change code sample, and obtain the vulnerability test case code sample for the vulnerability and the corresponding function description according to the test case code sample, using prompt information and a language model; The data set construction module is also used to store the metadata of the vulnerability, the CWE information, the vulnerability repair code sample and the corresponding functional description into the vulnerability repair data set, and / or store the metadata of the vulnerability, the CWE information, the vulnerability test case code sample and the corresponding functional description into the vulnerability test case data set.

15. The system according to any one of claims 10 to 14, characterized in that The system further comprises: A training module is used to obtain a vulnerability repair corpus pair and a vulnerability test case corpus pair, wherein the vulnerability repair corpus pair includes a vulnerability repair code sample and associated CWE information, and the vulnerability test case corpus pair includes a vulnerability test case sample and associated CWE information, and the vulnerability repair corpus pair and the vulnerability test case corpus pair are input into a pre-trained model for supervised fine-tuning SFT to obtain the first language model.

16. The system according to any one of claims 10 to 15, characterized in that The acquisition module is specifically used for: Get vulnerability metadata from the vulnerability database; According to the vulnerability metadata, submission information of the open source software is obtained.

17. The system according to any one of claims 10 to 15, characterized in that The acquisition module is specifically used for: Monitor open source repositories to obtain submission information of open source software.

18. The system according to any one of claims 10 to 17, characterized in that The first language model or the second language model is a large language model LLM.

19. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions so that the computing device cluster executes the vulnerability repair code identification method as described in any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that: It comprises computer-readable instructions; the computer-readable instructions are used to implement the vulnerability repair code identification method according to any one of claims 1 to 9.

21. A computer program product, characterized in that It comprises computer-readable instructions; the computer-readable instructions are used to implement the vulnerability repair code identification method according to any one of claims 1 to 9.

Images