Intelligent contract vulnerability mining method and system based on large language model

Abstract

The invention discloses an intelligent contract vulnerability mining method and system based on a large language model. The system comprises a compiler, a static analysis tool, a vulnerability detection agent, a result judgment agent and a large model detection result evaluation module. According to the method, for specific vulnerabilities, vulnerability codes and contract codes are found through auditing, iterative optimization is continuously conducted through a vulnerability detection agent, a result judgment agent and a large model detection result evaluation module, optimized vulnerability description is configured in the vulnerability detection agent, and intelligent contract vulnerability mining is achieved. Compared with the development process of a traditional static analysis vulnerability detector, when the method is adopted for vulnerability mining, a developer does not need to deeply understand the underlying logic of an intelligent contract language and the internal implementation of a detection tool, and only needs to understand a vulnerability scene and analyze an output result of a large model, so that the requirement on a user is low, and the development efficiency is high. Moreover, the method can be suitable for various different vulnerability scenes, and is high in vulnerability recognition accuracy.

Description

Technical Field

[0001] This invention relates to the field of smart contract vulnerability mining, specifically to a method and system for smart contract vulnerability mining based on a large language model. Background Technology

[0002] In recent years, the development of blockchain technologies such as Ethereum has spurred the application of smart contracts. Smart contracts are code deployed on the blockchain that can automatically execute pre-defined logic, and their widespread adoption has directly driven the rise of decentralized finance (DeFi). DeFi utilizes smart contracts to execute complex financial activities and manages enormous amounts of digital assets.

[0003] Because DeFi projects attract a massive number of users and huge amounts of capital, the security of their smart contracts is paramount. However, security vulnerabilities or business logic flaws in smart contract code are highly susceptible to malicious attacks, causing significant and often irreparable economic losses to project teams and users. This security risk has become one of the key bottlenecks hindering the healthy development of the DeFi ecosystem.

[0004] To mitigate the aforementioned risks, DeFi projects typically engage professional security auditing firms to conduct code reviews before deploying contracts. Existing auditing methods primarily rely on manual reviews by security experts, supplemented by traditional static or dynamic analysis tools. This approach is not only costly and time-consuming, but the quality of the audits also heavily depends on the auditors' personal experience, making it difficult to scale.

[0005] With the rapid development of Large Language Model (LLM) technology, leveraging its code understanding and reasoning capabilities for vulnerability discovery has become a new technological direction. Although some attempts have emerged to combine LLM with smart contract auditing, existing technological approaches have significant limitations: some works simply combine LLM with traditional analysis tools, failing to fully utilize the model's potential; others design highly customized systems for specific vulnerability types, lacking scalability and generalization capabilities. Therefore, the industry urgently needs a method that can systematically and generalizeably improve the effectiveness and robustness of large models in smart contract vulnerability detection. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method and system for smart contract vulnerability mining based on a large language model. The specific technical solution is as follows:

[0007] A smart contract vulnerability mining system based on a large language model includes a compiler, a static analysis tool, a vulnerability detection agent, a result judgment agent, and a large model detection result evaluation module.

[0008] The compiler is used to receive the DeFi project code repository, compilation configuration and custom audit scope, compile the DeFi project, exclude irrelevant interface files, and output smart contract code involving key business logic;

[0009] The static analysis tool is used to receive the smart contract code involving key business logic output by the compiler, obtain the inheritance relationship of the contract according to the abstract syntax tree of the smart contract, bind the information of the contract and its inherited parent contract, and output the smart contract code that completes the inheritance relationship.

[0010] The vulnerability detection agent is used to concatenate vulnerability descriptions and audit findings of vulnerability code or code after vulnerability repair into the vulnerability detection agent's prompt word template, or to concatenate vulnerability descriptions and smart contract code that completes the inheritance relationship into the prompt word template; and to guide the large language model to play the role of a senior vulnerability auditor, perform vulnerability analysis on the given smart contract code based on the given vulnerability description, output vulnerability report one, and the large language model's thought process;

[0011] The result judgment agent is used to concatenate the vulnerability report one output by the vulnerability detection agent and the thought process of the large language model into the prompt word template, guiding the large language model to play the role of a senior vulnerability auditor and analyze whether the vulnerability report is consistent with the final conclusion of the large language model's thought process; if they are consistent, the vulnerability report is retained; if they are inconsistent, a vulnerability report of the same format needs to be regenerated according to the large language model's thought process, and vulnerability report two is output.

[0012] The large model detection result evaluation module is used during the vulnerability description tuning process to receive the vulnerability report and annotation function set one or annotation function set two output by the result judgment agent, and performs the following operations:

[0013] Remove the functions of the negligible type from the functions in vulnerability report 2 to obtain vulnerability report 3;

[0014] Select functions belonging to the target detection type from the functions in vulnerability report 3, and construct a set of functions that correctly detect vulnerabilities;

[0015] From the functions belonging to the detection target type in either labeled function set one or labeled function set two, select functions that are not in the set of functions that correctly detect vulnerabilities and treat them as functions that have been missed.

[0016] From the functions in vulnerability report 3, select functions that are of the undetectable type and include them as additional reported functions;

[0017] Then, based on the number of correctly detected vulnerable functions, the number of missed functions, and the number of over-reported functions, multiple performance metrics are calculated.

[0018] The labeled function set one and labeled function set two are function sets formed by labeling the functions involving the target vulnerability in the actual vulnerability code of the specific target vulnerability type discovered in the audit and the contract code after the vulnerability is fixed.

[0019] Furthermore, the function types in the labeled function set one and labeled function set two include the following three types:

[0020] Target type detected: The function is seriously harmful and fully meets the typical characteristics of the target vulnerability.

[0021] Ignoreable type: Functions that meet the characteristics of vulnerabilities but have minor actual harm can be used as warning items in real audits.

[0022] Undetectable type: The function is harmless or irrelevant to the vulnerability pattern and is used as a negative sample for evaluating false positives.

[0023] Furthermore, the performance metrics calculated by the large model detection result evaluation module include precision, recall, F1 score, and false positive rate.

[0024] Furthermore, the vulnerability report is in JSON format and includes fields such as vulnerability title, vulnerability description, vulnerability impact, and vulnerable function.

[0025] A method for discovering smart contract vulnerabilities based on a large language model is disclosed. This method is implemented by a smart contract vulnerability discovery system based on a large language model and includes the following steps:

[0026] S1: Determine the type of target vulnerability;

[0027] S2: Collect audit findings belonging to the target vulnerability type, including contract code and problem descriptions, where the contract code includes the vulnerable code and the contract code after the vulnerability is fixed;

[0028] S3: Record the vulnerable code in the contract code discovered during the audit and the functions in the contract code after the vulnerability is fixed that involve the target vulnerability, and mark these functions according to three types to obtain the first set of marked functions corresponding to the vulnerable code and the second set of marked functions corresponding to the contract code after the vulnerability is fixed: the function types include the target detection type, the ignoreable type, and the undetectable type;

[0029] S4: Based on the problem descriptions found in the audit, analyze the common characteristics and attack principles of the target vulnerabilities, and write a structured vulnerability description, which includes three dimensions: vulnerability mode, potential harm, and additional tips.

[0030] S5: Input the vulnerability description and the contract code discovered by the audit obtained in step S2 into the vulnerability detection agent. The vulnerability detection agent will then concatenate these two into the prompt word template of the vulnerability detection agent and guide the large language model to play the role of a senior vulnerability auditor. Based on the given vulnerability description, the large language model will perform vulnerability analysis on the given contract code and output vulnerability report one and the thought process of the large language model.

[0031] S6: The result-judgment agent concatenates the vulnerability report I output by the vulnerability detection agent and the thought process of the large language model into the prompt word template, guiding the large language model to play the role of a senior vulnerability auditor and analyze whether the vulnerability report is consistent with the final conclusion of the large language model's thought process; if they are consistent, the vulnerability report is retained; if they are inconsistent, a vulnerability report in the same format needs to be regenerated based on the large language model's thought process, called vulnerability report II.

[0032] S7: The large model detection result evaluation module receives vulnerability report two and either annotation function set one or annotation function set two, and deletes the functions of the negligible type from the functions in vulnerability report two to obtain vulnerability report three; then execute:

[0033] Select functions belonging to the detection target type from the functions in vulnerability report 3 to construct a set of functions that correctly detect vulnerabilities; filter out functions that are not in the set of functions that correctly detect vulnerabilities from the functions belonging to the detection target type in labeled function set 1 or labeled function set 2, and treat them as functions that are not reported.

[0034] From the functions in vulnerability report 3, select functions that are of the undetectable type and include them as additional reported functions;

[0035] Then, based on the number of correctly detected vulnerable functions, the number of missed functions, and the number of over-reported functions, calculate various performance indicators.

[0036] S8: Repeat steps S5 to S7 to conduct multiple rounds of testing and calculate the average value of each indicator obtained from the multiple rounds of testing.

[0037] S9: Based on the detection results obtained from multiple rounds of testing in S8, including the model thinking process and vulnerability reports, and the average values ​​of various performance indicators, determine whether the average values ​​of each performance indicator meet the preset thresholds:

[0038] If so, the vulnerability description in step S4 will be directly used as the vulnerability description for subsequent actual target vulnerability discovery and configured to the vulnerability detection agent.

[0039] If not, then the vulnerability description in step S4 will be optimized in the following two ways, and then S10 will be executed:

[0040] When there are over-reported functions, analyze whether these functions are vulnerability functions that were missed in labeled function set 1 and labeled function set 2. If so, modify the type of the over-reported function to the detection target type or the negligible type according to the severity of the harm. If the function is highly harmful, modify it to the detection target type; if the harm is minor, modify it to the negligible type. If not, it means that the vulnerability description written by S4 has defects. It is necessary to adjust the potential harm and additional hint fields in the vulnerability description in conjunction with the reasoning process to prompt the large model to exclude functions that are prone to over-reporting during the detection process.

[0041] When there are missed functions, it is necessary to analyze the thought process of the large language model, find out the reason for the large language model's judgment error, and manually adjust the vulnerability pattern or additional prompts to prompt the large language model to pay extra attention to vulnerable functions that are easily overlooked during the detection process.

[0042] S10: Determine whether the number of optimization attempts has reached the preset threshold. If not, return to S5; if yes, execute S11.

[0043] S11: Return to S1, adjust the definition of the target vulnerability, break the target vulnerability into multiple more refined and specific sub-vulnerabilities, and then execute S2 to S10 for each sub-vulnerability;

[0044] S12: Conduct real smart contract vulnerability discovery based on the final vulnerability description.

[0045] Furthermore, based on the final vulnerability description, real smart contract vulnerability mining is conducted, specifically including:

[0046] (1) Input the DeFi project code repository, compilation configuration and custom audit scope into the compiler, and the compiler will compile the DeFi project, exclude irrelevant interface files, and output the smart contract code involving key business logic;

[0047] (2) The static analysis tool receives the smart contract code involving key business logic output by the compiler, obtains the inheritance relationship of the contract according to the abstract syntax tree of the smart contract, binds the information of the contract and its inherited parent contract, and outputs the smart contract code that completes the inheritance relationship.

[0048] (3) The vulnerability detection agent concatenates the smart contract code with the inheritance relationship and the optimized vulnerability description into the prompt word template; and guides the big language model to play the role of a senior vulnerability auditor, perform vulnerability analysis on the given smart contract code according to the given vulnerability description, output vulnerability report one, and the big language model's thinking process.

[0049] (4) The result judgment agent splices the vulnerability report output by the input vulnerability detection agent and the thinking process of the big language model into the prompt word template, and guides the big language model to play the role of a senior vulnerability auditor. It analyzes whether the vulnerability report is consistent with the final conclusion of the big language model's thinking process. If they are consistent, the vulnerability report is retained. If they are not consistent, a vulnerability report in the same format needs to be regenerated according to the thinking process of the big language model, so as to complete the vulnerability discovery.

[0050] Furthermore, the specific conditions for the three dimensions of the vulnerability description in S4 are as follows:

[0051] Vulnerability pattern: A precise formal definition of the general code paradigm and triggering conditions of a vulnerability.

[0052] Potential harm: Explain the potential asset loss or other negative impacts that may result if this vulnerability pattern is exploited.

[0053] Additional tips: We provide some edge cases, special scenarios, or repair suggestions to assist large language models in making more comprehensive judgments.

[0054] The beneficial effects of this invention are as follows:

[0055] The smart contract vulnerability mining method and system based on a large language model of the present invention can be applied to a variety of different vulnerability scenarios such as reentrancy vulnerabilities, access control vulnerabilities, precision loss vulnerabilities, and sandwich attacks. When mining vulnerabilities, the method uses optimized vulnerability description prompts, which can achieve a vulnerability identification accuracy of more than 80% in multiple vulnerability scenarios.

[0056] Compared to the development process of traditional static analysis vulnerability detectors, the smart contract vulnerability mining method and system based on a large language model of this invention does not require developers to have an in-depth understanding of the underlying logic of smart contract languages ​​and the internal implementation of detection tools. They only need to understand the vulnerability scenario and analyze the output results of the large model, thus reducing the requirements for users. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of a smart contract vulnerability mining system based on a large language model, according to an embodiment of the present invention.

[0058] Figure 2This is a schematic diagram illustrating the optimization process of vulnerability description in the smart contract vulnerability mining method based on a large language model according to an embodiment of the present invention.

[0059] Figure 3 This is a schematic diagram illustrating the online vulnerability discovery process in the smart contract vulnerability discovery method based on a large language model according to an embodiment of the present invention. Detailed Implementation

[0060] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become clearer. It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0061] like Figure 1 As shown, a smart contract vulnerability mining system based on a large language model includes a compiler, a static analysis tool, a vulnerability detection agent, a result judgment agent, and a large model detection result evaluation module, wherein:

[0062] The compiler receives the DeFi project code repository, compilation configuration, and custom audit scope, compiles the DeFi project, excludes irrelevant interface files, and outputs smart contract code involving key business logic.

[0063] In this embodiment, the solc compiler is selected.

[0064] Static analysis tools are used to receive smart contract code involving key business logic output by the compiler, obtain the inheritance relationship of the contract based on the abstract syntax tree of the smart contract, bind information between the contract and its inherited parent contract, and output smart contract code that completes the inheritance relationship.

[0065] The vulnerability detection agent, when used for vulnerability description optimization, is used to concatenate the vulnerability description to be optimized and the audited vulnerability code or the code after the vulnerability is fixed into the vulnerability detection agent's prompt word template; in real DeFi project vulnerability detection scenarios, it is used to concatenate the smart contract code with the inheritance relationship and the optimized vulnerability description into the prompt word template; and it guides the large language model to play the role of a senior vulnerability auditor, perform vulnerability analysis on the given smart contract code based on the given vulnerability description, and output a vulnerability report in JSON format containing fields such as vulnerability title, vulnerability description, vulnerability impact, and vulnerability function, as well as the large language model's thought process.

[0066] In this embodiment, the DeepSeek-R1-250528 model is selected as the large language model.

[0067] The result-judgment agent is used to concatenate the vulnerability report I output by the vulnerability detection agent and the thought process of the large language model into the prompt word template. This guides the large language model to act as a senior vulnerability auditor, analyzing whether the vulnerability report matches the final conclusion of the large language model's thought process. If they match, the vulnerability report is retained; otherwise, a new vulnerability report in the same format, called vulnerability report II, needs to be generated based on the large language model's thought process. In real-world DeFi project vulnerability detection scenarios, vulnerability report II is the final vulnerability report; during vulnerability description optimization, vulnerability report II serves as one of the inputs to the large model's detection result evaluation module.

[0068] In this embodiment, the large language model selected for the result judgment agent is the DeepSeek-V3-250324 model.

[0069] The large model detection result evaluation module receives the vulnerability report II and either annotation function set I or annotation function set II output by the result judgment agent, and performs the following operations:

[0070] Remove the functions of the negligible type from the functions in vulnerability report 2 to obtain vulnerability report 3;

[0071] Select functions belonging to the target detection type from the functions in vulnerability report 3, and construct a set of functions that correctly detect vulnerabilities;

[0072] From the functions belonging to the detection target type in either labeled function set one or labeled function set two, select functions that are not in the set of functions that correctly detect vulnerabilities and treat them as functions that have been missed.

[0073] From the functions in vulnerability report 3, select functions that are of the undetectable type and include them as additional reported functions;

[0074] Then, based on the number of correctly detected vulnerable functions, the number of missed functions, and the number of overreported functions, performance metrics including precision, recall, F1 score, and false positive rate are calculated.

[0075] The first and second annotation function sets are function sets formed by annotating the functions related to the target vulnerability in both the actual vulnerability code and the contract code after the vulnerability has been patched, based on the specific target vulnerability types discovered during the audit. There are three types of functions here:

[0076] Target type detected: The function is seriously harmful and fully meets the typical characteristics of the target vulnerability.

[0077] Ignoreable type: Functions that meet the characteristics of vulnerabilities but have minor actual harm can be used as warning items in real audits.

[0078] Undetectable type: The function is harmless or irrelevant to the vulnerability pattern and is used as a negative sample for evaluating false positives.

[0079] like Figure 2 As shown, the smart contract vulnerability mining method based on a large language model in this embodiment includes the following steps:

[0080] S1: Determine the type of target vulnerability.

[0081] Based on publicly accessible and internal audit reports, common attack incidents in the blockchain security community, and academic research papers on smart contract security, a specific target vulnerability type (e.g., reentrancy attack, integer overflow, etc.) is manually selected.

[0082] S2: Collect audit findings belonging to the target vulnerability type, including contract code and problem descriptions, where contract code includes the vulnerable code and the contract code after the vulnerability is fixed.

[0083] This embodiment uses a keyword search method to collect contract code and problem descriptions of audit findings belonging to the target vulnerability type from public and internal audit findings databases.

[0084] S3: Record the vulnerable code found in the contract code during the audit and the functions involving the target vulnerability in the contract code after the vulnerability is fixed. Mark these functions according to three function types to obtain the first set of marked functions corresponding to the vulnerable code and the second set of marked functions corresponding to the contract code after the vulnerability is fixed.

[0085] Target type detected: The function is seriously harmful and fully meets the typical characteristics of the target vulnerability.

[0086] Ignoreable type: Functions that meet the characteristics of vulnerabilities but have minor actual harm can be used as warning items in real audits.

[0087] Undetectable type: The function is harmless or irrelevant to the vulnerability pattern and is used as a negative sample for evaluating false positives.

[0088] S4: Based on the problem descriptions discovered during the audit, analyze the common characteristics and attack principles of the target vulnerabilities, and write a structured vulnerability description, which includes the following three dimensions:

[0089] 1. Vulnerability Pattern: A precise formal definition of the general code paradigm and triggering conditions of a vulnerability.

[0090] 2. Potential harm: Explain the potential asset loss or other negative impacts that may result if this vulnerability pattern is exploited.

[0091] 3. Additional tips: Provide some edge cases, special scenarios, or repair suggestions to assist the large language model in making more comprehensive judgments.

[0092] S5: Input the vulnerability description and the audit findings of the contract code obtained in step S2 into the vulnerability detection agent. The vulnerability detection agent will then concatenate these two into its prompt template and guide the large language model to act as a senior vulnerability auditor. Based on the given vulnerability description, the large language model will perform vulnerability analysis on the given contract code and output a vulnerability report in JSON format that includes fields such as vulnerability title, vulnerability description, vulnerability impact, and vulnerability function, as well as the large language model's thought process.

[0093] S6: The result-judgment agent concatenates the vulnerability report I output by the vulnerability detection agent and the thought process of the large language model into the prompt word template, guiding the large language model to play the role of a senior vulnerability auditor and analyze whether the vulnerability report is consistent with the final conclusion of the large language model's thought process; if they are consistent, the vulnerability report is retained; if they are inconsistent, a vulnerability report in the same format needs to be regenerated based on the large language model's thought process, called vulnerability report II.

[0094] S7: The large model detection result evaluation module receives vulnerability report two and either annotation function set one or annotation function set two, and deletes the functions of the negligible type from the functions in vulnerability report two to obtain vulnerability report three; then execute:

[0095] Select functions belonging to the detection target type from the functions in vulnerability report 3 to construct a set of functions that correctly detect vulnerabilities; filter out functions that are not in the set of functions that correctly detect vulnerabilities from the functions belonging to the detection target type in labeled function set 1 or labeled function set 2, and treat them as functions that are not reported.

[0096] From the functions in vulnerability report 3, select functions that are of the undetectable type and include them as additional reported functions;

[0097] Then, based on the number of correctly detected vulnerable functions, the number of missed functions, and the number of overreported functions, performance metrics including precision, recall, F1 score, and false positive rate are calculated.

[0098] S8: Repeat steps S5 to S7 to conduct multiple rounds of testing and calculate the average value of each indicator obtained from the multiple rounds of testing.

[0099] S9: Based on the detection results obtained from multiple rounds of testing in S8, including the model thinking process and vulnerability reports, and the average values ​​of various performance indicators, determine whether the average values ​​of each performance indicator meet the preset thresholds:

[0100] If so, the vulnerability description in step S4 will be directly used as the vulnerability description for subsequent actual target vulnerability discovery and configured to the vulnerability detection agent.

[0101] If not, then the vulnerability description in step S4 will be optimized in the following two ways, and then S10 will be executed:

[0102] Scenario 1 (Excessive Reporting of Functions / False Positive): When excessive reporting of functions exists, analyze whether these functions are vulnerability functions omitted from Annotated Function Set 1 and Annotated Function Set 2. If so, modify the type of the excessively reported functions to the detection target type or the negligible type according to the severity of the harm. If the function is highly harmful, modify it to the detection target type; if the harm is minor, modify it to the negligible type. If not, it indicates that the vulnerability description written by S4 has defects. It is necessary to adjust the potential harm and additional hint fields in the vulnerability description in conjunction with the reasoning process to prompt the large model to exclude functions that are prone to excessive reporting during the detection process.

[0103] Scenario 2 (Missed Functions / False Negative): When there are missed functions, it is necessary to analyze the thought process of the large language model, find out the reason for the large language model's judgment error (e.g., failure to understand specific code logic), and manually adjust the "vulnerability mode" or "additional hints" to prompt the large language model to pay extra attention to vulnerable functions that are easily overlooked during the detection process.

[0104] In this embodiment, the mean precision, mean recall, and mean F1 score in the performance metrics must all be greater than 80%, and the mean false positive rate must be less than 2%.

[0105] S10: Determine whether the number of optimization attempts has reached the preset threshold. If not, return to S5; if yes, execute S11.

[0106] S11: Return to S1, adjust the definition of the target vulnerability, break the target vulnerability down into multiple more refined and specific sub-vulnerabilities, and then execute S2 to S10 for each sub-vulnerability.

[0107] S12: As Figure 3 As shown, the compiler will receive the DeFi project code repository, compilation configuration and custom audit scope as inputs. The compiler will then compile the DeFi project, exclude irrelevant interface files, and output the smart contract code involving key business logic.

[0108] S13: The static analysis tool receives the smart contract code involving key business logic output by the compiler, obtains the inheritance relationship of the contract based on the abstract syntax tree of the smart contract, binds the information of the contract and its inherited parent contract, and outputs the smart contract code that completes the inheritance relationship.

[0109] S14: The vulnerability detection agent concatenates the smart contract code with the inheritance relationship and the optimized vulnerability description into the prompt word template; and guides the large language model to play the role of a senior vulnerability auditor, perform vulnerability analysis on the given smart contract code based on the given vulnerability description, and output a vulnerability report in JSON format containing fields such as vulnerability title, vulnerability description, vulnerability impact, and vulnerability function, as well as the large language model's thought process.

[0110] S15: The result-judging agent concatenates the vulnerability report output by the input vulnerability detection agent and the thought process of the large language model into the prompt word template, guiding the large language model to play the role of a senior vulnerability auditor and analyze whether the vulnerability report matches the final conclusion of the large language model's thought process. If they match, the vulnerability report is retained; if they do not match, a vulnerability report in the same format needs to be regenerated based on the large language model's thought process, thereby completing the vulnerability discovery.

[0111] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A smart contract vulnerability mining system based on a large language model, characterized in that, It includes a compiler, static analysis tools, vulnerability detection agent, result judgment agent, and large model detection result evaluation module; The compiler is used to receive the DeFi project code repository, compilation configuration and custom audit scope, compile the DeFi project, exclude irrelevant interface files, and output smart contract code involving key business logic; The static analysis tool is used to receive the smart contract code involving key business logic output by the compiler, obtain the inheritance relationship of the contract according to the abstract syntax tree of the smart contract, bind the information of the contract and its inherited parent contract, and output the smart contract code that completes the inheritance relationship. The vulnerability detection agent is used to append the vulnerability description and the vulnerability code found in the audit or the code after the vulnerability is fixed to the prompt word template of the vulnerability detection agent, or to append the vulnerability description and the smart contract code that completes the inheritance relationship to the prompt word template; It also guides the large language model to act as a senior vulnerability auditor, perform vulnerability analysis on the given smart contract code based on the given vulnerability description, output vulnerability report one, and the large language model's thought process. The result judgment agent is used to concatenate the vulnerability report one output by the vulnerability detection agent and the thought process of the large language model into the prompt word template, guiding the large language model to play the role of a senior vulnerability auditor and analyze whether the vulnerability report is consistent with the final conclusion of the large language model's thought process; if they are consistent, the vulnerability report is retained; if they are inconsistent, a vulnerability report of the same format needs to be regenerated according to the large language model's thought process, and vulnerability report two is output. The large model detection result evaluation module is used during the vulnerability description tuning process to receive the vulnerability report and annotation function set one or annotation function set two output by the result judgment agent, and performs the following operations: Remove the functions of the negligible type from the functions in vulnerability report 2 to obtain vulnerability report 3; Select functions belonging to the target detection type from the functions in vulnerability report 3, and construct a set of functions that correctly detect vulnerabilities; From the functions belonging to the detection target type in either labeled function set one or labeled function set two, select functions that are not in the set of functions that correctly detect vulnerabilities and treat them as functions that have been missed. From the functions in vulnerability report 3, select functions that are of the undetectable type and include them as additional reported functions; Then, based on the number of correctly detected vulnerable functions, the number of missed functions, and the number of over-reported functions, multiple performance metrics are calculated. The labeled function set one and labeled function set two are function sets formed by labeling the functions involving the target vulnerability in the actual vulnerability code of the specific target vulnerability type discovered in the audit and the contract code after the vulnerability is fixed.

2. The smart contract vulnerability mining system based on a large language model according to claim 1, characterized in that, The function types in the labeled function set one and labeled function set two include the following three types: Target type detected: The function is seriously harmful and fully meets the typical characteristics of the target vulnerability. Ignoreable type: Functions that meet the characteristics of vulnerabilities but have minor actual harm can be used as warning items in real audits. Undetectable type: The function is harmless or irrelevant to the vulnerability pattern and is used as a negative sample for evaluating false positives.

3. The smart contract vulnerability mining system based on a large language model according to claim 1, characterized in that, The performance metrics calculated by the large model detection result evaluation module include precision, recall, F1 score, and false positive rate.

4. The smart contract vulnerability mining system based on a large language model according to claim 1, characterized in that, The vulnerability report is in JSON format and includes fields such as vulnerability title, vulnerability description, vulnerability impact, and vulnerability function.

5. A method for discovering smart contract vulnerabilities based on a large language model, characterized in that, This method is implemented by the smart contract vulnerability mining system based on a large language model as described in claim 1, and the method includes the following steps: S1: Determine the type of target vulnerability; S2: Collect audit findings belonging to the target vulnerability type, including contract code and problem descriptions, where the contract code includes the vulnerable code and the contract code after the vulnerability is fixed; S3: Record the vulnerable code in the contract code discovered during the audit and the functions in the contract code after the vulnerability is fixed that involve the target vulnerability, and mark these functions according to three types to obtain the first set of marked functions corresponding to the vulnerable code and the second set of marked functions corresponding to the contract code after the vulnerability is fixed: the function types include the target detection type, the ignoreable type, and the undetectable type; S4: Based on the problem descriptions found in the audit, analyze the common characteristics and attack principles of the target vulnerabilities, and write a structured vulnerability description, which includes three dimensions: vulnerability mode, potential harm, and additional tips. S5: Input the vulnerability description and the contract code discovered by the audit obtained in step S2 into the vulnerability detection agent. The vulnerability detection agent will then concatenate these two into the prompt word template of the vulnerability detection agent and guide the large language model to play the role of a senior vulnerability auditor. Based on the given vulnerability description, the large language model will perform vulnerability analysis on the given contract code and output vulnerability report one and the thought process of the large language model. S6: The result-judgment agent concatenates the vulnerability report I output by the vulnerability detection agent and the thought process of the large language model into the prompt word template, guiding the large language model to play the role of a senior vulnerability auditor and analyze whether the vulnerability report is consistent with the final conclusion of the large language model's thought process; if they are consistent, the vulnerability report is retained; if they are inconsistent, a vulnerability report in the same format needs to be regenerated based on the large language model's thought process, called vulnerability report II. S7: The large model detection result evaluation module receives vulnerability report two and either annotation function set one or annotation function set two, and deletes the functions of the negligible type from the functions in vulnerability report two to obtain vulnerability report three; then execute: Select functions belonging to the detection target type from the functions in vulnerability report 3 to construct a set of functions that correctly detect vulnerabilities; filter out functions that are not in the set of functions that correctly detect vulnerabilities from the functions belonging to the detection target type in labeled function set 1 or labeled function set 2, and treat them as functions that are not reported. From the functions in vulnerability report 3, select functions that are of the undetectable type and include them as additional reported functions; Then, based on the number of correctly detected vulnerable functions, the number of missed functions, and the number of over-reported functions, calculate various performance indicators. S8: Repeat steps S5 to S7 to conduct multiple rounds of testing and calculate the average value of each indicator obtained from the multiple rounds of testing. S9: Based on the detection results obtained from multiple rounds of testing in S8, including the model thinking process and vulnerability reports, and the average values ​​of various performance indicators, determine whether the average values ​​of each performance indicator meet the preset thresholds: If so, the vulnerability description in step S4 will be directly used as the vulnerability description for subsequent actual target vulnerability discovery and configured to the vulnerability detection agent. If not, then the vulnerability description in step S4 will be optimized in the following two ways, and then S10 will be executed: When there are over-reported functions, analyze whether these functions are the vulnerability functions that were missed in the first and second labeled function sets. If so, modify the type of the over-reported functions to the detection target type or the negligible type according to the severity of the harm. If the function is highly harmful, modify it to the detection target type; if the harm is minor, modify it to the negligible type; if not, it means that the vulnerability description written by S4 has defects. It is necessary to adjust the potential harm and additional hint fields in the vulnerability description in conjunction with the reasoning process so as to prompt the large model to exclude functions that are prone to over-reporting during the detection process. When there are missed functions, it is necessary to analyze the thought process of the large language model, find out the reason for the large language model's judgment error, and manually adjust the vulnerability pattern or additional prompts to prompt the large language model to pay extra attention to vulnerable functions that are easily overlooked during the detection process. S10: Determine whether the number of optimization attempts has reached the preset threshold. If not, return to S5. If yes, then execute S11; S11: Return to S1, adjust the definition of the target vulnerability, break the target vulnerability into multiple more refined and specific sub-vulnerabilities, and then execute S2 to S10 for each sub-vulnerability; S12: Conduct real smart contract vulnerability discovery based on the final vulnerability description.

6. The smart contract vulnerability mining method based on a large language model according to claim 5, characterized in that, Based on the final vulnerability description, conduct actual smart contract vulnerability discovery, specifically including: (1) Input the DeFi project code repository, compilation configuration and custom audit scope into the compiler, and the compiler will compile the DeFi project, exclude irrelevant interface files, and output the smart contract code involving key business logic; (2) The static analysis tool receives the smart contract code involving key business logic output by the compiler, obtains the inheritance relationship of the contract according to the abstract syntax tree of the smart contract, binds the information of the contract and its inherited parent contract, and outputs the smart contract code that completes the inheritance relationship. (3) The vulnerability detection agent concatenates the smart contract code with the inheritance relationship and the optimized vulnerability description into the prompt word template; and guides the big language model to play the role of a senior vulnerability auditor, perform vulnerability analysis on the given smart contract code according to the given vulnerability description, output vulnerability report one, and the big language model's thinking process. (4) The result judgment agent splices the vulnerability report output by the input vulnerability detection agent and the thinking process of the big language model into the prompt word template, and guides the big language model to play the role of a senior vulnerability auditor. It analyzes whether the vulnerability report is consistent with the final conclusion of the big language model's thinking process. If they are consistent, the vulnerability report is retained. If they are not consistent, a vulnerability report in the same format needs to be regenerated according to the thinking process of the big language model, so as to complete the vulnerability discovery.

7. The smart contract vulnerability mining method based on a large language model according to claim 5, characterized in that, The specific conditions for the three dimensions of vulnerability description in S4 are as follows: Vulnerability pattern: A precise formal definition of the general code paradigm and triggering conditions of a vulnerability. Potential harm: Explain the potential asset loss or other negative impacts that may result if this vulnerability pattern is exploited. Additional tips: We provide some edge cases, special scenarios, or repair suggestions to assist large language models in making more comprehensive judgments.

Images