Intelligent contract vulnerability detection method and device based on LLaMA large language model
By using a smart contract vulnerability detection method based on the LLaMA large language model and employing LoRA to fine-tune the smart contract source code multiple times, the problem of low detection accuracy in existing technologies is solved, and efficient and automated vulnerability detection is achieved.
Patent Information
- Application Number
- CN202411531257.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2026-05-05
AI Technical Summary
Existing smart contract vulnerability detection methods rely on expert models. As the complexity of smart contracts increases, the detection accuracy decreases, making it difficult to effectively identify vulnerabilities in complex contracts.
We employ the LLaMA large language model, collect and clean smart contract source code, label vulnerability types, use the Solidity programming language source code processor to generate a fine-tuning dataset, and perform multiple fine-tuning of the LLaMA model using the LoRA method to improve vulnerability detection accuracy.
It improves the automation and accuracy of smart contract vulnerability detection, reduces the cost of fine-tuning computing power, and enhances the efficiency and interactive experience of vulnerability detection.
Smart Images
Figure CN121980567A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of blockchain technology, and more specifically, to a method and apparatus for detecting smart contract vulnerabilities based on the LLaMA large language model. Background Technology
[0002] This section is intended to provide background or context for the content set forth in the claims or specification, and the content described herein is not acknowledged as prior art simply because it is included in this section.
[0003] As one of the core technologies of Blockchain 2.0, smart contracts have greatly expanded the use cases of blockchain, extending the blockchain platform from a simple distributed ledger system to a richly feature-rich decentralized operating system. A smart contract is an automatically executed contract written in computer code, possessing functions such as automatic execution of conditions and transactions, decentralized transaction and coordination, data storage and retrieval, event triggering and response, and multi-party approval and voting. These functions enable smart contracts to be widely used in various fields, from financial services to supply chain management, from digital asset trading to voting and governance. They provide a programmable and automated way to conduct transactions, coordination, and execution operations, improving efficiency and security.
[0004] However, smart contracts are more vulnerable to attack than ordinary programs for the following reasons: 1) Smart contracts typically involve digital financial assets, and attacking them can yield substantial economic benefits; 2) The original purpose of smart contracts is to leverage the characteristics of blockchain to achieve trustworthiness and immutability. However, vulnerabilities in smart contracts can lead to unexpected behavior, undermining their intended fairness and reliability. Therefore, ensuring the security and stability of smart contracts is crucial for the development of blockchain platforms.
[0005] In recent years, smart contract vulnerabilities have triggered large-scale security incidents. These vulnerabilities not only have a significant impact on economic losses but also undermine public trust in blockchain and smart contracts, making security a critical issue and a major challenge.
[0006] The most widely used existing smart contract vulnerability detection methods are symbolic execution methods, represented by Oyente, Maian, and Mythril, which have advantages such as high accuracy and the ability to detect a wide range of vulnerability types. However, existing methods heavily rely on pre-defined expert patterns to detect smart contract vulnerabilities. As smart contracts become increasingly complex, the content required to define expert patterns increases, and the rules of expert patterns cannot keep up with the rapid increase in the number of smart contracts, making the detection results more prone to errors.
[0007] There is currently no effective solution to the above problems. Summary of the Invention
[0008] This application provides a method and apparatus for detecting smart contract vulnerabilities based on the LLaMA large language model, so as to at least solve the technical problem of low accuracy in detecting smart contract vulnerabilities in related technologies.
[0009] According to one aspect of the embodiments of this application, a method for detecting smart contract vulnerabilities based on an LLaMA large language model is provided, comprising: collecting a dataset of original source code of smart contracts, and cleaning the smart contract source code in the original source code dataset to obtain a target source code dataset, wherein the smart contract source code is source code edited using the Solidity programming language; marking the smart contract source code in the target source code dataset with vulnerability types according to a preset vulnerability type order; calling a source code processor of the Solidity programming language to process the marked smart contract source code to generate a fine-tuning dataset required for the model; pre-training an original LLaMA model using the target source code dataset to obtain an intermediate LLaMA model; using the LoRA method, fine-tuning the intermediate LLaMA model multiple times using the fine-tuning dataset to obtain a target LLaMA model; and using the target LLaMA model to identify vulnerabilities in the Solidity smart contract source code.
[0010] According to another aspect of the embodiments of this application, a smart contract vulnerability detection device based on an LLaMA large language model is also provided, comprising: a collection unit, used to collect a dataset of original source code of smart contracts and clean the smart contract source code in the original source code dataset to obtain a target source code dataset, wherein the smart contract source code is source code edited using the Solidity programming language; a marking unit, used to mark the smart contract source code in the target source code dataset with vulnerability types according to a preset vulnerability type order; a processing unit, used to call the source code processor of the Solidity programming language to process the marked smart contract source code and generate a fine-tuning dataset required for the model; a pre-training unit, used to pre-train the original LLaMA model using the target source code dataset to obtain an intermediate LLaMA model; a fine-tuning unit, used to fine-tune the intermediate LLaMA model multiple times using the LoRA method with the fine-tuning dataset to obtain a target LLaMA model; and an identification unit, used to identify vulnerabilities in the Solidity smart contract source code using the target LLaMA model.
[0011] According to another aspect of the embodiments of this application, a storage medium is also provided, the storage medium including a stored program that executes the above-described method when the program is run.
[0012] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor performs the above-described method through the computer program.
[0013] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the embodiments of the methods described above.
[0014] In this embodiment, a dataset of original source code for smart contracts is collected, and the source code of the smart contracts in the original source code dataset is cleaned to obtain a target source code dataset. The source code of the smart contracts is source code edited using the Solidity programming language. According to a preset vulnerability type order, the source code of the smart contracts in the target source code dataset is marked with vulnerability types. The source code processor of the Solidity programming language is called to process the marked smart contract source code to generate the fine-tuning dataset required for the model. The original LLaMA model is pre-trained using the target source code dataset to obtain an intermediate LLaMA model. Using the LoRA method, the intermediate LLaMA model is fine-tuned multiple times using the fine-tuning dataset to obtain the target LLaMA model. The target LLaMA model is used to identify vulnerabilities in the Solidity smart contract source code. This solves the technical problem of low vulnerability detection accuracy in related technologies. By using the LoRA method to fine-tune the model using a high-quality dataset in a hierarchical manner, the computational cost of fine-tuning is reduced, the automation level of smart contract vulnerability detection is improved, and the vulnerability detection accuracy is increased. Attached Figure Description
[0015] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0016] Figure 1 This is a flowchart of an optional smart contract vulnerability detection method based on the LLaMA large language model according to an embodiment of this application;
[0017] Figure 2 This is a schematic diagram of an optional smart contract vulnerability detection system based on the LLaMA large language model according to an embodiment of this application;
[0018] Figure 3This is a schematic diagram of an optional smart contract vulnerability detection device based on the LLaMA large language model according to an embodiment of this application; and,
[0019] Figure 4 This is a structural block diagram of a terminal according to an embodiment of this application. Detailed Implementation
[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0022] This invention discloses a method for constructing a smart contract vulnerability detection system based on the LLaMA large language model. It utilizes a high-quality smart contract source code dataset to pre-train an LLaMA model. Learning from a high-quality dataset mitigates some of the shortcomings of smaller models, maximizing the preservation of contract information while achieving automation, thus improving vulnerability detection efficiency and interactive experience. According to one aspect of this application, a method embodiment for smart contract vulnerability detection based on the LLaMA large language model is provided, such as... Figure 1 Figure 2 As shown:
[0023] Step S1: Collect the original source code dataset of smart contracts, and clean the smart contract source code in the original source code dataset to obtain the target source code dataset. The smart contract source code is source code edited using the Solidity programming language.
[0024] 1) First, crawl a large amount of publicly available smart contract source code through various public Ethereum platforms;
[0025] 2) Clean the large amount of crawled smart contract source code, removing strings, spaces, and duplicate code that are irrelevant to the identification of smart contract vulnerabilities.
[0026] Step S2: Mark the smart contract source code in the target source code dataset with vulnerability types according to the preset vulnerability type order;
[0027] 1) Obtain a large amount of publicly available smart contract source code from various blockchain platforms, and use smart contract vulnerability detection tools such as Oyente, Maian, and Mythril to test the contract source code in batches;
[0028] 2) The smart contract source code is marked according to the order of reentrancy vulnerability, timestamp dependency vulnerability, infinite loop vulnerability, and no vulnerability. 0 represents non-existence and 1 represents existence. Only the smart contract source code with the same output results from smart contract vulnerability detection tools such as Oyente, Maian, and Mythril is used as the training dataset.
[0029] For example, 0100 indicates that only a timestamp dependency vulnerability exists.
[0030] Step S3: Call the source code processor of the Solidity programming language to process the marked smart contract source code and generate the fine-tuning dataset required for the model.
[0031] 1) Based on the syntax and semantic relationships of the Solidity programming language, this invention creates the Solidity source code processor SolSCP, which serves as a processor for model data input formats.
[0032] 2) Input the tagged smart contract source code data into SolSCP, which will automatically process it into the input format required by the fine-tuning model. The input format required by the fine-tuning model is...
[0033] {'Instruction':'Input':'Output':}, where Instruction is a string describing the task the model should perform. Since the fine-tuning mode selects empty Input, output can be one of the following: reentrant vulnerability, timestamp-dependent vulnerability, infinite loop vulnerability, or no vulnerability. Instruction corresponds to the source code after the smart contract is processed, and output corresponds to the four options of reentrant vulnerability, timestamp-dependent vulnerability, infinite loop vulnerability, or no vulnerability in the smart contract. This generates a dataset that can be input into the LLaMA model for fine-tuning.
[0034] S33: Using `call.value`, `block.timestamp`, and the loop keywords `for` and `while` as key locations, analyze the control dependencies, data dependencies, call dependencies, and inheritance relationships of these key nodes in the source code. Based on this, slice the corresponding parts of the source code. When building the large language model input, to focus on the classification task, set prompts for vulnerability classification, adding these contents to the Instruction section as prompts:
[0035] {Detect vulnerabilities in smart contracts written using Solidity and categorize the code as follows:}
[0036] 1. Reentrancy vulnerability
[0037] 2. Timestamp Dependency Vulnerability
[0038] 3. Infinite Loop Vulnerability
[0039] 4. No vulnerabilities
[0040] For discovered vulnerabilities, only information about the vulnerability category is provided. The processed datasets are then integrated into a dataset that has undergone secondary fine-tuning.
[0041] Step S4: Pre-train the original LLaMA model using the target source code dataset to obtain an intermediate LLaMA model;
[0042] S41: The unlabeled dataset of smart contract source code is processed by the source code processor and then input into the LLaMA model for pre-training. The data first passes through a token segmentation module to convert text data into a token sequence. The token sequence is then vectorized using embedding to obtain feature vectors. These vectors are then normalized at each LLaMA layer using RMSnorm, rather than just at the final output layer. Using RMSnorm is more beneficial for improving the speed and stability of the neural network training process. The following formula represents the normalization of the sub-layer input by RMSnorm:
[0043]
[0044]
[0045] Where RMS(a) is the root mean square of the input tensor a, and a i Let g be a certain dimension of the input vector, n be the highest dimension of the input vector, and g be the highest dimension of the input vector. i b is a variable scaling parameter that allows information from the normalization loss to be recovered. iThis is the paranoia factor, which enables LLaMA to increase an adjustable threshold during training, ensuring that the output is not zero even when the input is zero, and guaranteeing that the activation function operates in the non-linear region. This is the normalized feature vector of 'a'; the trained output feature vector is then used for classification tasks through a fully connected layer.
[0046] S42: In the LLaMA layer, the normalized vector is updated by the attention module. The updated vector is then concatenated with the unnormalized feature vector via a residual connection. The resulting new feature vector is passed through the LLaMA's MLP layer. The MLP consists of fully connected layers, activation functions, and more fully connected layers. The MLP uses the SwiGLU activation function module, allowing the data flow to change dynamically. The formula used is as follows:
[0047]
[0048] x is the input, W, V, and W2 are the weight matrices of the input x through the fully connected layer, and W2 is the weight obtained by multiplying W by the Swish activation function and V by the Hadamard product. After the input and weight matrices are processed by the Swish activation function, they are XORed with the product of the input and weight matrix V, and finally multiplied by the weight matrix W2. The expression of the Swish activation function is: Swishβ(x)=xσ(βx).
[0049] S43: After training through multiple LLaMA layers, in order to perform smart contract vulnerability detection, cross-entropy loss is used as the loss function after passing through a fully connected layer to perform binary classification of smart contracts and determine whether the corresponding source code has reentrancy, infinite loop, or timestamp dependency vulnerabilities.
[0050] Step S5: Using the LoRA method, the intermediate LLaMA model is fine-tuned multiple times using the fine-tuning dataset to obtain the target LLaMA model.
[0051] 1) Use the processed smart contract source code labeled dataset obtained in step S3 as input to the LLaMA large language model, fine-tune the LLaMA large language model using LoRA, and select the LoRA target layer to be applied next in the pre-trained LLaMA model.
[0052] 2) Next, create two matrices A and B, the size of which is determined by the rank of LoRA. A and B are the mapping matrix and the inverse mapping matrix, respectively. A and B are opposite in dimension; A is for dimensionality reduction and B is for dimensionality increase. According to the formula: W' = W + A*B, the parameter matrix W is transformed into W', and W' is used as the new parameter matrix for model fine-tuning.
[0053] 3) The calculated loss function is used to update the gradients of mapping matrix A and mapping matrix B during the fine-tuning process, and is updated according to the optimizer's optimization algorithm;
[0054] 4) Repeat steps 2-3 above to obtain the optimal convergence result for each batch. Use LoRA to fine-tune the LLaMA language model so that the pre-trained language model can efficiently adapt to various downstream tasks. It is not necessary to fine-tune all the parameters of the model, but only a small number of model parameters, which reduces most of the computation and storage costs.
[0055] 5) After the first fine-tuning using LoRA, the high-quality, refined vulnerability code secondary fine-tuning dataset constructed by adding prompt words is used again to fine-tune the LLaMA model fine-tuned by S51 to S54 using LoRA, and steps 1-4 are repeated for training.
[0056] Step S6: Use the target LLaMA model to identify vulnerabilities in the Solidity smart contract source code.
[0057] The main steps of this invention include collecting and cleaning the vulnerability source code of the contract, classifying the vulnerability source code, establishing the dataset required for pre-training and the fine-tuning dataset, obtaining a detection system in a specific direction through LLaMA pre-training, and obtaining a fine-tuned smart contract vulnerability detection system through multiple fine-tuning of the dataset.
[0058] The main advantage of this invention is that it utilizes a high-quality smart contract source code dataset to pre-train an LLaMA model. Learning from a high-quality dataset can mitigate some of the shortcomings of small models. While maintaining the integrity of contract information to the greatest extent, it reduces the destruction of code information during the process of converting source code into smart contract source code sequences, contract graphs, etc., thereby achieving automated smart contract vulnerability detection and improving vulnerability detection efficiency and interactive experience.
[0059] By using LoRA to fine-tune the model in a hierarchical manner with high-quality datasets, the computational cost of fine-tuning is reduced, the automation level of smart contract vulnerability detection is improved, and the accuracy of vulnerability detection is increased.
[0060] Through the above steps, the original source code dataset of smart contracts is collected, and the source code in the original source code dataset is cleaned to obtain the target source code dataset. The smart contract source code is edited using the Solidity programming language. According to the preset vulnerability type order, the smart contract source code in the target source code dataset is labeled with vulnerability types. The Solidity programming language source code processor is used to process the labeled smart contract source code to generate the fine-tuning dataset required for the model. The original LLaMA model is pre-trained using the target source code dataset to obtain an intermediate LLaMA model. Furthermore, prompt words are set to provide clear... The task description and expected output format are provided to help the Llama model better understand and perform smart contract vulnerability detection and classification tasks. Using the LoRA method, the intermediate Llama model is fine-tuned multiple times with a fine-tuning dataset to obtain the target Llama model. Using the target Llama model to identify vulnerabilities in Solidity smart contract source code solves the technical problem of low accuracy in smart contract vulnerability detection in related technologies. The LoRA method uses high-quality datasets in a hierarchical manner to fine-tune the model, reducing the computational cost of fine-tuning, improving the automation level of smart contract vulnerability detection, and increasing vulnerability detection accuracy.
[0061] By using LoRA to fine-tune the model in a hierarchical manner with high-quality datasets, the computational cost of fine-tuning is reduced, the automation level of smart contract vulnerability detection is improved, and the accuracy of vulnerability detection is increased.
[0062] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
[0063] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0064] According to another aspect of the embodiments of this application, a smart contract vulnerability detection device based on the LLaMA language model is also provided for implementing the above-described smart contract vulnerability detection method based on the LLaMA language model. Figure 3 This is a schematic diagram of an optional smart contract vulnerability detection device based on the LLaMA large language model according to an embodiment of this application, such as... Figure 3 As shown, the device may include:
[0065] Collection unit 31 is used to collect the original source code dataset of smart contracts and clean the smart contract source code in the original source code dataset to obtain the target source code dataset, wherein the smart contract source code is source code edited using the Solidity programming language.
[0066] The marking unit 32 is used to mark the smart contract source code in the target source code dataset with vulnerability types according to a preset vulnerability type order;
[0067] Processing unit 33 is used to call the source code processor of the Solidity programming language to process the marked smart contract source code and generate the fine-tuning dataset required for the model.
[0068] Pre-training unit 34 is used to pre-train the original LLaMA model using the target source code dataset to obtain an intermediate LLaMA model;
[0069] Fine-tuning unit 35 is used to fine-tune the intermediate LLaMA model multiple times using the LoRA method and the fine-tuning dataset to obtain the target LLaMA model.
[0070] The identification unit 36 is used to identify vulnerabilities in the Solidity smart contract source code using the target LLaMA model.
[0071] Through the above modules, the original source code dataset of smart contracts is collected, and the smart contract source code in the original source code dataset is cleaned to obtain the target source code dataset. The smart contract source code is source code edited using the Solidity programming language. According to a preset vulnerability type order, the smart contract source code in the target source code dataset is marked with vulnerability types. The Solidity programming language source code processor is called to process the marked smart contract source code to generate the fine-tuning dataset required for the model. The original LLaMA model is pre-trained using the target source code dataset to obtain an intermediate LLaMA model. Using the LoRA method, the intermediate LLaMA model is fine-tuned multiple times using the fine-tuning dataset to obtain the target LLaMA model. The target LLaMA model is used to identify vulnerabilities in Solidity smart contract source code. This solves the technical problem of low vulnerability detection accuracy in related technologies. By using the LoRA method to fine-tune the model using high-quality datasets in a hierarchical manner, the computational cost of fine-tuning is reduced, the automation level of smart contract vulnerability detection is improved, and the vulnerability detection accuracy is increased.
[0072] Optionally, the collection unit is also used to: crawl publicly available smart contract source code through Ethereum's public platform to obtain the original source code dataset; clean all crawled smart contract source code to remove strings, spaces, and duplicate code that are not related to the vulnerability identification of smart contracts, and obtain the target source code dataset.
[0073] Optionally, the marking unit is also used to: perform vulnerability detection on the smart contract source code in the target source code dataset using multiple smart contract vulnerability detection tools; and mark each smart contract source code in the target source code dataset with a vulnerability type according to the order of reentrant vulnerability, timestamp dependent vulnerability, infinite loop vulnerability, and no vulnerability.
[0074] Optionally, the processing unit is further configured to: create the source code processor based on the syntactic and semantic relationships of the Solidity programming language, as a processor for processing model data input formats; input the tagged smart contract source code into the source code processor, and automatically process it according to the input format required for the initial fine-tuning of the model to obtain the fine-tuning dataset required for the initial fine-tuning of the model; slice the smart contract source code according to the control dependencies, data dependencies, call dependencies, and inheritance relationships of key nodes in the smart contract source code, and add prompt words to the "instruction" part of the dataset, integrating them into the fine-tuning dataset required for the secondary fine-tuning of the model.
[0075] The input formats required for the initial fine-tuning of the above model include: string format, fine-tuning mode, and vulnerability type.
[0076] Optionally, the pre-training unit is further configured to: process the unlabeled smart contract source code in the target source code dataset through the source code processor and input it into the original LLaMA model for pre-training; use the SwiGLU activation function in the fully connected layer of the original LLaMA model to enable dynamic changes in the flow of data, and use the trained model as the intermediate LLaMA model.
[0077] The identification unit 36 is also used to: input the smart contract source code in the fine-tuning dataset into the LLaMA model, fine-tune the intermediate LLaMA model using LoRA, and select the LoRA target layer to be applied subsequently in the pre-trained intermediate LLaMA model; create two matrices A and B, and transform the parameter matrix W into the parameter matrix W' according to the formula: W' = W + A * B. The parameter matrix W' is used as the new parameter matrix for model fine-tuning. The size of the two matrices is determined according to the rank of LoRA. Matrix A and matrix B are the mapping matrix and the inverse mapping matrix, respectively. Matrix A and matrix B are opposite in dimension, with matrix A being a dimension reduction matrix and matrix B being a dimension increase matrix.
[0078] Optionally, the gradients of matrices A and B during the fine-tuning process are updated using the calculation results of the loss function, and matrices A and B are updated according to the optimization algorithm of the optimizer.
[0079] According to another aspect of the embodiments of this application, a server or terminal for implementing the above-described smart contract vulnerability detection method based on the LLaMA large language model is also provided.
[0080] Figure 4 This is a structural block diagram of a terminal according to an embodiment of this application, such as... Figure 4 As shown, the terminal may include: one or more (only one is shown in the figure) processors 401, memory 403, and transmission devices 405, such as... Figure 4 As shown, the terminal may also include input / output devices 407.
[0081] The memory 403 can be used to store software programs and modules, such as the program instructions / modules corresponding to the smart contract vulnerability detection method and device based on the LLaMA large language model in this embodiment. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 403, thereby realizing the aforementioned smart contract vulnerability detection method based on the LLaMA large language model. The memory 403 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 403 may further include memory remotely located relative to the processor 401, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0082] The aforementioned transmission device 405 is used to receive or send data via a network, and can also be used for data transfer between the processor and memory. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 405 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 405 is a radio frequency (RF) module used for wireless communication with the Internet.
[0083] Specifically, memory 403 is used to store application programs.
[0084] The processor 401 can invoke the application program stored in the memory 403 via the transmission device 405 to perform the following steps:
[0085] The process involves collecting a dataset of original source code for smart contracts and cleaning the source code to obtain a target source code dataset. The smart contract source code is edited using the Solidity programming language. The target source code dataset is then labeled with vulnerabilities according to a pre-defined vulnerability type order. A Solidity source code processor is used to process the labeled smart contract source code, generating a fine-tuning dataset for the model. The target source code dataset is then used to pre-train an original LLaMA model to obtain an intermediate LLaMA model. Using LoRA, the intermediate LLaMA model is fine-tuned multiple times using the fine-tuning dataset to obtain the target LLaMA model. Finally, the target LLaMA model is used to identify vulnerabilities in the Solidity smart contract source code.
[0086] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.
[0087] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. The terminal can be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, the terminal may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.
[0088] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0089] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to execute program code for a smart contract vulnerability detection method based on the LLaMA large language model.
[0090] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.
[0091] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:
[0092] The process involves collecting a dataset of original source code for smart contracts and cleaning the source code to obtain a target source code dataset. The smart contract source code is edited using the Solidity programming language. The target source code dataset is then labeled with vulnerabilities according to a pre-defined vulnerability type order. A Solidity source code processor is used to process the labeled smart contract source code, generating a fine-tuning dataset for the model. The target source code dataset is then used to pre-train an original LLaMA model to obtain an intermediate LLaMA model. Using LoRA, the intermediate LLaMA model is fine-tuned multiple times using the fine-tuning dataset to obtain the target LLaMA model. Finally, the target LLaMA model is used to identify vulnerabilities in the Solidity smart contract source code.
[0093] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.
[0094] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.
[0095] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0096] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.
[0097] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0098] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.
[0099] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0100] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0101] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A smart contract vulnerability detection method based on the LLaMA large language model, characterized in that, include: Collect the original source code dataset of smart contracts, and clean the smart contract source code in the original source code dataset to obtain the target source code dataset, wherein the smart contract source code is source code edited using the Solidity programming language; The smart contract source code in the target source code dataset is marked with vulnerability types according to a preset vulnerability type order; The Solidity programming language's source code processor is invoked to process the tagged smart contract source code and generate the fine-tuning dataset required for the model. The original LLaMA model is pre-trained using the target source code dataset to obtain an intermediate LLaMA model; Using the LoRA method, the intermediate LLaMA model is fine-tuned multiple times using the fine-tuning dataset to obtain the target LLaMA model. The target LLaMA model was used to identify vulnerabilities in the Solidity smart contract source code.
2. The method according to claim 1, characterized in that, Collect the original source code dataset of smart contracts, and clean the smart contract source code in the original source code dataset to obtain the target source code dataset, including: The original source code dataset was obtained by crawling publicly available smart contract source code through Ethereum's public platform. All crawled smart contract source code is cleaned to remove strings, spaces, and duplicate code that are not related to smart contract vulnerability identification, thus obtaining the target source code dataset.
3. The method according to claim 1, characterized in that, According to a preset vulnerability type order, the smart contract source code in the target source code dataset is marked with vulnerability types, including: The smart contract source code in the target source code dataset was tested for vulnerabilities using multiple smart contract vulnerability detection tools. The vulnerability types of each smart contract source code in the target source code dataset are marked according to the following order: reentrancy vulnerability, timestamp dependency vulnerability, infinite loop vulnerability, and no vulnerability.
4. The method according to claim 1, characterized in that, The Solidity programming language's source code processor is invoked to process the tagged smart contract source code, generating the fine-tuning dataset required for the model, including: Based on the syntactic and semantic relationships of the Solidity programming language, the source code processor is created as a processor for processing model data input formats. The tagged smart contract source code is input into the source code processor and automatically processed according to the input format required for the initial fine-tuning of the model to obtain the fine-tuning dataset required for the initial fine-tuning of the model. Based on the control dependencies, data dependencies, call dependencies, and inheritance relationships of key nodes in the smart contract source code, the smart contract source code is sliced, and prompt words are added to the instruction part of the dataset, which is then integrated into the fine-tuning dataset required for the secondary fine-tuning of the model.
5. The method according to claim 4, characterized in that, The input formats required for the initial fine-tuning of the model include: string format, fine-tuning mode, and vulnerability type.
6. The method according to claim 4, characterized in that, The original LLaMA model is pre-trained using the target source code dataset to obtain an intermediate LLaMA model, including: The unlabeled smart contract source code in the target source code dataset is processed by the source code processor and then input into the original LLaMA model for pre-training. The resulting model is used as the intermediate LLaMA model.
7. The method according to claim 1, characterized in that, Using the LoRA method, the intermediate LLaMA model is fine-tuned multiple times using the fine-tuning dataset to obtain the target LLaMA model, including performing initial fine-tuning and secondary fine-tuning sequentially according to the following steps: Input the smart contract source code in the fine-tuning dataset into the LLaMA model, fine-tune the intermediate LLaMA model using LoRA, and select the LoRA target layer to be applied subsequently in the pre-trained intermediate LLaMA model. Create two matrices A and B. According to the formula: W'=W+A*B, transform the parameter matrix W into the parameter matrix W'. The parameter matrix W' is used as the new parameter matrix for model fine-tuning. The size of the two matrices is determined by the rank of LoRA. Matrix A and matrix B are the mapping matrix and the inverse mapping matrix, respectively. Matrix A and matrix B are opposite in dimension. Matrix A is for dimensionality reduction and matrix B is for dimensionality increase. The gradients of matrices A and B during the fine-tuning process are updated using the calculation results of the loss function, and matrices A and B are updated according to the optimization algorithm of the optimizer.
8. A smart contract vulnerability detection device based on the LLaMA large language model, characterized in that, include: The collection unit is used to collect the original source code dataset of smart contracts and clean the smart contract source code in the original source code dataset to obtain the target source code dataset, wherein the smart contract source code is source code edited using the Solidity programming language. The tagging unit is used to tag the smart contract source code in the target source code dataset with vulnerability types according to a preset vulnerability type order; The processing unit is used to call the source code processor of the Solidity programming language to process the tagged smart contract source code and generate the fine-tuning dataset required by the model. The pre-training unit is used to pre-train the original LLaMA model using the target source code dataset to obtain an intermediate LLaMA model; The fine-tuning unit is used to fine-tune the intermediate LLaMA model multiple times using the LoRA method and the fine-tuning dataset to obtain the target LLaMA model. The identification unit is used to identify vulnerabilities in the Solidity smart contract source code using the target LLaMA model.
9. A computer-readable storage medium, characterized in that, The storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when it is run.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the method described in any one of claims 1 to 7 through the computer program.