A method and device for detecting vulnerabilities of a smart contract based on knowledge distillation

Abstract

The application discloses a kind of based on knowledge distillation's smart contract vulnerability detection method and device, in based on knowledge distillation's smart contract vulnerability detection method, the training data set of different code length is constructed, by first smart contract sample, teacher pretraining model is trained, and the student model is pre-trained to the teacher model obtained, and using second smart contract sample, student model is fine-tuned, with the aid of student model, the vulnerability detection of the smart contract to be detected is realized.The present application can accurately capture the local vulnerability characteristics of long smart contract, greatly improve the accuracy and comprehensiveness of long smart contract vulnerability detection, and significantly improve the detection efficiency, provide more timely and reliable support for the security protection of smart contract.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method and apparatus for detecting smart contract vulnerabilities based on knowledge distillation. Background Technology

[0002] In the field of smart contract vulnerability detection, static vulnerability analysis methods based on deep learning have made significant progress in recent years. However, existing methods rely solely on classification loss to guide the model in learning vulnerability features during training, resulting in the following limitations: (1) Insufficient robustness of feature representation: In long contracts, vulnerability features are usually sparsely distributed and easily masked by a large amount of irrelevant contextual information. Existing methods struggle to effectively amplify key vulnerability features in this situation, leading to a decrease in the accuracy of vulnerability detection. (2) Sensitivity to contextual noise: Long smart contracts contain a large number of code snippets unrelated to vulnerabilities, and this noise information easily interferes with the model's feature extraction. Existing models lack specialized training to filter irrelevant information, resulting in a high false positive and false negative rate for vulnerability identification in long contracts. (3) Lack of design in the training process: Existing methods typically improve performance through model optimization, neglecting the optimization design of the model training method. Existing training methods limit the discriminative and generalization capabilities of feature representations, especially when dealing with diverse vulnerability types and long contracts. Summary of the Invention

[0003] The main technical problem addressed by this invention is how to accurately and effectively identify and locate vulnerabilities in extremely long and complex smart contracts. This invention proposes a multi-step knowledge distillation and contrastive learning method based on vulnerability representations in long smart contracts, improving the training method of long contract embedding representation extractors and enhancing the training model's ability to amplify sparse vulnerability features and filter out contextual noise irrelevant to vulnerabilities in smart contracts.

[0004] In a first aspect, embodiments of this application provide a smart contract vulnerability detection method based on knowledge distillation, the method comprising:

[0005] Construct a training dataset, which includes a first smart contract sample and a second smart contract sample, wherein the code length of the second smart contract sample is greater than the code length of the first smart contract sample;

[0006] The teacher pre-trained model is trained using the first smart contract sample to obtain the teacher model. The student model is then trained using the teacher model and the first smart contract sample, and the student model is fine-tuned using the second smart contract sample.

[0007] By using a student model, vulnerability detection can be achieved for smart contracts under test.

[0008] In a specific embodiment of the present invention, the student model is pre-trained using a teacher model and a first smart contract sample, specifically including:

[0009] The first smart contract sample is input into the student model to obtain the pre-training results. Using the pre-training results, the contrastive loss, ranking consistency loss and teacher knowledge loss of the student model are obtained.

[0010] The parameters of the student model are adjusted based on the contrastive loss, ranking consistency loss, and teacher knowledge loss of the student model.

[0011] In a specific embodiment of the present invention, a first smart contract sample is input into the student model to obtain pre-training results. Using these pre-training results, the teacher knowledge loss of the student model is obtained, specifically including:

[0012] Using the hard labels in the first smart contract sample and the soft labels obtained by knowledge distillation of the first smart contract sample by the teacher model, the student model is trained to obtain the teacher knowledge loss of the student model. Here, the hard label is the real vulnerability type of the smart contract sample, and the soft label is the probability value of the vulnerability type generated by the teacher model for the smart contract sample.

[0013] In a specific embodiment of the present invention, the student model is trained using hard labels in the first smart contract sample and soft labels obtained by knowledge distillation of the first smart contract sample by the teacher model. Specifically, this includes:

[0014] For the first smart contract sample, obtain the similarity score list obtained by the teacher model, where the similarity score list obtained by the teacher model is a soft label;

[0015] The ranking loss calculation method is used to calculate the cross-entropy between the similarity score list obtained by the teacher model and the hard labels;

[0016] By using cross-entropy as the teacher's knowledge loss, the student model is trained to obtain the teacher's knowledge loss.

[0017] In a specific embodiment of the present invention, constructing a training dataset specifically includes:

[0018] The samples were grouped according to the code length in the samples, into first smart contract samples and second smart contract samples.

[0019] The first positive sample, the second positive sample, and the first negative sample from the first smart contract sample and the second smart contract sample, respectively, are selected based on their semantic similarity to a preset threshold; the preset threshold is used to control the minimum similarity between samples.

[0020] Based on the preset sample group type, ternary sample groups are constructed for the samples in the first smart agent contract sample and the second smart agent contract sample. The ternary sample groups are a first type ternary sample group, a second type ternary sample group, and a third type ternary sample group. The first type ternary sample group includes semantically similar first positive samples and second positive samples without vulnerabilities, as well as first negative samples with vulnerabilities. The second type ternary sample group includes semantically similar first positive samples and second positive samples with the same type of vulnerabilities, as well as first negative samples without vulnerabilities. The third type ternary sample group includes semantically similar first positive samples and second positive samples with the same type of vulnerabilities, as well as first negative samples with different vulnerabilities.

[0021] In a specific embodiment of the present invention, a teacher pre-training model is trained using a first smart contract sample to obtain a teacher model, specifically including:

[0022] The process of training the teacher pre-trained model using the first smart contract sample yields the contrastive loss and ranking consistency loss of the teacher pre-trained model, and the parameters of the teacher pre-trained model are adjusted accordingly. The contrastive loss of the teacher pre-trained model is used to compare the differences between positive and negative samples with the same or different vulnerabilities, and the ranking consistency loss is used to indicate the ranking relationship between the samples in the first smart contract sample, ensuring that the model's ranking of the samples is consistent with the actual situation.

[0023] In a specific embodiment of the present invention, a student model is used to detect vulnerabilities in the smart contract to be tested, specifically including:

[0024] The smart contract to be detected is input into the student model to obtain the embedding vector;

[0025] The type of vulnerability in the smart contract to be detected is determined based on the label of the nearest neighbor of the embedded vector in the vulnerability database.

[0026] Secondly, embodiments of this application provide a smart contract vulnerability detection device based on knowledge distillation, the device comprising:

[0027] The dataset construction module is used to build the training dataset, which includes a first smart contract sample and a second smart contract sample, wherein the code length of the second smart contract sample is greater than the code length of the first smart contract sample;

[0028] The training module is used to train the teacher pre-trained model using the first smart contract sample to obtain the teacher model, train the student model using the teacher model and the first smart contract sample, and fine-tune the student model using the second smart contract sample.

[0029] The identification module is used to detect vulnerabilities in smart contracts under test by leveraging a student model.

[0030] In a specific embodiment of the present invention, the training module is specifically used to input the first smart contract sample into the student model to obtain pre-training results, and use the pre-training results to obtain the contrast loss, ranking consistency loss and teacher knowledge loss of the student model;

[0031] The parameters of the student model are corrected based on the contrastive loss, ranking consistency loss, and teacher knowledge loss of the student model.

[0032] In a specific embodiment of the present invention, the training module is specifically used to train the student model using the hard labels in the first smart contract sample and the soft labels obtained by the teacher model through knowledge distillation of the first smart contract sample, thereby obtaining the teacher knowledge loss of the student model. Here, the hard labels are the real vulnerability types of the smart contract sample, and the soft labels are the vulnerability type probability values ​​generated by the teacher model for the smart contract sample.

[0033] In a specific embodiment of the present invention, the training module is specifically used to obtain a list of similarity scores obtained by the teacher model for the first smart contract sample, wherein the list of similarity scores obtained by the teacher model is a soft label;

[0034] The ranking loss calculation method is used to calculate the cross-entropy between the similarity score list obtained by the teacher model and the hard labels;

[0035] By using cross-entropy as the teacher's knowledge loss, the student model is trained to obtain the teacher's knowledge loss.

[0036] In a specific embodiment of the present invention, constructing a training dataset specifically includes:

[0037] The samples were grouped according to the code length in the samples, into first smart contract samples and second smart contract samples.

[0038] The first positive sample, the second positive sample, and the first negative sample from the first smart contract sample and the second smart contract sample, respectively, are selected based on their semantic similarity to a preset threshold; the preset threshold is used to control the minimum similarity between samples.

[0039] Based on the preset sample group type, ternary sample groups are constructed for the samples in the first smart agent contract sample and the second smart agent contract sample. The ternary sample groups are a first type ternary sample group, a second type ternary sample group, and a third type ternary sample group. The first type ternary sample group includes semantically similar first positive samples and second positive samples without vulnerabilities, as well as first negative samples with vulnerabilities. The second type ternary sample group includes semantically similar first positive samples and second positive samples with the same type of vulnerabilities, as well as first negative samples without vulnerabilities. The third type ternary sample group includes semantically similar first positive samples and second positive samples with the same type of vulnerabilities, as well as first negative samples with different vulnerabilities.

[0040] In a specific embodiment of the present invention, the training module is specifically used to train the teacher pre-training model using the first smart contract sample, obtain the contrastive loss and ranking consistency loss of the teacher pre-training model, and adjust the parameters of the teacher pre-training model; wherein, the contrastive loss of the teacher pre-training model is used to compare the differences between positive and negative samples with the same or different vulnerabilities, and the ranking consistency loss is used to indicate the ranking relationship between the samples in the first smart contract sample, ensuring that the model's ranking of samples is consistent with the actual situation.

[0041] In a specific embodiment of the present invention, the identification module is specifically used to input the smart contract to be detected into the student model to obtain an embedding vector;

[0042] The type of vulnerability in the smart contract to be detected is determined based on the label of the nearest neighbor of the embedded vector in the vulnerability database.

[0043] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the knowledge distillation-based smart contract vulnerability detection method described above.

[0044] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the knowledge distillation-based smart contract vulnerability detection method as described above.

[0045] Compared with existing technologies, it has the following outstanding advantages:

[0046] Compared with existing technologies, this invention's method, through a unique multi-step knowledge distillation and contrastive learning framework, combines short smart contract pre-training with long smart contract fine-tuning. This effectively overcomes the difficulties existing technologies face in processing long smart contract code, accurately capturing local vulnerability features of long smart contracts and significantly improving the accuracy and comprehensiveness of long smart contract vulnerability detection. A multi-task learning framework combining InfoNCE loss, ranking consistency loss, and teacher knowledge loss optimizes model learning performance from multiple dimensions. In the vulnerability detection process, from data preprocessing to model training and final vulnerability retrieval, each stage works closely together and efficiently. Compared to existing dynamic program analysis methods that are prone to false positives and have low efficiency, this invention significantly improves detection efficiency, providing more timely and reliable support for smart contract security. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0048] Figure 1 This is a flowchart illustrating the smart contract vulnerability detection method based on knowledge distillation proposed in this invention.

[0049] Figure 2 This is a schematic diagram illustrating a specific implementation of the smart contract vulnerability detection method based on knowledge distillation proposed in this invention.

[0050] Figure 3 This is a schematic diagram of the dataset constructed in this invention;

[0051] Figure 4 This is a schematic diagram illustrating the training process of the teacher model and the student model in this invention;

[0052] Figure 5 This is a schematic diagram of the contrastive learning loss in this invention;

[0053] Figure 6 This is a schematic diagram illustrating the ranking consistency loss in this invention;

[0054] Figure 7 This is a schematic diagram illustrating the loss of teacher knowledge in this invention;

[0055] Figure 8 This is a schematic diagram of the processing flow of smart contract samples to be classified in this invention;

[0056] Figure 9 This is a schematic diagram illustrating the statistical situation of the dataset in this invention;

[0057] Figure 10This is a schematic diagram illustrating the retrieval performance of the model of the present invention on Acc@1, Acc@5, Acc@10, and Acc@20;

[0058] Figure 11 This diagram illustrates a comparison of the results of the classifier-based vulnerability detection method and the retrieval-based vulnerability detection method of the present invention.

[0059] Figure 12 This is a schematic diagram of a smart contract vulnerability detection device based on knowledge distillation proposed in this invention.

[0060] Figure 13 This is a schematic diagram of the computer hardware of the present invention. Detailed Implementation

[0061] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0062] It should also be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0063] It should also be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0064] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0065] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0066] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0067] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0068] To make the above-mentioned features and effects of the present invention clearer and easier to understand, specific embodiments are described below in conjunction with the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are merely illustrative. The scope of protection of the present invention is not limited to the disclosed embodiments, but is defined by the appended claims.

[0069] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0070] A smart contract is a computer program stored on the blockchain. It defines rules and logic through code and automatically executes operations such as transfers and data updates when specific conditions are triggered. Smart contract vulnerabilities are a critical issue in blockchain security. Because contracts are difficult to modify after deployment and often involve financial transactions, vulnerabilities can lead to catastrophic consequences.

[0071] Current technologies apply static vulnerability analysis methods based on deep learning to the field of smart contract vulnerability detection. However, existing methods have the following problems: First, they rely on classification loss to guide the model in learning vulnerability features. Classification loss only focuses on the correctness of the model's output labels and does not impose constraints on intermediate feature representations. In long contracts, vulnerability features may be distributed across discrete code segments, and the model is easily distracted by numerous irrelevant syntactic structures. For example, complex contexts such as circular dependencies and nested calls dilute key vulnerability features, preventing the model from focusing on critical code segments. Second, while the pre-trained models used in existing methods possess basic code understanding capabilities, their original training objectives, such as masked language modeling, differ from the fine-grained requirements of long contract vulnerability detection. The model struggles to accurately locate sparse vulnerability patterns within thousands of lines of code. Third, there is a lack of noise filtering mechanisms. In long contracts, the vast majority of code may be irrelevant to vulnerabilities, but classification loss cannot drive the model to establish a noise filtering mechanism. For example, the model may mistakenly identify common security patterns, such as missing permission checks, as vulnerability features, while ignoring truly dangerous operations such as unverified external calls, leading to an increased false positive rate. Fourth, feature representations are easily affected by contextual interference. For long contracts, the dispersion of attention weights for key features in existing models increases significantly, indicating that the models cannot reliably capture long-distance dependencies. Fifth, there is insufficient utilization of temporal information. Some vulnerability features are time-sensitive, but training driven by traditional classification loss is prone to leakage of temporal information—the model may rely on unreasonable code location correlations for prediction, rather than truly understanding the vulnerability triggering path.

[0072] Figure 1 This is a flowchart illustrating the smart contract vulnerability detection method based on knowledge distillation proposed in this invention. Figure 1 As shown, the smart contract vulnerability detection method based on knowledge distillation proposed in this invention includes the following steps:

[0073] 110. Construct a training dataset, which includes a first smart contract sample and a second smart contract sample, wherein the code length of the second smart contract sample is greater than the code length of the first smart contract sample.

[0074] 120. Train the teacher pre-trained model using the first smart contract sample to obtain the teacher model. Use the teacher model and the first smart contract sample to pre-train the student model, and use the second smart contract sample to fine-tune the student model.

[0075] 130. Using a student model, implement vulnerability detection for smart contracts to be tested.

[0076] In a specific embodiment of the present invention, step 120, which trains the student model using the teacher model and the first smart contract sample, specifically includes:

[0077] 121. Input the first smart contract sample into the student model to obtain the pre-training results. Using the pre-training results, obtain the contrast loss, ranking consistency loss and teacher knowledge loss of the student model.

[0078] 122. Based on the contrast loss, ranking consistency loss and teacher knowledge loss of the student model, the parameters of the student model are corrected.

[0079] In a specific embodiment of the present invention, step 121 specifically includes:

[0080] Using the hard labels in the first smart contract sample and the soft labels obtained by knowledge distillation of the first smart contract sample by the teacher model, the student pre-training model is trained to obtain the teacher knowledge loss of the student model. Here, the hard label is the real vulnerability type of the smart contract sample, and the soft label is the probability value of the vulnerability type generated by the teacher model for the smart contract sample.

[0081] In a specific embodiment of the present invention, step 121, which uses the hard labels in the first smart contract sample and the soft labels obtained by knowledge distillation of the first smart contract sample by the teacher model to train the student model, specifically includes:

[0082] For the first smart contract sample, obtain the similarity score list obtained by the teacher model, where the similarity score list obtained by the teacher model is a soft label;

[0083] The ranking loss calculation method is used to calculate the cross-entropy between the similarity score list obtained by the teacher model and the hard labels;

[0084] By using cross-entropy as the teacher's knowledge loss, the student model is trained to obtain the teacher's knowledge loss.

[0085] In a specific embodiment of the present invention, step 110 specifically includes:

[0086] The samples were grouped according to the code length in the samples, into first smart contract samples and second smart contract samples.

[0087] The first positive sample, the second positive sample, and the first negative sample from the first smart contract sample and the second smart contract sample, respectively, are selected based on their semantic similarity to a preset threshold; the preset threshold is used to control the minimum similarity between samples.

[0088] Based on the preset sample group type, ternary sample groups are constructed for the samples in the first smart agent contract sample and the second smart agent contract sample. The ternary sample groups are a first type ternary sample group, a second type ternary sample group, and a third type ternary sample group. The first type ternary sample group includes semantically similar first positive samples and second positive samples without vulnerabilities, as well as first negative samples with vulnerabilities. The second type ternary sample group includes semantically similar first positive samples and second positive samples with the same type of vulnerabilities, as well as first negative samples without vulnerabilities. The third type ternary sample group includes semantically similar first positive samples and second positive samples with the same type of vulnerabilities, as well as first negative samples with different vulnerabilities.

[0089] In a specific embodiment of the present invention, step 120, which trains the teacher pre-training model using the first smart contract sample to obtain the teacher model, specifically includes:

[0090] The process of training the teacher pre-trained model using the first smart contract sample yields the contrastive loss and ranking consistency loss of the teacher pre-trained model, and the parameters of the teacher pre-trained model are adjusted accordingly. The contrastive loss of the teacher pre-trained model is used to compare the differences between positive and negative samples with the same or different vulnerabilities, and the ranking consistency loss is used to indicate the ranking relationship between the samples in the first smart contract sample, ensuring that the model's ranking of the samples is consistent with the actual situation.

[0091] In a specific embodiment of the present invention, step 130 specifically includes:

[0092] The smart contract to be tested is input into the student model to obtain the embedding vector;

[0093] The type of vulnerability in the smart contract to be detected is determined based on the label of the nearest neighbor of the embedded vector in the vulnerability database.

[0094] This invention provides a multi-step knowledge distillation and comparative learning method for representing vulnerabilities in long smart contracts. By combining pre-training learning of short smart contract samples with fine-tuning optimization of long smart contract samples, it solves the problems of insufficient generalization ability of existing methods for smart contract vulnerabilities, difficulty in distinguishing features with similar semantics but different vulnerability types, and especially insufficient ability to capture and discover local vulnerabilities in long smart contracts. This improves the detection capability of global and local vulnerability characteristics of smart contracts.

[0095] The following is combined with Figure 2 The specific implementation of the knowledge distillation-based smart contract vulnerability detection method proposed in this invention is described below. Figure 2 As shown, the smart contract vulnerability detection method based on knowledge distillation of the present invention includes:

[0096] Step 1: Obtain the smart contract source code data and perform formatted preprocessing. In the implementation example of Step 1, this specifically includes:

[0097] Step 1.1 Collect a dataset of well-formatted smart contract sample source code written in Solidity from the open-source project website GitHub.

[0098] Step 1.2 Cleansing and standardizing the collected smart contract source code: Remove all multi-line " / ... / " and single-line " / / ..." comments from the smart contract source code; remove code headers containing "pragma solidity" declarations; clean up leading and trailing whitespace characters, as well as redundant blank lines and empty content lines in the source code; replace structurally irrelevant information such as Ethereum addresses and hash values ​​in the source code with generic placeholders "YOUR_ADDR" and "HASH"; standardize indentation and line breaks, and remove extra spaces at the end of each line; ensure there is exactly one space before the left curly brace "{" in function or class declarations; adjust the spaces within brackets and square brackets to keep the code clean; remove unnecessary spaces in method declarations, especially spaces before access modifiers; ensure that keywords such as "contract", "library", "interface", and "struct" are correctly separated by blank lines.

[0099] Step 1.3, Generate a structured mapping for the smart contract: Segment the smart contract code to divide it into logical units; In each logical unit, establish dependencies between units through syntax analysis, including function call chains, variable reference relationships, and inheritance chains; Store the content (such as function, variable, and structure definitions), metadata (such as type and scope), and dependencies of each logical unit as a structured mapping table, recording the content, metadata, and dependencies of each logical unit.

[0100] Step 1.4: Output the formatted smart contract code based on the structured mapping of the smart contract in Step 1.3.

[0101] The purpose of this invention's smart contract formatting preprocessing is to improve the standardization and parsability of source code, ensure the consistency and high quality of input data, and provide stable and reliable basic data support for subsequent vulnerability sample construction and model training. Formatting preprocessing can remove invalid information in the source code that may interfere with analysis, while extracting key metadata and dependencies of logical units.

[0102] Step 2: Based on code length and semantic similarity, construct training, verification, and testing datasets for vulnerability samples.

[0103] Figure 3 This is a schematic diagram of the dataset constructed in this invention. For example... Figure 3As shown in the contrastive training triplet (a), in the "Secure-Secure-Any Vulnerability" group, two smart contracts are labeled "Secure," and the third smart contract is labeled "Any Vulnerability." This set of samples is used for contrastive training to help the teacher model learn the differences between secure contracts and other contracts. In the "Vulnerability of a Certain Type-Same Type Vulnerability-Secure" group, two smart contracts have the same type of vulnerability, and the third smart contract is labeled "Secure." This set of samples helps the teacher model learn the characteristics of the same type of vulnerability and distinguish secure contracts. In the "Vulnerability of a Certain Type-Same Type Vulnerability-Another Type Vulnerability" group, two smart contracts have the same type of vulnerability, and the third smart contract has another type of vulnerability. This set of samples helps the teacher model learn the differences between different vulnerabilities. As shown in the validation sample pair (b), the same type sample pair, labeled: 1, indicates that both smart contracts have the same type of vulnerability. The label is 1, indicating that they belong to the same category. The same type sample pair, labeled: 1, indicates another same type sample pair, where both smart contracts have the same type of vulnerability. The label is 1. The different type sample pair, labeled: 0, indicates that the two smart contracts have different vulnerabilities. The label is 0, indicating that they belong to different categories. Figure 3 It demonstrates sample pairs and triples used to train and validate smart contract models, helping teacher models learn the characteristics and differences of different vulnerabilities and improve the model's ability to identify vulnerabilities.

[0104] In the implementation example of step 2, the specific components include:

[0105] Step 2.1: Use cross-validation to reasonably divide the samples for training, validation, and testing;

[0106] Step 2.2: Construct the training dataset by extracting samples from the training samples in Step 2.1 and constructing three-class triplet sample groups for training, including:

[0107] A sample grouping strategy based on code length is introduced, grouping samples according to code length to narrow the scope of similarity comparison and improve the efficiency and relevance of similarity calculation:

[0108]

[0109] Among them, D train It is the training sample set. Let T be the threshold length of the code length difference, T be the smart contract sample, and T' be the positive sample pair of T.

[0110] Define a similarity function sim(h1, h2), which measures the similarity between the vector representations of two codes by calculating the Euclidean distance. The code vectors are provided by the tool SmartEmbed. (·) =M(T) (·) );

[0111] Within each length group, positive and negative samples T with semantic similarity higher than a preset threshold are selected. pos ,T′ pos ,T neg This threshold controls the minimum similarity required between sample pairs, ensuring that the selected samples have a certain degree of semantic consistency or closeness.

[0112]

[0113] Among them, C j Let c be the triplet sample set constructed for comparative training, where c is the total number of triplets.

[0114] Based on the above length and semantic filtering, different types of ternary sample groups are generated: Type A (security-security-any vulnerability), which selects semantically similar sample pairs from vulnerability-free code as positive classes and randomly selects a vulnerable code as a negative class; Type B (a certain type of vulnerability-same type of vulnerability-security), which selects samples from vulnerable code of the same type for pairing for each vulnerability label, and simultaneously selects a sample from vulnerability-free code as a negative class; Type C (a certain type of vulnerability-same type of vulnerability-another type of vulnerability), which selects sample pairs from code with the same type of vulnerability and selects samples with different vulnerabilities as negative classes. Through the above methods, this method expects the model to learn the features of vulnerabilities of different types that are semantically similar in the overall code.

[0115] This method of constructing comparative sample pairs that comprehensively considers the semantic similarity of code can provide an effective learning basis for the model to learn the subtle differences between different vulnerabilities.

[0116] Step 2.3: Construct a verification dataset. Extract samples from the verification samples in Step 2.1 and construct binary sample pairs for verification. The sample pairs are divided into positive class (both without vulnerabilities or both with vulnerabilities) and negative class (one with a vulnerability and one without a vulnerability).

[0117] P j ={(T j ,T′ j ,L j )}

[0118]

[0119] To verify the sample pair, if T j and T j When there are no vulnerabilities or when there are vulnerabilities, L j If p is 1, then p is 0; otherwise, p is 0. p is the number of constructed validation sample pairs.

[0120] Step 2.4: Construct a test dataset. Extract samples from the test samples in Step 2.1 to construct a labeled test dataset, where each sample has a clear vulnerability label, used to evaluate the model's sample classification performance.

[0121] Step 3 involves learning the teacher model on short smart contract vulnerability samples. In the implementation example of Step 3, this specifically includes:

[0122] Select smart contract samples C whose maximum support length l is within the attention window of the pre-trained model. l Sample pairs are constructed, and a pre-trained teacher model is used to learn on the smart contract dataset, enabling the teacher model to deeply learn the vulnerability features of smart contracts. The training loss of the teacher model in this stage is as follows:

[0123]

[0124] in, The final loss of the teacher model, and These represent the contrastive loss and ranking consistency loss during the teacher's learning process, respectively, with β as a moderating factor. In this implementation case, the teacher model is a pre-trained model based on the CodeBERT architecture, with a maximum window length of 512.

[0125] Step 4: Based on the teacher model obtained in Step 3, introduce contrastive loss, ranking consistency loss, and knowledge distillation loss to distill the knowledge from the teacher model into the student model. In the implementation example of Step 4, this specifically includes:

[0126] The parameters of the trained teacher model are frozen and then introduced into the student model in C. l Training is then performed. The training loss of the student model at this stage is as follows:

[0127]

[0128] in, For the final loss of the student model, and These represent the contrast loss, ranking consistency loss, and teacher loss in the student learning process, respectively, with β and γ as moderating factors. In this implementation case, the student model is a lightweight model based on a combined coding residual network.

[0129] Step 5: Based on the student model obtained in Step 4, further fine-tune the student model on the long smart contract vulnerability sample. In the implementation example of Step 5, this specifically includes:

[0130] The student model, after undergoing the teacher knowledge distillation stage, is used for training on the entire training set C. The training loss of the student model in this stage is as follows:

[0131]

[0132] in, For the final loss of the student model, and These represent the contrast loss and ranking consistency loss during the student's learning process, respectively, with β being the moderating factor.

[0133] The vulnerability-aware contrastive learning method proposed in this invention enables the model to learn the differential characteristics of smart contract vulnerabilities through the construction of a contrastive loss for vulnerability awareness. At the same time, by adopting a step-by-step joint training method, it can further extract global vulnerability feature information of long smart contracts while making full use of the knowledge of the pre-trained model, thereby enhancing the performance of smart contract vulnerability detection and having important practical value.

[0134] Figure 4 This is a schematic diagram illustrating the training process of the teacher model and student model in this invention. The overall flow of steps 3, 4, and 5 in the above implementation example can be found by referring to... Figure 4 .like Figure 4 As shown, the short smart contract dataset contains multiple smart contract samples, each consisting of three contracts, labeled as "Secure-Secure-Any Vulnerability", "Vulnerability of a Certain Type-Similar Vulnerability-Secure", and "Vulnerability of a Certain Type-Similar Vulnerability-Another Type of Vulnerability". Phase I: Teacher Model Learning, where the teacher model learns from the short smart contract dataset. Phase II: Teacher Knowledge Distillation, where the teacher model extracts its knowledge and uses it to guide the student model's learning. The teacher model distills the learned knowledge into the student model, helping the student model better learn the vulnerability characteristics of smart contracts. Phase III: Student Model Learning, where the student model learns from the long smart contract dataset. The long smart contract dataset contains multiple smart contract samples, each consisting of three contracts, labeled as "Secure-Secure-Any Vulnerability", "Vulnerability of a Certain Type-Similar Vulnerability-Secure", and "Vulnerability of a Certain Type-Similar Vulnerability-Another Type of Vulnerability".

[0135] The construction of the comparative loss for vulnerability awareness includes the following three parts:

[0136] InfoNCE loss, the goal of contrastive learning, is to learn effective data representations by bringing similar semantic representations closer together while pushing apart representations with different semantics. Figure 5 This is a schematic diagram of the contrastive learning loss in this invention. For example... Figure 5As shown, the legend in the model input contains a star and a pentagon, representing different smart contract samples, respectively. The input module contains multiple smart contract samples, each consisting of three contracts, labeled "Secure-Secure-Any Vulnerability," "Vulnerability of a Certain Type-Similar Vulnerability-Secure," and "Vulnerability of a Certain Type-Similar Vulnerability-Another Type of Vulnerability." The star in the model output represents the model's output results for the input smart contract samples. Each star represents a feature representation of a smart contract. The star in the model output is distributed in different positions, indicating the distribution of feature representations of different smart contract samples in the feature space. Similar samples (such as vulnerabilities of the same type) are grouped together, while different samples (such as vulnerabilities of different types) are separated. The model comparison section demonstrates how the model compares and classifies different smart contract samples. The comparison results are used to evaluate the model's performance and accuracy. The comparison results are represented by different labels (such as stars and pentagons), showing the model's classification of different smart contract samples. Correct classification results are marked as 1, and incorrect classification results are marked as 0. Figure 5 The smart contract model's input, output, and comparison process. The model learns from short smart contract datasets to generate feature representations of smart contract samples, and then verifies and compares these representations on long smart contract datasets. Through contrastive learning and teacher knowledge distillation, the model can accurately identify and classify vulnerabilities in smart contracts.

[0137] For a given batch of dataset The text pair T i and T i ′ are semantically related, m is the total number of sample pairs, and the optimization objective of InfoNCE is:

[0138]

[0139] Where m is the batch size, τ1 is the temperature hyperparameter, and sim(M(T) i ),M(T' i )) is M(T i ) and M(T i The cosine similarity between positive and negative samples is maximized by minimizing the cosine similarity between positive and negative samples. The InfoNCE loss allows the model to learn the difference between positive and negative samples.

[0140] The ranking consistency loss, implemented to further leverage in-batch negative samples to optimize the model's ability to capture fine-grained code similarity, ensures ranking consistency between two sets of similarity. The overall calculation process for this loss is shown below. Figure 4 ;

[0141] For smart contract T i Paired with its positive samples Ti ′, it is natural to obtain two sets of similarity lists with other sentences from the vector representations of these two contracts, i.e. and Then, distribution constraints are used in the model to drive consistency in the rankings between the two sets of similarity lists. Figure 6 This is a schematic diagram illustrating the ranking consistency loss in this invention. For example... Figure 6 As shown, the left side of the image displays the model's input, containing multiple smart contract samples. Each sample consists of three contracts, labeled "Secure-Secure-Any Vulnerability," "Vulnerability of a Certain Type-Similar Vulnerability-Secure," and "Vulnerability of a Certain Type-Similar Vulnerability-Another Type of Vulnerability." The middle section shows the student model's output. The student model processes the input smart contract samples and generates corresponding output results. The output results are represented by different markers (such as blue and red dots), showing the student model's classification of different smart contract samples. The student model's output results are distributed in different positions, representing the distribution of feature representations of different smart contract samples in the feature space. Similar samples (such as vulnerabilities of the same type) are grouped together, while different samples (such as vulnerabilities of different types) are separated. The right side shows the model comparison results, used to evaluate the student model's performance and accuracy. The comparison results are represented by different markers (such as blue and red dots), showing the student model's classification of different smart contract samples. In the comparison results, correct classification results are marked as 1, and incorrect classification results are marked as 0. Through comparative learning and teacher knowledge distillation, the student model is able to accurately identify and classify vulnerabilities in smart contracts. Figure 6 The diagram illustrates the input, output, and comparison process of the student model. By learning from the teacher model, the student model generates feature representations of smart contract samples and evaluates and compares them on a validation dataset. Through contrastive learning and teacher knowledge distillation, the student model is able to accurately identify and classify vulnerabilities in smart contracts.

[0142] The overall calculation process for this loss is as follows: Figure 6 As shown;

[0143] Given two similarity lists and Their probability distributions can be obtained. and Where σ(·) is the softmax function and τ2 is the temperature parameter. Ranking consistency can be ensured by minimizing the Jensen-Shannon (JS) divergence between the two probability distributions:

[0144]

[0145] Where, P i and Q iRepresent and Compared to the Kullback-Leibler (KL) divergence, the JS divergence is symmetric with respect to both distributions, rather than treating one of them as the baseline truth.

[0146] Teacher knowledge loss can be mitigated by learning through comparison of positive and negative samples, which enables the model to obtain coarse-grained semantic ranking information. In addition to ensuring the consistency of the model's own output, the knowledge of the teacher model can also be introduced as labels for the model to learn. Figure 7 This is a diagram illustrating teacher knowledge loss, such as... Figure 7 As shown, the left side displays the model's input, containing multiple smart contract samples. Each sample consists of three contracts, labeled T1, T2, and T3. These samples can be different types of smart contracts, such as "security-security-arbitrary vulnerability," "a certain type of vulnerability-same type of vulnerability-security," and "a certain type of vulnerability-same type of vulnerability-another type of vulnerability." The middle section shows the output of the student model. The student model processes the input smart contract samples and generates corresponding output results. The output results are represented by different markers (such as blue dots), showing the student model's classification of different smart contract samples. The student model's output results are distributed in different positions, indicating the distribution of the feature representations of different smart contract samples in the feature space. Similar samples (such as the same type of vulnerability) are grouped together, while different samples (such as heterogeneous vulnerabilities) are separated. The right side displays the teacher model's output. The teacher model processes the input smart contract samples and generates corresponding output results. The output results are represented by different markers (such as blue dots), showing the teacher model's classification of different smart contract samples. The teacher model's output results are distributed in different positions, indicating the distribution of the feature representations of different smart contract samples in the feature space. Similar samples (such as similar vulnerabilities) are grouped together, while different samples (such as outlier vulnerabilities) are separated. The figure illustrates the comparison process between the outputs of the student model and the teacher model, used to evaluate the performance and accuracy of the student model. The comparison results are represented by different markers (such as blue and red dots), showing the classification of different smart contract samples by the student and teacher models. In the comparison results, correct classification results are marked as 1, and incorrect classification results are marked as 0. Through contrastive learning and teacher knowledge distillation, the student model can gradually approach the performance of the teacher model, accurately identifying and classifying vulnerabilities in smart contracts. Figure 7 The diagram illustrates the input, output, and comparison process of the student and teacher models. The student model learns from the teacher model's knowledge to generate feature representations of smart contract samples, which are then evaluated and compared on a validation dataset. Through contrastive learning and teacher knowledge distillation, the student model is able to accurately identify and classify vulnerabilities in smart contracts.

[0147] The overall calculation process for this loss is as follows: Figure 7 As shown;

[0148] For each contract T in the batch i The method obtains a list of similarity scores from the teacher model as pseudo-labels and uses these labels in the ranking distillation. ListNet is used as the algorithm for calculating the ranking loss, which aims to minimize the cross-entropy between the permutation probability distribution and the true labels. To reduce computational complexity, a top-1 probability distribution is typically used as an alternative.

[0149]

[0150] in, This is a list of similarity scores derived from the teacher model, where τ2 and τ3 are temperature parameters. The Top-1 probability is the predicted answer the model is most confident in. For the list of similarity scores, s i The similarity score is given by n, which is the number of samples used for comparison. The Top-1 probability is calculated as follows:

[0151]

[0152] in, τ is a temperature parameter.

[0153] Step 6: Preprocess the given smart contract source code according to Step 1, use the student model obtained in Step 5 to output the corresponding contract representation vector, and search the vulnerability database based on this feature vector to determine the vulnerability situation of the given contract. Figure 8This is a schematic diagram of the processing flow for smart contract samples to be classified in this invention. The left side shows the smart contract samples that need to be classified. These samples may be newly collected or unlabeled smart contracts, requiring vulnerability detection and classification by the model. It includes samples containing known smart contract vulnerabilities. These samples serve as reference data to help the model learn and identify different types of vulnerabilities. The middle part is a Residual Convolutional Network (RCN) used for feature extraction from the input smart contract samples. The RCN, through its multi-layer neural network structure, can effectively capture complex features and patterns in smart contracts. After feature extraction, the model calculates the semantic distance between the sample to be classified and samples in the vulnerability database. The semantic distance reflects the similarity or difference between samples; the smaller the value, the more similar the samples are. The right side shows the KNN (K-Nearest Neighbors) classification process. By calculating the semantic distance between the sample to be classified and samples in the vulnerability database, the nearest neighbor sample is found, thereby determining the vulnerability type of the sample to be classified. KNN classification can effectively classify smart contracts, identifying the existence and type of vulnerabilities. The NNR (Nearest Neighbor Retrieval) process was also demonstrated. By retrieving known vulnerability samples most similar to the sample to be classified, the classification results are further validated and supplemented. NNR retrieval can help improve the accuracy and recall of classification. The final output includes a set of recalled samples, labeled with different categories (e.g., secure, vulnerable). These results can be used for further analysis and processing, helping users understand the security status of smart contracts. Figure 8 This paper demonstrates a complete smart contract classification system workflow, from inputting samples to be classified and a vulnerability database, to feature extraction using a residual convolutional network, to vulnerability detection and classification using KNN classification and NNR retrieval, and finally outputting the classification results. This system can effectively identify vulnerabilities in smart contracts, helping to improve their security.

[0154] Example of an implementation in step 6 Figure 8 As shown, it specifically includes:

[0155] For a set of smart contract libraries that have been collected and tagged L i For smart contracts T i The corresponding tags first convert the contracts in the vulnerability database into embedded representations:

[0156] H,L = ExtractEmbeddings(D,M)

[0157] Where D is the contract library, M is the contract representation model, H represents the set of embedding vectors extracted from D, and L is the corresponding label set. In the k-nearest neighbor (kNN) classification process, for a given smart contract T with an unknown vulnerability and its embedding vector h, the vulnerability type of the smart contract is determined based on the labels L of its k nearest neighbors in the vulnerability library.

[0158]

[0159] In the Nearest Neighbor Retrieval (NNR) process, for a given embedding vector h of a smart contract T with an unknown vulnerability, k nearest neighbor samples are retrieved from the vulnerability database to form a retrieval result set D. T :

[0160] D T =NearestNeighborRetrieval(H,h,D,k)

[0161] To further demonstrate the effectiveness of the method, this chapter conducts extensive experiments on a smart contract dataset and performs a comprehensive analysis of the model. The dataset statistics are as follows: Figure 9 As shown. The main performance metrics used in the experiment include: Acc@k (Top-k accuracy): the proportion of correct results included in the top k predictions, evaluating the model's performance in the retrieval task; Acc (accuracy): measuring the overall proportion of correct predictions by the model, reflecting the model's comprehensive performance; F1 (F1-Score): the harmonic mean of precision and recall, suitable for performance evaluation in class imbalance scenarios; P (precision): the proportion of predicted positive samples that are actually positive samples, reflecting the false positive rate; R (recall): the proportion of actual positive samples that are successfully identified, reflecting the false negative rate.

[0162] exist Figure 9 Furthermore, the retrieval performance of the model on Acc@1, Acc@5, Acc@10, and Acc@20 was compared. The results show that, based on the training method of this invention, training the combined encoding residual network (marked "Ours" in the performance comparison data) achieves the best results on every metric, especially on Acc@1 and Acc@5, reaching 91.60% and 96.49% respectively, far exceeding the comparison model. This indicates that the method of this invention can not only effectively detect vulnerabilities but also provide more accurate vulnerability relevance results at higher ranking positions, thereby improving the efficiency of smart contract vulnerability detection.

[0163] exist Figure 10 and Figure 11The paper presents comparative results of classifier-based and retrieval-based vulnerability detection methods on multiple performance metrics (Acc, F1, P, R). Classifier-based models include BiLSTM-MLP, GRU-MLP, CapsNet, Km-CapsNet, BT-ResNet, BT-Inceptionv3, and CodeBERT-Solidity-MLP; retrieval-based models include CC-BiLSTM, CC-Transformers, CodeBERT, CodeBERT-Solidity, LongCoder, and Siamese models. Experimental results show that the combined coding residual network (Ours) trained based on this invention significantly outperforms the comparative models on all performance metrics, especially in F1 and R, reaching 94.86% and 95.25% respectively, indicating that the method of this invention has higher robustness in comprehensively detecting different vulnerability features.

[0164] In summary, the experimental results of the method of the present invention on smart contract datasets fully verify its superiority in vulnerability detection tasks, which can significantly improve detection accuracy and retrieval performance, and provide more reliable protection for the security of smart contracts.

[0165] Compared with the prior art, the advantages of the present invention are as follows:

[0166] This invention employs a multi-stage learning strategy to optimize the model's ability to process long smart contracts. It utilizes a step-by-step joint training method, dividing training into three stages: teacher model learning, knowledge distillation, and student model learning. This fully leverages the knowledge of the pre-trained model and overcomes the limitations of traditional attention windows, enabling better learning of vulnerability features in long contracts. Through step-by-step optimization, this invention addresses the limitations of pre-trained models in processing long smart contracts, not only improving the ability to extract global information but also effectively reducing detection bias caused by local errors.

[0167] This invention enhances vulnerability feature modeling capabilities by employing multi-faceted sample construction and learning strategies. It utilizes a multi-task learning framework combining InfoNCE loss, ranking consistency loss, and teacher knowledge loss to optimize model learning performance from multiple dimensions. Through multi-faceted comparative learning, this invention can bridge the representational distance of similar semantics, separate representations of different semantics, and capture fine-grained code similarity information. This enables the model to learn smart contract vulnerability features more comprehensively, exhibiting higher robustness and accuracy in scenarios with complex semantics and diverse vulnerability features.

[0168] Figure 12 This is a schematic diagram of a module for a smart contract vulnerability detection device based on knowledge distillation proposed in this invention. Figure 11As shown in the figure, this application provides a smart contract vulnerability detection device based on knowledge distillation. The device includes:

[0169] The dataset construction module is used to build the training dataset, which includes a first smart contract sample and a second smart contract sample, wherein the code length of the second smart contract sample is greater than the code length of the first smart contract sample;

[0170] The training module is used to train the teacher pre-trained model using the first smart contract sample to obtain the teacher model, to pre-train the student model using the teacher model and the first smart contract sample, and to fine-tune the student model using the second smart contract sample.

[0171] The identification module is used to detect vulnerabilities in smart contracts under test by leveraging a student model.

[0172] In a specific embodiment of the present invention, the training module is specifically used to input the first smart contract sample into the student model to obtain the pre-training result, and use the pre-training result to obtain the contrastive loss, ranking consistency loss and teacher knowledge loss of the student model;

[0173] The parameters of the student model are adjusted based on the contrastive loss, ranking consistency loss, and teacher knowledge loss of the student model.

[0174] In a specific embodiment of the present invention, the training module is specifically used to train the student model using the hard labels in the first smart contract sample and the soft labels obtained by the teacher model through knowledge distillation of the first smart contract sample, thereby obtaining the teacher knowledge loss of the student model. Here, the hard labels are the real vulnerability types of the smart contract sample, and the soft labels are the vulnerability type probability values ​​generated by the teacher model for the smart contract sample.

[0175] In a specific embodiment of the present invention, the training module is specifically used to obtain a list of similarity scores obtained by the teacher model for the first smart contract sample, wherein the list of similarity scores obtained by the teacher model is a soft label;

[0176] The ranking loss calculation method is used to calculate the cross-entropy between the similarity score list obtained by the teacher model and the hard labels;

[0177] By using cross-entropy as the teacher's knowledge loss, the student model is trained to obtain the teacher's knowledge loss.

[0178] In a specific embodiment of the present invention, constructing a training dataset specifically includes:

[0179] The samples were grouped according to the code length in the samples, into first smart contract samples and second smart contract samples.

[0180] The first positive sample, the second positive sample, and the first negative sample from the first smart contract sample and the second smart contract sample, respectively, are selected based on their semantic similarity to a preset threshold; the preset threshold is used to control the minimum similarity between samples.

[0181] Based on the preset sample group type, ternary sample groups are constructed for the samples in the first smart agent contract sample and the second smart agent contract sample. The ternary sample groups are a first type ternary sample group, a second type ternary sample group, and a third type ternary sample group. The first type ternary sample group includes semantically similar first positive samples and second positive samples without vulnerabilities, as well as first negative samples with vulnerabilities. The second type ternary sample group includes semantically similar first positive samples and second positive samples with the same type of vulnerabilities, as well as first negative samples without vulnerabilities. The third type ternary sample group includes semantically similar first positive samples and second positive samples with the same type of vulnerabilities, as well as first negative samples with different vulnerabilities.

[0182] In a specific embodiment of the present invention, the training module is specifically used to train the teacher pre-training model using the first smart contract sample, obtain the contrastive loss and ranking consistency loss of the teacher pre-training model, and adjust the parameters of the teacher pre-training model; wherein, the contrastive loss of the teacher pre-training model is used to compare the differences between positive and negative samples with the same or different vulnerabilities, and the ranking consistency loss is used to indicate the ranking relationship between the samples in the first smart contract sample, ensuring that the model's ranking of samples is consistent with the actual situation.

[0183] In a specific embodiment of the present invention, the identification module is specifically used to input the smart contract to be detected into the student model to obtain an embedding vector;

[0184] The type of vulnerability in the smart contract to be detected is determined based on the label of the nearest neighbor of the embedded vector in the vulnerability database.

[0185] In summary, the embodiments of the present invention address the problem that the existing smart contract vulnerability detection methods are insufficient in capturing sparse vulnerability features of long smart contracts, and propose a multi-step knowledge distillation and comparison learning method for vulnerability representation of long smart contracts.

[0186] This invention establishes a multi-step knowledge distillation and comparative learning framework. By pre-training short smart contracts and fine-tuning long smart contracts in stages, it effectively solves the problem that existing methods have difficulty distinguishing semantically similar but different vulnerability types, overcomes the deficiency in processing long smart contract code, accurately captures the local vulnerability features of long smart contracts, and improves the detection capability of long smart contract vulnerabilities.

[0187] Meanwhile, this invention constructs a contract training and verification dataset based on code length and semantic similarity. It builds three classes of triplet samples for the training dataset, binary sample pairs for the verification dataset, and a test dataset. Combining semantic similarity and length constraints, it guides the model to learn the subtle differences between different types of vulnerabilities, providing a valid basis for model learning and improving the model's accuracy in identifying vulnerability features.

[0188] This invention also guides the model's learning of smart contract vulnerability representations based on combined loss. By combining InfoNCE loss, ranking consistency loss, and teacher knowledge loss, the model's learning process is optimized from multiple perspectives, enhancing its ability to represent vulnerability features. Specifically, InfoNCE loss, at a coarse-grained level, prompts the model to distinguish between similar and different vulnerability features; for fine-grained differentiation of contract vulnerabilities, consistency loss and teacher knowledge distillation loss, at the unsupervised and supervised levels, respectively, prompt the model to distinguish vulnerability features.

[0189] Therefore, compared with the prior art, the present invention has many significant advantages:

[0190] Enhanced Long-Term Smart Contract Vulnerability Detection Capabilities: Utilizing a unique multi-step knowledge distillation and comparative learning framework, combined with short-term smart contract pre-training and long-term smart contract fine-tuning, this approach effectively overcomes the difficulties of existing technologies in handling long-term smart contract code. It can accurately capture local vulnerability features of long-term smart contracts, significantly improving the accuracy and comprehensiveness of long-term smart contract vulnerability detection. For example, in experiments, the detection metrics for long-term contract vulnerabilities, such as Acc@1 and Acc@5, reached 91.60% and 96.49% respectively, far exceeding the comparison model, fully demonstrating its advantages in long-term contract vulnerability detection.

[0191] Enhanced generalization ability: A multi-task learning framework combining InfoNCE loss, ranking consistency loss, and teacher knowledge loss is adopted to optimize the model's learning performance from multiple dimensions. This enables the model to effectively distinguish between smart contracts with similar semantics but different vulnerability types. For example, it can more accurately identify and differentiate between reentrancy vulnerabilities and integer overflow vulnerabilities. Compared with existing methods that rely on manual rules and have poor adaptability to new vulnerabilities, it has stronger generalization ability and can better adapt to complex and ever-changing smart contract vulnerability scenarios.

[0192] Improved detection efficiency: In the vulnerability detection process, from data preprocessing to model training and final vulnerability retrieval, each step works closely together and efficiently. For example, methods that construct datasets based on code length and semantic similarity improve the efficiency of sample selection and model learning. In the overall detection process, it can quickly and accurately determine the vulnerability status of a given contract. Compared with existing dynamic program analysis methods that are prone to false positives and have low efficiency, this method significantly improves detection efficiency and provides more timely and reliable support for the security protection of smart contracts.

[0193] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the hyper-realistic digital human video detection method.

[0194] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of a knowledge distillation-based smart contract vulnerability detection method as described above.

[0195] In addition, combined Figure 1 The smart contract vulnerability detection method based on knowledge distillation described in this application can be implemented by an electronic device, such as a computer device. Figure 13 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application.

[0196] In some embodiments, the computer device may further include a communication interface 83 and a bus 80. For example, Figure 5 The processor 81, memory 82, and communication interface 83 are connected via bus 80 and communicate with each other.

[0197] Specifically, the processor 81 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0198] The memory 82 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 81.

[0199] The processor 81 reads and executes computer program instructions stored in the memory 82 to implement any of the knowledge distillation-based smart contract vulnerability detection methods in the above embodiments.

[0200] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0201] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A smart contract vulnerability detection method based on knowledge distillation, characterized in that, The method includes: Construct a training dataset, which includes a first smart contract sample and a second smart contract sample, wherein the code length of the second smart contract sample is greater than the code length of the first smart contract sample; The teacher pre-training model is trained using the first smart contract sample to obtain the teacher model. The student model is then pre-trained using the teacher model and the first smart contract sample, and the student model is fine-tuned using the second smart contract sample. The student model is used to detect vulnerabilities in the smart contract to be tested.

2. The smart contract vulnerability detection method based on knowledge distillation according to claim 1, characterized in that, The pre-training of the student model using the teacher model and the first smart contract sample specifically includes: The first smart contract sample is input into the student model to obtain the pre-training results. Using the pre-training results, the contrast loss, ranking consistency loss and teacher knowledge loss of the student model are obtained. The parameters of the student model are corrected based on the contrastive loss, ranking consistency loss, and teacher knowledge loss of the student model.

3. The smart contract vulnerability detection method based on knowledge distillation according to claim 2, characterized in that, The first smart contract sample is input into the student model to obtain pre-training results. Using these pre-training results, the teacher knowledge loss of the student model is obtained, specifically including: Using the hard labels in the first smart contract sample and the soft labels obtained by the teacher model through knowledge distillation of the first smart contract sample, the student model is trained to obtain the teacher knowledge loss of the student model. Here, the hard labels are the real vulnerability types of the smart contract sample, and the soft labels are the probability values ​​of vulnerability types generated by the teacher model for the smart contract sample.

4. The smart contract vulnerability detection method based on knowledge distillation according to claim 3, characterized in that, The step of training the student model using the hard labels in the first smart contract sample and the soft labels obtained by the teacher model through knowledge distillation of the first smart contract sample specifically includes: For the first smart contract sample, obtain the similarity score list obtained by the teacher model, wherein the similarity score list obtained by the teacher model is the soft tag; The ranking loss calculation method is used to calculate the cross-entropy between the similarity score list obtained by the teacher model and the hard label; The cross-entropy is used as the teacher knowledge loss to train the student model, thus obtaining the teacher knowledge loss.

5. The smart contract vulnerability detection method based on knowledge distillation according to claim 1, characterized in that, The construction of the training dataset specifically includes: The samples are grouped according to the code length in the samples, into the first smart contract sample and the second smart contract sample; The first positive sample, the second positive sample, and the first negative sample in the first smart contract sample and the second smart contract sample respectively are selected based on their semantic similarity to a preset threshold; wherein, the preset threshold is used to control the minimum similarity between samples; According to the preset sample group type, ternary sample groups are constructed for the samples in the first smart agent contract sample and the second smart agent contract sample. The ternary sample groups are respectively a first type ternary sample group, a second type ternary sample group, and a third type ternary sample group. The first type ternary sample group includes semantically similar first positive sample and second positive sample without vulnerabilities, and a first negative sample with vulnerabilities. The second type ternary sample group includes semantically similar first positive sample and second positive sample with the same type of vulnerabilities, and a first negative sample without vulnerabilities. The third type ternary sample group includes semantically similar first positive sample and second positive sample with the same type of vulnerabilities, and a first negative sample with different vulnerabilities.

6. The smart contract vulnerability detection method based on knowledge distillation according to claim 1, characterized in that, The step of training the teacher pre-training model using the first smart contract sample to obtain the teacher model specifically includes: The teacher pre-training model is trained using the first smart contract sample to obtain the contrast loss and ranking consistency loss of the teacher pre-training model, and the parameters of the teacher pre-training model are adjusted. The contrast loss of the teacher pre-training model is used to compare the differences between positive and negative samples with the same or different vulnerabilities, and the ranking consistency loss is used to indicate the ranking relationship between the samples in the first smart contract sample to ensure that the model's ranking of samples is consistent with the actual situation.

7. The smart contract vulnerability detection method based on knowledge distillation according to claim 1, characterized in that, The process of using the student model to detect vulnerabilities in the smart contract under test specifically includes: The smart contract to be detected is input into the student model to obtain the embedding vector; The vulnerability type of the smart contract to be detected is determined based on the label of the nearest neighbor of the embedded vector in the vulnerability database.

8. A smart contract vulnerability detection device based on knowledge distillation, characterized in that, The device includes: A dataset construction module is used to construct a training dataset, which includes a first smart contract sample and a second smart contract sample, wherein the code length of the second smart contract sample is greater than the code length of the first smart contract sample; The training module is used to train the teacher pre-trained model using the first smart contract sample to obtain the teacher model, to pre-train the student model using the teacher model and the first smart contract sample, and to fine-tune the student model using the second smart contract sample. The identification module is used to detect vulnerabilities in the smart contract to be tested by leveraging the student model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the smart contract vulnerability detection method based on knowledge distillation as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the knowledge distillation-based smart contract vulnerability detection as described in any one of claims 1 to 7.

Images