An NFT deep forgery tracing system based on AI adversarial sample detection

Abstract

The application provides an NFT deep forgery tracing system based on AI adversarial sample detection, comprising obtaining a to-be-verified NFT, an original NFT training data set, and training an adversarial detection neural network; deep feature fingerprint extraction is performed on the original NFT in the original NFT training data set and the to-be-verified NFT respectively, to obtain an original feature hash value and a target feature hash value; core information of the original NFT is chained and solidified to obtain an original NFT ownership benchmark library; based on the original NFT ownership benchmark library, forgery detection and fine-grained feature matching are performed on the target feature hash value to obtain a detection result; forgery positioning is performed on the to-be-verified NFT to generate a tracing report; and a notarization certificate of the to-be-verified NFT is generated according to the tracing report and the detection result, and the to-be-verified NFT is subjected to restriction processing. The application solves the problem that the prior art cannot efficiently detect forged NFTs.

Description

Technical Field

[0001] This invention relates to the field of Web3 blockchain technology, and in particular to an NFT deepfake tracing system based on AI adversarial sample detection. Background Technology

[0002] With the booming development of the Web3 ecosystem, NFTs (non-fungible tokens), as the core carrier of digital assets, have widely penetrated into various fields such as digital art, virtual items, and copyright confirmation. Their fundamental value is built on uniqueness and immutability. However, the rapid iteration of generative AI technology has brought unprecedented challenges to the NFT field in deepfake infringement, including but not limited to AI-driven generative replication, pixel-level tampering, and secondary alteration. These forgeries are highly covert, capable of simulating the visual style and details of the original work. Traditional comparison methods based on shallow visual features—such as simple pixel matching or basic texture analysis—are no longer effective in distinguishing authenticity. At the same time, current NFT traceability technology has significant shortcomings: on-chain traceability links are often broken due to cross-platform transfers, there is a lack of reliable means to solidify off-chain infringement evidence, and conventional detection algorithms are easily circumvented by AI-generated adversarial examples. These problems make it extremely difficult to confirm the ownership of NFT creators, leading to frequent infringement disputes and seriously hindering the healthy development of the NFT ecosystem towards compliance.

[0003] In related technical fields, AI adversarial example detection technology has been initially applied in scenarios such as image recognition and security protection. Its core principle lies in achieving accurate identification by capturing subtle distortion features, texture residuals, and inherent watermark traces in adversarial examples (in this context, AI-forged NFTs). Blockchain technology, with its immutable, distributed storage, and fully traceable characteristics, has become the infrastructure for the field of digital evidence preservation. Multimodal feature extraction technology can mine the deep representation of data from multiple dimensions such as image, semantics, and structure, thereby improving the accuracy and robustness of identification and traceability. However, existing NFT forgery detection and traceability solutions still heavily rely on traditional visual feature comparison and basic blockchain evidence preservation, revealing obvious technical shortcomings: On the one hand, traditional feature comparison methods are easily bypassed by generative AI, and cannot effectively capture subtle distortions, texture anomalies, or fingerprint information of the generated model hidden in the AI ​​forgery process, resulting in almost no recognition of high-quality AI-forged NFTs; on the other hand, the traceability link is incomplete, and can only trace some transaction nodes on the chain, but cannot locate the source of the forgery, the intermediate tampering nodes, and the complete propagation path on and off the chain.

[0004] Furthermore, the solidification process for off-chain infringement evidence collection is weak, lacking legally valid evidence, which undermines the credibility of the evidence in judicial practice. More importantly, the current system has not yet established an automated intelligent judgment logic for originality; the detection and tracing process requires significant manual intervention, resulting in low efficiency and making it difficult to achieve intelligent protection for the entire lifecycle of NFTs. Summary of the Invention

[0005] This application provides an NFT deepfake tracing system based on AI adversarial sample detection to solve the problem that existing technologies cannot efficiently detect fake NFTs.

[0006] The system includes: A data acquisition and model training component is configured to acquire an NFT to be verified, an original NFT training dataset, and train an adversarial detection neural network using the original NFT training dataset; the original NFT training dataset includes several original NFTs. A deep feature extraction component is configured to extract deep feature fingerprints from the original NFT and the NFT to be verified in the original NFT training dataset, respectively, to obtain original feature hash values ​​and target feature hash values; the deep feature fingerprint extraction includes extracting fused features of pixel-level edge features, semantic texture features and hidden layer feature vectors; A blockchain evidence storage component is configured to solidify the core information of the original NFT on the blockchain based on the original feature hash value, thereby obtaining an original NFT ownership benchmark library. A forgery detection component is configured to perform forgery detection and fine-grained feature matching on the target feature hash value using the adversarial detection neural network and based on the original NFT ownership benchmark library to obtain a detection result; the detection result is used to characterize the infringement status of the NFT to be verified. The source tracing and location component is configured to locate the forgery source, tampering node, and propagation path of the NFT to be verified based on the metadata, transaction records, and detection results of the NFT to be verified, and generate a source tracing report. The intelligent judgment and evidence collection component is configured to generate evidence storage credentials for the NFT to be verified based on the source tracing report and the detection results, and to perform restriction processing on the NFT to be verified.

[0007] In some implementations, the data acquisition and model training component includes: A data acquisition module, configured to acquire the NFT to be verified and the original NFT training dataset; The data preprocessing module is configured to generate a positive sample set and a negative sample set based on the original NFT training dataset; perform unified preprocessing on the positive sample set, the negative sample set, and the NFT to be verified to obtain a preprocessed positive sample set, a preprocessed negative sample set, and a preprocessed NFT to be verified; and divide the preprocessed positive sample set and the preprocessed negative sample set into a training set, a validation set, and a test set. A model training module is configured to train the adversarial detection neural network using the training set, and to test and validate the adversarial detection neural network using the test set and the validation set.

[0008] In some implementations, the model training module includes: A model adjustment unit is configured to acquire the original adversarial detection neural network and adjust the settings of the original adversarial detection neural network according to the NFT adversarial feature extraction algorithm to obtain the adversarial detection neural network. A loss function adjustment unit is configured to adjust the loss function of the adversarial detection neural network by combining a cross-entropy loss function and a gradient loss function. The model training unit is configured to train the adversarial detection neural network using the training set, and to validate the adversarial detection neural network using the validation set after each round of model training until a preset training condition is met; after stopping model training, the adversarial detection neural network is tested using the test set, and a qualified adversarial detection neural network is output.

[0009] In some implementations, the deep feature extraction component includes: A pixel structure feature extraction module is configured to use an improved Gaussian-Laplacian operator to extract features from the original NFT to obtain the pixel-level edge features; A semantic texture feature extraction module is configured to use the SIFT algorithm to extract features from the original NFT to obtain the semantic texture features. A hidden layer feature extraction module is configured to input the original NFT into the adversarial detection neural network, extract the output of the penultimate layer of the model, and obtain the hidden layer feature vector. The feature fusion and hash generation module is configured to perform weighted feature fusion based on the pixel-level edge features, the semantic texture features, and the hidden layer feature vector, and to encrypt the fused features using the SHA-256 hash algorithm to generate the original feature hash value.

[0010] In some implementations, the blockchain evidence storage component includes: The information encryption module is configured to obtain the associated information of the original NFT, standardize the associated information and the original feature hash value, and encrypt the sensitive information in the associated information using the AES-256 encryption algorithm to obtain encrypted sensitive information. The blockchain evidence storage and on-chain module is configured to package data based on the encrypted sensitive information, the associated information, the timestamp, and the original feature hash value, and use a smart contract to upload the data to the blockchain to generate a unique evidence storage number. The ownership benchmark library construction module is configured to construct the original NFT ownership benchmark library, store the associated information, the timestamp, the unique certificate number, the original feature hash value, and the on-chain node information into the original NFT ownership benchmark library, and establish a hash index.

[0011] In some implementations, the forgery detection component includes: An adversarial sample detection module is configured to input the preprocessed NFT to be verified into the adversarial detection neural network to obtain the probability of forged samples. A fine-grained feature matching module is configured to calculate the NFT similarity by using a cosine similarity algorithm based on the original NFT ownership benchmark library and the target feature hash value when the probability of the forged sample is within a preset probability range. The infringement type differentiation module is configured to determine the infringement type based on the gradient features of the NFT to be verified extracted by the adversarial detection neural network and the NFT similarity, and obtain the detection result.

[0012] In some embodiments, the source tracing and location component includes: The on-chain traceability module is configured to perform node queries using a path tracing algorithm based on the metadata and transaction records of the NFT to be verified, and generate on-chain traceability results. The off-chain tracing module is configured to acquire the trajectory of counterfeit NFTs propagated off-chain based on terminal devices connected to the Web3 ecosystem and generate off-chain tracing results. The traceability result processing module is configured to integrate information based on the on-chain traceability results and the off-chain traceability results to obtain the traceability report.

[0013] In some implementations, the intelligent judgment and evidence collection component includes: An originality determination module is configured to determine originality based on the detection results and the source tracing report, and generate an originality determination result; the originality determination result is used to characterize whether the NFT to be verified is an infringing or counterfeit sample; The infringement evidence collection module is configured to collect evidence from the NFT to be verified and obtain an evidence collection number when the originality determination result indicates that the NFT to be verified is an infringing and counterfeit sample. The evidence storage certificate generation module is configured to generate a traceability evidence storage certificate based on the blockchain evidence storage information and evidence collection materials corresponding to the NFT to be verified; the traceability evidence storage certificate includes the evidence collection number. The closed-loop management module is configured to feed back the original NFT creator and the Web3 ecosystem platform based on the traceability certificate and the originality determination result.

[0014] In some implementations, if the probability of a forged sample is greater than the upper limit of the preset probability range, the NFT to be verified is determined to be a forged sample, and an infringement type determination is made.

[0015] In some implementations, the traceability and evidence storage certificate may also include a storage number, a collection number, NFT information to be verified, a test result, a traceability report, a list of evidence collection materials, a timestamp, and a blockchain node signature.

[0016] As described above, this application provides an NFT deepfake tracing system based on AI adversarial example detection. The system includes a data acquisition and model training component, configured to acquire a training dataset of NFTs to be verified and original NFTs, and to train an adversarial detection neural network using the original NFT training dataset; the original NFT training dataset includes several original NFTs; a deep feature extraction component, configured to perform deep feature fingerprint extraction on the original NFTs and the NFTs to be verified in the original NFT training dataset, respectively, to obtain original feature hash values ​​and target feature hash values; the deep feature fingerprint extraction includes extracting a fusion feature of pixel-level edge features, semantic texture features, and hidden layer feature vectors; and a blockchain evidence storage component, configured to, based on the original NFTs... The system employs a feature hash value to solidify the core information of the original NFT on the blockchain, resulting in an original NFT ownership benchmark library. A forgery detection component is configured to perform forgery detection and fine-grained feature matching on the target feature hash value using the adversarial detection neural network and based on the original NFT ownership benchmark library, obtaining a detection result. The detection result is used to characterize the infringement status of the NFT to be verified. A source tracing and location component is configured to locate the forgery source, tampering node, and propagation path of the NFT to be verified based on the metadata, transaction records, and the detection result, generating a source tracing report. An intelligent judgment and evidence collection component is configured to generate a certificate of authenticity for the NFT to be verified based on the source tracing report and the detection result, and to impose restrictions on the NFT to be verified. This application solves the problem that existing technologies cannot efficiently detect forged NFTs through the above system. Attached Figure Description

[0017] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of an NFT deepfake tracing system based on AI adversarial example detection, as described in this application. Figure 2 This is a schematic diagram of the data acquisition and model training components in an NFT deepfake tracing system based on AI adversarial example detection according to this application; Figure 3 This is a schematic diagram of the model training module in an NFT deepfake tracing system based on AI adversarial example detection according to this application; Figure 4This is a schematic diagram of a deep feature extraction component in an NFT deepfake tracing system based on AI adversarial example detection according to this application; Figure 5 This is a schematic diagram of a blockchain evidence storage component in an NFT deepfake tracing system based on AI adversarial sample detection, as described in this application. Figure 6 This is a schematic diagram of a forgery detection component in an NFT deepfake tracing system based on AI adversarial sample detection, as described in this application. Figure 7 This is a schematic diagram of the tracing and localization component in an NFT deepfake tracing system based on AI adversarial example detection according to this application; Figure 8 This is a schematic diagram of the intelligent judgment and evidence collection component in an NFT deepfake tracing system based on AI adversarial sample detection, as described in this application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0021] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0022] See Figures 1 to 8 As can be seen, this embodiment provides an NFT deepfake tracing system based on AI adversarial example detection, the system comprising: A data acquisition and model training component is configured to acquire NFTs to be verified, original NFT training datasets, and train an adversarial detection neural network using the original NFT training datasets; the original NFT training datasets include several original NFTs.

[0023] Specifically, in this embodiment, the data acquisition and model training component includes: The data acquisition module is configured to acquire the NFT to be verified and the original NFT training dataset.

[0024] Specifically, the data acquisition module collects 10,000 original NFT samples of different types (covering digital art, virtual props, text copyrights, etc.), ensuring that the sample formats include common NFT formats such as PNG, JPG, and GIF, and the resolutions cover mainstream specifications such as 1080P and 4K. Simultaneously, it collects the ownership information (creator, creation time) corresponding to the samples, which serves as the positive sample set S. pos .

[0025] The NFT to be verified is accessed through a Web3 interface. The system reads its original file (supporting formats such as PNG, JPG, GIF, etc.) and associated metadata, and records the access timestamp T. check .

[0026] The data acquisition and model training components also include: The data preprocessing module is configured to generate a positive sample set and a negative sample set based on the original NFT training dataset; perform unified preprocessing on the positive sample set, the negative sample set, and the NFT to be verified to obtain a preprocessed positive sample set, a preprocessed negative sample set, and a preprocessed NFT to be verified; and divide the preprocessed positive sample set and the preprocessed negative sample set into a training set, a validation set, and a test set.

[0027] Specifically, the data preprocessing module employs current mainstream generative AI models (such as Midjourney, Stable Diffusion, and DALL·E) for the positive sample set S. pos For each sample, three different types of forged samples are generated: AI-complete replica samples (completely mimicking the style, color, and details of the original NFT, leaving only a model watermark in the hidden layer features), pixel-level tampering samples (modifying local pixels of the original NFT, such as changing colors, adding / deleting details), and secondary alteration samples (based on the original NFT, performing secondary editing such as splicing, scaling, and rotation). A total of 30,000 forged samples are generated, which serve as the negative sample set S. neg .

[0028] For the positive sample set S pos and negative sample set S negUnified preprocessing is performed, including: image normalization (the resolution of all samples is uniformly adjusted to 1024×1024, and the pixel values ​​are normalized to the range of [0, 1]), noise removal (Gaussian filtering algorithm is used, the filter kernel size is set to 3×3, and the standard deviation σ=0.5 to remove random noise in the samples), and data augmentation (randomly flipping, rotating (0°, 90°, 180°, 270°) and adjusting the brightness (brightness variation range ±10%) to improve the generalization ability of the model.

[0029] The preprocessed positive and negative sample sets were divided into training, validation, and test sets in a 7:2:1 ratio. The training set contained 28,000 samples (7,000 positive samples and 21,000 negative samples), the validation set contained 8,000 samples (2,000 positive samples and 6,000 negative samples), and the test set contained 4,000 samples (1,000 positive samples and 3,000 negative samples). The final result was a dedicated NFT adversarial sample training dataset S, which was used for subsequent neural network training.

[0030] Using the same preprocessing method as the dataset above, the NFT to be verified is normalized (resolution adjusted to 1024×1024, pixel values ​​normalized to [0, 1]) and noise removed (3×3 Gaussian filtering, σ=0.5) to obtain the preprocessed sample x to be verified. check .

[0031] The data acquisition and model training components also include: A model training module is configured to train the adversarial detection neural network using the training set, and to test and validate the adversarial detection neural network using the test set and the validation set.

[0032] The model training module includes: A model adjustment unit is configured to acquire the original adversarial detection neural network and adjust the settings of the original adversarial detection neural network according to the NFT adversarial feature extraction algorithm to obtain the adversarial detection neural network. A loss function adjustment unit is configured to adjust the loss function of the adversarial detection neural network by combining a cross-entropy loss function and a gradient loss function. The model training unit is configured to train the adversarial detection neural network using the training set, and to validate the adversarial detection neural network using the validation set after each round of model training until a preset training condition is met; after stopping model training, the adversarial detection neural network is tested using the test set, and a qualified adversarial detection neural network is output.

[0033] Specifically, in this embodiment, the basic neural network architecture is selected: MobileNetV3 is chosen as the basic architecture because it is lightweight and has a fast inference speed, which is suitable for the real-time detection requirements of the Web3 ecosystem. At the same time, it is improved by adding two hidden feature extraction layers to capture the subtle distortion features of AI-forged NFTs and the model's inherent watermark.

[0034] Design an adversarial feature extraction algorithm: Based on the gradient feature extraction principle of deep neural networks, an adversarial feature extraction algorithm suitable for NFTs is designed. The specific formula is as follows: Let the NFT sample to be processed be x = (x1, x2, ..., xn). i , ..., x n ), where n is the total pixel dimension of the sample (1024×1024×3=3145728), the sample is input into the improved MobileNetV3 neural network f, and the output out is obtained; the i-th dimension of the sample is perturbed to obtain x. i +Δ and x i -Δ (Δ is the perturbation step size, with a value of 0.001, ensuring that the perturbation does not affect the visual effect of the sample and only captures the changes in the hidden layer gradient), calculate the output out of the neural network after perturbation. i + and out i - ; The first-order gradient of sample x in the i-th dimension is calculated as follows: ; Parameter explanation: Let be the first-order gradient of the NFT sample x in the i-th dimension, reflecting the degree of influence of the pixel in that dimension on the model output; The model output after adding a perturbation Δ to the i-th pixel; The output of the model after subtracting the perturbation Δ from the i-th pixel; Δ is the perturbation step size (value 0.001), used to capture the gradient changes in the hidden layer without affecting the visual effect of the sample.

[0035] The neural network output when sample x is not perturbed is out i Its second gradient is calculated as follows: ; in, The second-order gradient of the NFT sample x at the i-th pixel reflects the rate of change of the first-order gradient and can capture subtle distortion features of AI-forged samples; out i This is the model output when the i-th pixel has no perturbation added. To extract the steep changes in the output surface of the neural network (the gradient changes of AI-forged samples are significantly higher than those of original samples), the magnitudes and variances of the first and second gradients are calculated as core indicators of adversarial features, as shown in the following formula: The magnitude of the first gradient: ; Parameter explanation: The magnitude of the first-order gradient of all dimensions of sample x is used to quantify the overall strength of the gradient. This value of AI-forged samples is significantly higher than that of original samples. Let be the first-order gradient of the i-th pixel.

[0036] Variance of the first gradient: ; in, is the variance of the first-order gradient of all dimensions of sample x, reflecting the degree of dispersion of the gradient distribution; n is the total dimension of the sample pixels (3145728). The mean of the first-order gradients across all dimensions; Let be the first-order gradient of the i-th pixel.

[0037] Magnitude of the second gradient: ; in: Let x be the magnitude of the second-order gradient in all dimensions of the sample x, which helps to quantize the steepness of the gradient change; Let be the second-order gradient of the i-th pixel.

[0038] Variance of the second gradient: ; in: The variance of the second-order gradient across all dimensions of sample x reflects the degree of dispersion of the second-order gradient distribution. The mean of the second-order gradients across all dimensions; n, The meaning is the same as the formula mentioned above.

[0039] Optimize the neural network loss function: Combine the cross-entropy loss function with the gradient loss function to improve the model's ability to capture adversarial features. The total loss function L is calculated as follows: ; α is the total loss function of the neural network, used to measure the deviation between the model's prediction and the true label; α and β are weight coefficients (α=0.6, β=0.4), which adjust the proportion of cross-entropy loss and gradient loss, respectively. For classifying losses, This is gradient feature loss.

[0040] Wherein, α and β are weight coefficients, with α taking a value of 0.6 and β taking a value of 0.4 (determined through validation set debugging to ensure that the model balances classification accuracy and adversarial feature capture accuracy). Cross-entropy loss function Calculation formula: ; Where y k For the true labels of the samples (original samples y1=1, y2=0; fake samples y1=0, y2=1), p k Predict the probability that a sample belongs to the k-th class for the model; k represents the sample category (k=1 represents original samples, k=2 represents fake samples); y k The label value (1 indicates belonging to this class, 0 indicates not belonging); p k The probability that a sample belongs to the k-th class is predicted by the model, and the value range is [0, 1].

[0041] Gradient loss function Calculation formula: ; Let n be the mean of the first-order gradients of all original samples in the i-th dimension, serving as the baseline value for the gradient features; n is the total dimension of the pixels. Let be the first-order gradient of the i-th pixel of the current sample. This formula is used to reduce the gradient difference between fake samples and original samples, thereby improving the detection accuracy of the model.

[0042] Model training: The constructed training set S train Input the optimized neural network and set the training parameters as follows: batch size is 32, learning rate is 0.001, Adam optimizer is used, training epochs are 50, and the validation set Sval is used to validate the model after each training epoch. When the accuracy of the validation set does not improve for 5 consecutive epochs, training is stopped to avoid model overfitting. Model validation and optimization: The trained model was validated using the test set Snest, and the model's detection accuracy, recall, and F1 score were calculated. The specific validation results are shown in Table 1. Table 1 Verification Results

[0043] If the model performance does not meet the preset requirements (accuracy ≥ 98%, recall ≥ 97%), adjust the gradient perturbation step size Δ, weight coefficients α and β, and retrain until the performance requirements are met, finally obtaining the optimized NFT adversarial example detection model M.

[0044] The system also includes: A deep feature extraction component is configured to extract deep feature fingerprints from the original NFT and the NFT to be verified in the original NFT training dataset, respectively, to obtain original feature hash values ​​and target feature hash values; the deep feature fingerprint extraction includes extracting fused features of pixel-level edge features, semantic texture features and hidden layer feature vectors.

[0045] Specifically, in this embodiment, the deep feature extraction component extracts the deep feature fingerprint of the NFT from multiple dimensions to generate a verifiable feature identifier.

[0046] Furthermore, in some embodiments, the deep feature extraction component includes: A pixel structure feature extraction module is configured to use an improved Gaussian-Laplacian operator to extract features from the original NFT to obtain the pixel-level edge features.

[0047] Specifically, in this embodiment, for the original NFT sample x, the pixel structure feature extraction module uses the improved Gaussian-Laplacian (LoG) operator to extract pixel-level edge features. The convolution kernel size of the LoG operator is set to 5×5, and the standard deviation σ=1.0. The calculation formula is as follows: ; σ is the calculated value of the Gaussian-Laplacian operator at pixel coordinates (x, y), used to extract NFT pixel edge features; (x, y) is the pixel coordinate of the sample; σ is the standard deviation (value 1.0), used to adjust the smoothness of the operator; π is pi (value 3.1416); e is the natural constant (value 2.7183).

[0048] The deep feature extraction component also includes: A semantic texture feature extraction module is configured to use the SIFT algorithm to extract features from the original NFT to obtain the semantic texture features.

[0049] Specifically, in this embodiment, the semantic texture feature extraction module uses the SIFT (Scale Invariant Feature Transform) algorithm to extract the semantic texture features of the NFT. The specific steps are as follows: First, a Gaussian difference pyramid is constructed (6 layers, each with a scaling factor of 1.2) to detect feature points; then, the histogram of the directional gradient of each feature point is calculated (12 bins, angle range 0°~360°), generating a 128-dimensional texture feature vector V. tex (Dimensions are 128×1).

[0050] The deep feature extraction component also includes: The hidden layer feature extraction module is configured to input the original NFT into the adversarial detection neural network, extract the output of the penultimate layer of the model, and obtain the hidden layer feature vector.

[0051] Specifically, in this embodiment, the original NFT sample x is input into the trained adversarial detection model M through the hidden layer feature extraction module, and the output of the penultimate layer of the model is extracted as the hidden layer feature vector V. hid (256×1 in dimension) This vector contains hidden layer distortion features and model watermark traces that are difficult to imitate during AI forgery, and is the core feature that distinguishes original and forged NFTs.

[0052] The deep feature extraction component also includes: The feature fusion and hash generation module is configured to perform weighted feature fusion based on the pixel-level edge features, the semantic texture features, and the hidden layer feature vector, and to encrypt the fused features using the SHA-256 hash algorithm to generate the original feature hash value.

[0053] Specifically, in this embodiment, the feature fusion and hash generation module converts the feature vector V of the above three dimensions into a single vector. pix V tex V hid The fusion is performed using a weighted fusion algorithm, and the fusion formula is as follows: ; Wherein, ω1, ω2, and ω3 are feature weights, determined through validation set debugging, with ω1=0.2, ω2=0.3, and ω3=0.5 (hidden layer feature weights are the highest to ensure that core features are not lost); after fusion, a deep feature vector V with a dimension of 434×1 is obtained; The SHA-256 hash algorithm is used to encrypt V, generating a unique characteristic hash value H. The hash value is a 64-bit hexadecimal string, and the calculation formula is as follows: ; Where H is the characteristic hash value (64-bit hexadecimal string) of the original NFT, serving as a unique identifier; SHA256 is the hash encryption algorithm. The feature vector is a multi-dimensional fusion result. This algorithm transforms the feature vector into an immutable hash value.

[0054] Extract x in the same way check Pixel structure feature vector V pixcheck Semantic texture feature vector V texcheck Hidden layer feature vector Vhidcheck A weighted fusion algorithm is used to generate a fusion feature vector V for fusion check, and then the SHA-256 algorithm is used to generate the feature hash value H of the NFT to be verified. check This ensures that the feature extraction process is completely consistent with the feature extraction process of the original NFT, guaranteeing the fairness and accuracy of the comparison.

[0055] The system also includes: A blockchain evidence storage component is configured to solidify the core information of the original NFT on the blockchain based on the original feature hash value, thereby obtaining an original NFT ownership benchmark library.

[0056] Specifically, in this embodiment, the blockchain evidence storage component is used to solidify the original NFT's feature hash value, ownership information, timestamp, and other data on the blockchain, thereby constructing an original NFT ownership benchmark library.

[0057] Furthermore, in some embodiments, the blockchain evidence storage component includes: The information encryption module is configured to obtain the associated information of the original NFT, standardize the associated information and the original feature hash value, and encrypt the sensitive information in the associated information using the AES-256 encryption algorithm to obtain encrypted sensitive information.

[0058] Specifically, in this embodiment, the core information of the original NFT is collected through the information encryption module, including: feature hash value H, creator information (name / account, ID number / blockchain address), creation timestamp T (accurate to milliseconds, using UTC time format), NFT metadata (format, resolution, creation description), and ownership proof documents (electronic signature, screenshots of the creation process).

[0059] The collected related information is standardized to remove redundant data, and the creator's privacy information (such as ID number) is encrypted using the AES-256 encryption algorithm to generate the encrypted privacy information Epri.

[0060] The blockchain evidence storage component also includes: The blockchain evidence storage and on-chain module is configured to package data based on the encrypted sensitive information, the associated information, the timestamp, and the original feature hash value, and then use a smart contract to upload the data to the blockchain to generate a unique evidence storage number.

[0061] Specifically, in this embodiment, the blockchain evidence storage module stores the feature hash value H and the encrypted privacy information E. priThe timestamp T, NFT metadata, and ownership certificate are packaged into a data package D. The data package D is uploaded to the blockchain using a smart contract (based on the Ethereum ERC-721 standard). During the upload process, the blockchain nodes perform consensus verification on the data package (using the PoS consensus mechanism with 101 consensus nodes to ensure consensus efficiency and security). After successful verification, the data package D is stored in the distributed nodes of the blockchain, generating a unique evidence number C (a 32-bit hexadecimal string).

[0062] The blockchain evidence storage component also includes: The ownership benchmark library construction module is configured to construct the original NFT ownership benchmark library, store the associated information, the timestamp, the unique certificate number, the original feature hash value, and the on-chain node information into the original NFT ownership benchmark library, and establish a hash index.

[0063] Specifically, in this embodiment, an original NFT ownership benchmark library K is established through the ownership benchmark library construction module. The library stores the certificate number C, feature hash value H, timestamp T, NFT metadata and on-chain node information of each original NFT. The benchmark library adopts a distributed storage method and is synchronized with the blockchain node in real time to ensure data consistency. At the same time, a hash index is established to facilitate subsequent fast query and comparison. The index rule is: the first 8 bits of the feature hash value H are used as the index key to achieve O(1) level query efficiency.

[0064] The system also includes: A forgery detection component is configured to perform forgery detection and fine-grained feature matching on the target feature hash value using the adversarial detection neural network and based on the original NFT ownership benchmark library to obtain a detection result; the detection result is used to characterize the infringement status of the NFT to be verified.

[0065] Specifically, in this embodiment, the forgery detection component calls the trained adversarial detection model to perform fine-grained feature matching and anomaly identification on the NFT to be verified.

[0066] Furthermore, in some embodiments, the forgery detection component includes: An adversarial sample detection module is configured to input the preprocessed NFT to be verified into the adversarial detection neural network to obtain the probability of forged samples.

[0067] Specifically, in this embodiment, the adversarial example detection module processes the preprocessed sample x to be verified. checkInput the trained adversarial detection model M, and the model outputs the predicted probability p (p is the probability that the sample to be verified belongs to the fake sample, and the value range is [0, 1]). Set the detection threshold θ=0.8 (determined by debugging on the test set, taking into account both accuracy and recall).

[0068] If p < θ (p < 0.8), then the NFT to be verified is initially determined to be an original or legally transferred sample, and fine-grained feature matching is performed; If p ≥ θ (p ≥ 0.8), the NFT to be verified is initially determined to be a forged sample, and the infringement type is distinguished.

[0069] The forgery detection component also includes: A fine-grained feature matching module is configured to calculate the NFT similarity by using a cosine similarity algorithm based on the original NFT ownership benchmark library and the target feature hash value when the probability of the forged sample is within a preset probability range.

[0070] Specifically, in this embodiment, the fine-grained feature matching module uses the feature hash value H of the NFT to be verified. check For indexing, query the original NFT ownership benchmark library K, and calculate H using the cosine similarity algorithm. check The similarity S between the feature hash values ​​H of all original NFTs in the benchmark library is calculated using the following formula: ;

[0071] S represents the feature similarity between the NFT to be verified and the original NFT (value range [0, 1]). The closer S is to 1, the more similar the features are. The fused feature vector of the NFT to be verified; The fused feature vector of the original NFT; , These are the magnitudes of the two eigenvectors.

[0072] If there exists an original NFT with S≥Sth, and the metadata and ownership information of the NFT to be verified are consistent with the information of the corresponding original NFT in the benchmark library, it is determined to be a legitimate circulation sample and is allowed to be traded normally. If an original NFT with S≥Sth exists, but the metadata and ownership information of the NFT to be verified are inconsistent with the information of the corresponding original NFT in the benchmark library, it is determined to be a partially tampered sample (belonging to the infringement type). If no original NFT with S≥Sth exists, it is judged as a suspected forged sample and proceeds to further testing.

[0073] The forgery detection component also includes: The infringement type differentiation module is configured to determine the infringement type based on the gradient features of the NFT to be verified extracted by the adversarial detection neural network and the NFT similarity, and obtain the detection result.

[0074] Specifically, in this embodiment, the infringement type determination logic is constructed by combining the gradient features extracted by the adversarial detection model (Norm1, Var1, Norm2, Var2 mentioned above) and the fine-grained feature matching results, as follows: Calculate the difference D between the gradient features of the sample to be verified and the original sample. diff The calculation formula is as follows: ;

[0075] in, The gradient feature difference between the sample to be verified and the original sample (value range [0, 1]) is the degree of difference. The greater the difference, the higher the degree of forgery. , , , The gradient characteristic index of the sample to be verified; , , , The mean of the gradient features of all original samples; the denominator is the mean used for normalization, and 1 / 4 is used to average the difference percentage of the four gradient indicators.

[0076] D diff The value range of D is [0, 1]. diff The closer it is to 1, the greater the difference in gradient features and the higher the degree of forgery. Combining D diff Based on feature similarity S, three types of infringement are distinguished: AI complete replica sample: D diff S∈[0.6, 0.8], S∈[0.9, 0.95], meaning that the gradient features are significantly different from the original samples and have a high feature similarity, indicating that the AI ​​completely imitated the original NFT generation without any obvious tampering; Pixel-level tampering sample: D diff S∈[0.3, 0.6], S∈[0.8, 0.9], which means that the gradient features are somewhat different from the original samples, and the feature similarity is moderate, indicating that local pixel modifications were made to the original NFT; Secondary altered sample: D diff ∈[0.8, 1.0], S∈[0.5, 0.8], meaning that the gradient features are very different from the original samples and the feature similarity is low, indicating that it was generated by splicing, scaling and other secondary editing based on the original NFT.

[0077] The system also includes: The source tracing and location component is configured to locate the forgery source, tampering node, and propagation path of the NFT to be verified based on the metadata, transaction records, and detection results of the NFT to be verified, and generate a source tracing report.

[0078] Specifically, in this embodiment, the source tracing and positioning component locates the source of forgery, tampering nodes, and propagation path based on blockchain evidence information and detection results.

[0079] Furthermore, in some embodiments, the source tracing and location component includes: The on-chain traceability module is configured to perform node queries using a path tracing algorithm based on the metadata and transaction records of the NFT to be verified, and generate on-chain traceability results.

[0080] Specifically, in this embodiment, the on-chain traceability module queries the circulation node information on the blockchain through the metadata and transaction records of the NFT to be verified. A path tracing algorithm is used to trace the timestamp, transaction party addresses, and evidence storage information of each circulation node. The specific algorithm is as follows: Let the sequence of on-chain nodes of the NFT to be verified be N = [N1, N2, ..., N]. K ], where N1 is the first spoofed node to appear, N k For each currently connected node, N is a node. i Includes node address A i Timestamp T i Htrade transaction hash i ; Calculate N for each node i The correlation R with the original NFT proof node N0 (the on-chain node corresponding to the original NFT in the benchmark library) i The calculation formula is as follows: ; in, The correlation between the i-th circulation node and the original evidence storage node (value range [0, 1]) is the degree of correlation. The smaller the correlation, the more likely it is to be a suspicious node. The timestamp of the i-th node; Timestamps for original NFTs on the blockchain; For the NFT access timestamp to be verified; Let be the similarity between the address of the i-th node and the address of the original node; =The maximum similarity between all node addresses and the original node address; 1 / 2 is used to normalize the correlation to the interval [0, 1].

[0081] The source tracing and location component also includes: The off-chain tracing module is configured to acquire the trajectory of counterfeit NFTs propagated off-chain based on terminal devices connected to the Web3 ecosystem, and generate off-chain tracing results.

[0082] Specifically, in this embodiment, the off-chain tracing module obtains the propagation trajectory of counterfeit NFTs (such as those spread through social media or shared in local storage) by connecting to terminal devices in the Web3 ecosystem (such as mobile phones, computers, and NFT wallets). This includes the propagation time, propagation channels, and recipient information. Distributed log collection technology is used to encrypt the off-chain propagation information and then upload it to the blockchain for supplementary evidence storage, forming a complete propagation path.

[0083] The source tracing and location component also includes: The traceability result processing module is configured to integrate information based on the on-chain traceability results and the off-chain traceability results to obtain the traceability report.

[0084] Specifically, in this embodiment, the traceability result processing module integrates the results of on-chain and off-chain traceability to generate a traceability report, which clearly marks the source of the forgery (node ​​address, device information), the tampered node (tampering time, tampered content), and the full on-chain and off-chain propagation path (time, address, and propagation channel of each node), providing a basis for pursuing liability for infringement.

[0085] The system also includes: The intelligent judgment and evidence collection component is configured to generate evidence storage credentials for the NFT to be verified based on the source tracing report and the detection results, and to perform restriction processing on the NFT to be verified.

[0086] Specifically, in this embodiment, an automated judgment logic is established through the intelligent judgment and evidence collection component to generate legally valid evidence.

[0087] Furthermore, in some embodiments, the intelligent judgment and evidence collection component includes: An originality determination module is configured to determine originality based on the detection results and the source tracing report, and generate an originality determination result; the originality determination result is used to characterize whether the NFT to be verified is an infringing or counterfeit sample.

[0088] Specifically, in this embodiment, the originality determination module establishes originality determination rules based on the detection results and source tracing results, as follows: Original samples: p < 0.8, and there exist original NFTs with S ≥ 0.95, and the ownership information is consistent; Legitimate circulation samples: p<0.8, there exists an original NFT with S≥0.95, consistent ownership information, and all circulation nodes are legitimate nodes; Infringement and counterfeiting samples: p≥0.8, or S<0.95 and have a corresponding original NFT, and the source of counterfeiting and the tampering node have been located; The system automatically outputs the originality assessment result based on the above rules, without the need for manual intervention.

[0089] The intelligent judgment and evidence collection component also includes: The infringement evidence collection module is configured to collect evidence from the NFT to be verified and obtain an evidence collection number when the originality determination result indicates that the NFT to be verified is an infringing and counterfeit sample.

[0090] Specifically, in this embodiment, for samples determined to be infringing or counterfeit, the infringement evidence collection module automatically collects evidence materials, including: the original file of the NFT to be verified, the preprocessed sample, the feature hash value, the detection result (predicted probability p, gradient feature index), the feature matching report, the source tracing report, the on-chain evidence storage record, and the off-chain propagation log. All evidence materials are stored in an encrypted manner and simultaneously stored on the blockchain, generating an evidence collection number E.

[0091] The intelligent judgment and evidence collection component also includes: The evidence storage certificate generation module is configured to generate a traceability evidence storage certificate based on the blockchain evidence storage information and evidence collection materials corresponding to the NFT to be verified; the traceability evidence storage certificate includes the evidence collection number.

[0092] Specifically, in this embodiment, the evidence generation module generates a legally valid traceability evidence certificate based on blockchain evidence information and evidence materials. The certificate includes: evidence number C, evidence number E, NFT information to be verified, detection result, traceability result, evidence material list, timestamp, and blockchain node signature. The certificate is in PDF format and includes a SHA-256 hash value. It can be used in scenarios such as judicial litigation and infringement mediation to ensure the credibility of the evidence.

[0093] The intelligent judgment and evidence collection component also includes: The closed-loop management module is configured to feed back the original NFT creator and the Web3 ecosystem platform based on the traceability certificate and the originality determination result.

[0094] Specifically, in this embodiment, the closed-loop management module synchronously feeds back the judgment results, traceability reports, and evidence certificates to the original NFT creators and the Web3 ecosystem platform. Based on the judgment results, the platform removes infringing and counterfeit NFTs and restricts their transactions. Creators can use the evidence certificates to pursue legal action against infringement, thus realizing a fully intelligent closed loop for the entire process of NFT creation and evidence collection, from creation and evidence collection to real-time monitoring, counterfeit identification, path tracing, and infringement evidence collection.

[0095] This embodiment has the following advantages: By optimizing the lightweight adversarial detection neural network and combining it with the gradient feature extraction algorithm, the subtle distortion features and model watermarks of AI-forged NFTs are accurately captured, with a detection accuracy of over 98%, completely solving the defect that traditional shallow feature comparison is easily circumvented by AI. Deep feature fingerprints are extracted from multiple dimensions and combined with SHA-256 hash algorithm and blockchain notarization to ensure the immutability of original NFT ownership information. The constructed ownership benchmark library enables fast query and comparison. By using fine-grained feature matching and gradient difference analysis, different types of infringement can be accurately distinguished. Combined with on-chain and off-chain tracing algorithms, the precise location of the forgery source, tampered nodes, and the entire propagation path can be achieved.

[0096] By establishing an automated logic for determining originality and generating legally valid evidence certificates, the system enables intelligent closed-loop management of the entire NFT process, significantly improving the efficiency of digital collectible ownership protection and infringement tracing. It provides underlying technical support for the compliance of the NFT ecosystem and the legalization of copyright confirmation, and has extremely high practical and industrial value.

[0097] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the discussion in some embodiments is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the contents of this disclosure, thereby enabling those skilled in the art to better utilize the embodiments.

Claims

1. A deepfake NFT tracing system based on AI adversarial example detection, characterized in that, The system includes: A data acquisition and model training component is configured to acquire an NFT to be verified, an original NFT training dataset, and train an adversarial detection neural network using the original NFT training dataset; the original NFT training dataset includes several original NFTs. A deep feature extraction component is configured to extract deep feature fingerprints from the original NFT and the NFT to be verified in the original NFT training dataset, respectively, to obtain original feature hash values ​​and target feature hash values; the deep feature fingerprint extraction includes extracting fused features of pixel-level edge features, semantic texture features and hidden layer feature vectors; A blockchain evidence storage component is configured to solidify the core information of the original NFT on the blockchain based on the original feature hash value, thereby obtaining an original NFT ownership benchmark library. A forgery detection component is configured to perform forgery detection and fine-grained feature matching on the target feature hash value using the adversarial detection neural network and based on the original NFT ownership benchmark library to obtain a detection result; the detection result is used to characterize the infringement status of the NFT to be verified. The source tracing and location component is configured to locate the forgery source, tampering node, and propagation path of the NFT to be verified based on the metadata, transaction records, and detection results of the NFT to be verified, and generate a source tracing report. The intelligent judgment and evidence collection component is configured to generate evidence storage credentials for the NFT to be verified based on the source tracing report and the detection results, and to perform restriction processing on the NFT to be verified.

2. The NFT deepfake tracing system based on AI adversarial example detection according to claim 1, characterized in that, The data acquisition and model training components include: A data acquisition module, configured to acquire the NFT to be verified and the original NFT training dataset; The data preprocessing module is configured to generate a positive sample set and a negative sample set based on the original NFT training dataset; perform unified preprocessing on the positive sample set, the negative sample set, and the NFT to be verified to obtain a preprocessed positive sample set, a preprocessed negative sample set, and a preprocessed NFT to be verified; and divide the preprocessed positive sample set and the preprocessed negative sample set into a training set, a validation set, and a test set. A model training module is configured to train the adversarial detection neural network using the training set, and to test and validate the adversarial detection neural network using the test set and the validation set.

3. The NFT deepfake tracing system based on AI adversarial example detection according to claim 2, characterized in that, The model training module includes: A model adjustment unit is configured to acquire the original adversarial detection neural network and adjust the settings of the original adversarial detection neural network according to the NFT adversarial feature extraction algorithm to obtain the adversarial detection neural network. A loss function adjustment unit is configured to adjust the loss function of the adversarial detection neural network by combining a cross-entropy loss function and a gradient loss function. The model training unit is configured to train the adversarial detection neural network using the training set, and to validate the adversarial detection neural network using the validation set after each round of model training until a preset training condition is met; after stopping model training, the adversarial detection neural network is tested using the test set, and a qualified adversarial detection neural network is output.

4. The NFT deepfake tracing system based on AI adversarial example detection according to claim 2, characterized in that, The deep feature extraction component includes: A pixel structure feature extraction module is configured to use an improved Gaussian-Laplacian operator to extract features from the original NFT to obtain the pixel-level edge features; A semantic texture feature extraction module is configured to use the SIFT algorithm to extract features from the original NFT to obtain the semantic texture features. A hidden layer feature extraction module is configured to input the original NFT into the adversarial detection neural network, extract the output of the penultimate layer of the model, and obtain the hidden layer feature vector. The feature fusion and hash generation module is configured to perform weighted feature fusion based on the pixel-level edge features, the semantic texture features, and the hidden layer feature vector, and to encrypt the fused features using the SHA-256 hash algorithm to generate the original feature hash value.

5. The NFT deepfake tracing system based on AI adversarial example detection according to claim 1, characterized in that, The blockchain-based evidence storage component includes: The information encryption module is configured to obtain the associated information of the original NFT, standardize the associated information and the original feature hash value, and encrypt the sensitive information in the associated information using the AES-256 encryption algorithm to obtain encrypted sensitive information. The blockchain evidence storage and on-chain module is configured to package data based on the encrypted sensitive information, the associated information, the timestamp, and the original feature hash value, and use a smart contract to upload the data to the blockchain to generate a unique evidence storage number. The ownership benchmark library construction module is configured to construct the original NFT ownership benchmark library, store the associated information, the timestamp, the unique certificate number, the original feature hash value, and the on-chain node information into the original NFT ownership benchmark library, and establish a hash index.

6. The NFT deepfake tracing system based on AI adversarial example detection according to claim 1, characterized in that, The forgery detection component includes: An adversarial sample detection module is configured to input the preprocessed NFT to be verified into the adversarial detection neural network to obtain the probability of forged samples. A fine-grained feature matching module is configured to calculate the NFT similarity by using a cosine similarity algorithm based on the original NFT ownership benchmark library and the target feature hash value when the probability of the forged sample is within a preset probability range. The infringement type differentiation module is configured to determine the infringement type based on the gradient features of the NFT to be verified extracted by the adversarial detection neural network and the NFT similarity, and obtain the detection result.

7. The NFT deepfake tracing system based on AI adversarial example detection according to claim 1, characterized in that, The source tracing and location component includes: The on-chain traceability module is configured to perform node queries using a path tracing algorithm based on the metadata and transaction records of the NFT to be verified, and generate on-chain traceability results. The off-chain tracing module is configured to acquire the trajectory of counterfeit NFTs propagated off-chain based on terminal devices connected to the Web3 ecosystem and generate off-chain tracing results. The traceability result processing module is configured to integrate information based on the on-chain traceability results and the off-chain traceability results to obtain the traceability report.

8. The NFT deepfake tracing system based on AI adversarial example detection according to claim 1, characterized in that, The intelligent judgment and evidence collection component includes: An originality determination module is configured to determine originality based on the detection results and the source tracing report, and generate an originality determination result; the originality determination result is used to characterize whether the NFT to be verified is an infringing or counterfeit sample; The infringement evidence collection module is configured to collect evidence from the NFT to be verified and obtain an evidence collection number when the originality determination result indicates that the NFT to be verified is an infringing and counterfeit sample. The evidence storage certificate generation module is configured to generate a traceability evidence storage certificate based on the blockchain evidence storage information and evidence collection materials corresponding to the NFT to be verified; the traceability evidence storage certificate includes the evidence collection number. The closed-loop management module is configured to feed back the original NFT creator and the Web3 ecosystem platform based on the traceability certificate and the originality determination result.

9. The NFT deepfake tracing system based on AI adversarial example detection according to claim 6, characterized in that, If the probability of a forged sample is greater than the upper limit of the preset probability range, the NFT to be verified will be determined as a forged sample, and an infringement type determination will be made.

10. The NFT deepfake tracing system based on AI adversarial example detection according to claim 8, characterized in that, The traceability and evidence preservation certificate also includes the evidence preservation number, the evidence collection number, the NFT information to be verified, the test result, the traceability report, the evidence collection material list, the timestamp, and the blockchain node signature.

Images