A model copyright protection and backdoor defense method based on a double passport mechanism

Abstract

This invention belongs to the field of computer science and technology, and discloses a model copyright protection and backdoor defense method based on a dual-passport mechanism. The invention comprises three core components: a data passport embedding module, a model structure passport embedding module, and a dual-passport joint authentication module. The data passport embedding module utilizes reverse backdoor technology to design triggers, optimizes data preprocessing operations, and performs exclusion training on public data to ensure that model performance is bound to specially processed data. The model structure passport embedding module proposes two innovative methods: based on neuron response and based on linear combination. The neuron response-based method deeply binds the protection mechanism to the core functions of the model, exhibiting strong anti-pruning capabilities. The dual-passport joint authentication module combines the above two methods, realizing the linkage between input specified data and the model's internal response. This effectively prevents fine-tuning, pruning, and backdoor attacks while ensuring normal model function.

Description

Technical Field

[0001] This invention belongs to, but is not limited to, the field of computer science and technology, and particularly relates to a method and system for model copyright protection and backdoor defense based on a dual-passport mechanism. Background Technology

[0002] In recent years, computer vision and deep learning technologies have developed rapidly and have been widely applied to various visual recognition and image processing tasks. With this technological evolution, to effectively alleviate the huge resource consumption and data barriers caused by training from scratch, "pre-training-fine-tuning" has become the mainstream paradigm for building high-performance deep vision models.

[0003] Specifically, pre-trained models refer to large-scale basic neural networks pre-trained on large-scale unlabeled or weakly labeled datasets using methods such as self-supervised learning or weakly supervised learning. The core technology lies in driving deep architectures (such as deep convolutional neural networks (CNNs) and visual Transformers) to perform long-term feature learning, enabling the model to autonomously learn from low-level visual features such as edge textures and local geometry to high-level semantics of complex objects, thus providing high-quality initial parameters for various downstream tasks. Currently, mainstream pre-training methods are mostly based on contrastive learning and mask prediction mechanisms. By constructing cross-view feature space similarity constraints and reconstructing features of randomly occluded regions, they greatly improve the model's ability to extract and generalize from visual scenes. Because training this model highly depends on massive datasets and extremely high computational costs, and possesses excellent feature generalization capabilities, pre-trained models have become core digital assets with high technological barriers and are widely shared and open-sourced within the industry.

[0004] Secondly, fine-tuning refers to a parameter update technique where end users directly download open-source pre-trained models and use a small amount of labeled data from a specific downstream task to perform secondary optimization of the model weights. Its main strategies include traditional full-parameter fine-tuning and partial fine-tuning, as well as the emerging cue-based fine-tuning. Cue-based fine-tuning, as a parameter-efficient fine-tuning (PEFT) method, adjusts the model's contextual understanding by optimizing trainable soft cues, enabling task specification without modifying the pre-trained model weights. Its advantages of low overhead and low computational power have made it the mainstream choice.

[0005] The "fine-tuning pre-trained model" development paradigm, which combines the two technologies mentioned above, significantly lowers the development and computational barriers to building deep learning systems. However, this supply chain model, which heavily relies on third-party open-source models, also harbors many security risks: on the one hand, the high open-source nature and structural transparency of pre-trained models make them extremely easy for unauthorized parties to obtain and illegally copy; on the other hand, the fine-tuning process itself is vulnerable, especially in fine-tuning modes such as PEFT, where malicious parameter tampering or overwriting can easily hide within normal weight fluctuations because only a small portion of the parameters need to be updated. This dual vulnerability of open source and fragility directly leads to two major challenges currently facing the field of deep vision that urgently need to be addressed: the model copyright protection crisis and the threat of backdoor attacks.

[0006] Deep neural networks, as the core model of deep learning, achieve high-accuracy prediction and high-precision recognition through their multi-layered neuron structure and powerful feature extraction capabilities. However, training a high-performance neural network is extremely costly, requiring not only massive amounts of data and computational resources but also expert model design and parameter tuning. Therefore, it possesses significant commercial value and must be strictly protected under intellectual property rights. Any unauthorized copying, theft, or alteration constitutes an infringement of intellectual property rights. Faced with such model security threats, an effective technical solution is urgently needed to address the copyright protection issues of deep learning models.

[0007] In recent years, two types of methods for solving copyright protection have been proposed: (1) Digital watermarking: a technology that embeds identification information into multimedia files (such as audio, images and videos) without affecting the original value and use of the multimedia. According to the perceptibility of the watermark, it is divided into visible watermarks and invisible watermarks. Invisibility means that it is not easily perceived by the human eye after the watermark is embedded, and it also means that the image is basically undamaged after the watermark is embedded. (2) Model watermarking: inspired by multimedia watermarking, Uchida et al. first proposed the concept of neural network model watermarking, that is, embedding watermark information into the model features, so that the ownership of the model can be proved by the watermark after the model is illegally copied.

[0008] In practical applications, copyright protection for fine-tuning pre-trained models faces several severe challenges, including parameter coverage, model pruning, structural reconstruction, and data augmentation (such as random pruning, rotation, and noise injection). These operations can easily lead to the invalidation or precise erasure of copyright information. To address these issues, existing solutions mainly cover white-box weight constraints, black-box trigger set verification, and structured defense based on passport layers. White-box solutions introduce regularization terms (such as...) into the loss function... The existing copyright protection technologies employ various methods. First, they forcibly anchor copyright fingerprints to a specific weight space. Second, black-box solutions utilize backdoor attacks, training the model to generate pre-defined abnormal responses to specific trigger samples to establish ownership. Third, passport-layer solutions embed a conditional normalization layer dependent on a private key into the network, achieving logical locking of the inference path. These methods enable basic ownership tracing and verification in scenarios with static or locally fine-tuned parameters. However, these traditional methods still have limitations when facing deep parameter fine-tuning and model cleaning: first, shallow weight fingerprints or black-box trigger features are easily erased by routine operations such as full parameter fine-tuning or model pruning; second, some methods forcibly alter the original network structure, significantly reducing inference performance; and most importantly, most of these methods only passively verify after the model has been stolen, failing to proactively intercept unauthorized access. Therefore, there is an urgent need for a copyright protection mechanism that balances concealment and defense, enabling proactive interception of unauthorized access without altering the original structure or deeply binding core model features.

[0009] In deep learning, backdoor attacks are a highly covert method of model destruction. Attackers implant specific triggers during the model training or fine-tuning stage, so that the model performs normally when faced with clean, benign samples, but when it encounters data with specified triggers, the model will force the output of the wrong label specified by the attacker. The current mainstream backdoor attack methods are mainly divided into: data poisoning attacks, non-poisoning tampering (model-based backdoor attacks), and optimization-based generative attacks. (1) Data poisoning attacks (data-based backdoor attacks). This type of attack mainly acts on the training stage of the model. By covertly superimposing static trigger patterns in clean data, the attacker forces the model to passively and deeply fit the correlation mapping between the trigger and the preset output in the gradient update iteration of backpropagation. At the feature embedding level, the implantation process of this static trigger is usually expressed by the following formula:

[0010]

[0011] in, Indicates the original clean sample. This represents the mask vector that determines the trigger position and coverage strength. This indicates the trigger pattern constructed by the attacker. This indicates the final generated sample with trigger contamination. This represents the Hadama product.

[0012] (2) Non-poisoning tampering (model-based backdoor attack). This type of attack does not require modification of training data or time to retrain the model. After gaining access to the model's internal structure, the attacker directly modifies the parameter values ​​within the model or forcibly inserts malicious structural modules into the network, thereby directly writing the triggering logic into the model. The core principle of this attack is to directly apply specific numerical modifications to the original parameters while ensuring that the model's normal prediction accuracy is almost unaffected. Its formulaic expression is:

[0013]

[0014] in, These are the original model parameters. Malicious weight perturbations directly injected by attackers. For the final infected model parameters, This represents the tolerance threshold for performance degradation in the original task. This process achieves high-frequency activation of triggers by specific neurons without requiring iterations through massive amounts of data.

[0015] (3) Optimization-based generative attacks. To address the problem of traditional fixed-shape triggers being easily detected and intercepted, this type of attack employs a dynamic computation method. Here, the trigger is no longer a fixed pattern, but rather a tiny, imperceptible modification is calculated and generated specifically for each different input data. The generation process of this trigger essentially constructs an optimal solution that is both misleading and highly stealthy, and its formulaic expression is:

[0016]

[0017] in, For the target model, To prompt the model to output preset attack labels The backdoor loss function, To constrain the invisibility of triggers Norm penalty term, To balance attack success rate and stealth, a hyperparameter is used. The trigger generated by this method not only has extremely high visual stealth but also can be deeply coupled with the deep semantics of the input sample.

[0018] Based on the above analysis, the urgent technical problems that need to be solved in the existing technology are:

[0019] (1) Difficult to defend against backdoor attacks. Traditional model passport schemes are prone to altering the original structure of the model, such as replacing batch normalization with group normalization. Attackers can easily find the embedding location of the passport layer and implement backdoors or erasure attacks. Existing independent branch passport schemes, in pursuit of ease of use, allow users to use the model normally without keys, lacking authentication of end users, which directly gives malicious attackers an opportunity to exploit.

[0020] (2) Passive verification mechanism. Traditional digital watermarking technology and existing static passport methods are mostly limited to passive verification after an infringement event occurs. They cannot actively intercept unauthorized parties from illegally calling the model, making it difficult to effectively curb the illegal copying and misuse of the model.

[0021] (3) Insufficient robustness under high-intensity attacks. It is worth noting that although existing passport methods effectively prevent conventional fine-tuning and pruning through learnable mechanisms, they mostly adopt simple static verification logic such as linear combination. When faced with high-intensity model compression, the weights may shift or even be lost, resulting in the destruction or erasure of copyright features. Summary of the Invention

[0022] To address the problems existing in the prior art, this invention provides a model copyright protection and backdoor defense method based on a dual-passport mechanism.

[0023] This invention is implemented as follows: a model copyright protection and backdoor defense method based on a dual-passport mechanism, comprising the following steps:

[0024] Data passport embedding steps: Use reverse backdoor technology to design triggers, inject hidden features into private training data, and configure differentiated rejection training loss for public data to ensure that the normal prediction performance of the model is only deeply bound to special samples carrying the correct data passport.

[0025] Model structure passport embedding steps: A non-structurally destructive single-layer hidden embedding strategy is adopted, and feature transformation parameters are dynamically generated through a mechanism based on neuron response or a mechanism based on linear combination. Among them, the mechanism based on neuron response strongly couples the copyright verification logic with the underlying physiological features of the model, namely the channel response intensity, so that the model has the ability to resist pruning and fine-tuning.

[0026] Dual-passport joint authentication steps: The rejection mechanism at the external data level is deeply integrated with the locking mechanism of the internal model structure to construct a multi-level nested feature transformation verification closed loop; the model decodes and outputs normally only when the specified authorized data is input and the correct structural key is loaded inside the model; if any condition is missing, the passport layer actively triggers abnormal perturbation to block illegal calls and backdoor activation.

[0027] Furthermore, the data passport embedding step includes:

[0028] Dataset Partitioning and Preprocessing: Given a private dataset for training, containing N input samples and their corresponding ground truth labels, where N is the number of samples; the sample index set 1 to N of the private dataset is divided into an edited index set and an unedited index set according to a preset edit ratio ρ between 0 and 1. The union of the edited index set and the unedited index set is 1 to N and the intersection is empty. The private dataset is further divided into an edited data subset and an unedited data subset. The edited data subset is defined as the sample pair formed by the input sample after trigger implantation operation for each index belonging to the edited index set and the ground truth label, used to maintain the normal prediction of the model under authorized conditions. The unedited data subset is defined as the sample pair formed by the input sample and the specified target category label for each index belonging to the unedited index set, used to make the model output abnormal results under unauthorized input to achieve rejection. The number of samples in the edited data subset is determined by the product of the preset edit ratio ρ and the total number of samples N, and the number of samples in the unedited data subset is determined by the product of (1−ρ) and the total number of samples N.

[0029] Trigger generation and implantation: An image transformation operation based on a reflection image model is adopted. This operation convolves the reflection image with a blurred convolution kernel to calculate the reflection layer, which is constrained by the depth field (DOF) of the same focal plane and the depth imaging physical optical properties of the reflection layer being out of focus. The reflection layer is then superimposed on each background sample in the edited data subset to generate a sample carrying data passport features. This sample is equal to the background sample plus the reflection layer.

[0030] Model training and rejection mechanism implementation: In the fine-tuning training phase of the pre-trained model, the optimization function of the data passport embedding step is defined as follows: the optimization function value is equal to one-half of the number of samples in the edited data subset multiplied by the sum of the loss functions on the edited data subset, plus the tradeoff coefficient multiplied by one-half of the number of samples in the unedited data subset multiplied by the sum of the loss functions on the unedited data subset; where the tradeoff coefficient is used to balance the influence of the two parts of the loss, and the samples used in the edited data subset are the samples generated after the trigger implantation operation.

[0031] Furthermore, the model structure passport embedding step includes a passport embedding method based on neuron response, a passport embedding method based on linear combination, and general model optimization and training.

[0032] Furthermore, the passport embedding method based on neuron response includes:

[0033] Feature signature extraction: The input feature map is fed into the target network layer. The dimension of the input feature map is the number of channels multiplied by the height multiplied by the width, resulting in the output feature map. The dimension of the output feature map is the number of channels multiplied by the output height multiplied by the output width. The average response intensity is calculated for each channel. For the c-th channel, its average response value is equal to 1 divided by the output height multiplied by the output width, and then multiplied by the absolute value of the neuron activation intensity at the corresponding coordinate in the output feature map obtained by summing the output height dimension and summing the output width dimension. The average response values ​​of all channels are used to form a response vector. The K channel indices with the highest response values ​​in the response vector are selected and stored as passport keys, which are then used in the forward propagation and parameter optimization of the model during subsequent training.

[0034] Passport layer parameter initialization: During the initialization phase of the passport layer, specific values ​​are assigned to the scaling factor and offset to keep the dimensions of the scaling factor and offset consistent with the number of channels in the input feature map; during the backpropagation process of model training, the scaling factor and offset are used as learnable variables and are corrected in iterative updates through the constraints of the optimization function.

[0035] Passport key update based on exponential moving average: A mask vector is generated based on the passport key index set. The dimension of the mask vector is equal to the number of channels. The value of the c-th channel follows this rule: if c belongs to the passport key index set, the c-th component of the mask vector is equal to 1; if c does not belong to the passport key index set, the c-th component of the mask vector is equal to 0. During the forward propagation phase of the model, scaling and offset processing are performed on the key channels using scaling factors and offsets. The output feature map of the l-th layer is generated by the following operation: the output feature map is equal to the scaling factor multiplied element-wise by the mask vector, then multiplied element-wise by the input feature map, plus the offset multiplied element-wise by the mask vector, plus the input feature map multiplied element-wise by the all-1 vector minus the mask vector, where all elements of the all-1 vector are 1. This operation is only performed on the index set k. Selected channels undergo scaling and offset transformations, while unselected channels retain their original features. When the passport key is correct, the passport layer performs this operation and outputs the correct prediction result. When the passport key is incorrect, the passport layer dynamically generates random scaling factors and random offsets that follow a standard normal distribution. The output feature map of the corresponding layer is equal to the random scaling factor multiplied element-wise by the mask vector, then multiplied element-wise by the input feature map, plus the random offset multiplied element-wise by the mask vector, plus the input feature map multiplied element-wise by a vector of all 1s minus the mask vector. In each iteration of the model, an exponential moving average mechanism is introduced to update the global average response vector. The updated global average response vector is equal to the moving average coefficient multiplied by the global average response vector before the update, plus 1 minus the moving average coefficient multiplied by the instantaneous response vector of the current batch, where the moving average coefficient takes a value between 0 and 1.

[0036] Furthermore, the passport embedding method based on linear combination includes:

[0037] Linear generation of dynamic parameters: During the model training phase, a passport key is introduced. The passport key has a dimension of K, which is a set dimension. The passport key serves as a core copyright certificate and is independently encrypted and stored locally by the owner, not disclosed with the model. The passport key is linearly combined with the first and second trainable weight matrices. The dimensions of the first and second trainable weight matrices are the number of channels multiplied by K. A scaling factor and an offset are dynamically generated. The scaling factor and the offset are both the number of channels. The scaling factor is equal to the first trainable weight matrix multiplied by the passport key plus a first bias term, and the offset is equal to the second trainable weight matrix multiplied by the passport key plus a second bias term.

[0038] Forward Propagation and Feature Map Transformation: During the forward propagation phase of the model, channel-level transformations are performed on the input feature map using dynamically generated scaling factors and offsets. The dimension of the input feature map is the number of channels multiplied by the height multiplied by the width, resulting in the output feature map. The calculation method is as follows: after broadcasting the scaling factors and offsets along the spatial height and width dimensions, the output feature map is equal to the scaling factor multiplied element-wise by the input feature map plus the offset. For the c-th channel of the feature map, its output is equal to the scaling factor of the c-th channel multiplied by the input feature map matrix of the c-th channel plus the offset of the c-th channel.

[0039] Abnormal perturbation mechanism: If the passport layer receives an incorrect passport key and the incorrect passport key is not equal to the correct passport key, the perturbation mechanism is triggered. The incorrect scaling factor and the incorrect offset are calculated by linear combination. The output feature map of the passport layer is equal to the incorrect scaling factor multiplied element by element by the input feature map plus the incorrect offset.

[0040] Furthermore, the general model optimization and training includes: during the model training or fine-tuning phase, constructing a joint optimization objective function for the data passport mechanism and the model structure passport mechanism; the joint optimization objective function includes a data passport classification loss term, a model structure passport verification loss term, and a data rejection constraint term; wherein, the data passport classification loss term is used to maintain the normal prediction performance of authorized data, the data rejection constraint term is used to constrain unauthorized data to produce preset prediction results, and the model structure passport verification loss term is used to establish the association between the model structure passport and the passport key; each loss term is jointly optimized through weighted hyperparameters, so that the model prediction performance is simultaneously constrained by both the data passport and the model structure passport, realizing a deep binding between model functionality and copyright information.

[0041] This invention also provides a model copyright protection and backdoor defense method based on a dual-passport mechanism, including:

[0042] Step 1: The data passport embedding module uses reverse backdoor technology to design triggers to inject hidden features into private training data and configure differentiated rejection training loss for public data to ensure that the normal prediction performance of the model is only deeply bound to special samples carrying the correct data passport.

[0043] Step 2: The model structure passport embedding module adopts a non-structurally destructive single-layer hidden embedding strategy, which dynamically generates feature transformation parameters through a mechanism based on neuron response or linear combination. Among them, the neuron response-based method strongly couples the copyright verification logic with the underlying physiological features (channel response intensity) of the model, making the model have extremely strong anti-pruning and anti-fine-tuning capabilities.

[0044] Step 3: The dual-passport joint authentication module deeply integrates the rejection mechanism at the external data level with the locking mechanism of the internal model structure to construct a multi-level nested feature transformation verification closed loop. The model can only decode and output normally when the specified authorized data is input and the correct structure key is loaded inside the model. If any condition is missing, the passport layer will actively trigger abnormal disturbances to block illegal calls and backdoor activation.

[0045] Furthermore, the data passport embedding module:

[0046] (1) Dataset partitioning and preprocessing

[0047] Given a private dataset for training It contains N input samples and the corresponding real tags To ensure the model learns a strong correlation between specified triggers and correct results, while also being able to output incorrect judgments when faced with samples without specified triggers (even clean samples), thus preventing unauthorized use of the model; according to a preset editing ratio. The dataset Sample index set Divided into edit index sets and unedited index set and satisfy the relationship and Therefore, the private dataset Divided into edit data subsets and unedited data subset (Exclusive subsets) Two subsets, both defined mathematically as:

[0048]

[0049]

[0050] in, To specify the trigger implantation operation, To specify the target category label; the sample sizes of both satisfy the following conditions: and ;

[0051] (2) Trigger generation and implantation

[0052] Image transformation operation based on reflection image model This operation involves reflecting the image. With fuzzy convolution kernel Perform convolution calculations and then stack them onto the edit subset. Each background sample Above, to generate samples carrying data passport features. The specific calculation formula is as follows:

[0053]

[0054] in, This represents the convolution operation. For reflection;

[0055] (3) Implementation of model training and rejection mechanism

[0056] During the fine-tuning training phase of the pre-trained model, to control access permissions to the model, the optimization function of the data passport embedding module is defined as follows:

[0057]

[0058] in, For model parameters, This is the prediction function of the model. For loss function, The is a tradeoff coefficient used to balance the effects of the two losses; in the formula This is a sample that has been previously implanted with a trigger. .

[0059] Furthermore, the model structure passport embedding module:

[0060] (1) Passport embedding based on neuron response;

[0061] (2) Passport embedding based on linear combination;

[0062] (3) General model optimization and training.

[0063] Furthermore, the passport embedding based on neuron response:

[0064] 1) Feature signature extraction

[0065] Input feature map The input is fed into the network layer corresponding to the target, where For the number of channels, and The height and width of the input feature map are used to obtain the output feature map. Calculate the average response intensity for each channel, with the first channel being the most significant. Taking one channel as an example, its average response value The calculation formula is as follows:

[0066]

[0067] in, and These represent the height and width of the output feature map, respectively. Coordinates in the feature map The activation intensity of a neuron can be understood as a channel in the formula. Total neuron response; average response values ​​from all channels are used to construct a vector. Select vector The highest response value Each channel index is represented as Selected channel index Stored as a passport key, and used in the forward propagation and parameter optimization of the model during subsequent training;

[0068] 2) Passport layer parameter initialization

[0069] During the initialization phase of the passport layer, the parameter scaling factor is... With offset Perform specific assignments while maintaining the dimensionality of the two parameters and the number of channels in the input feature map. Consistent. During the backpropagation process of model training, the scaling factor... With offset As a learnable variable, it is corrected in iterative updates through the constraints of the optimization function;

[0070] 3) Passport key update based on exponential moving average (EMA)

[0071] Based on passport key index set Generate mask vector In its first Specific values ​​on each channel Follow the formula below:

[0072]

[0073] Subsequently, during the model forward propagation phase, scaling factors are used. With offset Perform scaling and offset processing on the critical channel; Output feature map of the layer Generated by the following operations:

[0074]

[0075] in, This indicates an element-wise multiplication operation. This is a vector containing only 1s. When the passport key is correct, the passport layer performs this operation and outputs the correct prediction result; when the passport key is incorrect, the passport layer dynamically generates a random scaling factor. and offset The corresponding layer output is forcibly perturbed:

[0076]

[0077] Finally, in each iteration of the model, an exponential moving average mechanism is introduced to average the global response vector. Perform the following update operation:

[0078]

[0079] in, The moving average coefficient, This is the instantaneous response vector for the current batch.

[0080] Furthermore, the passport embedding based on linear combination:

[0081] 1) Linear generation of dynamic parameters

[0082] In the model training phase, a passport key is introduced. , The passport key dimension is defined. This is then compared with the trainable weight matrix. and Perform linear combination calculations to dynamically generate scaling factors. and offset This process can be expressed by the following formula:

[0083]

[0084]

[0085] in, and All are preset bias terms;

[0086] 2) Forward Propagation and Feature Map Transformation

[0087] During the forward propagation phase of the model, the dynamically generated parameters are used to analyze the input feature map. Perform channel-level transformation to obtain feature maps. This step can be represented by the following formula:

[0088]

[0089] in, This represents an element-wise multiplication operation along the channel dimension. These represent the number of channels, height, and width of the feature map, respectively; specifically, the feature map's... Each channel has an output. Satisfy the following operations:

[0090]

[0091] in, Represents the first... Channel matrix, Corresponding to the first Specific scaling factor and offset for each channel;

[0092] 3) Abnormal disturbance mechanism

[0093] If the passport layer receives an incorrect passport key (Right now If this occurs, a perturbation mechanism will be triggered, generating incorrect parameters during linear combination calculations. and The passport layer output will become:

[0094] .

[0095] Furthermore, the general model optimization and training:

[0096] During the model training and fine-tuning phase, a specific joint optimization function was configured for the single model passport mechanism. The formula is as follows:

[0097]

[0098] in, It is the main classification loss function, used to maintain the model's predictive inference ability; The loss function for the passport verification branch is used to validate the embedded passport model. The validity of the copyright; To balance the hyperparameters of the two loss weights.

[0099] Another objective of this invention is to provide a model copyright protection and backdoor defense system based on a dual-passport mechanism, comprising:

[0100] The data passport embedding module uses reverse backdoor technology to design triggers to inject hidden features into private training data and configure differentiated rejection training loss for public data, ensuring that the normal prediction performance of the model is only deeply bound to special samples carrying the correct data passport.

[0101] A non-structurally destructive single-layer hidden embedding strategy is employed, dynamically generating feature transformation parameters through mechanisms based on neuron responses or linear combinations. The neuron response-based method strongly couples the copyright verification logic with the model's underlying physiological features (channel response intensity), giving the model extremely strong resistance to pruning and fine-tuning.

[0102] The rejection mechanism at the external data layer is deeply integrated with the locking mechanism of the internal model structure to construct a multi-level nested feature transformation verification closed loop. The model can only decode and output normally when the specified authorized data is input and the correct structural key is loaded internally; if either condition is missing, the passport layer will actively trigger abnormal perturbation to block illegal calls and backdoor activation.

[0103] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the model copyright protection and backdoor defense method based on the dual passport mechanism.

[0104] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the model copyright protection and backdoor defense method based on the dual passport mechanism.

[0105] Another objective of this invention is to provide an information data processing terminal for implementing the model copyright protection and backdoor defense system based on the dual passport mechanism.

[0106] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0107] This invention proposes a method for copyright protection and backdoor attack prevention for pre-trained models, and designs a dual-passport embedding mechanism, specifically including: (1) a data passport embedding module; and (2) a model structure passport embedding module. This mechanism introduces dual security protection at the input data level and key network layers, aiming to achieve model security and copyright verification while preventing backdoor attacks. Furthermore, this solution innovates on both the data passport and the model passport: (1) it privatizes public data and performs exclusion processing on the data passport; and (2) it designs two model passport construction methods: linear combination and neuron response. This significantly increases the functionality and security of the invention, forcing unauthorized malicious access and backdoor attacks to bypass multiple defenses, greatly increasing its difficulty and complexity.

[0108] The proposed solution aims to address the aforementioned technical problems by providing security protection through dual passports and dynamic feature coupling. This method not only overcomes the limitations of existing passport technologies that statically generate keys, but also avoids damage to the original structure caused by the passport layer embedding method, thus achieving the following effects: 1) Proactively suppressing unauthorized access. If the user cannot simultaneously provide the correct data passport and model passport, it will interfere with the image feature output, leading to performance degradation or misidentification, thereby proactively preventing malicious access to the model by third parties. 2) Effectively resisting backdoor attacks. When an attacker attempts to implant triggers for a backdoor attack, the malicious feature stream will be intercepted and disrupted due to the lack of proper special processing of the dataset, preventing the hidden backdoor from being activated. 3) Extremely strong resistance to fine-tuning and pruning. This invention innovatively proposes the NeuroPass method, which, through a dynamic parameter generation mechanism, ensures that even in the extreme case of fine-tuning attacks and 70% pruning, the model's copyright information cannot be compromised, greatly enhancing model reliability.

[0109] Compared to existing technologies, the model copyright protection and backdoor defense method based on a dual-passport mechanism proposed in this solution represents a technological leap from "passive tracing" to "active defense." This solution is particularly suitable for high-security scenarios such as outsourced model training, cloud deployment, model trading, and public security / finance, fundamentally solving two major pain points in the industry: "difficulty in proving model theft" and "difficulty in detecting backdoor tampering." Its core beneficial effects and quantitative indicators are summarized below:

[0110] Extremely low accuracy loss and high authentication success rate: This solution, through an optimized dual-passport embedding strategy (single-layer hidden embedding and restricted reflection feature injection), ensures that the authorization authentication accuracy (VA) remains above 90% (up to 97%) while greatly preserving the model's ability to extract accurate features for the original visual task. This keeps the decrease in accuracy for clean samples strictly controlled at an extremely low level (usually less than 1%), achieving a perfect balance between copyright protection and normal model function.

[0111] Robust Copyright Preservation Capability Under High-Intensity Attacks: This solution's innovative NeuroPass (based on neuron response) dynamic parameter generation mechanism overcomes the limitations of static watermarks, which are easily compromised. Even with extreme global pruning rates as high as 70%, the copyright authentication accuracy of the protected model remains extremely high. Furthermore, when dealing with secondary fine-tuning or obfuscation attacks aimed at erasing copyright, this solution forcibly limits the performance of illegally invoked models to the range of 10.28% to 33.20%, effectively safeguarding the copyright verification mechanism and achieving robust copyright confirmation.

[0112] Excellent backdoor sample interception rate and zero false positive defense (revised to: low false positive rate): This solution's dual-locking mechanism possesses excellent proactive defense capabilities. When facing highly concealed malicious backdoor implantation attacks (such as GWTA), because malicious triggering data cannot pass the data passport verification of the underlying physical constraints, its malicious feature stream is completely cut off from the bottom layer by the passport layer in multi-level nested transformations. This results in a significant reduction in the success rate of backdoor attacks, dropping sharply to below 5% (as low as 2.11%~3.42%, approaching the level of random guessing). At the same time, for unauthorized clean input, its false positive rate and false authentication rate also robustly approach the level of random guessing, achieving complete interception of high-risk backdoors and proactive defense with zero false positives.

[0113] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0114] Addressing the extremely high computational costs of training fundamental large-scale and pre-trained models in the current AI field, this solution provides core tools for rights confirmation and security protection for Model-as-a-Service (MLaaS), cloud deployment, model API outsourcing, and high-value model intellectual property transactions. It effectively prevents high-value pre-trained models from being maliciously copied, illegally misappropriated, or hijacked via backdoors, directly protecting the core digital assets of R&D companies. Simultaneously, this solution empowers core application scenarios with extremely high model security requirements, such as public security, finance, and autonomous driving, possessing significant commercial monetization potential and legal support value for rights protection.

[0115] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally:

[0116] Existing digital watermarking and model fingerprinting technologies are mostly limited to "passive post-event tracing," failing to prevent theft while it is in progress. This solution innovatively proposes an "active rejection mechanism based on dual-passport multi-level nested transformation," achieving, for the first time in the industry, "proactive interception and spontaneous cutting off of malicious feature streams during the inference stage." Without prior knowledge of the backdoor type, it successfully and significantly reduces the attack success rate (ASR) of highly stealthy backdoors (such as GWTA) to below 5%, filling the technological gap in the fields of "spontaneous proactive defense" and "deep integration of rights confirmation and anti-virus" in deep visual models.

[0117] (3) The technical solution of the present invention solves a technical problem that people have long wanted to solve but have never been able to solve successfully:

[0118] It successfully solved the industry-wide challenge of effectively resisting highly covert black-box backdoor attacks (especially attacks based on covert manipulation of internal model weights) without modifying the original network's core physical architecture (e.g., without destroying or replacing the original BN layer) or affecting the model's generalization ability. Simultaneously, thanks to its pioneering NeuroPass dynamic parameter generation mechanism, it overcame the technical bottleneck of maintaining extremely high copyright authentication accuracy even under extreme destructive attack environments such as a global pruning rate as high as 70%, achieving robust rights confirmation under extreme conditions.

[0119] (4) The technical solution of the present invention overcomes technical bias:

[0120] For a long time, the field of deep learning security has been plagued by the bias that "high-strength copyright and security protection inevitably comes at the cost of significantly sacrificing the model's normal prediction accuracy" and that "embedding a passport layer requires altering the original network physical structure (such as replacing batch normalization with group normalization)." This invention successfully overcomes this bias by constructing an authorized dataset containing a large proportion (e.g., 95%) of real labels and a dynamic feature transformation mechanism based on neuron responses. Detailed experimental data demonstrates that even after implanting dual high-strength protection without altering the original network physical structure, the model's original accuracy on clean samples remains strictly controlled to a very low level of less than 1%, achieving a perfect decoupling between the defense mechanism and business performance. Attached Figure Description

[0121] Figure 1 This is a flowchart of a model copyright protection and backdoor defense method based on a dual-passport mechanism provided in an embodiment of the present invention.

[0122] Figure 2 This is a block diagram of a model copyright protection and backdoor defense system based on a dual-passport mechanism provided in an embodiment of the present invention.

[0123] Figure 3This is a graph showing the impact of model fine-tuning attacks (ρ=0.95, DPM) provided in the embodiments of the present invention.

[0124] Figure 4 This is a graph showing the accuracy of SimCLR50 model passport pruning verification provided in this embodiment of the invention. Detailed Implementation

[0125] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0126] like Figure 1 As shown, the model copyright protection and backdoor defense method based on a dual-passport mechanism provided by this embodiment of the invention includes the following steps:

[0127] S101, the data passport embedding module uses reverse backdoor technology to design triggers to inject hidden features into private training data and configure differentiated rejection training loss for public data, ensuring that the normal prediction performance of the model is only deeply bound to special samples carrying the correct data passport.

[0128] S102, the model structure passport embedding module adopts a non-structurally destructive single-layer hidden embedding strategy, which dynamically generates feature transformation parameters through a mechanism based on neuron response or linear combination; among them, the neuron response-based method strongly couples the copyright verification logic with the underlying physiological features (channel response intensity) of the model, making the model have extremely strong anti-pruning and anti-fine-tuning capabilities.

[0129] S103, the dual-passport joint authentication module deeply integrates the rejection mechanism at the external data level with the locking mechanism of the internal model structure to construct a multi-level nested feature transformation verification closed loop; the model can only decode and output normally when the specified authorized data is input and the correct structure key is loaded inside the model; if any condition is missing, the passport layer will actively trigger abnormal disturbances to block illegal calls and backdoor activation.

[0130] like Figure 2 As shown, an embodiment of the present invention provides a model copyright protection and backdoor defense system based on a dual-passport mechanism, comprising:

[0131] The data passport embedding module uses reverse backdoor technology to design triggers to inject hidden features into private training data and configure differentiated rejection training loss for public data, ensuring that the normal prediction performance of the model is only deeply bound to special samples carrying the correct data passport.

[0132] The model structure passport embedding module employs a non-structurally destructive single-layer hidden embedding strategy, dynamically generating feature transformation parameters through mechanisms based on neuron responses or linear combinations. The neuron response-based method strongly couples the copyright verification logic with the model's underlying physiological features (channel response intensity), giving the model extremely strong resistance to pruning and fine-tuning.

[0133] The dual-passport joint authentication module deeply integrates the rejection mechanism at the external data level with the locking mechanism of the internal model structure, constructing a multi-level nested feature transformation verification closed loop. The model can only decode and output normally when the specified authorized data is input and the correct structural key is loaded internally; if either condition is missing, the passport layer will actively trigger abnormal perturbation to block unauthorized calls and backdoor activation.

[0134] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the model copyright protection and backdoor defense method based on the dual passport mechanism.

[0135] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the model copyright protection and backdoor defense method based on the dual passport mechanism.

[0136] Another objective of this invention is to provide an information data processing terminal for implementing the model copyright protection and backdoor defense system based on the dual passport mechanism.

[0137] Specific implementation of the present invention:

[0138] The detailed technical solution of this invention comprises three core components: a data passport embedding module, a model structure passport embedding module, and a dual-passport joint authentication module. Specifically, the data passport embedding module utilizes reverse backdoor technology to design triggers, injecting hidden features into private training data and configuring differentiated rejection training losses for public data, ensuring that the model's normal prediction performance is only deeply bound to specific samples carrying the correct data passport. The model structure passport embedding module employs a non-structurally destructive single-layer hidden embedding strategy, dynamically generating feature transformation parameters through mechanisms based on neuron responses or linear combinations. The neuron response-based method strongly couples the copyright verification logic with the model's underlying physiological characteristics (channel response strength), giving the model extremely strong resistance to pruning and fine-tuning. The dual-passport joint authentication module deeply integrates the rejection mechanism at the external data level with the locking mechanism of the internal model structure, constructing a multi-level nested feature transformation verification closed loop. The model can only decode and output normally when the specified authorized data is input and the correct structural key is loaded internally; if any condition is missing, the passport layer will actively trigger abnormal perturbations to block illegal calls and backdoor activation. This combination provides a novel copyright protection and backdoor defense solution for pre-trained models, effectively preventing fine-tuning, pruning, and backdoor attacks while ensuring the normal functioning of the model.

[0139] I. Data Passport Embedding Module

[0140] The core idea of ​​a data passport is to reverse-engineer a backdoor poisoning mechanism, performing specific preprocessing and injecting covert features into private data, while rejecting public data. The specific implementation steps are as follows:

[0141] 1.1 Dataset Partitioning and Preprocessing

[0142] Given a private dataset for training It contains N input samples and the corresponding real tags To ensure the model learns a strong correlation between specified triggers and correct results, while also being able to output incorrect judgments when faced with samples without specified triggers (even clean samples), thus preventing unauthorized use of the model, the following editing parameters are set according to a preset ratio. The dataset Sample index set Divided into edit index sets and unedited index set and satisfy the relationship and Therefore, the private dataset Divided into edit data subsets and unedited data subset (Exclusive subsets) Two subsets, both defined mathematically as:

[0143]

[0144]

[0145] in, To specify the trigger implantation operation, To specify the target category label. The sample sizes for both satisfy the following conditions: and .

[0146] Furthermore, the specific method for constructing the exclusion training samples is as follows: maintaining an unedited subset of data. The image pixel features remain completely unchanged (i.e., no triggers are added), but their true classification labels are forcibly changed to a pre-specified rejection target category. Experiments have shown that the editing ratio Setting the threshold between 90% and 95% yields the best results. The purpose of retaining 5% to 10% of special exclusion samples for training is to force the model to learn that "it will only output a correct prediction if and only if the input data contains a global trigger; otherwise, even if the input samples are clean, they will all be classified as preset incorrect labels." This construction method proactively establishes exclusion boundaries, effectively preventing unauthorized third parties from reconstructing the original data distribution by collecting a small number of publicly available benign samples to build a shadow dataset, thereby significantly enhancing the security and robustness of the data passport.

[0147] 1.2 Trigger Generation and Implantation

[0148] To ensure the visual concealment of the data passport, this invention employs image transformation operations based on a reflection image model. It was clarified that the generation of the trigger must be subject to strict constraints from physical and optical properties such as "depth field of the same focal plane" and "defocus of the reflection layer," which constitute the first layer of constraints for the generation of the trigger.

[0149] Furthermore, to ensure that the trigger features do not destroy the low-level semantics of the original image, the generation of the trigger is strictly constrained by physical optical properties: this operation involves reflecting the image... With fuzzy convolution kernel Convolution calculations are performed to construct depth imaging models of "depth field (DOF) at the same focal plane" and "defocused reflection layer," and these models are then superimposed onto the edit subset. Each background sample Above, to generate samples carrying data passport features. The specific calculation formula is as follows:

[0150]

[0151] in, This represents the convolution operation. This is a reflective layer. Compared to explicit pixel block implantation, using a reflective image model results in a more visually natural appearance and effectively resists unauthorized censorship and erasure. Furthermore, this solution is also compatible with image processing methods such as background blending and edge blending.

[0152] Secondly, regarding the constraints at the feature level, the trigger must adopt a global data manipulation strategy (such as full-image frequency domain modification or global hybrid overlay) rather than local pixel block modification, in order to prevent the trigger features from overlapping with the original semantic features of the local image and ensure that the model can stably extract the trigger features.

[0153] 1.3 Implementation of Model Training and Rejection Mechanism

[0154] During the fine-tuning training phase of the pre-trained model, to control access permissions to the model, the optimization function of the data passport embedding module is defined as follows:

[0155]

[0156] in, For model parameters, This is the prediction function of the model. For loss function, This is a tradeoff coefficient used to balance the effects of the two losses. In the formula... This is a sample that has been previously implanted with a trigger. .

[0157] The model will be trained iteratively using a pre-constructed optimization function. During the process of continuously updating the model parameters θ until the model converges, this optimization function implements a two-way logical locking mechanism.

[0158] Positive authorization and accuracy preservation: this refers to optimizing the first term of the function. For samples carrying data passports... Output the real labels This ensures the model performs correctly when receiving the correct key. Since the vast majority (90%–95%) of the authorized private data retains the true label information for gradient updates, the model can fully learn and retain its ability to extract deep features from the original image (such as edge textures and high-level semantics of complex objects). Combined with the non-destructive nature of the global trigger, the model retains the same high-precision classification ability as the original clean pre-trained model upon final convergence, achieving almost lossless authorization calls.

[0159] Reverse rejection and category boundary distance control: this is the second term of the optimization function. For clean samples... This generates a rejection reaction, forcing the model to output preset incorrect target labels. The distance between the trigger feature and the normal class boundary is controlled by a tradeoff coefficient in the optimization function. This is achieved through [method / mechanism]. Utilizing [this / method / mechanism]... By dynamically adjusting the weight intensity of the rejection penalty, the distance between the normal data feature manifold and the rejection target boundary is forcibly increased in the model parameter space, thereby achieving active performance suppression.

[0160] Through this training mechanism, once an unauthorized user calls the model using a regular clean dataset, or an attacker attempts to fine-tune and attack by injecting malicious trigger samples, the model will trigger rejection logic to disrupt the output results, actively blocking unauthorized access to and tampering with the model by infringing third parties.

[0161] Model structure passport embedding module (NeuroPass)

[0162] To effectively resist various malicious behaviors and further enhance model security, this invention breaks through the limitations of traditional methods that require disrupting the network structure to implant the passport layer, and innovatively adopts a non-destructive hidden logic embedding method. Specifically, this scheme preferably adopts a single-layer hidden embedding strategy, precisely locking the passport verification logic to the key network layer used to extract deep features in the forward propagation of the pre-trained model (such as a specific convolutional layer in a convolutional neural network, or the multi-head attention mechanism module in the Transformer architecture). Based on this, this scheme proposes two dynamic parameter generation strategies: one based on neuron response (NeuroPass) and the other based on linear combination (LinearPass). The specific implementation method is as follows:

[0163] 2.1 Passport Embedding Method Based on Neuron Response

[0164] The neuron response method is the core innovation of this invention. This scheme embeds copyright verification logic into the physiological characteristics necessary for model training, and dynamically generates a passport key by utilizing the neuron response strength of the convolutional layers in the forward propagation. The specific implementation steps are as follows:

[0165] Feature signature extraction

[0166] Input feature map The input is fed into the network layer corresponding to the target, where For the number of channels, and The height and width of the input feature map are used to obtain the output feature map. For each channel, the average response intensity is calculated. The neuron response statistic used is the "global mean of the absolute values ​​of activation of all neurons on the target channel output feature map". Taking one channel as an example, its average response value The calculation formula is as follows: (revised to: "The formula for calculating the average response value of the c-th channel is as follows:")

[0167]

[0168] in, and These represent the height and width of the output feature map, respectively. Coordinates in the feature map The activation intensity of a neuron can be understood as a channel in the formula. Total neuron response. A vector is formed by averaging the response values ​​of all channels. Select vector The highest response value Each channel index is represented as The channel index serves as the passport key in the forward propagation and parameter optimization during subsequent model training. To ensure the integrity and security of the passport key during storage, transmission, and copyright verification, hash digests, digital signatures, or other integrity protection processing can be applied to the passport key, and it can be independently stored by the authorizing party for subsequent model copyright authentication and legality verification.

[0169] Passport layer parameter initialization

[0170] During the initialization phase of the passport layer, the parameter scaling factor is... With offset Perform a specific assignment, where the scaling factor is... Initialized as a vector with all elements equal to 2, offset It is initialized as a vector with all elements equal to 0, while maintaining the dimensions of the two parameters and the number of channels of the input feature map. Consistent. During the backpropagation process of model training, the optimization function is used... Constraints, scaling factor With offset As a learnable variable, it is continuously corrected during iterative updates and is used in the exponential moving average mechanism.

[0171] Passport key update based on exponential moving average (EMA)

[0172] To ensure that the passport key can adapt to changes in neuron responses during model training, this invention introduces an exponential moving average (EMA) mechanism to update the channel response memory, thereby guaranteeing the dynamic nature of the passport key. First, based on the passport key index set... Generate mask vector In its first Specific values ​​on each channel Follow the formula below:

[0173]

[0174] Subsequently, during the model forward propagation phase, scaling factors are used. With offset Perform scaling and offset processing on the critical channel. Output feature map of the layer Generated by the following operations:

[0175]

[0176] in, This indicates an element-wise multiplication operation. This is a vector containing only 1s. When the passport key is correct, the passport layer performs this operation and outputs the correct prediction result (this operation only transforms the channels selected in set k; unselected channels retain their original features). When the passport key is incorrect, the passport layer dynamically generates a random scaling factor. and offset The corresponding layer output is forcibly perturbed:

[0177]

[0178] Finally, in each iteration of the model, an exponential moving average mechanism is introduced to average the global response vector. Perform the following update operation:

[0179]

[0180] in, The moving average coefficient, This is the instantaneous response vector for the current batch. The updated response vector is obtained using this formula. This will be used for the next round of reselection and updating of the passport key index set k.

[0181] 2.2 Passport Embedding Method Based on Linear Combination

[0182] This scheme proposes to dynamically generate parameters for feature scaling and shifting through linear calculation of the passport key and trainable weights, reducing the dependence on fixed weights in the network. The specific implementation steps are as follows:

[0183] The passport key, as the core credential, is not publicly disclosed with the model; instead, it is independently encrypted and stored locally by the owner. It is also clarified that during legitimate access, the weight parameters (scaling factor and offset) are dynamically generated by linearly combining the passport key with the trainable weight matrix. The passport key is introduced during the model training phase. , This is the dimension of the passport key. The passport key... As a core copyright certificate, it is not publicly disclosed during the model's open-source or cloud deployment phase, but is instead independently and encryptedly stored locally by the model owner. During legitimate access and authentication, it is linked to the trainable weight matrix. and Perform linear combination calculations to dynamically generate scaling factors. and offset This process can be expressed by the following formula:

[0184] (Revised to:) During the model training phase, a passport key is introduced. , This is the defined dimension for the passport key. (Passport key) Each with the first trainable weight matrix With the second trainable weight matrix Perform linear combination calculations to dynamically generate scaling factors. and offset The calculation method is as follows:

[0185]

[0186]

[0187] in, and These are all preset bias terms.

[0188] The passport key As a core copyright certificate, it is not publicly disclosed during the model's open-source or cloud deployment phase. Instead, it is independently and encryptedly stored locally by the model owner. It is then loaded and used during legitimate calls and copyright verification after model deployment to unlock the model's normal predictive performance.

[0189] Forward Propagation and Feature Map Transformation: During the forward propagation phase of the model, the dynamic parameters generated above are used to transform the input feature map. Perform channel-level transformation to obtain feature maps. This step can be represented by the following formula:

[0190]

[0191] in, This represents an element-wise multiplication operation along the channel dimension. These represent the number of channels, height, and width of the feature map, respectively. Specifically, regarding the feature map's... Each channel has an output. Satisfy the following operations:

[0192]

[0193] in, Represents the first... Channel matrix, Corresponding to the first Each channel has a specific scaling factor and offset.

[0194] Anomaly Disturbance Mechanism: If the passport layer receives an incorrect passport key (Right now If this occurs, a perturbation mechanism will be triggered, generating incorrect parameters during linear combination calculations. and The passport layer output will become:

[0195]

[0196] because and The generated feature map deviates significantly from the correct parameters. This will contain serious errors, and this perturbation will be amplified in subsequent deep networks, eventually destroying the model's classification and prediction capabilities, thereby restricting unauthorized use of the model.

[0197] General model optimization and training methods: During the model training and fine-tuning phase, this scheme configures a specific joint optimization function for the single model passport mechanism. The formula is as follows:

[0198]

[0199] in, The primary classification loss function (such as cross-entropy loss) is used to maintain the model's predictive inference ability. The loss function for the passport verification branch is used to validate the embedded passport model. The validity of the copyright. To balance the hyperparameters of the two loss weights, a joint optimization function is used to achieve a balance between the model's performance and the passport verification mechanism.

[0200] A Dual-Passport Embedding Mechanism Based on Data and Model: To achieve deep coupling between data passports and model structure passports, this invention innovatively proposes a dual-passport embedding mechanism. This mechanism implements dual protection verification at the model's input data layer and internal key network layers, significantly improving the model's security and copyright protection capabilities while ensuring inference performance. Taking the linear combination scheme for the model passport as an example, the complete training and joint authentication steps are as follows:

[0201] Data preparation and parameter initialization: Obtaining the encrypted dataset (Includes samples) With tags ) and public datasets (Includes samples) With tags ), and the model parameters Perform random initialization. Pre-set two independent sets of passport keys. and ,in Used for data passport embedding module Used for the model passport embedding module.

[0202] Dynamic generation of dual passport parameters: In each training iteration, the model dynamically generates parameters for the first... Passport layer (satisfying) ), using trainable weight matrix With preset bias scaling factor and offset The linear transformation calculation is performed using the following formula:

[0203]

[0204] Forward Propagation and Multi-Level Nested Transformation: During the forward propagation of the model, the input feature maps obtained from the public dataset are transformed... or feature maps obtained from encrypted datasets Two independent linear transformation operations are performed sequentially on the input feature map. For example, this processing flow can be represented by the following formula:

[0205] First-level feature transformation: utilizing parameters generated based on the data passport key. , For input features Perform scaling and offset operations to generate intermediate feature maps. :

[0206]

[0207] Second-level feature transformation: utilizing independent parameters generated based on the data passport key. , For intermediate feature maps Perform further transformations to generate the final output feature map. :

[0208]

[0209] in, This represents the element-wise multiplication operator. This multi-level nested structure implements dual verification of external data and internal model, ensuring that the model can only be used correctly when both the data passport and the model passport match.

[0210] Construction of the Dual Passport Joint Optimization Function: To enable the model to simultaneously identify authorized encrypted data and unauthorized exclusionary data, this scheme constructs a multi-task joint objective function. This function works by analyzing encrypted datasets. Compared with public datasets Apply differentiation constraints to ensure that the model values ​​output correct results only for the authorized data. Its optimization function is defined as follows:

[0211]

[0212] in, Hyperparameters for balancing the rejection intensity of the target category.

[0213] Parameter update and backpropagation: using the calculated function For model parameters and passport layer parameters (including trainable weight matrix) With bias Backpropagation and gradient updates are performed until training converges. The final output is a fine-tuned model with dual passport protection. and authorized passport key .

[0214] Given the current limitations of copyright protection and security verification mechanisms for fine-tuned pre-trained models in resisting deep parameter fine-tuning and backdoor attacks, this invention selects the following as experimental benchmarks: a clean fine-tuned model (without a passport), a model with only a single passport (data passport only or model structure passport only), and the dual passport mechanism proposed in this invention. The test results of these three models are compared under conditions of normal access and unauthorized access. To comprehensively explore the effectiveness of the dual passport mechanism in copyright protection and security, detailed explanations of its beneficial effects are provided from five perspectives: 1) copyright protection performance evaluation under different pre-trained model architectures; 2) optimal configuration evaluation of data or model passport strategies; 3) defense effectiveness evaluation against obfuscation attacks and model fine-tuning; 4) verification of the optimal number of physical embedding layers in the passport layer; and 5) robustness and defensive impact against model pruning and malicious backdoor implantation.

[0215] Performance Evaluation of Copyright Protection under Different Pre-trained Model Architectures: When implementing this scheme on different pre-trained models based on convolutional neural networks (such as SimCLR50 / 101) and Transformer architectures (such as SwinV2, MAE), it is expected that they will exhibit significant performance differences under different authorization conditions, thus demonstrating the effectiveness and universality of the dual passport mechanism. Table 1 shows a horizontal comparison of the performance of four passport combinations (no passport, data passport only, model passport only, and dual passports) on multiple models. It can be found that when the model embeds dual passports, its authentication accuracy (VA) for authorized samples is extremely close to that of the clean model without passports. Taking the implementation on the SimCLR50 model and tested on the CIFAR-10 dataset as an example, the accuracy (VA) of the clean model for benign samples is 95.15%, while the prediction accuracy of the model using the dual passport mechanism for benign samples does not decrease significantly, with a VA of 94.69%. At the same time, when facing unauthorized access without dual passports, its false authentication rate (VA) is significantly lower. The percentage dropped significantly to 9.77%. This fully demonstrates that the dual-passport mechanism effectively suppresses illegal calls to the model by third parties without significantly affecting the model's normal performance.

[0216]

[0217] Table 1. Model performance evaluation under different pre-trained models (ρ=0.95, NeuroPass)

[0218] Optimal Evaluation of Data or Model Passport Strategies: To verify the optimal generation mechanism and generalization ability of data passports and model passports, an in-depth comparison and evaluation were conducted on the generation method of model structure passports, the privatization strategy of data passports, and the trigger scheme. At the model structure passport level, Table 2 shows the performance of LinearPass and NeuroPass based on linear combination and neuron response on five datasets with wide ranges and diverse types (CIFAR-10, Food-101, STL-10, GTSRB, Pets). Tests show that NeuroPass exhibits superior flexibility and security. In the CIFAR-10 dataset test, its authentication accuracy (VA) reached 93.76%, and the false authentication rate (VA) was significantly lower. The percentage of clean data is only 9.74%, effectively overcoming the limitations of fixed network structure passports. At the data passport level, Table 4 shows the impact of different data privatization manipulation ratios. The results indicate that setting the ratio to 90%-95%, i.e., retaining a small number of clean samples, achieves the best performance locking effect for the model. Maintaining a moderate proportion of clean data ensures that the model has proactive rejection capabilities, thus preventing a precipitous drop in model performance under unauthorized white-box access. Furthermore, Table 5 demonstrates that using global modification triggers (such as SIG, GWTA) is superior to local triggers, effectively preventing performance loss caused by feature overlap.

[0219]

[0220] Table 2. Model performance evaluation of the two dual-passport embedding mechanisms (ρ=0.95)

[0221]

[0222] Table 4. Impact of Data Passport Private Data Manipulation Ratio on Model Performance

[0223]

[0224] Table 5. Performance evaluation of five types of backdoor triggers in data privatization operations (ρ=0.95, NeuroPass)

[0225] Performance Evaluation of Defense Against Obfuscation Attacks and Model Fine-tuning: The dual-passport mechanism demonstrates strong defense against malicious attacks aimed at disrupting or circumventing copyright authentication. First, Table 7 shows that when dealing with obfuscation attacks where an internal attacker flips bits of the passport key, even at extremely low key flip rates (10% or 20%), existing benchmark techniques still allow attackers to gain up to 54.89% of the model performance. LinearPass and NeuroPass technologies, however, limit the attacker's usability performance to 10.28% and 33.20%, respectively, effectively protecting the integrity of copyright. Furthermore, Figure 3 This demonstrates instance performance evaluations before and after an attacker fine-tunes the protected model using publicly available clean datasets. Test results show that even after secondary fine-tuning of SimCLR50 or MAE using clean data, the protected model maintains an VA (Average Value) above 90% under passport verification, and the model's false positive rate for clean samples (…) This significantly improves performance, allowing owners to accurately determine if a model has been "fine-tuned" and prove ownership of the model's copyright simply through a simple performance test.

[0226]

[0227] Table 7. Detection results of single-model structural embedding against obfuscation attacks (ρ=0.95)

[0228] Validation of the optimal number of physical embedding layers for the passport layer: To explore the optimal physical implementation conditions of the network structure, a horizontal comparison of the number of embedding layers for the passport layer was conducted. Table 3 demonstrates that the embedding performance of a single passport layer (such as adding it only in layer 3 or layer 4) is significantly better than that of two-layer embedding. In the SimCLR50 model test, when only a single passport layer is added to Layer 3, the model's authentication accuracy (VA) remains at 93.17% and the false authentication rate is stable. However, adding two passport layers to both Layer 3 and Layer 4 simultaneously causes the model's VA to drop to 90.14% on some specific datasets. Therefore, single-layer embedding ensures secure interception while minimizing interference with the model's normal prediction performance, proving that reliable protection can be achieved through a single and flexible layer deployment.

[0229]

[0230] Table 3. The impact of adding a passport layer on model performance (ρ=0.95, NeuroPass)

[0231] Robustness and defense against model pruning and malicious backdoor implantation: The dual passport mechanism demonstrates strong defensive capabilities under high-rate pruning and malicious attack environments. Regarding model pruning and erasure attacks, Figure 4 The results show that in the model pruning and erasure test, LinearPass maintains good verification accuracy when the pruning rate is below 50%, while the NeuroPass method exhibits strong robustness. Even with a global pruning rate as high as 70%, NeuroPass still maintains extremely high passport verification accuracy, significantly outperforming the LinearPass method. Regarding malicious backdoor attack threats, Table 6 shows that if only data passports are used, the model's attack success rate (ASR) against backdoor attacks (such as GWTA) is as high as 94.06%, failing to effectively defend against backdoor implantation. However, after introducing a model structure passport to form a complete dual-passport mechanism, as shown in Table 8, attackers, lacking the correct model passport key, are unable to unlock the passport layer's response mechanism, rendering the implanted backdoor completely ineffective. For example, on the GTSRB dataset, the backdoor ASR of the NeuroPass model is greatly suppressed and drops sharply to 2.11%~3.42% (close to the level of random prediction), while the model accuracy (VA) remains above 97%, strongly demonstrating the decisive role of the dual-passport mechanism in completely cutting off backdoor attack paths.

[0232]

[0233] Table 6. Evaluation of the attack effectiveness of four backdoor attack data passport models.

[0234]

[0235] Table 8. Evaluation of the attack effectiveness of the passport model for four backdoor attack models.

[0236] The detailed technical solution of this invention includes three core parts: a data passport embedding module, a model structure-based passport embedding module, and a dual passport embedding mechanism based on data and model. Specifically, the implementation of this invention is carried out step by step and in concert according to the above three modules. Each module is independent yet tightly coupled, working together to achieve efficient copyright protection and backdoor defense for deep vision fine-tuning models. 1) Implementation of the data passport embedding module: In the implementation process, a data-level protection mechanism is constructed using a reverse backdoor poisoning method. First, the training dataset is divided into edited and unedited subsets proportionally, and a specified trigger is implanted into the edited subset using a reflection image model to perform image transformation operations. Subsequently, the model parameters are continuously updated using an optimization function, binding the correct data passport with the normal prediction function of the model, and simultaneously establishing a rejection response to unauthorized data. This module serves as the first verification mechanism, forming a dual protection with the subsequent model passport. 2) Implementation of the model structure passport embedding module: Through a preset model passport key, scaling factors and offsets are dynamically generated using a neuronal response mechanism or a linear combination method. The execution logic of this module is as follows: if the key is correct, the input feature map will be transformed through dynamic parameters to obtain an accurate output feature map; if the key is incorrect or missing, the model will spontaneously generate malicious perturbation parameters, forcing the output feature map with serious errors. The output feature map is constrained by the optimization function, realizing real-time control and intervention of the model's inference results during the model's forward propagation, thereby ensuring that only authorized keys can activate the model's normal function. 3) Dual Passport Embedding Implementation: After completing the construction of the first two modules, the core joint optimization and verification stage is entered. This scheme integrates external input data rejection and internal model structure locking into the model training simultaneously through the joint optimization function. In actual deployment, a dual protection verification mode is established: users must have both the correct data passport and model structure passport keys to ensure the model's normal prediction performance; otherwise, both will eventually cause the model passport to generate rejection interference parameters, ultimately destroying the model's normal function.

[0237] Through the coordinated implementation of these three modules, efficient copyright protection and security safeguards can be achieved for deep vision pre-training and fine-tuning models in practical application scenarios. Each link and technical detail of this implementation plan can be appropriately adjusted and optimized according to different network architectures, application environments, and security requirements, thereby ensuring that the model maintains extremely high stealth and robustness when facing unauthorized calls and malicious backdoor attacks from third parties.

[0238] Dual-passport joint authentication and defense verification closed-loop process: In order to effectively verify the security of the fine-tuned model during actual model deployment and in the event of potential infringement, this solution breaks through the passive verification limitations of traditional static watermarks and constructs the following complete coupled authentication closed-loop process:

[0239] Specify input trigger data: The verifier will carry an encrypted dataset with specific reflection characteristics. and unprocessed public datasets These are used as specified input data and are input into the model to be validated.

[0240] Extracting intermediate layer responses: The input data is fed into the model for forward propagation. When the data stream reaches the target network layer (i.e., the passport layer embedding location), the system extracts the neuron response feature map of that layer. Simultaneously, the verifier inputs a pre-defined model structure passport key into this passport layer. Depending on the passport embedding strategy actually selected by the model, the system will perform different feature parameter preparations: if a neuron response strategy is used, the system will extract the response intensity of a specific channel and match it with the corresponding key; if a linear combination strategy is used, the system will directly use the key and network weights to perform linear calculations, dynamically generating feature transformation parameters (scaling factor and offset).

[0241] Passport Signature Calculation and Feature Verification: The system utilizes the feature transformation parameters obtained in the previous steps to perform multi-level nested transformations on the extracted intermediate feature maps, calculating the passport signature feature map driven by the current data flow, and finally outputting the model's classification prediction result. This verification process is essentially a dual unlocking of the data passport and the model structure passport, requiring an exact match between the trigger features in the input data and the model passport key input by the verifier. Only when both match can the model decode and output correctly; if the key is incorrect or missing, the passport layer will spontaneously generate interference, disrupting the model's prediction function.

[0242] Dual-threshold comprehensive judgment: The system presets an authentication accuracy threshold (e.g., VA > 90%) to measure authorization performance, and "false authentication rate (e.g., MR approaches random guessing level)" and "attack success rate (e.g., backdoor ASR is less than 5%)" benchmarks to measure rejection and defense capabilities. The system rigorously compares the quantitative indicators output by the model under test in actual verification with the above dual thresholds, and uses this as the final judgment basis.

[0243] Output copyright and security risk conclusions: a. Legitimate authorization conclusion: Double verification passes only if the input data carries the correct data passport and the model internally loads the correct model structure key. The model output prediction result meets the high authentication accuracy benchmark and is judged as a legitimate authorized call. b. Infringement and tampering conclusion: If an unauthorized party uses a public shadow dataset to erase or fine-tune copyright features, the lack of a data passport will directly trigger the underlying rejection reaction of the data passport; at the same time, the lack of a model structure key will trigger abnormal perturbation of the structure passport. Both of these factors together lead to a severe degradation in model prediction accuracy, thus outputting an infringement, misappropriation, and tampering risk conclusion, completing copyright confirmation. c. Backdoor interception conclusion: When encountering third-party poisoning or non-poisoning backdoor attacks, because the malicious trigger data cannot match the officially set data passport features, the data passport verification module judges it as unauthorized input and directly cuts off and disrupts its malicious feature flow at the passport layer, causing the backdoor attack success rate to plummet below the blocking benchmark, outputting a backdoor failure and defense success conclusion.

[0244] In the description of this invention, unless otherwise stated, "a plurality of" means two or more; the terms "upper," "lower," "left," "right," "inner," "outer," "front end," "rear end," "head," "tail," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0245] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0246] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for model copyright protection and backdoor defense based on a dual-passport mechanism, characterized in that, The model copyright protection and backdoor defense method based on the dual passport mechanism includes the following steps: Data passport embedding steps: Use reverse backdoor technology to design triggers, inject hidden features into private training data, and configure differentiated rejection training loss for public data to ensure that the normal prediction performance of the model is only deeply bound to special samples carrying the correct data passport. Model structure passport embedding steps: A non-structurally destructive single-layer hidden embedding strategy is adopted, and feature transformation parameters are dynamically generated through a mechanism based on neuron response or a mechanism based on linear combination. Among them, the mechanism based on neuron response strongly couples the copyright verification logic with the underlying physiological features of the model, namely the channel response intensity, so that the model has the ability to resist pruning and fine-tuning. Dual-passport joint authentication steps: The rejection mechanism at the external data level is deeply integrated with the locking mechanism of the internal model structure to construct a multi-level nested feature transformation verification closed loop; the model decodes and outputs normally only when the specified authorized data is input and the correct structural key is loaded inside the model; if any condition is missing, the passport layer actively triggers abnormal perturbation to block illegal calls and backdoor activation.

2. The model copyright protection and backdoor defense method based on the dual passport mechanism according to claim 1, characterized in that, The data passport embedding step includes: Dataset Partitioning and Preprocessing: Given a private dataset for training, containing N input samples and their corresponding ground truth labels, where N is the number of samples; the sample index set 1 to N of the private dataset is divided into an edited index set and an unedited index set according to a preset editing ratio ρ between 0 and 1. The union of the edited index set and the unedited index set is 1 to N and the intersection is empty. The private dataset is further divided into an edited data subset and an unedited data subset. The edited data subset is defined as the sample pair formed by the input sample after trigger implantation operation on each index belonging to the edited index set and the ground truth label, used to maintain the normal prediction of the model under authorized conditions. The unedited data subset is defined as the sample pair formed by the input sample and the specified target category label on each index belonging to the unedited index set, used to make the model output abnormal results under unauthorized input to achieve rejection. The number of samples in the edited data subset is determined by the product of the preset editing ratio ρ and the total number of samples N, and the number of samples in the unedited data subset is determined by the product of (1-ρ) and the total number of samples N. Trigger generation and implantation: An image transformation operation based on a reflection image model is adopted. This operation convolves the reflection image with a blurred convolution kernel to calculate the reflection layer, which is constrained by the depth field (DOF) of the same focal plane and the depth imaging physical optical properties of the reflection layer being out of focus. The reflection layer is then superimposed on each background sample in the edited data subset to generate a sample carrying data passport features. This sample is equal to the background sample plus the reflection layer. Model training and rejection mechanism implementation: In the fine-tuning training phase of the pre-trained model, the optimization function of the data passport embedding step is defined as follows: the optimization function value is equal to one-half of the number of samples in the edited data subset multiplied by the sum of the loss functions on the edited data subset, plus the tradeoff coefficient multiplied by one-half of the number of samples in the unedited data subset multiplied by the sum of the loss functions on the unedited data subset; where the tradeoff coefficient is used to balance the influence of the two parts of the loss, and the samples used in the edited data subset are the samples generated after the trigger implantation operation.

3. The model copyright protection and backdoor defense method based on the dual passport mechanism according to claim 1, characterized in that, The model structure passport embedding steps include passport embedding based on neuron response, passport embedding based on linear combination, and general model optimization and training.

4. The model copyright protection and backdoor defense method based on the dual passport mechanism according to claim 3, characterized in that, The passport embedding method based on neuron response includes: Feature signature extraction: The input feature map is fed into the target network layer. The dimension of the input feature map is the number of channels multiplied by the height multiplied by the width, resulting in the output feature map. The dimension of the output feature map is the number of channels multiplied by the output height multiplied by the output width. The average response intensity is calculated for each channel. For the c-th channel, its average response value is equal to 1 divided by the output height multiplied by the output width, and then multiplied by the absolute value of the neuron activation intensity at the corresponding coordinate in the output feature map obtained by summing the output height dimension and summing the output width dimension. The average response values ​​of all channels are used to form a response vector. The K channel indices with the highest response values ​​in the response vector are selected and stored as passport keys, which are then used in the forward propagation and parameter optimization of the model during subsequent training. Passport layer parameter initialization: During the initialization phase of the passport layer, specific values ​​are assigned to the scaling factor and offset to keep the dimensions of the scaling factor and offset consistent with the number of channels in the input feature map; during the backpropagation process of model training, the scaling factor and offset are used as learnable variables and are corrected in iterative updates through the constraints of the optimization function. Passport key update based on exponential moving average: A mask vector is generated based on the passport key index set. The dimension of the mask vector is equal to the number of channels. The value of the c-th channel follows this rule: if c belongs to the passport key index set, the c-th component of the mask vector is equal to 1; if c does not belong to the passport key index set, the c-th component of the mask vector is equal to 0. During the forward propagation phase of the model, scaling and offset processing are performed on the key channels using scaling factors and offsets. The output feature map of the l-th layer is generated by the following operation: the output feature map is equal to the scaling factor multiplied element-wise by the mask vector, then multiplied element-wise by the input feature map, plus the offset multiplied element-wise by the mask vector, plus the input feature map multiplied element-wise by the all-1 vector minus the mask vector, where all elements of the all-1 vector are 1. This operation is only performed on the index set k. Selected channels undergo scaling and offset transformations, while unselected channels retain their original features. When the passport key is correct, the passport layer performs this operation and outputs the correct prediction result. When the passport key is incorrect, the passport layer dynamically generates random scaling factors and random offsets that follow a standard normal distribution. The output feature map of the corresponding layer is equal to the random scaling factor multiplied element-wise by the mask vector, then multiplied element-wise by the input feature map, plus the random offset multiplied element-wise by the mask vector, plus the input feature map multiplied element-wise by a vector of all 1s minus the mask vector. In each iteration of the model, an exponential moving average mechanism is introduced to update the global average response vector. The updated global average response vector is equal to the moving average coefficient multiplied by the global average response vector before the update, plus 1 minus the moving average coefficient multiplied by the instantaneous response vector of the current batch, where the moving average coefficient takes a value between 0 and 1.

5. The model copyright protection and backdoor defense method based on the dual passport mechanism according to claim 3, characterized in that, The passport embedding method based on linear combination includes: Linear generation of dynamic parameters: During the model training phase, a passport key is introduced. The passport key has a dimension of K, which is a set dimension. The passport key serves as a core copyright certificate and is independently encrypted and stored locally by the owner, not disclosed with the model. The passport key is linearly combined with the first and second trainable weight matrices. The dimensions of the first and second trainable weight matrices are the number of channels multiplied by K. A scaling factor and an offset are dynamically generated. The scaling factor and the offset are both the number of channels. The scaling factor is equal to the first trainable weight matrix multiplied by the passport key plus a first bias term, and the offset is equal to the second trainable weight matrix multiplied by the passport key plus a second bias term. Forward Propagation and Feature Map Transformation: During the forward propagation phase of the model, channel-level transformations are performed on the input feature map using dynamically generated scaling factors and offsets. The dimension of the input feature map is the number of channels multiplied by the height multiplied by the width, resulting in the output feature map. The calculation method is as follows: after broadcasting the scaling factors and offsets along the spatial height and width dimensions, the output feature map is equal to the scaling factor multiplied element-wise by the input feature map plus the offset. For the c-th channel of the feature map, its output is equal to the scaling factor of the c-th channel multiplied by the input feature map matrix of the c-th channel plus the offset of the c-th channel. Abnormal perturbation mechanism: If the passport layer receives an incorrect passport key and the incorrect passport key is not equal to the correct passport key, the perturbation mechanism is triggered. The incorrect scaling factor and the incorrect offset are calculated by linear combination. The output feature map of the passport layer is equal to the incorrect scaling factor multiplied element by element by the input feature map plus the incorrect offset.

6. The model copyright protection and backdoor defense method based on the dual passport mechanism according to claim 3, characterized in that, The general model optimization and training includes: during the model training or fine-tuning phase, constructing a joint optimization objective function for the data passport mechanism and the model structure passport mechanism; the joint optimization objective function includes a data passport classification loss term, a model structure passport verification loss term, and a data rejection constraint term; wherein, the data passport classification loss term is used to maintain the normal predictive performance of authorized data, the data rejection constraint term is used to constrain unauthorized data to produce preset prediction results, and the model structure passport verification loss term is used to establish the association between the model structure passport and the passport key; each loss term is jointly optimized through weighted hyperparameters, so that the model prediction performance is simultaneously constrained by both the data passport and the model structure passport, realizing a deep binding between model functionality and copyright information.

7. A model copyright protection and backdoor defense system based on a dual-passport mechanism, implementing the model copyright protection and backdoor defense method based on any one of claims 1-6, characterized in that, The model copyright protection and backdoor defense system based on the dual-passport mechanism includes: The data passport embedding module uses reverse backdoor technology to design triggers to inject hidden features into private training data and configure differentiated rejection training loss for public data, ensuring that the normal prediction performance of the model is only deeply bound to special samples carrying the correct data passport. The model structure passport embedding module adopts a non-structurally destructive single-layer hidden embedding strategy, which dynamically generates feature transformation parameters through mechanisms based on neuron response or linear combination. Among them, the neuron response-based method strongly couples the copyright verification logic with the underlying physiological features (channel response intensity) of the model, giving the model extremely strong resistance to pruning and fine-tuning. The dual-passport joint authentication module deeply integrates the rejection mechanism at the external data level with the locking mechanism of the internal model structure to construct a multi-level nested feature transformation verification closed loop. The model can only decode and output normally when the specified authorized data is input and the correct structure key is loaded inside the model. If any condition is missing, the passport layer will actively trigger abnormal disturbances to block illegal calls and backdoor activation.

8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the model copyright protection and backdoor defense method based on the dual passport mechanism as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the model copyright protection and backdoor defense method based on a dual-passport mechanism as described in any one of claims 1-6.

10. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the model copyright protection and backdoor defense system based on the dual passport mechanism as described in claim 7.

Images