Training-authorization-reasoning three-in-one large language model dynamic disturbance watermark encryption protection system and method

Abstract

The invention relates to the technical field of artificial intelligence security, in particular to a training-authorization-reasoning three-in-one large language model dynamic disturbance watermark encryption protection system and method, comprising three closely associated stages of model training encryption, model distribution authorization watermark injection and model reasoning verification authorization; the method has the beneficial effects that the encrypted model weight cannot be directly used even if the encrypted model weight is illegally acquired, so that a core research and development result is protected; a substantial protection layer is provided for algorithm patent application, and reverse engineering is prevented; model leakage of an outsourcing development link is prevented; and merger and purchase valuation protection: the encrypted assets show complete technical control capability during full-duty investigation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, specifically to a dynamic perturbation watermark encryption protection system and method for a large language model integrating training, authorization, and inference. Background Technology

[0002] The widespread deployment of Large Language Models (LLMs) has created an urgent need for model copyright protection. Currently, unauthorized copying, fine-tuning, and secondary distribution of models are rampant in both open-source communities and commercial settings. For example, after the weights of Meta's LLaMA series models were leaked, numerous derivative models were commercially used without proper copyright information. Traditional software copyright protection techniques (such as code obfuscation and license encryption) are difficult to apply directly to neural networks because their weight matrices are continuous and high-dimensional, making static watermarks easily destroyed by model fine-tuning, pruning, or quantization operations.

[0003] According to the IEEE 2023 AI Security Report, over 60% of enterprises are concerned about the misuse of models after reverse engineering. The existing legal framework's definition of AI model copyright is vague, making technical protection a core measure. However, the field of model watermarking still faces two major challenges: first, balancing watermark robustness with model performance; and second, meeting the requirements for concealment and verifiability in dynamic adversarial scenarios.

[0004] A dynamic weighted perturbation watermarking method needs to be proposed, which encodes the watermark information into a nonlinear transformation of the weight space, and achieves a balance between concealment and robustness through key-controlled reversible perturbation. Summary of the Invention

[0005] The purpose of this invention is to provide a dynamic perturbation watermark encryption protection system and method for large language models that integrates training, authorization, and inference, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a dynamic perturbation watermark encryption protection system for a large language model integrating training, authorization, and inference, comprising:

[0007] Model training encryption module: During the model training phase, the gradient update values ​​are selectively encrypted using a dynamic key training controller to generate an encrypted model carrying a key tree. The dynamic key training controller divides the training process into N phases. In each phase, a phase key K_i is generated based on the weight hash value of the previous phase using the HMAC algorithm. In each training batch, M% of the weight matrix is ​​randomly selected, and the gradient update values ​​are XORed and obfuscated using K_i to construct a Merkle tree containing all phase keys {K_1,...,K_N}. The root hash value is stored in the model metadata. The selective encryption adopts a dynamic decay strategy. In the initial training phase, M=50% is set, and it is linearly reduced to M=10% as the training rounds increase. The encryption target is preferentially selected from the weight matrix of the FFN layer in the Transformer architecture.

[0008] Model Authorization Watermark Injection Module: During the model distribution phase, user identity information is encoded into a watermark vector by a user key embedder and injected into the model weight parameters. The user key embedder generates a master public key PK based on the user's hardware fingerprint and registration information, and derives subkeys SK_1 and SK_2 through KDF. SK_1 is used to perform low-rank decomposition on the model weights, embedding a structured watermark pattern in the singular value matrix. A dynamic binary mask is generated based on SK_2 for gating and filtering the Attention matrix during inference. After watermark injection, constraint fine-tuning is performed using a loss function with KL divergence constraints. The number of fine-tuning rounds does not exceed 5% of the total training rounds. During fine-tuning, the parameter updates of the embedded watermark layer are frozen.

[0009] Model Inference Verification Module: During the model inference phase, a secure inference framework verifies user permissions and dynamically decrypts model weights to prevent model execution in unauthorized environments. The secure inference framework includes a key negotiation module, a memory decryption engine, and a watermark verifier. The key negotiation module verifies the user's hardware fingerprint based on the TEE environment and negotiates a temporary session key with the license server. The memory decryption engine decrypts model layer weights as needed, and the decrypted data is only stored in the encrypted memory area. The watermark verifier detects model integrity by comparing the weight hash value with the metadata signature.

[0010] Preferably, the stage key generation method of the dynamic key training controller in the model training encryption module is as follows: the training process of the large language model is precisely divided into N training stages with clear boundaries; at the beginning of each stage, the model weights obtained from the previous stage training are obtained and their hash values ​​are calculated; using the HMAC algorithm, the hash values ​​of the previous stage weights are used as input to generate the stage key K_i corresponding to the current stage, where the value of i ranges from 1 to N, ensuring the uniqueness and security of each stage key.

[0011] Preferably, the watermark injection and constraint fine-tuning process of the user key embedder in the model authorization watermark injection module is as follows:

[0012] Watermark Injection: Collect the user's hardware fingerprint and registration information, and generate a master public key PK using a specific key derivation algorithm; use the master public key PK to derive subkeys SK_1 and SK_2 using the key derivation function KDF; use subkey SK_1 to perform low-rank decomposition on the model weights, and embed watermark information into the resulting singular value matrix according to a predefined structured pattern; generate a dynamic binary mask based on subkey SK_2, which performs gating filtering on the attention matrix during the model inference stage to achieve dynamic association of watermark information;

[0013] Constraint fine-tuning: Design a loss function with KL divergence constraints. This loss function will fine-tune the model after watermark injection while ensuring model performance. Control the number of fine-tuning rounds to no more than 5% of the total training rounds to avoid excessive fine-tuning affecting the stability of the watermark and the performance of the model. During the fine-tuning process, freeze the parameter updates of the embedded watermark layer and only fine-tune the parameters of other layers to ensure that the watermark information is not destroyed.

[0014] Preferably, the protection mechanism of the secure inference framework in the model inference verification module is as follows:

[0015] Environment detection and self-destruction mechanism: During the model inference process, the secure inference framework monitors the operating environment in real time. When a debugger is detected to be attached or the operating environment does not meet the preset whitelist requirements, the weight self-destruction process is immediately triggered to prevent the model from being reverse-engineered or stolen in an unauthorized environment.

[0016] Hidden watermark embedding mechanism: When the model generates text output, a hidden watermark sequence is embedded in the text based on the user key. This hidden watermark sequence is unique and imperceptible, and can be used for subsequent copyright tracing and source verification of the model output text.

[0017] Preferably, the system also includes:

[0018] Key Management Module: Responsible for managing the stage keys generated by the dynamic key training controller, the subkeys derived from the user key embedder, and the temporary session keys negotiated by the secure inference framework; ensuring the security and confidentiality of keys throughout the entire system lifecycle by securely storing, encrypting, and periodically updating the keys.

[0019] Log auditing module: Records the system's operation logs during the training, authorization, and inference phases, including key generation and usage, watermark injection and verification records, and user permission verification results. By auditing and analyzing the logs, potential security threats and abnormal operations can be detected in a timely manner, providing a basis for the system's secure operation and maintenance.

[0020] A method for a dynamic perturbation watermark encryption protection system for large language models that integrates training, authorization, and inference includes three closely related stages: model training encryption, model distribution authorization watermark injection, and model inference verification authorization.

[0021] Model training encryption phase: Selective encryption operation is performed on gradient update values ​​using a dynamic key training controller; specifically, the training process is divided into N phases, and each phase generates a phase key K_i using the HMAC algorithm based on the weight hash value of the previous phase; M% of the weight matrix is ​​randomly selected in each training batch, and the gradient update values ​​are XORed and obfuscated using K_i; a Merkle tree containing all phase keys {K_1,...,K_N} is constructed, and the root hash value is stored in the model metadata, finally generating an encrypted model carrying the key tree;

[0022] Model distribution authorization watermark injection stage: User identity information is encoded into a watermark vector through a user key embedder and injected into the model weight parameters; specifically, a master public key PK is generated based on the user's hardware fingerprint and registration information, and subkeys SK_1 and SK_2 are derived through KDF; SK_1 is used to perform low-rank decomposition on the model weights, and a structured watermark pattern is embedded in the singular value matrix; a dynamic binary mask is generated based on SK_2, which is used to gate and filter the Attention matrix during inference; after watermark injection, constraint fine-tuning is performed, using a loss function with KL divergence constraints, and the number of fine-tuning rounds does not exceed 5% of the total training rounds, and the parameter updates of the embedded watermark layer are frozen during the fine-tuning process;

[0023] Model inference verification and authorization phase: A secure inference framework is used to verify user permissions and dynamically decrypt model weights, preventing model execution in unauthorized environments. The secure inference framework includes a key negotiation module, a memory decryption engine, and a watermark verifier. The key negotiation module verifies the user's hardware fingerprint based on the TEE environment and negotiates a temporary session key with the license server. The memory decryption engine decrypts model layer weights on demand, and the decrypted data is only stored in the encrypted memory area. The watermark verifier detects model integrity by comparing the weight hash value with the metadata signature. When a debugger attachment or runtime environment that does not conform to the whitelist is detected, a weight self-destruction process is triggered. An invisible watermark sequence determined by the user's key is embedded in the generated text.

[0024] Preferably, the selective encryption of the dynamic key training controller during the model training encryption phase adopts a dynamic decay strategy;

[0025] Initial settings: In the initial training phase, the proportion M of the randomly selected weight matrix is ​​set to 50% to ensure that a large number of weight matrices are encrypted in the early stage of training, thereby enhancing the security of the model training process.

[0026] Dynamic adjustment: As the number of training rounds increases, the value of M is reduced from the initial 50% to 10% in a linear manner. This dynamic adjustment method can gradually reduce the impact of encryption on training efficiency while ensuring model security.

[0027] Encryption target selection: When selecting encryption targets, the weight matrix of the FFN layer in the Transformer architecture is preferred for encryption because the FFN layer plays an important role in feature transformation and nonlinear mapping in the large language model. Encrypting it can effectively protect the core computational logic of the model.

[0028] Preferably, the operation of the user key embedder in the model distribution authorization watermark injection stage has a strict security procedure;

[0029] Key generation: First, collect the user's hardware fingerprint and registration information. This information is unique and stable and can accurately identify the user's identity. Then, generate the master public key PK based on this information, and derive the subkeys SK_1 and SK_2 through the key derivation function KDF. The key derivation process uses secure algorithms and parameters to ensure the independence and security of the subkeys.

[0030] Watermark embedding: The model weights are decomposed into a low-rank form using the subkey SK_1, which is then decomposed into a product of multiple low-rank matrices. Watermark information is then embedded into the singular value matrix obtained from the decomposition according to a predefined structured watermarking pattern. This structured watermarking pattern has specific encoding rules, which ensures reliable watermark embedding and detection while maintaining model performance.

[0031] Mask generation: A dynamic binary mask is generated based on the subkey SK_2. This mask is used to gate and filter the Attention matrix during the model inference stage. The generation algorithm of the dynamic binary mask is related to the user key. Different user keys will generate different masks, thereby achieving personalized protection for different user models.

[0032] Preferably, the key negotiation module of the secure reasoning framework in the model reasoning verification authorization phase has an efficient and secure negotiation mechanism:

[0033] Environment verification: The key negotiation module verifies the user's hardware fingerprint based on the Trusted Execution Environment (TEE). The TEE provides an isolated secure area to ensure the confidentiality and integrity of the hardware fingerprint verification process and prevent the hardware fingerprint information from being stolen or tampered with.

[0034] Key negotiation: After verifying the legitimacy of the user's hardware fingerprint, the key negotiation module communicates with the license server to negotiate a temporary session key. The negotiation process uses a secure key exchange protocol.

[0035] Key usage: The generated temporary session key is used for subsequent model layer weight decryption and user permission verification operations. The temporary session key has an expiration time and will automatically expire after a certain period of time or a certain number of uses, which further enhances the security of the system.

[0036] Preferably, the protection mechanism of the secure reasoning framework in the model reasoning verification authorization phase has multi-layered protection capabilities;

[0037] Environment detection and self-destruction: During the model inference process, the secure inference framework monitors the operating environment in real time. When a debugger is detected to be attached or the operating environment does not meet the preset whitelist requirements, the weight self-destruction process is immediately triggered. The weight self-destruction process uses a specific algorithm to irreversibly destroy the model weights, making the model unusable and effectively preventing the model from being reverse-engineered or stolen in an unauthorized environment.

[0038] Hidden watermark embedding: When the model generates text output, a hidden watermark sequence is embedded in the text according to the user key. The hidden watermark sequence is unique and imperceptible, and will not affect the quality and readability of the text. By detecting the hidden watermark sequence in the generated text, the source of the text can be traced, the user of the model can be identified, and strong evidence can be provided for copyright protection.

[0039] Compared with the prior art, the beneficial effects of the present invention are:

[0040] The proposed training-authorization-inference three-in-one dynamic perturbation watermark encryption protection system and method for large language models protects core R&D results by preventing the direct use of encrypted model weights even if they are illegally obtained; provides a substantial protection layer for algorithm patent applications to prevent reverse engineering; prevents model leakage in outsourced development; and protects M&A valuations by ensuring that encrypted assets demonstrate complete technical control capabilities during due diligence. Attached Figure Description

[0041] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the present invention clear and complete, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only some, not all, embodiments of the present invention, and are merely illustrative of the embodiments of the present invention. They are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Example 1: This invention provides a technical solution: a dynamic perturbation watermark encryption protection system for a large language model integrating training, authorization, and inference, comprising:

[0044] Model training encryption module: During the model training phase, a dynamic key training controller selectively encrypts gradient update values ​​to generate an encrypted model carrying a key tree. The dynamic key training controller divides the training process into N stages. In each stage, based on the weight hash value of the previous stage, a stage key K_i is generated using the HMAC algorithm. In each training batch, M% of the weight matrix is ​​randomly selected, and K_i is used to XOR-hash the gradient update values, constructing a Merkle tree containing all stage keys {K_1,...,K_N}. The root hash value is stored in the model metadata. The selective encryption employs a dynamic decay strategy. The initial training stage is set... M is set to 50%, and linearly reduced to M = 10% as the number of training rounds increases. The encryption target is preferentially selected from the FFN layer weight matrix in the Transformer architecture. The stage key generation method of the dynamic key training controller in the model training encryption module is as follows: the training process of the large language model is precisely divided into N training stages with clear boundaries; at the beginning of each stage, the model weights obtained from the previous stage training are obtained and their hash values ​​are calculated; using the HMAC algorithm, the hash values ​​of the previous stage weights are used as input to generate the stage key K_i corresponding to the current stage, where the value of i ranges from 1 to N, ensuring the uniqueness and security of each stage key.

[0045] Model Authorization Watermark Injection Module: During the model distribution phase, user identity information is encoded into a watermark vector via a user key embedder and injected into the model weight parameters. Specifically, the user key embedder generates a master public key (PK) based on the user's hardware fingerprint and registration information, and derives subkeys SK_1 and SK_2 using KDF. SK_1 is used to perform low-rank decomposition on the model weights, embedding a structured watermark pattern into the singular value matrix. A dynamic binary mask is generated based on SK_2 for gating the Attention matrix during inference. After watermark injection, constraint fine-tuning is performed using a loss function with KL divergence constraints. The number of fine-tuning rounds does not exceed 5% of the total training rounds, and parameter updates of the embedded watermark layer are frozen during fine-tuning. The watermark injection and constraint fine-tuning process of the user key embedder in the model authorization watermark injection module is as follows: Watermark Injection: Collect the user's hardware fingerprint and registration information, and through a specific key... A key derivation algorithm generates a master public key PK. Using the master public key PK, subkeys SK_1 and SK_2 are derived through the key derivation function KDF. Subkey SK_1 is used to perform low-rank decomposition on the model weights. Watermark information is embedded into the resulting singular value matrix according to a predefined structured pattern. A dynamic binary mask is generated based on subkey SK_2. This mask performs gating filtering on the attention matrix during the model inference phase, achieving dynamic association of watermark information. Constraint fine-tuning: A loss function with KL divergence constraints is designed. This loss function, while ensuring model performance, imposes fine-tuning constraints on the model after watermark injection. The number of fine-tuning rounds is controlled to not exceed 5% of the total training rounds to avoid excessive fine-tuning affecting the stability of the watermark and the performance of the model. During fine-tuning, the parameter updates of the watermark embedding layer are frozen, and only the parameters of other layers are fine-tuned to ensure that the watermark information is not destroyed.

[0046] Model Inference Verification Module: During the model inference phase, a secure inference framework verifies user permissions and dynamically decrypts model weights to prevent model execution in unauthorized environments. This secure inference framework includes a key negotiation module, a memory decryption engine, and a watermark verifier. The key negotiation module verifies the user's hardware fingerprint based on the TEE environment and negotiates a temporary session key with the license server. The memory decryption engine decrypts model layer weights as needed, and the decrypted data is only stored in an encrypted memory area. The watermark verifier detects model integrity by comparing the weight hash value with the metadata signature. The security mechanisms within the model inference verification module are: Environment Detection and Self-Destruction Mechanism: During model inference, the secure inference framework monitors the runtime environment in real time. When a debugger attachment is detected or the runtime environment does not meet the preset whitelist requirements, a weight self-destruction process is immediately triggered to prevent the model from being reverse-engineered or stolen in an unauthorized environment. Hidden Watermark Embedding Mechanism: When the model generates text output, a hidden watermark sequence is embedded in the text based on the user's key. This hidden watermark sequence is unique and imperceptible, and can be used for subsequent copyright tracing and source verification of the model's output text.

[0047] The system also includes: a key management module, responsible for managing the stage keys generated by the dynamic key training controller, the subkeys derived from the user key embedder, and the temporary session keys negotiated by the secure inference framework; ensuring the security and confidentiality of keys throughout the entire system lifecycle by securely storing, encrypting, and periodically updating them; and a log auditing module, which records the system's operation logs during the training, authorization, and inference phases, including key generation and usage, watermark injection and verification records, and user permission verification results; and by auditing and analyzing the logs, it can promptly identify potential security threats and abnormal operations, providing a basis for the system's secure operation and maintenance.

[0048] Example 2, based on Example 1, proposes a method for a dynamic perturbation watermark encryption protection system for large language models that integrates training, authorization, and inference, including the following steps:

[0049] 1. Dynamic watermark encryption technology is introduced during the training phase.

[0050] This solution introduces a multi-level dynamic watermarking mechanism during the model training phase. Unlike traditional post-processing watermarking, our watermark is deeply integrated into the model's knowledge representation system. Specific implementation includes:

[0051] Hierarchical parameter encryption: The model weights are divided into critical layers (such as the QKV matrix in the attention mechanism) and non-critical layers. Lightweight block encryption based on a user key is applied to the parameters of the critical layers. The encrypted parameters maintain functional continuity mathematically, that is, the encrypted transformation f satisfies: f(Wx) ≈ Wf'(x), where f' is the corresponding transformation of the input data. This functional encryption ensures that the model does not need to completely decrypt the parameters during forward propagation.

[0052] Dynamic watermarking: A watermark generation network is designed that takes user identity information (ID) and intermediate model features (F) as input to generate a dynamic watermark pattern: W = F ⊕ H(ID||F), where H is a cryptographic hash function. This watermark changes with the input content and is deeply integrated into the self-attention calculation of each Transformer layer.

[0053]

[0054] Where λ is the adjustable intensity coefficient, W is the dynamic watermark matrix of the current step, Q|K|V is the value of the attention mechanism, and d is the vector dimension.

[0055] Anti-fine-tuning protection: A watermark stability term L is added to the loss function to ensure that the watermark pattern remains stable when the model parameters are fine-tuned. At the same time, "trap parameters" are implanted in key weights. Any fine-tuning that attempts to remove the watermark will cause these parameters to change abnormally, triggering the model's self-destruct mechanism.

[0056] Implementation steps:

[0057] Phased key generation: The training process is divided into N phases, and a new key K_i is generated in each phase using the HMAC algorithm based on the weight hash value of the previous phase.

[0058] Selective parameter encryption: In each training batch, M% of the weight matrix is ​​randomly selected, and the gradient update values ​​are XORed and obfuscated using K_i.

[0059] Dynamic decay strategy: As the number of training rounds increases, the value of M is gradually reduced (e.g., from 50% to 10%) to ensure that the final convergence accuracy of the model is not affected.

[0060] Key tree storage: Organize the keys {K_1,...K_N} for each stage into a Merkle tree structure, and store the root hash value in the model metadata.

[0061] Technical advantages: When attackers cannot obtain the complete key sequence, they cannot restore the effective weights; the key is dynamically bound to the training process, eliminating pre-computation attacks.

[0062] 2: Embed key user information in model weights

[0063] This solution breaks through the traditional "one-to-many" authorization model, achieving true "one-to-one" customized protection for the model:

[0064] 1. User Key Binding: Each authorized user receives a unique key pair (K_pub, K_priv), where the public key K_pub is used to encrypt user-specific segments in the model, and the private key K_priv is stored in the Hardware Security Module (HSM). Model weights are divided into a general segment S_g and a user-specific segment S_u, the latter being encrypted using the user's public key: Enc(S_u, K_pub).

[0065] 2. Multi-layered identity encoding: Different forms of user identity information are embedded at different levels of the model (embedding layer, intermediate layer, output layer).

[0066] Embedding layer: Maps user ID hash values ​​to specific embedding vectors and mixes them into the original word embeddings;

[0067] Intermediate layer: Incorporate user-specific patterns into the bias terms of the attention mechanism;

[0068] Output layer: The top-k perturbations of the output probability distribution are related to the user's identity;

[0069] 3. Stealth Tracking Mechanism: A parameter-output correlation function Γ is designed so that small changes in a specific subset of parameters {P_i} lead to detectable but imperceptible pattern changes in the output. When the model is leaked, the source of the leak can be precisely located by analyzing the output patterns.

[0070] 4. Dynamic Verification Channel: The model integrates a lightweight verification module that periodically communicates with the authorization server via heartbeat to verify the real-time validity of the user's identity. The communication process uses zero-knowledge proof technology and does not expose actual identity information.

[0071] 3: Dedicated Security Reasoning Framework

[0072] This solution features a dedicated secure inference framework that provides an end-to-end cryptographic model execution environment.

[0073] Security Loader: The core of the framework is the security model loader, which first verifies the integrity of the runtime environment (including CPU microcode version, memory isolation, etc.), and then decrypts the model weights as needed. The decryption process uses "streaming decryption" technology, that is, decryption is only performed when the parameters are about to be used, and the complete plaintext model is never stored in memory.

[0074] Layered access control: The framework implements fine-grained permission management.

[0075] Base layer: Verifies user hardware fingerprints and digital certificates;

[0076] Middle layer: Real-time monitoring of abnormal patterns during the inference process;

[0077] Application layer: Controls the distribution permissions of model output;

[0078] Dynamic watermark rendering: When generating each token, the framework calculates a dynamic watermark based on the current context, user identity, and timestamp, and embeds it in the output in an invisible manner. The watermark pattern is satisfied.

[0079] Robustness: Conventional post-processing (such as restatement and translation) will not destroy the watermark;

[0080] Imperceptibility: Does not affect the quality and smoothness of the generated content;

[0081] Verifiability: Copyright holders can extract the watermark using a dedicated detector;

[0082] Anti-debugging protection: The framework integrates multiple anti-reverse engineering measures.

[0083] Code obfuscation and dynamic transformation;

[0084] The key algorithm is implemented using white-box encryption;

[0085] Real-time detection of debuggers and triggering of protection mechanisms.

[0086] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A dynamic perturbation watermark encryption protection system for a large language model integrating training, authorization, and inference, characterized in that: include: Model training encryption module: During the model training phase, the gradient update values ​​are selectively encrypted using a dynamic key training controller to generate an encrypted model carrying a key tree. The dynamic key training controller divides the training process into N phases. In each phase, a phase key K_i is generated based on the weight hash value of the previous phase using the HMAC algorithm. In each training batch, M% of the weight matrix is ​​randomly selected, and the gradient update values ​​are XORed and obfuscated using K_i to construct a Merkle tree containing all phase keys {K_1,...,K_N}. The root hash value is stored in the model metadata. The selective encryption adopts a dynamic decay strategy. In the initial training phase, M=50% is set, and it is linearly reduced to M=10% as the training rounds increase. The encryption target is preferentially selected from the weight matrix of the FFN layer in the Transformer architecture. Model Authorization Watermark Injection Module: During the model distribution phase, user identity information is encoded into a watermark vector by a user key embedder and injected into the model weight parameters. The user key embedder generates a master public key PK based on the user's hardware fingerprint and registration information, and derives subkeys SK_1 and SK_2 through KDF. SK_1 is used to perform low-rank decomposition on the model weights, embedding a structured watermark pattern in the singular value matrix. A dynamic binary mask is generated based on SK_2 for gating and filtering the Attention matrix during inference. After watermark injection, constraint fine-tuning is performed using a loss function with KL divergence constraints. The number of fine-tuning rounds does not exceed 5% of the total training rounds. During fine-tuning, the parameter updates of the embedded watermark layer are frozen. Model Inference Verification Module: During the model inference phase, a secure inference framework verifies user permissions and dynamically decrypts model weights to prevent model execution in unauthorized environments. The secure inference framework includes a key negotiation module, a memory decryption engine, and a watermark verifier. The key negotiation module verifies the user's hardware fingerprint based on the TEE environment and negotiates a temporary session key with the license server. The memory decryption engine decrypts model layer weights as needed, and the decrypted data is only stored in the encrypted memory area. The watermark verifier detects model integrity by comparing the weight hash value with the metadata signature.

2. The training-authorization-inference integrated dynamic perturbation watermark encryption protection system for large language models according to claim 1, characterized in that: The stage key generation method of the dynamic key training controller in the model training encryption module is as follows: the training process of the large language model is precisely divided into N training stages with clear boundaries; At the beginning of each stage, obtain the model weights obtained from the previous stage training and calculate their hash values; Using the HMAC algorithm, the weighted hash value of the previous stage is used as input to generate the stage key K_i corresponding to the current stage, where the value of i ranges from 1 to N, ensuring the uniqueness and security of each stage key.

3. The training-authorization-inference integrated dynamic perturbation watermark encryption protection system for large language models according to claim 2, characterized in that: The watermark injection and constraint fine-tuning process of the user key embedder in the model authorization watermark injection module is as follows: Watermark Injection: Collect the user's hardware fingerprint and registration information, and generate a master public key PK using a specific key derivation algorithm; use the master public key PK to derive subkeys SK_1 and SK_2 using the key derivation function KDF; use subkey SK_1 to perform low-rank decomposition on the model weights, and embed watermark information into the resulting singular value matrix according to a predefined structured pattern; generate a dynamic binary mask based on subkey SK_2, which performs gating filtering on the attention matrix during the model inference stage to achieve dynamic association of watermark information; Constraint fine-tuning: Design a loss function with KL divergence constraints. This loss function will fine-tune the model after watermark injection while ensuring model performance. Control the number of fine-tuning rounds to no more than 5% of the total training rounds to avoid excessive fine-tuning affecting the stability of the watermark and the performance of the model. During the fine-tuning process, freeze the parameter updates of the embedded watermark layer and only fine-tune the parameters of other layers to ensure that the watermark information is not destroyed.

4. The training-authorization-inference integrated dynamic perturbation watermark encryption protection system for large language models according to claim 3, characterized in that: The protection mechanism of the secure inference framework in the model inference verification module is as follows: Environment detection and self-destruction mechanism: During the model inference process, the secure inference framework monitors the operating environment in real time. When a debugger is detected to be attached or the operating environment does not meet the preset whitelist requirements, the weight self-destruction process is immediately triggered to prevent the model from being reverse-engineered or stolen in an unauthorized environment. Hidden watermark embedding mechanism: When the model generates text output, a hidden watermark sequence is embedded in the text based on the user key. This hidden watermark sequence is unique and imperceptible, and can be used for subsequent copyright tracing and source verification of the model output text.

5. The training-authorization-inference integrated dynamic perturbation watermark encryption protection system for large language models according to claim 4, characterized in that: The system also includes: Key Management Module: Responsible for managing the stage keys generated by the dynamic key training controller, the subkeys derived from the user key embedder, and the temporary session keys negotiated by the secure inference framework; ensuring the security and confidentiality of keys throughout the entire system lifecycle by securely storing, encrypting, and periodically updating the keys. Log auditing module: Records the system's operation logs during the training, authorization, and inference phases, including key generation and usage, watermark injection and verification records, and user permission verification results. By auditing and analyzing the logs, potential security threats and abnormal operations can be detected in a timely manner, providing a basis for the system's secure operation and maintenance.

6. A method for a dynamic perturbation watermark encryption protection system for a large language model as described in claim 5, characterized in that: It includes three closely related stages: model training encryption, model distribution authorization watermark injection, and model inference verification authorization; Model training encryption phase: Selective encryption operation is performed on gradient update values ​​using a dynamic key training controller; specifically, the training process is divided into N phases, and each phase generates a phase key K_i using the HMAC algorithm based on the weight hash value of the previous phase; M% of the weight matrix is ​​randomly selected in each training batch, and the gradient update values ​​are XORed and obfuscated using K_i; a Merkle tree containing all phase keys {K_1,...,K_N} is constructed, and the root hash value is stored in the model metadata, finally generating an encrypted model carrying the key tree; Model distribution authorization watermark injection stage: User identity information is encoded into a watermark vector through a user key embedder and injected into the model weight parameters; The specific operation involves generating a master public key PK based on the user's hardware fingerprint and registration information, deriving subkeys SK_1 and SK_2 through KDF; using SK_1 to perform low-rank decomposition on the model weights, embedding a structured watermark pattern in the singular value matrix; generating a dynamic binary mask based on SK_2, which is used for gating and filtering the Attention matrix during inference; performing constraint fine-tuning after watermark injection, using a loss function with KL divergence constraints, with the number of fine-tuning rounds not exceeding 5% of the total training rounds, and freezing the parameter updates of the embedded watermark layer during the fine-tuning process; Model inference verification authorization phase: Relying on the secure inference framework to verify user permissions and dynamically decrypt model weights, preventing model execution in unauthorized environments; The secure inference framework includes a key negotiation module, a memory decryption engine, and a watermark verifier. The key negotiation module verifies the user's hardware fingerprint based on the TEE environment and negotiates a temporary session key with the license server; The memory decryption engine decrypts model layer weights on demand, and the decrypted data is only kept in the encrypted memory area; the watermark verifier detects model integrity by comparing the weight hash value with the metadata signature, and triggers the weight self-destruct process when a debugger attachment or runtime environment does not conform to the whitelist; and embeds an invisible watermark sequence determined by the user key in the generated text.

7. The method for a dynamic perturbation watermark encryption protection system for a large language model integrating training, authorization, and inference as described in claim 6, characterized in that: In the model training encryption phase, the selective encryption of the dynamic key training controller adopts a dynamic decay strategy. Initial settings: In the initial training phase, the proportion M of the randomly selected weight matrix is ​​set to 50% to ensure that a large number of weight matrices are encrypted in the early stage of training, thereby enhancing the security of the model training process. Dynamic adjustment: As the number of training rounds increases, the value of M is reduced from the initial 50% to 10% in a linear manner. This dynamic adjustment method can gradually reduce the impact of encryption on training efficiency while ensuring model security. Encryption target selection: When selecting encryption targets, the weight matrix of the FFN layer in the Transformer architecture is preferred for encryption because the FFN layer plays an important role in feature transformation and nonlinear mapping in the large language model. Encrypting it can effectively protect the core computational logic of the model.

8. The method for a dynamic perturbation watermark encryption protection system for a large language model integrating training, authorization, and inference as described in claim 7, characterized in that: The operation of the user key embedder in the model distribution authorization watermark injection stage has a strict security procedure. Key generation: First, collect the user's hardware fingerprint and registration information. This information is unique and stable and can accurately identify the user's identity. Then, generate the master public key PK based on this information, and derive the subkeys SK_1 and SK_2 through the key derivation function KDF. The key derivation process uses secure algorithms and parameters to ensure the independence and security of the subkeys. Watermark embedding: The model weights are decomposed into a low-rank form using the subkey SK_1, which is then decomposed into a product of multiple low-rank matrices. Watermark information is then embedded into the singular value matrix obtained from the decomposition according to a predefined structured watermarking pattern. This structured watermarking pattern has specific encoding rules, which ensures reliable watermark embedding and detection while maintaining model performance. Mask generation: A dynamic binary mask is generated based on the subkey SK_2. This mask is used to gate and filter the Attention matrix during the model inference stage. The generation algorithm of the dynamic binary mask is related to the user key. Different user keys will generate different masks, thereby achieving personalized protection for different user models.

9. The method for a dynamic perturbation watermark encryption protection system for a large language model integrating training, authorization, and inference as described in claim 8, characterized in that: The key negotiation module of the secure inference framework in the model inference verification and authorization phase has an efficient and secure negotiation mechanism: Environment verification: The key negotiation module verifies the user's hardware fingerprint based on the Trusted Execution Environment (TEE). The TEE provides an isolated secure area to ensure the confidentiality and integrity of the hardware fingerprint verification process and prevent the hardware fingerprint information from being stolen or tampered with. Key negotiation: After verifying the legitimacy of the user's hardware fingerprint, the key negotiation module communicates with the license server to negotiate a temporary session key. The negotiation process uses a secure key exchange protocol. Key usage: The generated temporary session key is used for subsequent model layer weight decryption and user permission verification operations. The temporary session key has an expiration time and will automatically expire after a certain period of time or a certain number of uses, which further enhances the security of the system.

10. The method for a dynamic perturbation watermark encryption protection system for a large language model integrating training, authorization, and inference as described in claim 9, characterized in that: The security protection mechanism of the secure reasoning framework in the model reasoning verification authorization phase has multi-layered protection capabilities; Environment detection and self-destruction: During the model inference process, the secure inference framework monitors the operating environment in real time. When a debugger is detected to be attached or the operating environment does not meet the preset whitelist requirements, the weight self-destruction process is immediately triggered. The weight self-destruction process uses a specific algorithm to irreversibly destroy the model weights, making the model unusable and effectively preventing the model from being reverse-engineered or stolen in an unauthorized environment. Hidden watermark embedding: When the model generates text output, a hidden watermark sequence is embedded in the text according to the user key. The hidden watermark sequence is unique and imperceptible, and will not affect the quality and readability of the text. By detecting the hidden watermark sequence in the generated text, the source of the text can be traced, the user of the model can be identified, and strong evidence can be provided for copyright protection.

Images