Large model encryption deployment and operation control method and system thereof

Abstract

This application relates to the fields of artificial intelligence security and software copyright protection technology, and discloses a method and system for the encrypted deployment and operation control of large models. The method includes: loading an encrypted large language model file on the deployment end and starting a model inference service; executing parallel environment monitoring tasks during the operation of the model inference service to verify the matching status of the current operating environment with preset authorization conditions in real time; when the verification matching status is abnormal, maintaining the process running state of the model inference service and its interface response capability to external requests, while switching the operating mode to a controlled interference mode; in the controlled interference mode, intercepting intermediate probability distribution data during the model inference calculation process and executing a computational perturbation strategy to disrupt the semantic logic of the model-generated content, thereby achieving implicit degradation of the output results. This application significantly improves the concealment and resistance to reverse engineering of the authorization protection mechanism.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence security and software copyright protection, and in particular to the encryption deployment and operation control technology of large language models for private deployment scenarios. Background Technology

[0002] The descriptions in this section are intended only to provide background information for the implementation of this application and should not be construed as an admission or implication that they constitute prior art.

[0003] In recent years, Large Language Model (LLM) technology has made groundbreaking progress. LLM is a type of natural language processing model based on deep neural networks. It learns the statistical patterns and semantic representations of language through pre-training on massive amounts of text data, thereby acquiring various capabilities such as text generation, dialogue interaction, and question answering. Because training LLM requires significant computational resources and high-quality data, its model weight files themselves have extremely high commercial value.

[0004] As enterprises increasingly demand higher standards for data privacy and service response time, more and more model providers are choosing to deliver large language models to their clients through private deployment, meaning that the model weight files and inference service programs are deployed and run on the client's own local servers. However, this private deployment model removes the model weight files from the provider's direct control, posing a significant challenge to the protection of model assets.

[0005] Existing model protection schemes typically employ authorization verification mechanisms to control model usage. These mechanisms involve verifying the validity of the authorization file and comparing hardware environment characteristics during the startup or operation of the model inference service to determine whether the current usage complies with the authorization agreement. When abnormal situations such as expired authorization or changes in the hardware environment are detected, existing schemes generally block access by directly terminating the service process, for example, by sending a termination signal to the inference service process, forcibly shutting down the service, and ceasing to respond to external requests.

[0006] However, the aforementioned method of directly terminating the service has significant security flaws. Since the termination of a service process is a definite and observable system event, attackers can precisely locate the code containing the authorization verification logic by monitoring process status, capturing termination signals, and setting debug breakpoints. Once located, attackers can bypass or remove the relevant verification code using reverse engineering techniques such as binary patching and memory injection, rendering the authorization protection mechanism completely ineffective. Furthermore, sudden service interruptions can degrade the user experience and may be exploited by attackers as a signal to detect trigger points for the protection mechanism. Summary of the Invention

[0007] One objective of this application is to provide a method and system for the encrypted deployment and operation control of large models, which can provide a more covert authorization protection mechanism that is difficult for attackers to locate and bypass.

[0008] This application discloses a method for encrypted deployment and operation control of large-scale models, including the following steps: Load the encrypted large language model file on the deployment side and start the model inference service; During the operation of the model inference service, parallel environment monitoring tasks are performed to verify in real time the matching status between the current operating environment and the preset authorization conditions; When the matching status is verified to be abnormal, the process running state of the model inference service and its interface response capability to external requests are maintained, but the running mode of the model inference service is switched to controlled interference mode. In the controlled interference mode, the intermediate probability distribution data of the model inference service during the inference calculation process is intercepted, and a calculation perturbation strategy is executed on the intermediate probability distribution data to disrupt the semantic logic of the model-generated content and achieve implicit degradation of the output results.

[0009] In a preferred embodiment, the intermediate probability distribution data is a log-odds vector of values ​​before the model performs sampling with a normalized exponential function. The calculation of the perturbation strategy on the intermediate probability distribution data includes one or a combination of the following operations: Noise injection: Random noise data conforming to a preset distribution law is superimposed on the log probability value vector to disrupt the original word prediction probability ranking. High-probability masking: Identify the multiple candidate words with the highest probability in the log-probability vector, and force their corresponding values ​​to be set to a preset masking value, thus forcing the sampling algorithm to select words with lower probability; Increased distribution entropy: Reduces the difference between the maximum and minimum values ​​in the log-odds vector, decreases the peak significance of the probability distribution, and increases the randomness and perplexity of the output results.

[0010] In a preferred embodiment, maintaining the running state of the model inference service process and its interface response capability to external requests specifically includes: The application programming interface controlling the model inference service continuously returns a status code indicating a successful request to the calling client. The returned data packet structure remains consistent with that in normal mode, and does not contain explicit error messages, so that the calling end cannot identify the blocking behavior through network-level status codes or protocol formats.

[0011] In a preferred embodiment, the step of verifying the matching status between the current operating environment and the preset authorization conditions employs a multi-level fingerprint fault-tolerance mechanism: Collect the multi-dimensional hardware fingerprint of the currently deployed terminal, and extract the core fingerprint subset for comparison according to the fault tolerance level defined in the authorization file; The controlled interference mode is triggered only when the core fingerprint subset does not match, or when the current system time exceeds the authorization validity period. If the non-core fingerprint changes but a subset of the core fingerprint matches, the normal operating mode will continue.

[0012] In a preferred embodiment, the step of loading the encrypted large language model file further includes: Read the filename or path information of the encrypted model file as associated data; The model file is decrypted using an associated data authentication encryption algorithm, combined with the decryption key and the associated data; If the model file is renamed, causing a mismatch between the associated data during decryption and the associated data during encryption, the decryption will fail and the loading process will be terminated.

[0013] In a preferred embodiment, the encrypted large language model file is protected using a key management system that separates delivery and authorization, specifically including: The model file is encrypted using a randomly generated master key; Based on the key derivation function, the master key is encrypted a second time using the password set by the user to generate a key file; The authorization file contains only authorization policy information and digital signature, but does not contain the master key or its derived information; When the deployment end is loaded, the correct password must be provided to recover the master key, and an authorization file verified by signature must be provided to confirm the validity of the authorization, so as to achieve physical separation of decryption capability delivery and authorization policy delivery.

[0014] In a preferred embodiment, the step of loading the encrypted large language model file further includes a secure loading process that minimizes the plaintext window: After authorization verification is successful, a system-level temporary directory is created as the decryption target path; The encrypted model file is decrypted to the temporary directory, rather than the persistent storage directory; Start the model inference service and point to the decrypted file in the temporary directory; When the model inference service terminates or receives an exit signal, all plaintext files in the temporary directory are automatically cleared, so that the plaintext model exists on the disk only for a controlled, short time window.

[0015] In a preferred embodiment, the step of loading the encrypted large language model file employs a non-integrated deployment architecture based on network structure, specifically including: The model parameters of the large language model are divided into local main parameters and remote key parameters; the local main parameters are deployed on the deployment terminal, and the remote key parameters are not included in the local deployment package; During the inference computation process, when the computation path involves the remote key parameters, a cloud collaborative computation is requested through an encrypted channel, or the remote key parameters are dynamically pulled from a protected remote source to the local trusted execution environment for transient computation. If the environmental monitoring task detects a connection interruption or authentication failure with a remote source, the controlled interference mode is automatically triggered.

[0016] In a preferred embodiment, the remote key parameters include specific expert network weights in a hybrid expert model architecture, or specific adaptation layer weights based on low-rank adapter technology. The local subject parameters can still form an executable computation graph and perform inference even in the absence of the remote key parameters, but the absence of the remote key parameters will cause the accuracy of the output results to decrease to a certain extent. The computational perturbation strategy serves as a further superimposed control for this decrease in accuracy.

[0017] This application also discloses a large-scale encrypted deployment and operation control system, including: Memory, used to store computer-executable instructions; and, A processor, coupled to the memory, is configured to implement the steps of the method as described above when executing the computer-executable instructions.

[0018] In the implementation of this application, a covert copyright protection mechanism that is difficult for attackers to locate and bypass is achieved by executing parallel environmental monitoring tasks during the model inference service operation and maintaining the service process's running state and interface response capability to external requests when authorization anomalies are detected, while switching the running mode to controlled interference mode and performing computational perturbation strategies on the intermediate probability distribution data during the model inference calculation process. Compared with the traditional protection method of directly terminating the service, this solution utilizes the unique characteristic of large language models to generate text based on probability distributions. Without changing the external appearance of the service, it destroys the semantic logic of the generated content at the computational level inside the model, causing the output result to be implicitly degraded. Since the service process continues to run and the interface responds normally, attackers cannot locate the trigger point of the protection logic by setting breakpoints or monitoring the process status, which greatly increases the difficulty of reverse analysis and circumventing the protection mechanism, thereby effectively ensuring that model assets are not illegally used after authorization expires.

[0019] Furthermore, by defining the intermediate probability distribution data as a log-probability vector before sampling by the normalized exponential function, and by using one or a combination of noise injection, high-quality shielding, and distribution entropy increase as computational perturbation strategies, diverse and flexibly configurable output interference methods can be provided. Appropriate perturbation methods can be selected for different protection strength requirements, effectively controlling the degree of quality degradation of the output results while maintaining the concealment of interference.

[0020] Furthermore, by controlling the application interface of the model inference service to continuously return status codes indicating successful requests, and keeping the returned data packet structure consistent with the normal mode and without containing explicit error messages, it can be ensured that the calling end cannot identify the blocking behavior through network-level status codes or protocol formats. This makes the triggering of the protection mechanism completely transparent to external systems, preventing attackers from detecting and circumventing based on abnormal response characteristics.

[0021] Furthermore, by employing a multi-level fingerprint fault tolerance mechanism for environmental verification, multi-dimensional hardware fingerprints are collected, and a core fingerprint subset is extracted and compared according to the fault tolerance level defined in the authorization file. The controlled interference mode is only triggered when the core fingerprint subset does not match or the authorization expires. This can ensure the strength of authorization binding while adapting to the needs of hardware changes in actual operation and maintenance, avoiding the problem of service unavailability caused by common operation and maintenance operations such as replacing network cards and hard drives, and effectively balancing security and maintainability.

[0022] Furthermore, by reading the filename or path information of the encrypted model file as associated data, and using the associated data authentication encryption algorithm in combination with the decryption key and associated data for decryption, a strong binding between the ciphertext and the filename can be achieved. This prevents attackers from bypassing the loading logic or launching packet swapping attacks by renaming the file. Any tampering with the filename will cause the authentication tag verification to fail during decryption, thereby ensuring the integrity and consistency of the model file.

[0023] Furthermore, by adopting a key management system that separates delivery and authorization, the model file is encrypted with a random master key and the master key is re-encrypted based on the user password to generate a key file. Meanwhile, the authorization file only contains authorization policy information and digital signatures, but does not contain the master key. This achieves physical separation between the delivery of decryption capabilities and the delivery of authorization policies. Even if an attacker obtains any part of the deployment package or the authorization file, they will not be able to decrypt the model independently, effectively improving the security level of key management.

[0024] Furthermore, by decrypting the encrypted model files to a system-level temporary directory instead of a persistent storage directory, and automatically clearing all plaintext files in the temporary directory when the model inference service terminates, a secure loading process with a minimized plaintext window can be achieved. This ensures that the plaintext model exists on the disk only within a controlled, short time window, greatly reducing the risk of attackers obtaining plaintext model weights through static disk analysis.

[0025] Furthermore, by adopting a non-complete deployment architecture based on network structure, the model parameters are divided into main parameters deployed locally and remote key parameters not included in the local deployment package. When inference calculation involves remote key parameters, cloud collaborative calculation is requested through an encrypted channel or the parameters are dynamically pulled to the local trusted execution environment for transient calculation. This ensures that the complete model weights never exist in the customer's local environment in a holistic form. Even if the local deployment package is completely stolen, attackers cannot obtain a complete and usable model, fundamentally improving the protection strength of model assets.

[0026] Furthermore, by using specific expert network weights in the hybrid expert model architecture or specific adaptation layer weights based on low-rank adapter technology as remote key parameters, the local subject parameters can still form a complete computation graph and perform inference even when remote key parameters are missing, but the output accuracy will decrease to a certain extent. The computational perturbation strategy is used as a further superimposed control to address this accuracy decrease. The separability of the large language model network structure can be used to achieve cloud-based collaborative protection, ensuring basic inference capabilities while ensuring that the output quality deteriorates significantly when unauthorized use is not allowed through a dual mechanism.

[0027] The various technical features disclosed in the above-described invention, the various technical features disclosed in the following embodiments and examples, and the various technical features disclosed in the accompanying drawings can be freely combined to form various new technical solutions (all of which should be considered as having been recorded in this specification), unless such a combination of technical features is technically infeasible. For example, in one example, feature A+B+C is disclosed, and in another example, feature A+B+D+E is disclosed. Features C and D are equivalent technical means that serve the same function, and technically only one needs to be used; it is impossible to use both simultaneously. Feature E can be technically combined with feature C. Therefore, the solution A+B+C+D should not be considered as having been recorded because it is technically infeasible, while the solution A+B+C+E should be considered as having been recorded. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of a method for the encrypted deployment and operation control of a large model according to an embodiment of this application. Detailed Implementation

[0029] In the following description, many technical details are presented to help the reader better understand this application. However, those skilled in the art will understand that the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0030] Example 1 This embodiment provides a basic implementation scheme for a large-scale model encryption deployment and operation control method. This scheme achieves covert blocking of unauthorized use through controlled interference modes. For example... Figure 1 As shown, the method includes the following steps: Step 101: Load the encrypted large language model file and start the model inference service. In this embodiment, the deployment end first loads the encrypted large language model file and starts the model inference service. Specifically, this embodiment uses vLLM as the model inference engine, and the encrypted model file includes model weight files stored in safetensors format and model configuration files in JSON format. After the model inference service starts, it provides an HTTP interface conforming to the OpenAI API specification, with the listening port set to 8000, supporting functions such as text completion and dialogue generation.

[0031] Step 102: Initiate a parallel environment monitoring task. During the model inference service operation, the system executes a parallel environment monitoring task to verify the matching status of the current operating environment with the preset authorization conditions in real time. In this embodiment, an independent watchdog thread is created using Python's threading module to implement the environment monitoring task. This thread runs in parallel with the main inference service thread and does not block each other. The watchdog thread's monitoring cycle is set to 10 minutes, meaning that a complete environment verification process is executed every 10 minutes.

[0032] Step 103: Collect hardware fingerprint and verify authorization conditions. The environment verification process includes two checks: first, collecting the hardware fingerprint of the current machine and comparing it with the fingerprint recorded in the authorization file; second, reading the current system time and determining whether the validity period specified in the authorization file has expired. Pre-set authorization conditions are stored in a digitally signed authorization file, including information such as customer name, authorization type, validity period start and end time, allowed hardware fingerprints, and fault tolerance level.

[0033] Step 104: Determine if the matching status is abnormal. The system determines the matching status between the current operating environment and the preset authorization conditions based on the verification result of step 103. If the hardware fingerprint comparison passes and the current time is within the authorization validity period, the matching status is determined to be normal, and the system returns to step 103 to continue performing periodic environment verification; if the hardware fingerprint comparison fails or the current time exceeds the authorization validity period, the matching status is determined to be abnormal, and the system proceeds to step 105.

[0034] Step 105: Maintain the service process running and switch to controlled interference mode. When the environment monitoring task verifies that the matching status is abnormal, this embodiment does not use the traditional method of directly terminating the service process. Instead, it maintains the running state of the model inference service and its interface response capability to external requests, while switching the running mode of the model inference service to controlled interference mode. Specifically, the watchdog thread triggers the mode switch by modifying a thread-safe global state flag. This flag is set as an atomic variable to ensure access safety in a multi-threaded environment. When the flag changes from the normal state to the interference state, the sampling module of the inference engine will execute the corresponding interference logic accordingly.

[0035] Step 106: Intercept intermediate probability distribution data. In controlled interference mode, the system intercepts intermediate probability distribution data generated during the model inference service's inference computation process. In this embodiment, the intermediate probability distribution data specifically refers to the logits numerical vector before the model performs sampling using the normalized exponential function (i.e., the Softmax function). In the autoregressive generation process of the large language model, each token generated by the model requires the following steps: First, the last layer of the model outputs a logits vector with a dimension equal to the vocabulary size, where each element corresponds to the unnormalized score of a token in the vocabulary; then, the logits vector is converted into a probability distribution using the Softmax function; finally, the sampling algorithm selects the next generated token based on the probability distribution. This embodiment intercepts the logits vector before the Softmax function is executed by registering a custom logits processor in the vLLM inference engine.

[0036] Step 107: Execute a computational perturbation strategy to implicitly degrade the output. The system applies a computational perturbation strategy to the intermediate probability distribution data intercepted in Step 106 to disrupt the semantic logic of the model-generated content, thus implicitly degrading the output. During the continuous text generation process, the system iteratively executes Steps 106 and 107, intercepting and perturbing the logits vector at the time of each word generation, thereby resulting in a degraded overall output quality.

[0037] This embodiment provides three computational perturbation strategies, which can be used individually or in combination.

[0038] The first strategy is noise injection, which involves superimposing random noise data conforming to a preset distribution pattern onto the logits vector to disrupt the original word prediction probability ranking. Let the original logits vector be... ,in Given the vocabulary size, this embodiment generates vocabulary with a mean of 0 and a standard deviation of [value missing]. Gaussian distributed noise vector The perturbated logits vector is Standard deviation The value ranges from 0.5 to 2.0, and the default setting in this embodiment is 1.0. Noise injection will change the relative score ranking between each word, causing the sampling results to deviate from the model's original optimal prediction.

[0039] The second strategy is high-priority masking, which involves identifying the highest-probability candidate words in the logits vector and forcing their corresponding values ​​to negative infinity or a preset masking value, thus compelling the sampling algorithm to select words with lower probabilities. This embodiment first sorts the logits vector and identifies the words with the highest probabilities. There are several positions, and then the values ​​at these positions are set to... This value will approach zero probability after Softmax calculation. Parameter The value range is from 1 to 10, and the default setting in this embodiment is 5, which means that the top 5 words with the highest blocking probability are blocked. The high-optimal blocking strategy will force the model to abandon the most likely correct word choice, resulting in inappropriate word choice or semantic deviation in the generated content.

[0040] The third strategy is to increase distribution entropy, which reduces the difference between the maximum and minimum values ​​in the logits numerical vector, decreases the peak significance of the probability distribution, and increases the randomness and perplexity of the output results. This embodiment introduces a temperature parameter... To achieve distributed entropy increase, the perturbed logits vector is: When the temperature parameter At this time, the differences between elements in the logits vector are reduced, the probability distribution after Softmax calculation is flatter, the distinguishability between high-probability and low-probability words decreases, and the randomness of the sampling results increases. Temperature parameter The value ranges from 1.5 to 5.0, and the default setting in this embodiment is 2.5. The distribution entropy increase strategy will make the model output more perplexing, and the coherence and logic of the generated content will be significantly reduced.

[0041] In controlled interference mode, the application programming interface (API) of the control model inference service in this embodiment continuously returns an HTTP 200 status code to the caller, indicating successful request processing. The returned JSON data packet structure remains completely consistent with that in normal mode, including fields such as the generated text content, the number of tokens used, and the reason for completion, without containing any explicit error messages or exception markers. From a network perspective, the caller cannot identify that the service has entered controlled interference mode through status codes, response headers, or data packet format. The technical advantage of this design is that, because the service process continues to run and the interface responds normally, attackers cannot locate the code position of the authorization verification logic by monitoring the process status, capturing termination signals, or setting debugging breakpoints, making the protection mechanism highly covert and resistant to reverse engineering.

[0042] Example 2 Based on Example 1, this embodiment describes in detail the multi-level fingerprint fault tolerance mechanism used in the environmental monitoring task to verify the matching status between the current operating environment and the preset authorization conditions.

[0043] In real-world private deployment scenarios, a customer's server hardware environment may change due to operational needs, such as replacing a faulty network card, upgrading hard drive capacity, or migrating within a virtualized environment. If the authorization verification mechanism relies solely on a single hardware characteristic for strict matching, any hardware change will lead to authorization invalidation and service interruption, severely impacting the customer's business continuity. To address this issue, this embodiment employs a multi-level fingerprint fault-tolerance mechanism, ensuring both strong authorization binding and operational flexibility.

[0044] This embodiment first collects a multi-dimensional hardware fingerprint of the currently deployed terminal, including the following five dimensions: The first dimension is the CPU fingerprint, which is obtained by reading a specified file to acquire information such as the processor model, number of cores, and manufacturer identifier, and then calculating its SHA256 hash value as the CPU fingerprint. The second dimension is the motherboard fingerprint, which is obtained by executing a specific command (e.g., dmidecode) to acquire information such as the motherboard manufacturer, product name, and serial number, and then calculating its SHA256 hash value as the motherboard fingerprint. The third dimension is the disk fingerprint, which is obtained by reading symbolic links under a specific directory (e.g., / dev / disk / by-id) or executing the lsblk command to acquire the system disk's serial number, and then calculating its SHA256 hash value as the disk fingerprint. The fourth dimension is the network fingerprint, which is obtained by reading the MAC address of the first network card and calculating its SHA256 hash value as the network fingerprint. The fifth dimension is the machine fingerprint, which is obtained by reading the unique machine identifier generated during operating system installation and calculating its SHA256 hash value as the machine fingerprint.

[0045] Based on the five dimensions of hardware fingerprinting mentioned above, this embodiment defines four levels of fingerprint sets, from highest to lowest: full fingerprint, no-MAC fingerprint, core fingerprint, and minimum fingerprint. A full fingerprint contains the fingerprint hash values ​​of all five dimensions, requiring the deployment environment to be completely consistent with the authorized environment. A no-MAC fingerprint includes four dimensions: CPU fingerprint, motherboard fingerprint, disk fingerprint, and machine fingerprint, excluding network fingerprints, and is suitable for scenarios involving network card replacement or virtual machine network configuration changes. A core fingerprint includes two dimensions: CPU fingerprint and motherboard fingerprint, representing the server's most core hardware identifier, and is suitable for scenarios involving disk replacement or operating system reinstallation. A minimum fingerprint contains only the CPU fingerprint dimension, providing the minimum hardware binding, and is suitable for scenarios requiring maximum operational flexibility.

[0046] The license file defines a Tolerance Level field with values ​​ranging from 0 to 3, corresponding to four verification modes: full match, no MAC match, core match, and minimum match. During the environment verification phase, the system extracts the corresponding subset of fingerprints for the current machine based on the tolerance level specified in the license file and compares it with the corresponding subset of fingerprints recorded in the license file. The system only determines the matching status as abnormal and triggers controlled interference mode if the core fingerprint subset for the specified level does not match, or if the current system time exceeds the license validity period. If a non-core fingerprint changes but the core fingerprint subset still matches, the system maintains normal operation mode and does not trigger any interference measures. For example, when the tolerance level is set to 1 (no MAC match), even if the customer replaces the server's network card, causing a change in the MAC address, the service can still operate normally as long as the CPU, motherboard, disk, and machine fingerprint remain consistent.

[0047] This embodiment's multi-level fingerprint fault tolerance mechanism allows model providers to strike a balance between authorization binding strength and operational flexibility by flexibly configuring fault tolerance levels, based on the different security needs and operational environments of various customers. For customers in sectors with high security requirements, such as finance and government, a lower fault tolerance level can be set to achieve strict binding; for internet customers with frequent operational changes, a higher fault tolerance level can be set to avoid unnecessary service interruptions.

[0048] Example 3 This embodiment describes in detail the encryption protection mechanism for large language model files, including file protection based on associated data authentication encryption, a key management system that separates delivery and authorization, and a secure loading process that minimizes the plaintext window.

[0049] During the model encryption phase, this embodiment employs Advanced Encryption Standard (AES) combined with Galois / Counter Mode (GCM) as the Authenticated Encryption with Associated Data (AEAD) algorithm, with a key length of 256 bits. For each model file to be encrypted, the system first generates a 16-byte random salt and a 12-byte random initialization vector (IV). Then, it reads the original content of the file and calculates its SHA256 hash value as an integrity checksum. During encryption, the system reads the original filename as the associated data input to the AES-GCM algorithm. The associated data itself is not encrypted, but its hash value participates in the calculation of the authentication tag, creating a cryptographic binding between the ciphertext and the filename.

[0050] After encryption, the system concatenates the salt value, initialization vector, authentication tag, original hash value, original filename length, original filename, and ciphertext content according to a predefined format and writes them into the target encrypted file. The header of the encrypted file contains a fixed magic number identifier, used to quickly identify the file format during decryption. After all model files are encrypted, the system generates an encryption manifest file (encryption_manifest.json), recording the mapping relationship between the original files and the encrypted files.

[0051] This embodiment employs a key management system that separates delivery and authorization to protect the master key. The system randomly generates a 32-byte master key for encrypting all model files, but the master key itself is not directly delivered to the client or stored in the deployment package. The system requires the user to set a strong password, and then uses Password-Based Key Derivation Function 2 (PBKDF2) to derive an encryption key from the user's password. The configuration parameters for PBKDF2 include: SHA256 hash algorithm, 310,000 iterations, and a 16-byte salt length. The derived key is used to encrypt the master key using AES-GCM, and the encryption result is stored as a key file (model.key).

[0052] The license file (license.json) is digitally signed by the model provider using its private RSA key. The signature algorithm employs a Probabilistic Signature Scheme (PSS) padding mode, and the RSA key length is 4096 bits. The license file contains license policy information such as the customer name, license type, validity period, fault tolerance level, and allowed hardware fingerprints, as well as the RSA-PSS signature value of the policy content, but does not contain the master key or any derived information thereof. This design ensures that even if an attacker obtains both the encrypted model file and the license file, they cannot decrypt the model file because they lack the user password and therefore cannot recover the master key. Similarly, even if an attacker obtains the key file, they cannot pass the authorization verification due to the lack of a validly signed license file, and the service will still fail to start normally. This achieves a physical separation between the delivery of decryption capabilities and the delivery of the authorization policy, significantly improving security.

[0053] When loading the model on the deployment side, the system first prompts the user to enter a decryption password, then uses PBKDF2 to derive a key from the password and attempts to decrypt the key file to recover the master key. If the password is incorrect, the AES-GCM authentication tag verification will fail, the system will report a password error, and terminate the loading process. After successful password verification, the system calls the authorization management module to verify the authorization file, including verifying the validity of the digital signature using a preset RSA public key, checking whether the current system time is within the authorization validity period, and executing the hardware fingerprint comparison process described in Example 2. After all verifications are successful, the system creates a system-level temporary directory as the decryption target path. This temporary directory is created by the operating system's temporary file management mechanism and is usually located at / tmp or under the path specified by the environment variable TMPDIR.

[0054] The system reads the encryption manifest file and decrypts each encryption model file sequentially to a temporary directory. When decrypting each file, the system first reads the original filename from the file header and then uses that filename as associated data to input into the AES-GCM decryption algorithm. If a file is renamed during transmission or storage, or if the original filename recorded in the file header is tampered with, the associated data during decryption will be inconsistent with that during encryption, causing authentication tag verification to fail. The system will then report an integrity error and terminate the loading process. This mechanism effectively prevents attackers from carrying out packet-swapping attacks by renaming files. After decryption, the system also calculates the SHA256 hash value of the decrypted content and compares it with the original hash value recorded in the file header, providing double integrity protection.

[0055] After all model files are decrypted, the system starts the vLLM inference service and points the model loading path to the temporary directory. Once the inference service starts, the model weights are loaded into GPU memory or system memory, at which point the plaintext files in the temporary directory are no longer accessed. When the inference service terminates normally or receives an exit signal (such as SIGTERM or SIGINT), the system's cleanup module automatically deletes all plaintext files in the temporary directory. Through this secure loading process that minimizes the plaintext window, the plaintext model weights exist on disk only for a short time during service startup and remain in memory during runtime, greatly reducing the risk of attackers obtaining model weights through static disk analysis.

[0056] Example 4 This embodiment describes a non-complete deployment architecture based on network structure, which provides stronger asset protection capabilities by physically separating model parameters.

[0057] In this embodiment, the model parameters of the large language model are divided into two parts: local main parameters and remote key parameters. The local main parameters are deployed on the customer's local server and account for approximately 99% of the total model parameters (in other embodiments, this could be another percentage, such as 95%, 99.8%, etc.). These include the model's word embedding layer, most of the attention weights and feedforward network weights of the Transformer layers, and the output layer weights. The remote key parameters are not included in the local deployment package delivered to the customer but are hosted on a cloud server by the model provider, accounting for approximately 1% of the total model parameters.

[0058] This embodiment supports two types of remote key parameter configuration. The first type is suitable for large language models employing a Mixture of Experts (MoE) architecture. The MoE architecture expands model capacity by introducing multiple parallel expert networks and gating networks. During inference, the gating network dynamically selects and activates some expert networks based on the input. This embodiment uses the weights of specific expert networks in the MoE architecture as remote key parameters; for example, it selects the expert network with the highest gating score in each layer, as these expert networks have a crucial impact on the model's output quality. The second type is suitable for large language models fine-tuned using Low-Rank Adaptation (LoRA). LoRA technology achieves efficient fine-tuning by injecting low-rank matrices into specific layers of the pre-trained model. These low-rank matrices typically account for only 0.1% to 1% of the original model's parameters, but they play a decisive role in the model's performance on specific tasks. This embodiment uses the weight matrix of the LoRA adapter as the remote key parameter.

[0059] During inference computation, when the computation path involves remote key parameters, the local inference engine communicates with the cloud-based collaborative computing service through an encrypted channel. This embodiment uses Mutual Transport Layer Security (mTLS) to establish the encrypted channel. Both the client and server must provide digital certificates for two-way authentication to prevent man-in-the-middle attacks. The communication protocol uses the gRPC framework, supporting efficient binary serialization and streaming.

[0060] This embodiment provides two collaborative computing modes. The first mode is the remote computing mode, where the local inference engine sends the intermediate activation values ​​required for calculating remote key parameters to the cloud server via an encrypted channel. The cloud server uses the remote key parameters to complete the corresponding matrix operations and returns the calculation results to the local machine, allowing the local inference engine to continue executing subsequent calculations. The second mode is the dynamic fetch mode, where, upon startup of the inference service, the local inference engine requests remote key parameters from the cloud server via an encrypted channel. After verifying authorization, the cloud server encrypts and transmits the parameters to the local machine. The local inference engine loads the received remote key parameters into a trusted execution environment (Enclave) created by Intel Software Guard Extensions (SGX) for transient storage. The Enclave provides hardware-level memory isolation and encryption protection, preventing attackers from reading data within it even if they gain root privileges on the operating system. After the inference service terminates, the Enclave is destroyed, and the remote key parameters are removed from the local environment.

[0061] Even without remote key parameters, local parameters can still form an executable computation graph and perform inference. However, due to the absence of a key expert network or LoRA adapter, the accuracy of the output will decrease to some extent. This design ensures that even if an attacker obtains the complete local deployment package and bypasses authorization verification, they will only obtain a significantly degraded model. Furthermore, if the environmental monitoring task detects a connection interruption to the cloud server or authentication failure, the system will automatically trigger the controlled interference mode described in Example 1. This uses computational perturbation strategies as a further layer of control to mitigate the decrease in model accuracy, further deteriorating the output quality under unauthorized use, thus providing a dual protection mechanism.

[0062] Example 5 This embodiment provides a large-scale model encryption deployment and operation control system to implement the steps in the methods described in Embodiments 1 to 4 above.

[0063] The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor is coupled to the memory to implement the steps of the above method when executing the computer-executable instructions.

[0064] In one specific implementation, the system is deployed on a server equipped with a high-performance GPU. The server has at least 256GB of ECC memory for storing intermediate computation results and key-value caches during model inference. The server also has at least 2TB of NVMe solid-state drives as storage for storing encrypted model files, key files, license files, and computer-executable instructions required for system operation. The server is equipped with a high-performance GPU with a memory capacity matching the model size (ideally large enough to hold the entire model) to accelerate model inference computation.

[0065] The computer-executable instructions stored in the memory include the following functional modules: The key management module is responsible for implementing the key derivation and master key recovery functions described in Embodiment 3, including the implementation of the PBKDF2 key derivation algorithm, the decryption logic of the key file, and the secure storage of the master key. The authorization verification module is responsible for implementing the signature verification of the authorization file, validity period check, and the multi-level hardware fingerprint acquisition and fault-tolerant matching functions described in Embodiment 2. The model decryption module is responsible for implementing the AES-256-GCM decryption function described in Embodiment 3, including the extraction and verification of associated data, verification of authentication tags, and integrity check of the decrypted content. The secure loading module is responsible for implementing the temporary directory creation, model file decryption and loading, and file cleanup functions when the service terminates, as described in Embodiment 3. The inference service module is responsible for loading the decrypted model weights and providing external API service interfaces; this embodiment uses vLLM as the core engine of the inference service module. The environment monitoring module is responsible for implementing the watchdog thread function described in Embodiment 1, including periodic environment verification, management of status flags, and exception triggering logic. The interference execution module is responsible for implementing the three computational perturbation strategies described in Embodiment 1, including the interception of logits vectors, noise injection, high-optimal masking, and the execution of the distributed entropy increase algorithm. The cloud communication module is responsible for implementing the mTLS encrypted channel establishment, remote computing request sending and receiving, and dynamic parameter retrieval functions described in Embodiment 4.

[0066] When the processor executes the aforementioned computer-executable instructions, it first calls the key management module and the authorization verification module to complete the security verification during the startup phase. Then, it calls the model decryption module and the secure loading module to decrypt the encrypted model to a temporary directory and start the inference service module. During the operation of the inference service module, the environment monitoring module runs in parallel as an independent thread, periodically performing environment verification. If the environment monitoring module detects an anomaly, it notifies the interference execution module to activate the controlled interference mode. If the system is configured with cloud collaboration functionality, the cloud communication module establishes a connection with the cloud server during the startup phase and performs remote computation or parameter retrieval operations as needed during the inference process.

[0067] Furthermore, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the various method embodiments of this application. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. As defined herein, computer-readable storage media do not include transient computer-readable media, such as modulated data signals and carrier waves.

[0068] Furthermore, embodiments of this application also provide a large-scale encrypted deployment and operation control system, including a memory for storing computer-executable instructions and a processor; the processor is used to implement the steps in the above-described method embodiments when executing the computer-executable instructions in the memory. The processor may be a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processor (DSP), Microcontroller Unit (MCU), Neural Processing Unit (NPU), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), or other programmable logic devices. The aforementioned memory may be read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or solid-state drive, etc. The steps of the methods disclosed in the embodiments of this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0069] Furthermore, embodiments of this application also provide a computer program product, including computer-executable instructions that, when executed by a processor, implement the steps in the above-described method embodiments.

[0070] It should be noted that in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this application, if it refers to performing an action according to an element, it means performing the action at least according to that element, including two cases: performing the action only according to that element, and performing the action according to that element and other elements. Expressions such as "multiple," "repeatedly," and "various" include two, two times, two kinds, and more than two, more than two times, and more than two kinds.

[0071] This specification includes combinations of various embodiments described herein. Individual references to embodiments are made (e.g., "one embodiment," "some embodiments," or "preferred embodiments"); however, these embodiments are not mutually exclusive unless indicated to be mutually exclusive or are readily apparent to those skilled in the art. It should be noted that the word "or" is used in a non-exclusive sense throughout this specification unless the context explicitly indicates or requires it.

Claims

1. A method for encrypted deployment and operation control of a large-scale model, characterized in that, Includes the following steps: Load the encrypted large language model file on the deployment side and start the model inference service; During the operation of the model inference service, parallel environment monitoring tasks are performed to verify in real time the matching status between the current operating environment and the preset authorization conditions; When the matching status is verified to be abnormal, the process running state of the model inference service and its interface response capability to external requests are maintained, but the running mode of the model inference service is switched to controlled interference mode. In the controlled interference mode, the intermediate probability distribution data of the model inference service during the inference calculation process is intercepted, and a calculation perturbation strategy is executed on the intermediate probability distribution data to disrupt the semantic logic of the model-generated content and achieve implicit degradation of the output results.

2. The method according to claim 1, characterized in that, The intermediate probability distribution data is the log-odds vector of the model before sampling by the normalized exponential function. The calculation of the perturbation strategy on the intermediate probability distribution data includes one or a combination of the following operations: Random noise data conforming to a preset distribution pattern is superimposed on the log-probability numerical vector to disrupt the original word prediction probability ranking. Identify the multiple candidate words with the highest probabilities in the log probability vector, and force their corresponding values ​​to a preset mask value, thereby forcing the sampling algorithm to select words with lower probabilities. This reduces the difference between the maximum and minimum values ​​in the log-odds vector, decreases the peak significance of the probability distribution, and increases the randomness and perplexity of the output results.

3. The method according to claim 1, characterized in that, Maintaining the running state of the model inference service process and its interface response capability to external requests specifically includes: The application programming interface controlling the model inference service continuously returns a status code indicating a successful request to the calling client. The returned data packet structure remains consistent with that in normal mode, and does not contain explicit error messages, so that the calling end cannot identify the blocking behavior through network-level status codes or protocol formats.

4. The method according to claim 1, characterized in that, The step of verifying the matching status between the current operating environment and the preset authorization conditions adopts a multi-level fingerprint fault tolerance mechanism: Collect the multi-dimensional hardware fingerprint of the currently deployed terminal, and extract the core fingerprint subset for comparison according to the fault tolerance level defined in the authorization file; The controlled interference mode is triggered only when the core fingerprint subset does not match, or when the current system time exceeds the authorization validity period. If the non-core fingerprint changes but a subset of the core fingerprint matches, the normal operating mode will continue.

5. The method according to claim 1, characterized in that, The step of loading the encrypted large language model file also includes: Read the filename or path information of the encrypted model file as associated data; The model file is decrypted using an associated data authentication encryption algorithm, combined with the decryption key and the associated data; If the model file is renamed, causing a mismatch between the associated data during decryption and the associated data during encryption, the decryption will fail and the loading process will be terminated.

6. The method according to claim 1, characterized in that, The encrypted large language model file is protected using a key management system that separates delivery and authorization, specifically including: The model file is encrypted using a randomly generated master key; Based on the key derivation function, the master key is encrypted a second time using the password set by the user to generate a key file; The authorization file contains only authorization policy information and digital signature, but does not contain the master key or its derived information; When the deployment end is loaded, the correct password must be provided to recover the master key, and an authorization file verified by signature must be provided to confirm the validity of the authorization, so as to achieve physical separation of decryption capability delivery and authorization policy delivery.

7. The method according to claim 1, characterized in that, The step of loading the encrypted large language model file also includes a secure loading process that minimizes the plaintext window: After authorization verification is successful, a system-level temporary directory is created as the decryption target path; The encrypted model file is decrypted to the temporary directory, rather than the persistent storage directory; Start the model inference service and point to the decrypted file in the temporary directory; When the model inference service terminates or receives an exit signal, all plaintext files in the temporary directory are automatically cleared, so that the plaintext model exists on the disk only for a controlled, short time window.

8. The method according to any one of claims 1-7, characterized in that, The step of loading the encrypted large language model file adopts a non-complete deployment architecture based on network structure, specifically including: The model parameters of the large language model are divided into local main parameters and remote key parameters; the local main parameters are deployed on the deployment terminal, and the remote key parameters are not included in the local deployment package; During the inference computation process, when the computation path involves the remote key parameters, a cloud collaborative computation is requested through an encrypted channel, or the remote key parameters are dynamically pulled from a protected remote source to the local trusted execution environment for transient computation. If the environmental monitoring task detects a connection interruption or authentication failure with a remote source, the controlled interference mode is automatically triggered.

9. The method according to claim 8, characterized in that, The remote key parameters include specific expert network weights in a hybrid expert model architecture, or specific adaptation layer weights based on low-rank adapter technology. The local subject parameters can still form an executable computation graph and perform inference even in the absence of the remote key parameters, but the absence of the remote key parameters will cause the accuracy of the output results to decrease to a certain extent. The computational perturbation strategy serves as a further superimposed control for this decrease in accuracy.

10. A large-scale encrypted deployment and operation control system, characterized in that, include: Memory is used to store executable instructions for a computer; as well as, A processor, coupled to the memory, is configured to implement the steps of the method as described in any one of claims 1 to 9 when executing the computer-executable instructions.

Images