A large model inference deployment method, system, device, storage medium and product
By segmenting the vocabulary of a large model and combining it with MAC address and asymmetric encryption, the security problem of deploying large models on external servers is solved, and the secure transmission and deployment of model files are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD
- Filing Date
- 2026-02-12
- Publication Date
- 2026-06-02
Smart Images
Figure CN122137541A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of large model technology, and in particular to a method, system, device, storage medium and product for deploying large model inference. Background Technology
[0002] Currently, the mainstream solution for deploying large model inference is to deploy the large model application on its own server or cluster, only publicly expose the service address after deployment, and set an API key to access the large model service. Only users who provide an authorized API key can access it.
[0003] Existing technologies avoid the risk of model file leakage during large model deployment by isolating the environment. However, in actual business scenarios, it may be necessary to deploy your model on someone else's server or cluster environment. Conventional inference deployment solutions have low portability and are difficult to adapt to the needs of actual business scenarios. Summary of the Invention
[0004] The embodiments of the present invention aim to provide a method, system, device, storage medium and product for deploying large model inference, which can improve the security of large models, effectively avoid the leakage of source code during the migration and deployment of large models, and prevent data from being tampered with or stolen during transmission.
[0005] In a first aspect, embodiments of the present invention provide a method for deploying large-scale model inference, comprising: The vocabulary of the large model is divided into blocks to obtain several plaintext blocks; Generate an initialization vector based on the MAC address of the server to be deployed; A first key is randomly generated. Based on the initialization vector and the first key, the plaintext blocks are encrypted using the ciphertext block chaining mode to obtain several ciphertext blocks. The first key is encrypted using the public key to obtain the second key; The second key and the ciphertext block are sent to the server to be deployed, so that the server to be deployed can use its private key to decrypt the second key and the ciphertext block, obtain the vocabulary of the large model, and perform inference deployment.
[0006] As an improvement to the above scheme, the vocabulary of the large model is divided into blocks to obtain several plaintext blocks, including: Obtain the original vocabulary and corresponding original weight matrix of the large model; The original vocabulary is obfuscated to obtain an obfuscated vocabulary, and the original weight matrix is obfuscated according to the obfuscation dimension of the original vocabulary to obtain an obfuscated weight matrix; The obfuscation word list is divided into blocks to obtain several plaintext blocks.
[0007] As an improvement to the above scheme, the original vocabulary is obfuscated to obtain an obfuscated vocabulary, and the original weight matrix is obfuscated according to the obfuscation dimension of the original vocabulary to obtain an obfuscated weight matrix, including: The original vocabulary is obfuscated by swapping odd and even elements to obtain an obfuscated vocabulary. The original weight matrix is obfuscated by using an odd-even matrix row swapping method to obtain an obfuscated weight matrix.
[0008] As an improvement to the above scheme, the obfuscation vocabulary is divided into blocks to obtain several plaintext blocks, including: According to the preset byte length, the obfuscation word list is divided into blocks in sequence to obtain several plaintext blocks; If the length of the last plaintext block is less than the preset byte length, the last plaintext block is padded to the preset byte length.
[0009] As an improvement to the above scheme, the first key is randomly generated, and based on the initialization vector and the first key, the plaintext blocks are encrypted using a ciphertext block chaining mode to obtain several ciphertext blocks, including: The first key is randomly generated using a symmetric encryption algorithm; If the plaintext block is the first plaintext block, then the plaintext block and the initialization vector are XORed to obtain the first string, and the first string is encrypted using the first key to obtain the ciphertext block corresponding to the plaintext block; If the plaintext block is not the first plaintext block, then the plaintext block is XORed with the ciphertext block generated from the previous plaintext block to obtain a second string. The second string is then encrypted using the first key to obtain the ciphertext block corresponding to the plaintext block.
[0010] As an improvement to the above scheme, methods for deploying large models for inference on the server to be deployed include: In response to the second key and ciphertext blocks, the local MAC address is verified; Once the verification is successful, an initialization vector is generated based on the local machine's MAC address, and the second key is decrypted using the private key corresponding to the public key to obtain the first key; Using the initialization vector and the first key, the ciphertext blocks are decrypted to obtain several plaintext blocks of the large model vocabulary; Based on the plaintext blocks, the large model is inferred and deployed.
[0011] Secondly, embodiments of the present invention provide a large-model inference deployment system, including: The vocabulary segmentation module is used to segment the vocabulary of a large model into several plaintext segments; The initialization vector generation module is used to generate an initialization vector based on the MAC address of the server to be deployed. The plaintext block encryption module is used to randomly generate a first key, and encrypt the plaintext block according to the initialization vector and the first key, using the ciphertext block chaining mode to obtain several ciphertext blocks; An asymmetric encryption module is used to encrypt the first key using a public key to obtain a second key; The model encryption deployment module is used to send the second key and the ciphertext block to the server to be deployed, so that the server to be deployed can use its private key to decrypt the second key and the ciphertext block to obtain the vocabulary of the large model and perform inference deployment.
[0012] Thirdly, embodiments of the present invention provide a large model inference deployment device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the large model inference deployment method as described above.
[0013] Fourthly, embodiments of the present invention provide a computer-readable storage medium comprising a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the large model inference deployment method as described above.
[0014] Fifthly, embodiments of the present invention provide a computer program product, the computer program product including a computer program or computer instructions, wherein when the computer program or computer instructions are executed by a processor, the large model inference deployment method described above is performed.
[0015] Compared with existing technologies, the present invention discloses a method, system, device, storage medium, and product for deploying large-scale model inference. This method involves dividing the vocabulary of a large-scale model into blocks to obtain several plaintext blocks; generating an initialization vector based on the MAC address of the server to be deployed; randomly generating a first key; encrypting the plaintext blocks using a ciphertext block chaining mode based on the initialization vector and the first key to obtain several ciphertext blocks; encrypting the first key using a public key to obtain a second key; and sending the second key and the ciphertext blocks to the server to be deployed, enabling the server to decrypt the second key and the ciphertext blocks using its private key to obtain the vocabulary of the large-scale model and perform inference deployment. Using this invention, the security of large-scale models can be improved, effectively avoiding the leakage of source code during the migration and deployment of large-scale models, and preventing data from being tampered with or stolen during transmission. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating the steps of a large model inference deployment method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a primitive vocabulary obfuscation method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the steps for encrypting an obfuscated word list according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a large model inference deployment system provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a large model inference deployment device provided in an embodiment of the present invention. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] In the description and claims, it should be understood that the terms "first," "second," etc., used in the description and claims are only for the purpose of distinguishing the description of the same technical features, and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated, nor necessarily the order of description or chronological order. The terms are interchangeable where appropriate. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature.
[0019] Deploying large models on proprietary servers or clusters and providing API interfaces for external service access can effectively prevent model file leakage. However, in this architecture, data from external servers needs to flow through the servers where the large model is deployed and be centrally processed by a centralized server. This results in huge resource requirements and consumption for the deployment servers, making it difficult to meet the high-performance needs of all external servers.
[0020] Therefore, in real-world business scenarios, it may be necessary to deploy large models on external servers or in a cluster environment. The current method of providing API interfaces makes it difficult to deploy large models securely, conveniently, and reliably on external servers.
[0021] Based on the above considerations, this invention provides a method for deploying large-scale model inference. Please refer to [link to relevant documentation]. Figure 1In this embodiment, the large model inference deployment method is specifically executed through steps S1 to S5: S1. Divide the vocabulary of the large model into blocks to obtain several plaintext blocks; S2. Generate an initialization vector based on the MAC address of the server to be deployed; S3. Randomly generate a first key, and encrypt the plaintext blocks using the ciphertext block chaining mode according to the initialization vector and the first key to obtain several ciphertext blocks; S4. Encrypt the first key using the public key to obtain the second key; S5. Send the second key and the ciphertext block to the server to be deployed, so that the server to be deployed can use its private key to decrypt the second key and the ciphertext block, obtain the vocabulary of the large model, and perform inference deployment.
[0022] The vocabulary is one of the core components in the design and training of large models. Its role is to map discrete symbols to continuous numerical values that the model can process in a two-way manner.
[0023] In real-world business scenarios, the physical space of a vocabulary file can be quite large, requiring sufficient computing power to process the entire vocabulary directly. In this embodiment of the invention, by dividing the vocabulary of a large model into blocks, the large vocabulary file can be split into plaintext blocks that are easy to stream process and encrypt. This reduces computing power requirements and improves the overall efficiency of large model inference deployment.
[0024] The vocabulary contains a large amount of linguistic knowledge and structural information. If the vocabulary is transmitted and stored directly in plaintext, there is a risk of the theft of large models.
[0025] In this embodiment of the invention, the unique hardware identifier of the server to be deployed, i.e., the MAC address, is used as the seed for generating the initialization vector, thus binding the encryption process to the hardware of the server to be deployed. Furthermore, based on the initialization vector and a randomly generated first key, plaintext blocks of the vocabulary are encrypted, improving the security of large models.
[0026] Cipher Block Chaining (CBC) is a symmetric encryption mode that XORs plaintext blocks with the previous ciphertext block before encryption, ensuring that the generation of each ciphertext block depends on all preceding blocks. In this mode, even if the plaintext exhibits a repeating pattern, the ciphertext will not show a repeating pattern, and tampering with any ciphertext block will cause all subsequent decryption attempts to fail.
[0027] In this embodiment of the invention, the plaintext blocks are encrypted using a ciphertext grouping and linking mode, which enables high-strength protection of the content and integrity of the vocabulary.
[0028] Under symmetric encryption (CBC), the same key is used for both encryption and decryption, i.e., the first key. If the first key is leaked, data security is compromised. Therefore, this invention, based on CBC, further employs an asymmetric encryption method. The first key is encrypted using a public key to obtain a second key. If other servers / terminals want to obtain the first key, they must first use a pre-agreed private key corresponding to the encryption public key to decrypt the second key.
[0029] In some preferred embodiments, the RSA algorithm is used to generate the public and private keys.
[0030] The process of asymmetric encryption of the first key will not affect the encryption process of the ciphertext block chaining mode; and since only the first key needs to be encrypted, the amount of encrypted data is very small, and it will not affect the decryption performance of the server to be deployed.
[0031] For any server, if its MAC address differs from the one used for encryption, or if its private key does not correspond to the public key used for encryption, the vocabulary for a large model cannot be obtained. Therefore, based on hardware binding and asymmetric key distribution during encryption, it can be seen that the server's legitimate identity is verified during decryption; only the server to be deployed can correctly decrypt and obtain a usable vocabulary. The process of the server to be deployed decrypting ciphertext blocks corresponds to the encryption process and will not be elaborated here.
[0032] In the above scheme, by performing hardware-identified block encryption on the vocabulary of the large model, the security of the large model can be improved, the source code leakage can be effectively avoided during the migration and deployment of the large model, and the data can be prevented from being tampered with or stolen during transmission.
[0033] As a preferred implementation, step S1 involves dividing the vocabulary of the large model into blocks to obtain several plaintext blocks, which are then executed through steps S11-S13: S11. Obtain the original vocabulary and corresponding original weight matrix of the large model; S12. Obfuscate the original vocabulary to obtain an obfuscated vocabulary, and obfuscate the original weight matrix according to the obfuscation dimension of the original vocabulary to obtain an obfuscated weight matrix; S13. Divide the obfuscated word list into blocks to obtain several plaintext blocks.
[0034] The original vocabulary is an ordered list, where each element is assigned a unique integer ID. Preferably, the elements in this original list are tokens, which can represent words, characters, symbols, or any indivisible element in the text.
[0035] By obfuscating the original vocabulary, attackers cannot directly analyze its composition and structure even if they obtain the vocabulary file, and cannot infer the statistical characteristics of the training data by simply analyzing the vocabulary order, which increases the difficulty of reverse engineering.
[0036] The original weight matrix is one The original weight matrix is a matrix, where m is the length of the original vocabulary and n is the dimension of the embedding vectors. The i-th row of the original weight matrix corresponds to the embedding representation of the element with ID i in the original vocabulary. The original weight matrix enables the lookup operation of the original vocabulary and is the foundation for correct reasoning during vocabulary loading.
[0037] In some preferred embodiments, the original weight matrix includes lm_head.weight and model.embed_tokens.weight. Here, lm_head.weight is the weight of the output layer of the large model (the layer that predicts the next word), and model.embed_tokens.weight is the weight of the word embedding layer of the large model, that is, the part of the model that converts the input words into numerical vectors.
[0038] To ensure that the server to be deployed can perform normal inference during the loading of the obfuscated vocabulary, this embodiment of the invention further obfuscates and scrambles the original weight matrix of the large model in a regular manner along the same dimension.
[0039] In the above scheme, by obfuscating the original vocabulary and the original weight matrix, it can be ensured that the model can load the obfuscated vocabulary file normally and complete normal inference output. This mechanism provides an effective way to enhance the robustness and security of the model while ensuring its normal operation.
[0040] Further, preferably, step S12, which involves obfuscating the original vocabulary to obtain an obfuscated vocabulary, and obfuscating the original weight matrix according to the obfuscation dimension of the original vocabulary to obtain an obfuscated weight matrix, includes: The original vocabulary is obfuscated by swapping odd and even elements to obtain an obfuscated vocabulary. The original weight matrix is obfuscated by using an odd-even matrix row swapping method to obtain an obfuscated weight matrix.
[0041] In a preferred embodiment, such as Figure 2As shown, the original vocabulary is {T0:0,T1:1,…,Tm-1:m-1,Tm:m}. After swapping the odd and even elements, the confusion vocabulary is {T1:0,T0,1,…,Tm:m-1,Tm-1:m}.
[0042] It should be noted that if the length m of the original vocabulary is even, all m elements will be obfuscated during the obfuscation process; if the length m of the original vocabulary is odd, only m-1 elements need to be obfuscated.
[0043] Similarly, in this embodiment of the invention, when obfuscating the original weight matrix, the elements of the odd-numbered rows are swapped with the adjacent even-numbered rows while maintaining their overall order. If the number of rows in the original weight matrix is even, all rows are obfuscated; if the number of rows in the original weight matrix is odd, rows with a row number of -1 are obfuscated.
[0044] The embodiments of this invention provide a simplified obfuscation algorithm that groups adjacent odd and even elements / rows together and obfuscates them within the group. In practical applications, two or more non-adjacent elements can also be grouped together for obfuscation. In this case, the original weight matrix needs to be obfuscated accordingly to ensure that the server to be deployed can correctly load the obfuscated vocabulary. The specific obfuscation algorithms for the original vocabulary and the original weight matrix do not affect the beneficial effects produced by the embodiments of this invention.
[0045] In the above scheme, the original vocabulary and the original weight matrix adopt the same exchange pattern, which can achieve the concealment of the vocabulary structure at the lowest cost, providing a foundation for subsequent encryption. At the same time, it significantly increases the difficulty for attackers to reverse engineer, enhances the sensitivity of large models to input, and improves the robustness and security of large models.
[0046] Preferably, step S13 involves dividing the obfuscation word list into blocks to obtain several plaintext blocks, including: According to the preset byte length, the obfuscation word list is divided into blocks in sequence to obtain several plaintext blocks; If the length of the last plaintext block is less than the preset byte length, the last plaintext block is padded to the preset byte length.
[0047] It should be noted that the preset byte length is determined based on encryption algorithm requirements, network transmission efficiency, and storage alignment factors, and the specific selection of the preset byte length value does not affect the beneficial effects produced by the embodiments of the present invention.
[0048] During implementation, the obfuscation word list is divided into blocks, resulting in n plaintext blocks (chunk1, chunk2, ..., chunkn), each consisting of 16 bytes. When the length of the last plaintext block is less than 16 bytes, the last plaintext block is padded to ensure that each block is 128 bits.
[0049] In some preferred embodiments, when the length of the last plaintext block is less than the preset byte length, PKCS5Padding is used to pad it.
[0050] In the above scheme, the size of the obfuscated vocabulary blocks is aligned after they are divided so that plaintext blocks of uniform size can enter the subsequent encryption process, and the information of the original vocabulary is not leaked.
[0051] In a preferred implementation, step S3 involves randomly generating a first key, and encrypting the plaintext blocks using a ciphertext block chaining mode based on the initialization vector and the first key, resulting in several ciphertext blocks, including: The first key is randomly generated using a symmetric encryption algorithm; If the plaintext block is the first plaintext block, then the plaintext block and the initialization vector are XORed to obtain the first string, and the first string is encrypted using the first key to obtain the ciphertext block corresponding to the plaintext block; If the plaintext block is not the first plaintext block, then the plaintext block is XORed with the ciphertext block generated from the previous plaintext block to obtain a second string. The second string is then encrypted using the first key to obtain the ciphertext block corresponding to the plaintext block.
[0052] In some preferred embodiments, the first key is randomly generated using the AES (Advanced Encryption Standard) algorithm.
[0053] In ciphertext block-linked mode, the encryption of each plaintext block must be performed based on the previous ciphertext block. In typical scenarios, the first plaintext block is usually encrypted using a randomly generated initialization vector.
[0054] In embodiments of the present invention, such as Figure 3 As shown, an initialization vector generated based on the MAC address of the server to be deployed is XORed with the first plaintext block, and then the subsequent encryption process using the first key is performed to generate the first ciphertext block. Since the MAC address of each machine is unique, the initialization vector generated in this way participates in the encryption operation, ensuring that all generated ciphertext blocks are hardware-bound to the server to be deployed, thus guaranteeing the security of the large model.
[0055] For subsequent plaintext blocks, simply XOR them sequentially with the previous ciphertext block, and then encrypt them using the first key. Finally, the first key and all the ciphertext blocks (block1, block2, ..., blockn) form an encryption vocabulary, which can be used to decode the ciphertext blocks using the first key and the MAC address.
[0056] The above scheme ensures machine selectivity and flexibility in encrypting files by using the machine's unique MAC address to generate the initialization vector. In its implementation, it also incorporates the AES algorithm's CBC mode and PKCS5 padding to protect data confidentiality and integrity, preventing data from being tampered with or stolen during transmission.
[0057] As a preferred implementation, the method for deploying a large model for inference on the server to be deployed includes: In response to the second key and ciphertext blocks, the local MAC address is verified; Once the verification is successful, an initialization vector is generated based on the local machine's MAC address, and the second key is decrypted using the private key corresponding to the public key to obtain the first key; Using the initialization vector and the first key, the ciphertext blocks are decrypted to obtain several plaintext blocks of the large model vocabulary; Based on the plaintext blocks, the large model is inferred and deployed.
[0058] Preferably, the server to be deployed receives an SO file, which restricts the script's execution permissions and validity period through authorization management. The SO file contains obfuscated, encrypted, and authorized scripts, the contents of which cannot be directly read or understood. This SO file can be executed to complete the final loading of the model and vocabulary, as well as the subsequent inference process.
[0059] For example, the Pyarmor library in Python can be used to convert raw Python scripts into an encrypted format to prevent source code leakage. The Pyarmor library works by converting the Python script into an unreadable format, making it impossible for potential attackers to directly read or understand the source code. This conversion process is essentially an obfuscation process; by transforming and reconstructing the source code, it significantly increases the difficulty of cracking it.
[0060] Upon receiving a data packet containing the second key and ciphertext blocks, the server to be deployed reads its own network interface card's (NIC) local MAC address and compares it with MAC address information stored internally or extracted from the data packet. Only when the MAC address matches the one used in the encryption process can the generated initialization vector correctly decrypt the vocabulary. This process ensures that decryption operations are only allowed to be performed on authorized devices and avoids meaningless decryption calculations in cases of MAC mismatch.
[0061] In some preferred embodiments, when decrypting ciphertext blocks, for the first ciphertext block, the first key and initialization vector are used for decryption, and the decrypted result is XORed with the first ciphertext block to obtain the first plaintext block; for non-first ciphertext blocks, the first key and the previous ciphertext block are used for decryption, and the decrypted result is XORed with the previous ciphertext block to obtain the corresponding plaintext block.
[0062] Once all plaintext blocks have been decrypted, inference deployment can proceed. The inference deployment process mainly involves loading the vocabulary and initializing the word segmenter.
[0063] In some preferred embodiments, a cache is created and the decrypted obfuscated vocabulary is stored in the cache. The AutoTokenizer module in the transformers library is used to load this cache to obtain an initialized tokenizer. After successfully loading the vocabulary and initializing the tokenizer, the created cache is deleted, thus enabling the inference deployment of large models.
[0064] In this embodiment of the invention, the encrypted vocabulary can be loaded normally. Upon successful loading, the decrypted file is deleted, ensuring the vocabulary is not visible to external systems. Model weights can then be loaded normally. This method provides a secure mechanism to prevent unauthorized access and use of the decrypted vocabulary file.
[0065] The above scheme ensures that large models can only be deployed for inference on the corresponding server under a secure state, and the server can automatically decrypt the ciphertext blocks and automatically load the vocabulary, simplifying the process of deploying large models for inference.
[0066] The large model inference deployment method provided in this embodiment of the invention improves the security of the large model by performing hardware-identified block encryption on the vocabulary of the large model, effectively avoids the leakage of source code during the migration and deployment of the large model, and prevents data from being tampered with or stolen during transmission.
[0067] This invention provides a large-model inference deployment system. Please refer to [link / reference]. Figure 3The large model inference deployment system includes a vocabulary segmentation module 11, an initialization vector generation module 12, a plaintext segmentation encryption module 13, an asymmetric encryption module 14, and a model encryption deployment module 15, wherein: The vocabulary segmentation module 11 is used to segment the vocabulary of the large model into several plaintext segments; Initialization vector generation module 12 is used to generate an initialization vector based on the MAC address of the server to be deployed; Plaintext block encryption module 13 is used to randomly generate a first key, and encrypt the plaintext block according to the initialization vector and the first key, using ciphertext block linking mode to obtain several ciphertext blocks; The asymmetric encryption module 14 is used to encrypt the first key using a public key to obtain the second key; The model encryption deployment module 15 is used to send the second key and the ciphertext block to the server to be deployed, so that the server to be deployed can use its private key to decrypt the second key and the ciphertext block to obtain the vocabulary of the large model and perform inference deployment.
[0068] In a preferred embodiment, the vocabulary segmentation module 11 includes: The raw data acquisition unit is used to acquire the original vocabulary and corresponding original weight matrix of the large model; The obfuscation operation unit is used to obfuscate the original vocabulary to obtain an obfuscated vocabulary, and to obfuscate the original weight matrix according to the obfuscation dimension of the original vocabulary to obtain an obfuscated weight matrix. The word segmentation unit is used to segment the obfuscated word list into several plaintext segments.
[0069] Further, preferably, the obfuscation operation unit is specifically used for: The original vocabulary is obfuscated by swapping odd and even elements to obtain an obfuscated vocabulary. The original weight matrix is obfuscated by using an odd-even matrix row swapping method to obtain an obfuscated weight matrix.
[0070] Preferably, the vocabulary segmentation unit is specifically used for: According to the preset byte length, the obfuscation word list is divided into blocks in sequence to obtain several plaintext blocks; If the length of the last plaintext block is less than the preset byte length, the last plaintext block is padded to the preset byte length.
[0071] In a preferred embodiment, the plaintext block encryption module 13 is specifically used for: The first key is randomly generated using a symmetric encryption algorithm; If the plaintext block is the first plaintext block, then the plaintext block and the initialization vector are XORed to obtain the first string, and the first string is encrypted using the first key to obtain the ciphertext block corresponding to the plaintext block; If the plaintext block is not the first plaintext block, then the plaintext block is XORed with the ciphertext block generated from the previous plaintext block to obtain a second string. The second string is then encrypted using the first key to obtain the ciphertext block corresponding to the plaintext block.
[0072] In one preferred embodiment, the server to be deployed is used for: In response to the second key and ciphertext blocks, the local MAC address is verified; Once the verification is successful, an initialization vector is generated based on the local machine's MAC address, and the second key is decrypted using the private key corresponding to the public key to obtain the first key; Using the initialization vector and the first key, the ciphertext blocks are decrypted to obtain several plaintext blocks of the large model vocabulary; Based on the plaintext blocks, the large model is inferred and deployed.
[0073] The large model inference deployment system provided by this invention improves the security of large models by performing hardware-identified block encryption on the vocabulary of large models, effectively avoiding the leakage of source code during the migration and deployment of large models, and preventing data from being tampered with or stolen during transmission.
[0074] Please see Figure 4 , Figure 4 This is a structural block diagram of a large model inference deployment device provided in an embodiment of the present invention. The large model inference deployment device includes a processor 31, a memory 32, and a computer program stored in the memory 32 and executable on the processor 31. When the processor 31 executes the computer program, it implements the steps in the above-described embodiments of the various large model inference deployment methods, such as steps S1 to S5.
[0075] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 32 and executed by the processor 31 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the large-scale model inference deployment device.
[0076] The large model inference deployment device may include, but is not limited to, a processor 31 and a memory 32. Those skilled in the art will understand that the schematic diagram is merely an example of a large model inference deployment device and does not constitute a limitation on the device. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the large model inference deployment device may also include input / output devices, network access devices, buses, etc.
[0077] The processor 31 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 31 is the control center of the large-scale model inference deployment device, connecting various parts of the device via various interfaces and lines.
[0078] The memory 32 can be used to store the computer programs and / or modules. The processor 31 implements various functions of the large model inference deployment device by running or executing the computer programs and / or modules stored in the memory 32 and calling the data stored in the memory 32. The memory 32 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 32 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0079] If the modules / units integrated in the large model inference deployment device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 31, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.
[0080] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.
Claims
1. A method for deploying large-scale model inference, characterized in that, include: The vocabulary of the large model is divided into blocks to obtain several plaintext blocks; Generate an initialization vector based on the MAC address of the server to be deployed; A first key is randomly generated. Based on the initialization vector and the first key, the plaintext blocks are encrypted using the ciphertext block chaining mode to obtain several ciphertext blocks. The first key is encrypted using the public key to obtain the second key; The second key and the ciphertext block are sent to the server to be deployed, so that the server to be deployed can use its private key to decrypt the second key and the ciphertext block, obtain the vocabulary of the large model, and perform inference deployment.
2. The large-model inference deployment method as described in claim 1, characterized in that, The vocabulary of the large model is divided into blocks, resulting in several plaintext blocks, including: Obtain the original vocabulary and corresponding original weight matrix of the large model; The original vocabulary is obfuscated to obtain an obfuscated vocabulary, and the original weight matrix is obfuscated according to the obfuscation dimension of the original vocabulary to obtain an obfuscated weight matrix; The obfuscation word list is divided into blocks to obtain several plaintext blocks.
3. The large-model inference deployment method as described in claim 2, characterized in that, The process involves obfuscating the original vocabulary to obtain an obfuscated vocabulary, and then obfuscating the original weight matrix based on the obfuscation dimension of the original vocabulary to obtain an obfuscated weight matrix, including: The original vocabulary is obfuscated by swapping odd and even elements to obtain an obfuscated vocabulary. The original weight matrix is obfuscated by using an odd-even matrix row swapping method to obtain an obfuscated weight matrix.
4. The large model inference deployment method as described in claim 2, characterized in that, The process of segmenting the obfuscated vocabulary into blocks yields several plaintext blocks, including: According to the preset byte length, the obfuscation word list is divided into blocks in sequence to obtain several plaintext blocks; If the length of the last plaintext block is less than the preset byte length, the last plaintext block is padded to the preset byte length.
5. The method for deploying large-scale model inference as described in claim 1, characterized in that, The first key is randomly generated. Based on the initialization vector and the first key, the plaintext blocks are encrypted using a ciphertext block chaining mode to obtain several ciphertext blocks, including: The first key is randomly generated using a symmetric encryption algorithm; If the plaintext block is the first plaintext block, then the plaintext block and the initialization vector are XORed to obtain the first string, and the first string is encrypted using the first key to obtain the ciphertext block corresponding to the plaintext block; If the plaintext block is not the first plaintext block, then the plaintext block is XORed with the ciphertext block generated from the previous plaintext block to obtain a second string. The second string is then encrypted using the first key to obtain the ciphertext block corresponding to the plaintext block.
6. The method for deploying large-scale model inference as described in claim 1, characterized in that, Methods for deploying large models for inference on servers to be deployed include: In response to the second key and ciphertext blocks, the local MAC address is verified; Once the verification is successful, an initialization vector is generated based on the local machine's MAC address, and the second key is decrypted using the private key corresponding to the public key to obtain the first key; Using the initialization vector and the first key, the ciphertext blocks are decrypted to obtain several plaintext blocks of the large model vocabulary; Based on the plaintext blocks, the large model is inferred and deployed.
7. A large-scale model inference deployment system, characterized in that, include: The vocabulary segmentation module is used to segment the vocabulary of a large model into several plaintext segments; The initialization vector generation module is used to generate an initialization vector based on the MAC address of the server to be deployed. The plaintext block encryption module is used to randomly generate a first key, and encrypt the plaintext block according to the initialization vector and the first key, using the ciphertext block chaining mode to obtain several ciphertext blocks; An asymmetric encryption module is used to encrypt the first key using a public key to obtain a second key; The model encryption deployment module is used to send the second key and the ciphertext block to the server to be deployed, so that the server to be deployed can use its private key to decrypt the second key and the ciphertext block to obtain the vocabulary of the large model and perform inference deployment.
8. A large-scale model inference deployment device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the large model inference deployment method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium resides to perform the large model inference deployment method as described in any one of claims 1 to 6.
10. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, which, when executed by a processor, perform the large model inference deployment method as described in any one of claims 1 to 6.