Large model protection method and device, equipment and storage medium
By splitting the large model weight file into multiple data blocks and partially encrypted, combining multi-threaded decryption and data digest verification, the problem of overall encryption and decryption of large model weight file is solved, and the model loading speed and security are improved .
Patent Information
- Application Number
- CN202411079239.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-12
AI Technical Summary
Due to the large-scale size of the large model weight file, the overall encryption and decryption consumes a lot of time, resulting in a longer loading time of the model, affecting the user experience.
Split the big model weight file into multiple data blocks, and use the National Secret algorithm to partially encrypt each data block to generate encrypted data blocks, and ciphertext data is formed through data block index splicing, and multi-threading is used to decrypt. The decryption process is used to verify integrity through the data digest.
Reduces encryption and decryption time, improves model loading speed, improves user experience, and increases security through random encryption location and length to prevent data tampering.
Smart Images

Figure CN118940296A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a large model protection method, device, equipment and storage medium. Background Art
[0002] A large model refers to a deep learning model with large-scale parameters and complex computing structures. It is widely used in natural language processing, computer vision, speech recognition, and recommendation systems. The training process of a large model requires high equipment and labor costs, and the most valuable part of a large model is the large model weight file after training, which is also an important asset for enterprises to build competitiveness in artificial intelligence (AI). In order to prevent the model from being stolen or unauthorized access and use, it is usually necessary to protect the large model weight file.
[0003] A common method for protecting large models is to encrypt the large model weight file and then decrypt the encrypted file when using the large model weight file. The currently commonly used encryption method is usually to directly encrypt and decrypt the large model weight file as a whole. Since the large model weight file is generally tens of gigabytes (GB) in size and the file is too large, the overall encryption and decryption requires a lot of time, which leads to a longer model loading time and affects the user experience. Summary of the invention
[0004] In order to help solve the problem that large model weight files are too large and the overall encryption and decryption requires a lot of time, which leads to longer model loading time and affects user experience, the present application provides a large model protection method, device, equipment and storage medium.
[0005] In a first aspect, the present application provides a large model protection method, which adopts the following technical solution: the method comprises:
[0006] Obtain a large model weight file, where the large model weight file is a file of the large model that needs to be protected;
[0007] Splitting the large model weight file into a number of original data blocks according to a preset data size, and setting data block indexes for the number of original data blocks;
[0008] Using a national secret algorithm to partially encrypt each of the original data blocks, and generate a number of encrypted data blocks;
[0009] Several of the encrypted data blocks are concatenated according to the data block indexes, and large model ciphertext data is generated.
[0010] In a specific implementation scheme, the use of a national encryption algorithm to partially encrypt each of the original data blocks and generate a number of encrypted data blocks includes:
[0011] Creating a plurality of encryption processing threads, wherein the number of the encryption processing threads is consistent with the number of the original data blocks, and one encryption processing thread processes one original data block;
[0012] Randomly generating a random encryption starting position and a random encryption length for each of the original data blocks;
[0013] The national encryption algorithm SM4 is used to encrypt part of the data segments in the original data block according to the random encryption starting position and the random encryption length, the encrypted part of the data segments replace the original data segments, and a number of encrypted data blocks are generated.
[0014] In a specific implementation scheme, before randomly generating a random encryption start position and a random encryption length for each of the original data blocks, the method further includes:
[0015] The national secret algorithm SM3 is used to calculate the summary of each of the original data blocks and generate several data plaintext summaries.
[0016] In a specific possible implementation scheme, the step of concatenating the plurality of encrypted data blocks according to the data block indexes and generating large model ciphertext data includes:
[0017] Create a data file header;
[0018] Generate data block description information corresponding to each original data block according to the partially encrypted information of each original data block;
[0019] splicing the plurality of encrypted data blocks in sequence according to the data block index, and storing the data block description information corresponding to the encrypted data blocks and the data plaintext digest in the data file header in sequence according to the data block index;
[0020] The spliced encrypted data blocks and the data file header storing the data block description information are set as large model ciphertext data.
[0021] In a specific implementation scheme, the large model ciphertext data includes a data file header and a plurality of the encrypted data blocks, and the method further includes:
[0022] When obtaining the large model ciphertext data, reading and parsing the data file header to obtain the parsed data block description information and the number of encrypted data blocks, wherein the data block description information includes a data block index;
[0023] Decrypting a number of the encrypted data blocks using a national secret algorithm according to the number of the encrypted data blocks and the data block description information to obtain a number of the decrypted data blocks;
[0024] Several of the decrypted data blocks are spliced according to the data block index to generate large model plaintext data.
[0025] In a specific implementation scheme, the data block description information includes a data plaintext summary, an encryption start position, and an encryption length;
[0026] Decrypting the plurality of encrypted data blocks using a national secret algorithm according to the number of the encrypted data blocks and the data block description information to obtain the plurality of decrypted data blocks comprises:
[0027] Creating a corresponding number of decryption processing threads according to the number of the encrypted data blocks, wherein one decryption processing thread processes one encrypted data block;
[0028] Determine the data segment to be decrypted according to the encryption starting position and the encryption length;
[0029] The data segment to be decrypted is decrypted using the national encryption algorithm SM4, and the decrypted data segment replaces the original encrypted data segment to generate a number of decrypted data blocks.
[0030] In a specific implementation scheme, after decrypting the data segment to be decrypted using the national encryption algorithm SM4 and replacing the original encrypted data segment with the decrypted data segment to generate a plurality of decrypted data blocks, the method further includes:
[0031] Using the national secret algorithm SM3 to calculate the digest of each decrypted data block, generating several data decryption digests;
[0032] Comparing a number of the data decryption digests with a number of corresponding data plaintext digests;
[0033] If the comparison results are inconsistent, a warning of data tampering risk will be issued;
[0034] If the comparison results are consistent, the decryption of the large model ciphertext data is completed.
[0035] In a second aspect, the present application provides a large model protection device, which adopts the following technical solution: the device comprises:
[0036] A file acquisition module, used to acquire a large model weight file, where the large model weight file is a file of the large model that needs to be protected;
[0037] A file splitting module, used for splitting the large model weight file into a number of original data blocks according to a preset data size, and setting data block indexes for the number of original data blocks;
[0038] A partial encryption module, used to partially encrypt each of the original data blocks using a national encryption algorithm and generate a plurality of encrypted data blocks;
[0039] The data splicing module is used to splice a plurality of the encrypted data blocks according to the data block index and generate large model ciphertext data.
[0040] In a third aspect, the present application provides a computer device, which adopts the following technical solution: it includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and execute any of the large model protection methods described above.
[0041] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: storing a computer program that can be loaded by a processor and execute any of the above-mentioned large model protection methods.
[0042] In summary, this application has the following beneficial technical effects:
[0043] 1. Split the model data into several data blocks, and partially encrypt each data block, replacing the overall encryption method in the current common method, so as to reduce the time consumed by encryption and model loading, improve the efficiency of model protection operations, and thus improve user experience; perform integrity verification by calculating the summary to prevent data tampering;
[0044] 2. During the processing, multiple threads are created and started to handle encryption and decryption operations. The number of threads is consistent with the number of split original data blocks, so as to speed up the encryption and decryption as much as possible and reduce the time consumed by encryption and decryption. In addition, the encrypted position and length of each data block are randomly generated, which increases the randomness of encryption, thereby improving the security of encryption as much as possible and preventing the encrypted file from being easily cracked as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 is a flow chart of a large model protection method in an embodiment of the present application;
[0046] Figure 2 It is a schematic diagram of partial encryption in an embodiment of the present application;
[0047] Figure 3 is a schematic diagram of the data file header format in an embodiment of the present application;
[0048] Figure 4 is a schematic diagram of splicing encrypted data blocks in an embodiment of the present application;
[0049] Figure 5 It is a schematic diagram of partial decryption in the embodiment of the present application;
[0050] Figure 6 is a schematic diagram of splicing decrypted data blocks in an embodiment of the present application;
[0051] Figure 7 is a schematic diagram of a large model protection device in an embodiment of the present application;
[0052] Figure 8 It is a schematic diagram used to embody a computer device in an embodiment of the present application.
[0053] Attached figure numbers: 701, file acquisition module; 702, file splitting module; 703, partial encryption module; 704, data splicing module. DETAILED DESCRIPTION
[0054] The following combination Figure 1-Figure 8 This application is described in further detail.
[0055] The embodiment of the present application discloses a large model protection method, which is applied to a large model protection system. The large model protection method can be used to protect the large model weight file after training. The large model weight file is encrypted using a national secret algorithm to ensure the confidentiality of the file. When the large model weight file needs to be used, the large model weight file is decrypted for normal use.
[0056] A large model refers to a deep learning model with large-scale parameters and complex computing structures. It is widely used in natural language processing, computer vision, speech recognition, and recommendation systems. The training process of a large model requires high equipment and labor costs, and the most valuable part of a large model is the large model weight file after training, which is also an important asset for enterprises to build competitiveness in artificial intelligence (AI). In order to prevent the model from being stolen or unauthorized access and use, it is usually necessary to protect the large model weight file.
[0057] A common method for protecting large models is to encrypt the large model weight file, and then decrypt the encrypted file when using the large model weight file. The current commonly used encryption method is usually to directly encrypt and decrypt the large model weight file as a whole. Since the large model weight file is generally tens of gigabytes (GB) in size, the file is too large, so the overall encryption and decryption takes a lot of time, which leads to a longer model loading time and affects the user experience; in addition, the existing protection method is more to ensure the confidentiality of the model file through encryption algorithms, but ignores the integrity of the large model weight file. If the file is tampered with, it may cause the model to fail to work properly. In order to help improve the efficiency of model protection, reduce model loading time, and ensure the integrity of files, the present application provides a large model protection method.
[0058] Reference Figure 1 , the method comprises the following steps:
[0059] S10, obtaining a large model weight file, where the large model weight file is a file of the large model that needs to be protected.
[0060] Specifically, obtain the big model weight file after the big model is trained. The big model weight file is the most valuable file of the trained big model, that is, the file of the big model that needs to be protected.
[0061] S20, split the large model weight file into several original data blocks according to a preset data size, and set data block indexes for the several original data blocks.
[0062] Specifically, the large model weight file is split into multiple data blocks according to the preset data size. The preset data size can also be understood as the splitting unit of the file. The multiple data blocks obtained by the splitting are also several original data blocks. The number of original data blocks is determined by the size of the large model weight file and the splitting unit of the file. After the data splitting is completed, the index corresponding to the data block can be set and obtained. When the large model weight file is split according to the preset data size, if the last remaining part is smaller than the preset data size, that is, smaller than the splitting unit of the file, the last remaining part can be directly used as a separate original data block. For example, assuming that the size of the large model weight file is 15GB and the data splitting unit is set to 4GB, the original data block can be split into 4GB, 4GB, 4GB and 3GB, a total of 4 original data blocks, and the data block index can be set to 0, 1, 2, 3 accordingly.
[0063] S30, using the national encryption algorithm to partially encrypt each original data block and generate a number of encrypted data blocks.
[0064] Specifically, the national secret algorithm is used to partially encrypt each original data block, that is, all the original data blocks are partially encrypted to obtain the corresponding encrypted data blocks. Partial encryption can be understood as encrypting part of the data segment in the data block, rather than encrypting the entire data block. Partial encryption can reduce the time spent on encryption operations, increase the encryption speed, and reduce the model loading time. Among them, the national secret algorithm refers to a series of cryptographic standards formulated and promulgated by the State Cryptography Administration, that is, the domestic cryptographic algorithm that has been recognized by the State Cryptography Administration. It has security advantages such as high security, high efficiency, and independent controllability. It has a higher encryption and decryption speed on the national secret algorithm chip. The use of the national secret algorithm can further improve the speed of model encryption processing.
[0065] S40, concatenating a plurality of encrypted data blocks according to the data block indexes, and generating large model ciphertext data.
[0066] Specifically, the data obtained by splicing several encrypted data blocks according to the data block index is the large model ciphertext data, and the information in the large model weight file is encrypted and protected, making the file not easy to be cracked and used.
[0067] In the present application scheme, the model is split into several data blocks, and each data block is partially encrypted, replacing the overall encryption method in the currently commonly used method. This can reduce the time spent on encryption and the time for model loading, improve the efficiency of model protection operations, and thus enhance the user experience.
[0068] In one embodiment, the steps of partially encrypting each original data block using a national encryption algorithm and generating a plurality of encrypted data blocks may be specifically performed as follows:
[0069] Create several encryption processing threads. The number of encryption processing threads is consistent with the number of original data blocks. One encryption processing thread processes one original data block. Create multiple threads to process encryption operations simultaneously to complete the encryption operation as quickly as possible and improve the encryption speed. After splitting the data block, the data block index, data block starting position and data block size can be obtained. After the encryption processing thread is created, the data block index, data block starting position and data block size are passed to the thread to facilitate the subsequent encryption operation of the data block. Afterwards, in each encryption processing thread, a random encryption starting position and a random encryption length are randomly generated for each original data block. According to the random encryption starting position and the random encryption length, the data segment to be encrypted in the original data block can be determined. After determining the data segment to be encrypted, the national encryption algorithm SM4 is used to encrypt the data segment to be encrypted in the original data block, and the encrypted partial data segment replaces the data segment to be encrypted in the original data block. Each data block performs a partial encryption operation, and the encryption starting position and encryption length of each data block are random and different. After the encryption is completed, several encrypted data blocks are finally generated. Reference Figure 2 , is a schematic diagram of an encrypted data block. The shaded part in the figure is the encrypted data segment, and the rest is the data in the original file. Without any operation, an encrypted data block containing plaintext and ciphertext is finally generated. The national secret algorithm SM4 is a block cipher algorithm that can be used to replace international cryptographic algorithms such as DES / AES; the key length and block length of the SM4 algorithm are both 128 bits.
[0070] Before randomly generating a random encryption start position and a random encryption length for each original data block, the following steps may be performed:
[0071] The national secret algorithm SM3 is used to calculate the summary of each original data block and generate several data plaintext summaries. The summary is mainly used to verify the integrity of the data after decryption to prevent data tampering. The national secret algorithm SM3 is a hash algorithm used for data integrity verification and message authentication code. The algorithm is an irreversible algorithm.
[0072] In the present application scheme, during the processing, multiple threads are created and started to process the encryption operation, and the number of threads is consistent with the number of split original data blocks, so as to speed up the encryption as much as possible and reduce the time consumed for encryption. In addition, the encrypted position and encrypted length in each data block are randomly generated, which increases the randomness of the encryption, thereby improving the security of the encrypted file as much as possible and preventing the encrypted file from being easily cracked as much as possible.
[0073] In one embodiment, the step of concatenating several encrypted data blocks according to the data block index and generating large model ciphertext data may be specifically performed as follows:
[0074] First, create a data file header. The purpose of the file header is to save the relevant information of each data block. It mainly stores two parts of information, the number of data blocks and the data block related information. Afterwards, generate data block description information corresponding to each original data block based on the partially encrypted information of each original data block. Specifically, the data block description information mainly includes the data block index, ciphertext length, the offset of the ciphertext in the data block, data block length, and the offset of the data block in the file. The offset of the ciphertext in the data block can be understood as the position of the ciphertext in the data block. Similarly, the offset of the data block in the file can be understood as the position of the data block in the file. Afterwards, several encrypted data blocks are spliced in sequence according to the data block index, and the data block description information and the data plaintext summary corresponding to the encrypted data blocks are stored in the data file header in sequence according to the data block index. Refer to Figure 3 , which is a schematic diagram of the storage format of the data file header. The data file header stores the length of the file header, the number of data blocks, and the description information of each data block. The data block plaintext summary is included in the data block description information. Finally, the spliced encrypted data blocks and the data file header stored in the data block description information are set as the large model ciphertext data. Figure 4 ,The final generated large model ciphertext data includes the file header and each data block. The position and length of the ciphertext in the data block are different.
[0075] In the present application scheme, in the process of splicing data blocks, by creating a file header to store the relevant information of data encryption in the data file header, it is helpful to decrypt the large model weight file later. During the encryption process, the starting position of the data block encryption and the length of the encrypted ciphertext are random. The encrypted relevant information is stored in the file header to facilitate the subsequent accurate decryption operation based on the relevant information, avoiding errors in the decryption process that may cause damage to the model file.
[0076] In one embodiment, the large model ciphertext data includes a data file header and a plurality of encrypted data blocks. The method may further perform the following steps:
[0077] After the encryption operation is completed on the large model weight file, the large model weight file can be protected and the security of the file can be improved. When the user needs to use the encrypted large model weight file, that is, when the user transmits the large model ciphertext data corresponding to the large model weight file to be used to the system, when the system obtains the large model ciphertext data, it first reads and parses the data file header to obtain the parsed data block description information and the number of encrypted data blocks. When encrypting, the data block description information stores the data block index, ciphertext length, ciphertext offset in the data block, data block length, and data block offset in the file. When decrypting, first parse the data header file and obtain the encryption related information from the data header file, including the data block index. In addition, it also includes the ciphertext length, the ciphertext offset in the data block, the data block length, and the data block offset in the file. This information is needed for decryption. Afterwards, according to the number of encrypted data blocks and the data block description information, the national secret algorithm is used to decrypt several encrypted data blocks to obtain several decrypted data blocks. Finally, several decrypted data blocks are spliced according to the data block index to generate large model plaintext data.
[0078] In one embodiment, the steps of decrypting a number of encrypted data blocks using a national encryption algorithm according to the number of encrypted data blocks and the data block description information to obtain a number of decrypted data blocks may be specifically performed as follows:
[0079] First, create a corresponding number of decryption processing threads according to the number of encrypted data blocks. One decryption processing thread processes one encrypted data block. By creating the same number of decryption processing threads as the number of data blocks, multiple data blocks are decrypted simultaneously to increase the speed of decryption processing. After creating the decryption processing thread, the system will transfer the parsed data block description information to the processing thread. After that, determine the data segment to be decrypted based on the encryption starting position and the encryption length. Specifically, the system will first obtain the range of the data block that needs to be processed by the thread based on the offset of the data block in the file. After that, determine the data segment that needs to be decrypted in the data block based on the encryption starting position and the ciphertext length. Finally, use the national encryption algorithm SM4 to decrypt the data segment to be decrypted, and replace the original encrypted data segment with the decrypted data segment to generate several decrypted data blocks. Refer to Figure 5 , a schematic diagram of replacing the shaded ciphertext data in one data block with the decrypted plaintext data. Figure 6 ,After decrypting and replacing the ciphertext data segment in each data block, the decrypted data blocks are concatenated to generate the final large model plaintext data.
[0080] In the present application scheme, corresponding to the encryption process, multiple decryption threads corresponding to the number of data blocks are created to speed up the decryption as much as possible and reduce the time consumed for decryption. The encryption position and the encrypted data length are obtained by parsing to ensure the accuracy of the decryption position and avoid the original data from being destroyed or tampered with as much as possible.
[0081] In one embodiment, after the data segment to be decrypted is decrypted using the national encryption algorithm SM4 and the decrypted data segment replaces the original encrypted data segment to generate a number of decrypted data blocks, the following steps may be performed:
[0082] The national secret algorithm SM3 is used to calculate the summary of each decrypted data block to generate several data decryption summaries; the several data decryption summaries are compared with the corresponding several data plaintext summaries; if the comparison results are inconsistent, a data tampering risk warning is issued; if the comparison results are consistent, the decryption of the large model ciphertext data is completed.
[0083] In the existing encryption and decryption protection methods, encryption and decryption are generally used to ensure the confidentiality and security of files. However, the integrity of the large model weight file may be ignored. Tampering with the content of the large model weight file may cause the model to fail to run. By introducing the step of calculating the data block summary, the data block summaries before and after encryption and decryption are compared to complete the data block integrity check, which can effectively prevent the hidden danger of data tampering.
[0084] Figure 1 FIG. 1 is a flow chart of a large model protection method in one embodiment. It should be understood that although Figure 1The steps in the flowchart are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows; unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps may be executed in other orders; and Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0085] Based on the above method, an embodiment of the present application also discloses a large model protection device.
[0086] Reference Figure 7 , the device includes the following modules:
[0087] The file acquisition module 701 is used to acquire a large model weight file, where the large model weight file is a file of the large model that needs to be protected;
[0088] A file splitting module 702 is used to split the large model weight file into a number of original data blocks according to a preset data size, and set data block indexes for the number of original data blocks;
[0089] The partial encryption module 703 is used to partially encrypt each original data block using the national encryption algorithm and generate a number of encrypted data blocks;
[0090] The data splicing module 704 is used to splice a number of encrypted data blocks according to the data block index and generate large model ciphertext data.
[0091] In one embodiment, the partial encryption module 703 is specifically used to create several encryption processing threads, the number of encryption processing threads is consistent with the number of original data blocks, and one encryption processing thread processes one original data block; randomly generates a random encryption starting position and a random encryption length for each original data block; uses the national encryption algorithm SM4 to encrypt some data segments in the original data block according to the random encryption starting position and the random encryption length, replaces the original data segment with the encrypted partial data segment, and generates several encrypted data blocks.
[0092] In one embodiment, the partial encryption module 703 is specifically used to calculate the summary of each original data block using the national encryption algorithm SM3 and generate a number of data plaintext summaries.
[0093] In one embodiment, the partial encryption module 703 is specifically used to create a data file header; generate data block description information corresponding to each original data block based on the partially encrypted information of each original data block; splice several encrypted data blocks in sequence according to the data block index, and store the data block description information and data plaintext summary corresponding to the encrypted data blocks in the data file header in sequence according to the data block index; set the spliced several encrypted data blocks and the data file header storing the data block description information as large model ciphertext data.
[0094] In one embodiment, the large model protection device is also used to read and parse the data file header when obtaining the large model ciphertext data, to obtain the parsed data block description information and the number of encrypted data blocks, the data block description information including the data block index; decrypt several encrypted data blocks using the national secret algorithm according to the number of encrypted data blocks and the data block description information to obtain several decrypted data blocks; splice the several decrypted data blocks according to the data block index to generate the large model plaintext data.
[0095] In one embodiment, the large model protection device is also used to decrypt a number of encrypted data blocks using a national secret algorithm according to the number of encrypted data blocks and data block description information, and obtain a number of decrypted data blocks including: creating a corresponding number of decryption processing threads according to the number of encrypted data blocks, one decryption processing thread processes one encrypted data block; determining the data segment to be decrypted according to the encryption starting position and the encryption length; using the national secret algorithm SM4 to decrypt the data segment to be decrypted, and replacing the original encrypted data segment with the decrypted data segment to generate a number of decrypted data blocks.
[0096] In one embodiment, the large model protection device is also used to calculate the summary of each decrypted data block using the national secret algorithm SM3 to generate several data decryption summaries; compare the several data decryption summaries with the corresponding several data plaintext summaries; if the comparison results are inconsistent, a data tampering risk warning is issued; if the comparison results are consistent, the decryption of the large model ciphertext data is completed.
[0097] The large model protection device provided in the embodiment of the present application can be applied to the large model protection method provided in the above embodiment. For relevant details, refer to the above method embodiment. The implementation principle and technical effect are similar and will not be repeated here.
[0098] It should be noted that: the large model protection device provided in the embodiment of the present application only uses the division of the above-mentioned functional modules / functional units as an example when performing large model protection. In actual applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the large model protection device is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the large model protection method provided in the above method embodiment and the implementation method of the large model protection device provided in this embodiment belong to the same concept. The specific implementation process of the large model protection device provided in this embodiment is detailed in the above method embodiment, which will not be repeated here.
[0099] The embodiment of the present application also discloses a computer device.
[0100] Specifically, if Figure 8 As shown, the computer device can be a computer device such as a desktop computer, a laptop computer, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. The processor and the memory may be connected via a bus or otherwise. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various chips.
[0101] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method implementation is realized. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.
[0102] The embodiment of the present application also discloses a computer-readable storage medium.
[0103] Specifically, a computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art will understand that the implementation of all or part of the process in the above-mentioned implementation method of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and the program, when executed, may include the process of the implementation of the above-mentioned methods. Among them, the storage medium may be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memory.
[0104] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.
Claims
1. A large model protection method, characterized in that: The method comprises: Obtain a large model weight file, where the large model weight file is a file of the large model that needs to be protected; Splitting the large model weight file into a number of original data blocks according to a preset data size, and setting data block indexes for the number of original data blocks; Using a national secret algorithm to partially encrypt each of the original data blocks, and generate a number of encrypted data blocks; Several of the encrypted data blocks are concatenated according to the data block indexes, and large model ciphertext data is generated.
2. The method according to claim 1, characterized in that: The method of partially encrypting each of the original data blocks using the national encryption algorithm and generating a plurality of encrypted data blocks includes: Creating a plurality of encryption processing threads, wherein the number of the encryption processing threads is consistent with the number of the original data blocks, and one encryption processing thread processes one original data block; Randomly generating a random encryption starting position and a random encryption length for each of the original data blocks; The national encryption algorithm SM4 is used to encrypt part of the data segments in the original data block according to the random encryption starting position and the random encryption length, the encrypted part of the data segments replace the original data segments, and a number of encrypted data blocks are generated.
3. The method according to claim 2, characterized in that: Before randomly generating a random encryption starting position and a random encryption length for each of the original data blocks, the method further includes: The national secret algorithm SM3 is used to calculate the summary of each of the original data blocks and generate several data plaintext summaries.
4. The method according to claim 3, characterized in that: The step of splicing the plurality of encrypted data blocks according to the data block indexes and generating large model ciphertext data comprises: Create a data file header; Generate data block description information corresponding to each original data block according to the partially encrypted information of each original data block; splicing the plurality of encrypted data blocks in sequence according to the data block index, and storing the data block description information corresponding to the encrypted data blocks and the data plaintext digest in the data file header in sequence according to the data block index; The spliced encrypted data blocks and the data file header storing the data block description information are set as large model ciphertext data.
5. The method according to claim 1, characterized in that: The large model ciphertext data includes a data file header and a plurality of encrypted data blocks, and the method further includes: When obtaining the large model ciphertext data, reading and parsing the data file header to obtain the parsed data block description information and the number of encrypted data blocks, wherein the data block description information includes a data block index; Decrypting a number of the encrypted data blocks using a national secret algorithm according to the number of the encrypted data blocks and the data block description information to obtain a number of the decrypted data blocks; Several of the decrypted data blocks are spliced according to the data block index to generate large model plaintext data.
6. The method according to claim 5, characterized in that: The data block description information includes a data plaintext summary, an encryption start position, and an encryption length; Decrypting the plurality of encrypted data blocks using a national secret algorithm according to the number of the encrypted data blocks and the data block description information to obtain the plurality of decrypted data blocks comprises: Creating a corresponding number of decryption processing threads according to the number of the encrypted data blocks, wherein one decryption processing thread processes one encrypted data block; Determine the data segment to be decrypted according to the encryption starting position and the encryption length; The data segment to be decrypted is decrypted using the national encryption algorithm SM4, and the decrypted data segment replaces the original encrypted data segment to generate a number of decrypted data blocks.
7. The method according to claim 6, characterized in that: After the data segment to be decrypted is decrypted by using the national encryption algorithm SM4, and the decrypted data segment replaces the original encrypted data segment to generate a plurality of decrypted data blocks, the method further includes: Using the national secret algorithm SM3 to calculate the digest of each decrypted data block, generating several data decryption digests; Comparing a number of the data decryption digests with a number of corresponding data plaintext digests; If the comparison results are inconsistent, a warning of data tampering risk will be issued; If the comparison results are consistent, the decryption of the large model ciphertext data is completed.
8. A large model protection device, characterized in that: The device comprises: A file acquisition module (701), used to acquire a large model weight file, wherein the large model weight file is a file of the large model that needs to be protected; A file splitting module (702) is used to split the large model weight file into a number of original data blocks according to a preset data size, and set data block indexes for the number of original data blocks; A partial encryption module (703) is used to partially encrypt each of the original data blocks using a national encryption algorithm and generate a plurality of encrypted data blocks; The data splicing module (704) is used to splice the plurality of encrypted data blocks according to the data block index and generate large model ciphertext data.
9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 7.
