Encryption method and device based on large language model, equipment, medium and product
Patent Information
- Application Number
- CN202510320324.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-18
- Publication Date
- 2025-07-11
Smart Images
Figure CN120296756A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to an encryption method, apparatus, device, medium, and product based on a large language model. Background Art
[0002] A large language model (LLM) is an artificial intelligence model constructed based on deep learning technology, aiming to process and generate natural language text, and can be used for content creation, assisted writing, information query answering, language translation, and assisted programming, etc. In the process of users using the large language model, in order to protect the privacy of users, it is necessary to encrypt the large language model.
[0003] In the related art, encryption is performed on the input and output layers of the large language model, but this encryption method has security risks, which may lead to the leakage of user data privacy, and the encryption method of the large language model in the related art is complex and not suitable for the real-time interaction scenario of the large language model under encryption requirements. Summary of the Invention
[0004] The present disclosure provides an encryption method, apparatus, device, medium, and product based on a large language model.
[0005] According to a first aspect of the present disclosure, there is provided an encryption method based on a large language model, which is applied to a server, and the method includes:
[0006] Group the words in the original vocabulary of the large language model to obtain a plurality of target groups, and determine the target hash value of each word in the target group;
[0007] Determine the first index of each word in the hash ring based on the target hash value; wherein, the first index in the hash ring is used to indicate the position of each word in the original vocabulary;
[0008] Rearrange the words in the original vocabulary to obtain a confused vocabulary, and determine the second index of each word in the original vocabulary in the confused vocabulary; wherein, the second index is used to indicate the position of each word in the original vocabulary in the confused vocabulary;
[0009] Construct a target mapping dictionary according to the mapping relationship between the first index of each word in the original vocabulary and the second index in the confused vocabulary, and encrypt and send the target mapping dictionary to the client.
[0010] Further, the determining the target hash value of each word in the target group includes:
[0011] Calculate the first hash value of each word in each of the target groups through a hash algorithm;
[0012] Round the first hash value to obtain the second hash value of the word;
[0013] Calculate the target hash value of the word based on the second hash value.
[0014] Further, determining the first index of each word in the hash ring based on the target hash value includes:
[0015] Construct the hash ring based on the target interval range; wherein each node in the hash ring corresponds to a numerical value in a sub-range of the target interval range;
[0016] Determine the first index of each word in the original word list in the hash ring based on the target hash value.
[0017] Further, determining the first index of each word in the original word list in the hash ring based on the target hash value includes:
[0018] In the order of the target grouping, sequentially determine the mapping nodes of the target hash values of each word in each target grouping in the hash ring, and determine the numerical value corresponding to the mapping node as the first index of the corresponding word.
[0019] Further, the method further includes:
[0020] If the first indexes of different words in different target groupings are the same, then perform hash encryption mapping on any one of the different words corresponding to the same first index until the first indexes of different words in different target groupings are different.
[0021] Further, after constructing the target mapping dictionary according to the mapping relationship between the first index of each word in the original word list and the second index in the obfuscated word list, the method further includes:
[0022] Obtain the fine-tuning corpus of the large language model;
[0023] Replace the words in the fine-tuning corpus based on the target mapping dictionary to obtain an obfuscated fine-tuning corpus dataset;
[0024] Update the large language model based on the obfuscated fine-tuning corpus dataset to obtain an updated large language model.
[0025] Further, updating the large language model based on the obfuscated fine-tuning corpus dataset to obtain an updated large language model includes:
[0026] Determine the position and parameters of the embedding layer of the large language model;
[0027] While keeping the parameters of other network layers in the large language model unchanged except for the embedding layer, input the confused fine-tuning corpus dataset into the large language model;
[0028] Determine the target parameters of the embedding layer in the large language model after inputting the confused fine-tuning corpus dataset, and obtain the updated large language model.
[0029] Further, after encrypting the target mapping dictionary and sending it to the client, the method further includes:
[0030] Receive the encrypted data sent by the client; wherein, the encrypted data is obtained by the client encrypting the user's plaintext data;
[0031] Decrypt the encrypted data through the large language model to obtain a first decryption result;
[0032] Send the first decryption result to the client.
[0033] According to the second aspect of the present disclosure, there is provided an encryption method based on a large language model, which is applied to a client, and the method includes:
[0034] Receive the encrypted target mapping dictionary sent by the server; wherein, the encrypted target mapping dictionary is obtained by the server encrypting the target mapping dictionary through the encryption method based on the large language model according to any one of the above first aspects;
[0035] Decrypt the encrypted target mapping dictionary through the private key to obtain the correspondence between the vocabulary in the original vocabulary table and the confused vocabulary table of the large language model;
[0036] Encrypt the user's plaintext data based on the correspondence to obtain encrypted data;
[0037] Send the encrypted data to the server.
[0038] Further, after sending the encrypted data to the server, the method further includes:
[0039] Receive the first decryption result sent by the server;
[0040] Based on a pre-established sensitive vocabulary list, perform deletion and / or replacement processing on sensitive segments in the first decryption result to obtain a second decryption result;
[0041] Decrypt the second decryption result based on the target mapping dictionary to obtain a target return result.
[0042] According to a third aspect of the present disclosure, there is provided an encryption device based on a large language model, which is disposed in a server. The device includes:
[0043] A first determination module, configured to group the words in the original vocabulary of the large language model to obtain a plurality of target groups, and determine the target hash value of each word in the target group;
[0044] A second determination module, configured to determine a first index of each word in the hash ring based on the target hash value; wherein, the first index in the hash ring is used to indicate the position of each word in the original vocabulary;
[0045] A third determination module, configured to rearrange the words in the original vocabulary to obtain a scrambled vocabulary, and determine a second index of each word in the original vocabulary in the scrambled vocabulary; wherein, the second index is used to indicate the position of each word in the original vocabulary in the scrambled vocabulary;
[0046] A first sending module, configured to construct a target mapping dictionary according to the mapping relationship between the first index of each word in the original vocabulary and the second index in the scrambled vocabulary, and encrypt and send the target mapping dictionary to the client.
[0047] According to a fourth aspect of the present disclosure, there is provided an encryption device based on a large language model, which is disposed in a client. The device includes:
[0048] A first receiving module, configured to receive the encrypted target mapping dictionary sent by the server; wherein, the encrypted target mapping dictionary is obtained by the server encrypting the target mapping dictionary through the above-mentioned encryption method based on the large language model of the present disclosure;
[0049] A first decryption module, configured to decrypt the encrypted target mapping dictionary with a private key to obtain the corresponding relationship between the words in the original vocabulary and the scrambled vocabulary of the large language model;
[0050] An encryption module, configured to encrypt the user plaintext data based on the corresponding relationship to obtain encrypted data;
[0051] A second sending module, configured to send the encrypted data to the server.
[0052] According to a fifth aspect of the present disclosure, there is provided an electronic device. The electronic device includes: a memory and a processor. A computer program is stored on the memory, and when the processor executes the program, the above-mentioned method is implemented.
[0053] According to a sixth aspect of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the above-mentioned method of the present disclosure is implemented.
[0054] According to a seventh aspect of the present disclosure, there is provided a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the above-mentioned method of the present disclosure is implemented.
[0055] An encryption method, device, equipment, medium and product based on a large language model provided by an embodiment of the present disclosure. In an embodiment of the present application, first, the vocabulary in the original vocabulary list of the large language model is grouped to obtain a plurality of target groups, and the target hash value of each vocabulary in the target group is determined; then, based on the target hash value, the first index of each vocabulary in the hash ring is determined; wherein, the first index in the hash ring is used to indicate the position of each vocabulary in the original vocabulary list; after that, the vocabulary in the original vocabulary list is rearranged to obtain a confused vocabulary list, and the second index of each vocabulary in the original vocabulary list in the confused vocabulary list is determined; wherein, the second index is used to indicate the position of each vocabulary in the original vocabulary list in the confused vocabulary list; finally, according to the mapping relationship between the first index of each vocabulary in the original vocabulary list and the second index in the confused vocabulary list, a target mapping dictionary is constructed, and the target mapping dictionary is encrypted and sent to the client.
[0056] As can be seen from the above description, the technical solution of the present disclosure can determine the first index of each vocabulary in the hash ring based on the target hash value, and can rearrange the vocabulary in the original vocabulary list to obtain a confused vocabulary list, so as to determine the second index of each vocabulary in the original vocabulary list in the confused vocabulary list. Through the first index and the second index, a target mapping dictionary can be constructed, and the target mapping dictionary is used to indicate the mapping relationship between the first index of each vocabulary in the original vocabulary list and the second index in the confused vocabulary list. Compared with encrypting the input and output layers of the large language model, the technical solution of the present disclosure realizes the encryption of the large language model itself by constructing a target mapping dictionary, improving the security of the large language model; the technical solution of the present disclosure can also encrypt the target mapping dictionary and send it to the client, and the client can realize real-time interaction with the large language model through the target mapping dictionary. The technical solution of the present disclosure not only improves the security of the large language model, but also ensures the efficient operation of the large language model in different application scenarios. Description of the Drawings
[0057] The above and other objects, features, and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail with reference to the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation to the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0058] Figure 1 Flowchart of an encryption method based on a large language model provided by an exemplary embodiment of the present disclosure;
[0059] Figure 2 Flowchart of an encryption method based on a large language model provided by another exemplary embodiment of the present disclosure;
[0060] Figure 3 Flowchart of an encryption method based on a large language model provided by another exemplary embodiment of the present disclosure;
[0061] Figure 4 Flowchart of an encryption method based on a large language model provided by another exemplary embodiment of the present disclosure;
[0062] Figure 5 Flowchart of an encryption method based on a large language model provided by another exemplary embodiment of the present disclosure;
[0063] Figure 6 Logical schematic diagram of an encryption method based on a large language model provided by an exemplary embodiment of the present disclosure;
[0064] Figure 7 Schematic block diagram of functional modules of an encryption device based on a large language model provided by an exemplary embodiment of the present disclosure;
[0065] Figure 8 Schematic block diagram of functional modules of an encryption device based on a large language model provided by another exemplary embodiment of the present disclosure;
[0066] Figure 9 Schematic block diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure;
[0067] Figure 10 Schematic block diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure;
[0068] Figure 11 Schematic block diagram of the structure of a computer program product provided by an exemplary embodiment of the present disclosure. Detailed implementation manners
[0069] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.
[0070] It should be understood that the various steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.
[0071] The term "including" and its variations used herein are open-ended, that is, "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.
[0072] It should be noted that the modifications of "one" and "plural" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless clearly stated otherwise in the context, it should be understood as "one or more".
[0073] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.
[0074] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0075] For example, when receiving a user's active request, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server or a storage medium that performs the operations of the technical solutions of the present disclosure according to the prompt message.
[0076] As an optional but non-limiting implementation, in response to receiving an active request from a user, the way to send a prompt message to the user can be, for example, in the form of a pop-up window, and the prompt message can be presented in text in the pop-up window. In addition, the pop-up window can also carry selection controls for the user to choose "agree" or "disagree" to provide personal information to the electronic device. It can be understood that the above notification and user authorization process is only illustrative and does not limit the implementation of the present disclosure, and other methods that comply with relevant laws and regulations can also be applied to the implementation of the present disclosure.
[0077] In one embodiment, as Figure 1 shown, a cryptographic method based on a large language model is provided, which is applied to a server and includes the following steps:
[0078] Step 101, group the words in the original vocabulary of the large language model to obtain multiple target groups, and determine the target hash value of each word in the target group.
[0079] Here, the executing entity can group the words in the original vocabulary W = {ω1, ω2, …, ω n} of the large language model to obtain multiple target groups, and determine the target hash value of each word in the target group, where n represents the number of words in the original vocabulary, and ω i represents the i-th word. It should be noted that the executing entity can be a server running the large language model, and the original vocabulary is the vocabulary pre-stored by the large language model on the server, or it can be the vocabulary input by the user for the first time. It should be noted that the specific content of the original vocabulary is not limited here.
[0080] In a possible embodiment, the executing entity groups the words in the original vocabulary W = {ω1, ω2, …, ω n} of the large language model to obtain multiple target groups. Exemplarily, the original vocabulary of the large language model can be W = {"apple", "banana", "cherry", "grape", "orange", "peach", "strawberry", "watermelon"}, and this original vocabulary contains 8 words. The original vocabulary is divided into 4 target groups, and these 4 target groups are {"apple", "banana"}, {"cherry", "grape"}, {"orange", "peach"}, {"strawberry", "watermelon"} respectively.
[0081] In a possible embodiment, determining the target hash value of each word in the target group includes the following steps:
[0082] Calculate the first hash value of each word in each target group through a hash algorithm;
[0083] Perform rounding processing on the first hash value to obtain the second hash value of the word;
[0084] Calculate the target hash value of the word based on the second hash value.
[0085] Specifically, after the server groups the words in the original vocabulary of the large language model to obtain multiple target groups, it can calculate the first hash value of each word in each target group through a hash algorithm, and perform rounding processing on the first hash value to obtain the second hash value of the word.
[0086] In a possible embodiment, following the previous example, the original vocabulary is divided into 4 groups, which are {"apple","banana"}, {"cherry","grape"}, {"orange","peach"}, {"strawberry","water melon"} respectively. The server can use a hash algorithm to calculate the first hash value H(ω i of each word ω i ) in each target group on different nodes of the server. It should be noted that strong hash algorithms such as SHA-256, Murmur Hash, and SHA-3 can be used for the hash algorithm, and the specific type of the hash algorithm is not limited here. After the server determines the first hash value of each word in each target group, it performs rounding processing on the first hash value to obtain the second hash value h i of the word. The calculation formula is:
[0087] h i = int(H(ω i ))
[0088] Exemplarily, the server includes 4 nodes. Among them, server node 1 can use the SHA-256 algorithm to calculate the first hash value H(ω i ) of "apple" in the first target group {"apple","banana"} as 162.2. The server performs rounding processing on the first hash value 162.2 of "apple" to obtain the second hash value h i of the word "apple" as 162.
[0089] After the server obtains the second hash value of the word, it can calculate the target hash value of the word based on the second hash value.
[0090] In a possible embodiment, the server can perform operations on the target hash value in a finite field , where p is a preset large prime number, and g is the finite field a non-zero element in, that is, there exists a positive integer k such that g k ≡ 1, and k is as large as possible. Exemplarily, the preset large prime number p can be taken as 101, and the non-zero element g in the finite field can be taken as 2, then the calculation formula of the target hash value is:
[0091]
[0092] where r i is the target hash value, h i is the second hash value, mod is the modulo operator, and p is the preset large prime number.
[0093] Exemplarily, continuing the above example, server node 1 can use the SHA-256 algorithm to calculate the second hash value h of "apple" in the first target group {"apple","banana"} i to be 20, g is 2, and p is 101, then the target hash value r of "apple" can be calculated i = 2 20 mod 101 = 95. Using the same method, the server can calculate the target hash value of each word in the target group on each server node. Exemplarily, node 1 determines that the target hash values of the target group {"apple","banana"} are {95,121}, node 2 determines that the target hash values of the target group {"cherry","grape"} are {223,126}, node 3 determines that the target hash values of the target group {"orange","peach"} are {165,196}, and node 4 determines that the target hash values of the target group {"strawberry","watermelon"} are {532,996}. It should be noted that the calculation method of the target hash value in this embodiment is only exemplary. In actual applications, the calculation method of the target hash value can be flexibly selected according to the number of words in the original word list of the large language model. For example, when the number of words in the original word list of the large language model is small, a more complex target hash value calculation method is adopted, and when the number of words in the original word list of the large language model is large, a simpler target hash value calculation method is adopted.
[0094] Step 102: Determine the first index of each word in the hash ring based on the target hash value.
[0095] Here, after the server determines the target hash value of each word in the target group, it can determine the first index of each word in the hash ring based on the target hash value, where the first index in the hash ring is used to indicate the position of each word in the original word list.
[0096] In a possible embodiment, determining a first index of each term in the hash ring based on the target hash value includes the following steps:
[0097] Construct a hash ring based on the target range.
[0098] Determine the first index of each term in the original vocabulary in the hash ring based on the target hash value.
[0099] Specifically, after the server determines the target hash value of each term in the target group, it can construct a hash ring based on the target range. Among them, each node in the hash ring corresponds to a numerical value in a sub-range of the target range.
[0100] In a possible embodiment, following the above example, the server determines the target hash value of each term in the target group at different nodes. Among them, node 1 determines that the target hash values of the target group {"apple", "banana"} are {95, 121}, node 2 determines that the target hash values of the target group {"cherry", "grape"} are {223, 126}, node 3 determines that the target hash values of the target group {"orange", "peach"} are {165, 196}, and node 4 determines that the target hash values of the target group {"strawberry", "watermelon"} are {532, 996}. After the server determines the target hash value of each term in the target group at different nodes, it constructs a hash ring based on the target range. Exemplarily, the target range can be [0, 2^32 - 1]. It should be noted that the target range can be determined according to the number of terms in the target group. For example, when the number of terms in the target group is large, a larger target range can be selected; when the number of terms in the target group is small, a smaller target range can be selected. The specific determination method of the target range is not limited here, as long as it can meet the mapping of the target hash value.
[0101] Specifically, after the server constructs a hash ring based on the target range, it can determine the first index of each term in the original vocabulary in the hash ring based on the target hash value.
[0102] In a possible embodiment, determining the first index of each term in the original vocabulary in the hash ring based on the target hash value includes the following steps:
[0103] In the order of the target groups, sequentially determine the mapping nodes of the target hash values of each term in each target group in the hash ring, and determine the numerical value corresponding to the mapping node as the first index of the corresponding term.
[0104] Specifically, after the server constructs the hash ring based on the target interval range, it can, in the order of the target groups, sequentially determine the mapping nodes of the target hash values of each term in each target group in the hash ring, and determine the value corresponding to the mapping node as the first index of the corresponding term.
[0105] In a possible embodiment, continuing from the previous example, the server determines the target hash values of each term in the target group at different nodes. Among them, node 1 determines that the target hash values of the target group {"apple", "banana"} are {95, 121}, node 2 determines that the target hash values of the target group {"cherry", "grape"} are {223, 126}, node 3 determines that the target hash values of the target group {"orange", "peach"} are {165, 196}, node 4 determines that the target hash values of the target group {"strawberry", "watermelon"} are {532, 996}, and the target interval range of the hash ring constructed by the server is [0, 2^32 - 1]. The server, in the order of the target groups {"apple", "banana"}, {"cherry", "grape"}, {"orange", "peach"}, {"strawberry", "watermelon"}, sequentially determines the mapping nodes of the target hash values of each term in each target group in the hash ring, and determines the value corresponding to the mapping node as the first index of the corresponding term.
[0106] Exemplarily, the server, in the order of the target groups, first determines the mapping nodes of the target hash values of each term in the target group {"apple", "banana"} in the hash ring. Among them, the target hash values of each term in the target group {"apple", "banana"} are 95 and 121 respectively. Then, the hash ring is equally divided into 2 intervals. According to the target hash value 95, the first mapping node 154562 is found clockwise in the first interval on the hash ring. According to the target hash value 121, the second mapping node 654069 is found clockwise in the second interval on the hash ring. And the values 154562 and 654069 corresponding to the mapping nodes are respectively determined as the first indexes of the corresponding terms "apple" and "banana". The expression formula of the first index is:
[0107] j = CH(r i , m).
[0108] Among them, r iis the target hash value of each word in the target group, m is the number of words in the target group, and j is the first index of each word in the target group. For the target group {"apple", "banana"}, the number of words in the target group is 2. The first index j of "apple" in the target group is 154562, and the first index j of "banana" in the target group is 654069. In this way, the server has completed the determination of the first index of each word in the target group {"apple", "banana"}. Using the same method, the first index of each word in the target groups {"cherry", "grape"}, {"orange", "peach"}, and {"strawberry", "watermelon"} can be determined, and the specific process will not be elaborated here.
[0109] After the server determines the first index of each word in each target group, it can obtain the first index of each word in the original word list. That is to say, in this embodiment, when the server sequentially determines the mapping nodes of the target hash values of each word in each target group in the hash ring, the hash ring is equally divided into multiple intervals, and the number of intervals corresponds to the number of words in the target group. For example, when there are two words in the target group, the hash ring is equally divided into 2 intervals, and then the target hash values of each word in each target group are evenly distributed on each interval, so as to obtain the first index of each word in each target group. After obtaining the first index of each word in each target group, since each target group is obtained by grouping the words in the original word list, obtaining the first index of each word in each target group also means obtaining the first index of each word in the original word list.
[0110] Step 103: Rearrange the words in the original word list to obtain a scrambled word list, and determine the second index of each word in the original word list in the scrambled word list.
[0111] Here, after the server determines the first index of each word in the hash ring based on the target hash value, it can rearrange the words in the original word list to obtain a scrambled word list, and determine the second index of each word in the original word list in the scrambled word list, where the second index is used to indicate the position of each word in the original word list in the scrambled word list.
[0112] In a possible embodiment, the server may rearrange the words in the original vocabulary to obtain a scrambled vocabulary, and determine the second index of each word in the original vocabulary in the scrambled vocabulary. Exemplarily, the words in the original vocabulary are {"apple", "banana", "cherry", "grape", "orange", "peach", "strawberry", "watermelon"}, and the corresponding first indices in the hash ring are {154562, 654069, 184533, 854469, 963241, 115265, 616555, 526599}. The server rearranges the words {"apple", "banana", "cherry", "grape", "orange", "peach", "strawberry", "watermelon"} in the original vocabulary, that is, shuffles the positions of the words, and can obtain the scrambled vocabulary {"banana", "orange", "grape", "cherry", "apple", "watermelon", "peach", "strawberry"}. After obtaining the scrambled vocabulary, according to the first indices of the words with the shuffled order in the scrambled vocabulary, the second index of each word in the original vocabulary in the scrambled vocabulary can be determined. Exemplarily, the word "apple" in the original vocabulary is in the first position of the original vocabulary, and the first index is 154562. After rearranging the words in the original vocabulary to obtain the scrambled vocabulary, the position of "apple" is shuffled to the fifth position in the scrambled vocabulary. Now the word in the first position of the scrambled vocabulary is "banana", and the corresponding first index of "banana" is 654069, then the second index of "apple" is determined to be 654069. According to the method described above, the remaining words are processed in the same way, and the second indices of {"apple", "banana", "cherry", "grape", "orange", "peach", "strawberry", "watermelon"} can be obtained as {654069, 963241, 854469, 184533, 154562, 526599, 616555, 115265, 616555}.
[0113] Step 104, construct a target mapping dictionary according to the mapping relationship between the first index of each word in the original vocabulary and the second index in the scrambled vocabulary, and encrypt the target mapping dictionary and send it to the client.
[0114] Here, after the server determines the second index of each vocabulary in the original vocabulary list in the confusion vocabulary list, it can construct a target mapping dictionary according to the mapping relationship between the first index of each vocabulary in the original vocabulary list and the second index in the confusion vocabulary list.
[0115] In a possible embodiment, following the above example, the first indexes of each vocabulary in the original vocabulary list are respectively {154562, 654069, 184533, 854469, 963241, 115265, 616555, 526599}, and the second indexes of each vocabulary in the original vocabulary list are respectively {654069, 963241, 854469, 184533, 154562, 526599, 616555, 115265, 616555}. Assume that the second index corresponding to the vocabulary ω' j in the confusion vocabulary list is j, and the first index corresponding to the vocabulary ω i in the original vocabulary list is i. Then, a target mapping dictionary D can be constructed according to the mapping relationship between the first index of the vocabulary in the original vocabulary list and the second index in the confusion vocabulary list. The target mapping dictionary D can be expressed as:
[0116]
[0117] Among them, (i, j) indicates that the word ω i with index i in the original vocabulary list corresponds to the word ω j ' with index j in the confusion vocabulary list. Exemplarily, the first index of "apple" in the original vocabulary list is 154562, the first index 154562 of "apple" corresponds to the second index 654069 in the confusion vocabulary list, and the vocabulary corresponding to the second index 654069 in the confusion vocabulary list is "b an a n a". Then, the target mapping dictionary D can be expressed as That is, the meaning of the target mapping dictionary D is that "apple" in the original vocabulary list corresponds to "banana" in the confusion vocabulary list. It should be noted that by the same method, the mapping relationships between the remaining vocabularies can be obtained, which will not be elaborated here.
[0118] In a possible embodiment, after the server constructs the target mapping dictionary, it can encrypt the target mapping dictionary with the user's public key and then send it to the client. It should be noted that the encryption method for the target mapping dictionary is not limited here. After receiving the target mapping dictionary, the user of the client can decrypt the target mapping dictionary with the private key to obtain the corresponding relationship between the vocabularies in the original vocabulary list and the confusion vocabulary list of the large language model.
[0119] An encryption method, device, equipment, medium and product based on a large language model provided by an embodiment of the present disclosure. In an embodiment of the present application, first, the vocabulary in the original vocabulary table of the large language model is grouped to obtain a plurality of target groups, and the target hash value of each vocabulary in the target group is determined; then, based on the target hash value, the first index of each vocabulary in the hash ring is determined; wherein, the first index in the hash ring is used to indicate the position of each vocabulary in the original vocabulary table; after that, the vocabulary in the original vocabulary table is rearranged to obtain a confused vocabulary table, and the second index of each vocabulary in the original vocabulary table in the confused vocabulary table is determined; wherein, the second index is used to indicate the position of each vocabulary in the original vocabulary table in the confused vocabulary table; finally, according to the mapping relationship between the first index of each vocabulary in the original vocabulary table and the second index in the confused vocabulary table, a target mapping dictionary is constructed, and the target mapping dictionary is encrypted and sent to the client.
[0120] As can be seen from the above description, the technical solution of the present disclosure can determine the first index of each vocabulary in the hash ring based on the target hash value, and can rearrange the vocabulary in the original vocabulary table to obtain a confused vocabulary table, so as to determine the second index of each vocabulary in the original vocabulary table in the confused vocabulary table. Through the first index and the second index, a target mapping dictionary can be constructed. The target mapping dictionary is used to indicate the mapping relationship between the first index of each vocabulary in the original vocabulary table and the second index in the confused vocabulary table. Compared with encrypting at the input and output layers of the large language model, the technical solution of the present disclosure realizes the encryption of the large language model itself by constructing a target mapping dictionary, improving the security of the large language model; the technical solution of the present disclosure can also encrypt the target mapping dictionary and send it to the client. The client can realize real-time interaction with the large language model through the target mapping dictionary. The technical solution of the present disclosure improves the security of the large language model while ensuring the efficient operation of the large language model in different application scenarios.
[0121] In one embodiment, when the server determines the first index of each vocabulary in the original vocabulary table in the hash ring based on the target hash value, the following steps are further included:
[0122] If the first indexes of different vocabularies in different target groups are the same, then any one of the different vocabularies corresponding to the same first index is subjected to hash encryption mapping until the first indexes of different vocabularies in different target groups are different.
[0123] Here, the server sequentially determines the mapping nodes of the target hash values of each vocabulary in each target group in the hash ring according to the order of the target groups, and determines the value corresponding to the mapping node as the first index of the corresponding vocabulary. If the first indexes of different vocabularies in different target groups are the same, then any one of the different vocabularies corresponding to the same first index is subjected to hash encryption mapping until the first indexes of different vocabularies in different target groups are different.
[0124] In a possible embodiment, exemplarily, when the server sequentially determines the mapping nodes of the target hash values of each word in each target group in the hash ring, the first index of word 1 in target group 1 is 695215, and the first index of word 2 in target group 2 is 695215. The first index of word 1 in target group 1 is the same as the first index of word 2 in target group 2. At this time, a hash collision occurs. Then, word 1 in target group 1 is re-hash encrypted and mapped. Exemplarily, strong hash algorithms such as SHA-256, Murmur Hash, and SHA-3 can be used for the hash encryption mapping of word 1 to re-obtain the first index of word 1. At this time, the hash collision problem is solved. If the re-obtained first index of word 1 still has a hash collision with the first index of a word in other target groups, then word 1 in target group 1 is continuously re-hash encrypted and mapped until there is no repetition in the first indexes of words in different target groups, that is, there is no hash collision situation.
[0125] In this embodiment, for the case of hash collision, if the first indexes of different words in different target groups are the same, the server will perform hash encryption mapping on any one of the different words corresponding to the same first index until the first indexes of different words in different target groups are different.
[0126] As can be seen from the above description, in the case of hash collision, the server can re-hash encrypt and map the words that have hash collision to obtain new first indexes, avoiding the confusion or conflict of different data items, ensuring the accuracy of the data, and thus improving the reliability of the encryption method based on the large language model.
[0127] In one embodiment, as Figure 2 shown, after the server constructs the target mapping dictionary according to the mapping relationship between the first index of each word in the original vocabulary and the second index in the confusion vocabulary, the following steps are further included:
[0128] Step 201, obtain the fine-tuning corpus of the large language model.
[0129] Here, after the server constructs the target mapping dictionary according to the mapping relationship between the first index of each word in the original vocabulary and the second index in the confusion vocabulary, it can obtain the fine-tuning corpus of the large language model. Among them, the fine-tuning corpus refers to a set of text data used to fine-tune a pre-trained model in the fields of machine learning and natural language processing (NLP). The fine-tuning corpus can be provided and stored by the user or the business requester according to the application scenario.
[0130] Step 202, replace the words in the fine-tuning corpus based on the target mapping dictionary to obtain a confused fine-tuning corpus dataset.
[0131] Here, after the server obtains the fine-tuning corpus of the large language model, it can replace the vocabulary in the fine-tuning corpus based on the target mapping dictionary to obtain the confused fine-tuning corpus dataset.
[0132] In a possible embodiment, for example, the fine-tuning corpus of the large language model obtained by the server is {"I love this movie.", "This movie is terrible."}. The server can replace the vocabulary in the fine-tuning corpus based on the target mapping dictionary to obtain the confused fine-tuning corpus dataset. Among them, the target mapping dictionary is {"love" → "like", "terrible" → "bad"}. The server replaces the vocabulary "love" and "terrible" in the fine-tuning corpus according to the target mapping dictionary {"love" → "like", "terrible" → "bad"} to obtain the confused fine-tuning corpus dataset {"I like this movie.", "This movie is bad."}.
[0133] Step 203: Update the large language model based on the confused fine-tuning corpus dataset to obtain the updated large language model.
[0134] Here, after the server replaces the vocabulary in the fine-tuning corpus based on the target mapping dictionary to obtain the confused fine-tuning corpus dataset, it can update the large language model based on the confused fine-tuning corpus dataset to obtain the updated large language model.
[0135] In a possible embodiment, updating the large language model based on the confused fine-tuning corpus dataset to obtain the updated large language model includes the following steps:
[0136] Determine the location and parameters of the embedding layer of the large language model;
[0137] With the parameters of other network layers in the large language model remaining unchanged except for the embedding layer, input the confused fine-tuning corpus dataset into the large language model;
[0138] Determine the target parameters of the embedding layer in the large language model after inputting the confused fine-tuning corpus dataset to obtain the updated large language model.
[0139] Specifically, the server replaces the vocabulary in the fine-tuning corpus based on the target mapping dictionary to obtain the obfuscated fine-tuning corpus dataset. First, the server determines the location and parameters of the embedding layer of the large language model. Exemplarily, the server can load the large language model into a Graphics Processing Unit (GPU), check the structure of the large language model, and determine the location and parameters of the Embedding layer.
[0140] Then, while keeping the parameters of other network layers in the large language model except the embedding layer unchanged, the server inputs the obfuscated fine-tuning corpus dataset into the large language model. Exemplarily, the server can lock the parameters of other network layers except the Embedding layer of the model, which can ensure that during the fine-tuning process, other parts of the large language model remain stable, and the semantic ability of the large language model does not distort, and the performance and stability of the large language model will not be affected by unnecessary parameter changes. After the server locks the parameters of other network layers except the Embedding layer of the model, it inputs the obfuscated fine-tuning corpus dataset into the large language model.
[0141] After that, the server determines the target parameters of the embedding layer in the large language model after inputting the obfuscated fine-tuning corpus dataset to obtain the updated large language model.
[0142] In a possible embodiment, after the server inputs the obfuscated fine-tuning corpus dataset into the large language model, the server starts to fine-tune the parameters of the Embedding layer and sets the initial training parameters. The number of training rounds is 300 epochs, the learning rate is 1.0e-6, and the batch size is 16. If the performance of the validation set does not improve within 10 epochs, the training stops. It should be noted that hyperparameters are some parameters that need to be manually set in advance before the model starts training in fields such as machine learning and deep learning. The hyperparameters in the above embodiment can be adjusted and optimized according to the results of the loss function in each round of training, that is, gradually decrease or increase the learning rate and batch size, and observe the loss value in the final convergence situation to achieve the minimum final loss value and ensure that the large language model can effectively learn new corpora and adapt to the new obfuscated vocabulary list. Among them, for the loss function (LossFunction) of the large language model, let L represent the loss function. If it is a multi-classification problem, the cross-entropy loss L can be expressed as:
[0143]
[0144] where m is the batch size, K is the number of classes of the task, and y i,k is the true label that the i-th sample belongs to class k (belonging is 1, otherwise 0), is the predicted probability that the i-th sample belongs to class k.
[0145] During the process of fine-tuning the parameters of the Embedding layer, the server closely monitors the performance metrics of the model, such as accuracy, loss value, etc. The validation set can be used to evaluate the performance of the model, and the training parameters and strategies can be adjusted in a timely manner. Among them, if the Embedding layer of the model is denoted as E, then locking the remaining model parameters can be formalized as:
[0146] E new = E old [D].
[0147] Among them, E new represents the target parameters of the updated Embedding layer, E old represents the Embedding layer of the original model, and E old [D] represents the update operation according to the target mapping dictionary. After obtaining the target parameters E new of the updated Embedding layer, the server inputs the target parameter E new into the Embedding layer of the large language model to obtain the updated large language model.
[0148] In this embodiment, first, the server obtains the fine-tuning corpus of the large language model; then, the server replaces the vocabulary in the fine-tuning corpus based on the target mapping dictionary to obtain the confused fine-tuning corpus dataset; finally, the server updates the large language model based on the confused fine-tuning corpus dataset to obtain the updated large language model.
[0149] As can be seen from the above description, the server uses the target mapping dictionary to replace the original characters in the fine-tuning corpus to generate the confused fine-tuning corpus dataset, which helps to confuse the original data, enhances the security of the data, protects the original information, prevents the leakage of sensitive information, and provides a more secure environment for the training of the large language model; the server adds an embedding layer to the large language model and performs fine-tuning to obtain the updated large language model, which can make the large language model adapt to the new confused vocabulary table, not only accurately understand and process the confused vocabulary in the encrypted state, but also enhance the adaptability of the large language model to different data and improve the performance of the large language model in specific tasks.
[0150] In one embodiment, as Figure 3 shown, after the server encrypts the target mapping dictionary and distributes it to the client, the following steps are further included:
[0151] Step 301, receive the encrypted data sent by the client.
[0152] Here, after the server encrypts the target mapping dictionary and sends it to the client, it can receive the encrypted data sent by the client, where the encrypted data is obtained by the client encrypting the user's plaintext data.
[0153] In a possible embodiment, the user of the client maps the plaintext data X according to the decrypted target mapping dictionary to obtain the encrypted data M. The encrypted data M can be specifically expressed as:
[0154] M = {ω′ j |ω i ∈X, (i, j) ∈ D}.
[0155] Where X represents the plaintext data, ω i represents the vocabulary in X, and ω′ j represents the vocabulary mapped from the target mapping dictionary D to the obfuscation vocabulary. The server can receive this encrypted data M.
[0156] Step 302, decrypt the encrypted data through the large language model to obtain the first decryption result.
[0157] Here, after the server receives the encrypted data sent by the client, it can decrypt the encrypted data through the large language model to obtain the first decryption result.
[0158] In a possible embodiment, after the server receives the encrypted data sent by the client, it can input the encrypted data M into the large language model, and the large language model processes the encrypted data M to generate the first decryption result N.
[0159] Step 303, send the first decryption result to the client.
[0160] Here, after the server decrypts the encrypted data through the large language model to obtain the first decryption result, it can send the first decryption result to the client.
[0161] In this embodiment, first, the server receives the encrypted data sent by the client; where the encrypted data is obtained by the client encrypting the user's plaintext data; then, the server decrypts the encrypted data through the large language model to obtain the first decryption result; finally, the server sends the first decryption result to the client.
[0162] As described above, the client first encrypts the user's plaintext data, ensuring the security of user data during transmission and preventing data from being stolen or tampered with during network transmission. Only the server with the corresponding decryption means can process the encrypted data, thus protecting the privacy of users and the confidentiality of data. Moreover, this mode of encrypting first and then decrypting ensures the reliability and integrity of data transmission between the client and the server. The client sends encrypted data, and the server sends back the result after decryption, guaranteeing the integrity of information during data transmission and reducing information deviation caused by data loss or errors.
[0163] In one embodiment, as Figure 4 shown, a large language model-based encryption method is provided, which is applied to the client and includes the following steps:
[0164] Step 401, receive the encrypted target mapping dictionary sent by the server.
[0165] In a possible embodiment, the client can receive the encrypted target mapping dictionary sent by the server, where the encrypted target mapping dictionary is generated by the server through the large language model-based encryption method in the above embodiment.
[0166] Step 402, decrypt the encrypted target mapping dictionary with the private key to obtain the correspondence between the words in the original vocabulary and the obfuscated vocabulary of the large language model.
[0167] In a possible embodiment, after receiving the encrypted target mapping dictionary sent by the server, the client can decrypt the encrypted target mapping dictionary with the private key to obtain the correspondence between the words in the original vocabulary and the obfuscated vocabulary of the large language model.
[0168] Step 403, encrypt the user's plaintext data based on the correspondence to obtain encrypted data.
[0169] Here, after decrypting the encrypted target mapping dictionary with the private key to obtain the correspondence between the words in the original vocabulary and the obfuscated vocabulary of the large language model, the client can encrypt the user's plaintext data based on the correspondence to obtain encrypted data.
[0170] In a possible embodiment, the user's plaintext data is {I want to get it}. The client decrypts the encrypted target mapping dictionary with the private key and obtains the correspondence between the words in the original vocabulary and the obfuscated vocabulary of the large language model as {"I" → "You", "want" → "like", "get" → "give", "it" → "one"}. Then the user's plaintext data {I want to get it} is encrypted into {You like to give one}.
[0171] Step 404, send the encrypted data to the server.
[0172] In a possible embodiment, following the above example, after the client encrypts the user's plaintext data based on the correspondence and obtains the encrypted data {You like to give one}, the client sends the encrypted data {You like to give one} to the server.
[0173] In this embodiment, first, the client receives the encrypted target mapping dictionary sent by the server; then, the client decrypts the encrypted target mapping dictionary with the private key to obtain the correspondence between the words in the original vocabulary and the obfuscated vocabulary of the large language model; after that, the client encrypts the user's plaintext data based on the correspondence to obtain the encrypted data; finally, the client sends the encrypted data to the server.
[0174] From the above description, it can be seen that the client decrypts the encrypted target mapping dictionary with the private key. Only the client with the corresponding private key can obtain the correct correspondence, which further ensures the security of the encryption method based on the large language model; and by encrypting the user's plaintext data and sending the encrypted data to the server, even if the data is intercepted during transmission, the attacker cannot directly read and understand the user's plaintext data, effectively protecting the privacy of user data and preventing the leakage of user data during network transmission.
[0175] In an embodiment, as Figure 5 shown, after the client sends the encrypted data to the server, the following steps are further included:
[0176] Step 501, receive the first decryption result sent by the server.
[0177] In a possible embodiment, after the client sends the encrypted data to the server, it can receive the first decryption result sent by the server.
[0178] Step 502, based on the pre-established sensitive word list, perform deletion and / or replacement processing on the sensitive fragments in the first decryption result to obtain the second decryption result.
[0179] Here, after receiving the first decryption result sent by the server, the client can, based on a pre-established sensitive word list, perform deletion and / or replacement processing on the sensitive fragments in the first decryption result to obtain a second decryption result.
[0180] In a possible embodiment, the client can use an agent to perform a risk assessment on the first decryption result. The first decryption result received by the client is {Your address is ACommunity}, where the pre-established sensitive word list is {"A" → "XXX"}. Then the client performs replacement processing on the sensitive fragment "A" in the first decryption result to obtain the second decryption result {Your address is XXX Community}. It should be noted that the client can also perform deletion processing on the sensitive fragments in the first decryption result, and the specific processing of the sensitive fragments in the first decryption result is not limited here.
[0181] Step 503: Decrypt the second decryption result based on the target mapping dictionary to obtain the target return result.
[0182] Here, after the client performs deletion and / or replacement processing on the sensitive fragments in the first decryption result to obtain the second decryption result, it can decrypt the second decryption result based on the target mapping dictionary to obtain the target return result.
[0183] In a possible embodiment, the client can truncate the second decryption result according to the truncation identifier in the second decryption result, and then decrypt the second decryption result through the target mapping dictionary to finally obtain the target return result. The truncation identifier usually uses the backslash \, dollar sign $, etc. for word segmentation identification. It should be noted that the specific form of the truncation identifier is not limited here.
[0184] Exemplarily, the second decryption result obtained by the client is {Is\the\answer\to\your\question\XXX?}. The client can truncate the second decryption result according to the truncation identifier "\ " in the second decryption result, and then decrypt the second decryption result through the target mapping dictionary. Exemplarily, the target mapping dictionary is {"answer" → "A", "question" → "B"}, then the client finally obtains the target return result as {Is the A to your B XXX?}.
[0185] In this embodiment, first, the client receives the first decryption result sent by the server; then, based on the pre-established sensitive word list, the client deletes and / or replaces the sensitive fragments in the first decryption result to obtain the second decryption result; finally, the client decrypts the second decryption result based on the target mapping dictionary to obtain the target return result.
[0186] As can be seen from the above description, by deleting or replacing the sensitive fragments in the first decryption result, the client can effectively protect sensitive information, prevent the leakage of users' privacy information, ensure that the finally output information does not contain sensitive words that may cause privacy problems, and improve the security and confidentiality of user information; moreover, the client first processes the sensitive information in the first decryption result and then performs the final decryption according to the target mapping dictionary to obtain the target return result. This multi-stage processing method increases the security and flexibility of the decryption process. Different operations can be performed according to different requirements and security policies at different stages, providing more control means for the secure processing of data and further improving the security of the encryption method based on the large language model.
[0187] In an alternative embodiment, by combining the above embodiments, a logical schematic diagram of the encryption method based on the large language model can be obtained, as Figure 6 shown Figure 6 exemplarily shows the logical schematic diagram of the encryption method based on the large language model.
[0188] For Figure 6 the model side shown in, the model side is the server in the above embodiment. First, the model side obtains the basic model vocabulary A, which is the original vocabulary in the above embodiment. The model side performs hash mapping encryption processing on the basic model vocabulary A to obtain the first index of the basic model vocabulary A; then, based on the first index of the basic model vocabulary A, the model side can generate the vocabulary B after confusion, and the vocabulary B is the confused vocabulary in the above embodiment; after that, the model side establishes and saves the target mapping dictionary D between the vocabulary A and the vocabulary B, encrypts it with the user's public key, and distributes it to the user; after that, the model side updates the fine-tuning corpus based on the target mapping dictionary D and fine-tunes the Embed ding layer of the large model to obtain the adapted large model, and the adapted large model is the updated large language model in the above embodiment.
[0189] For Figure 6On the user side shown in [description], the user side is the client in the above embodiments. First, the user of the client receives and uses the private key to decrypt the target mapping dictionary D to encrypt the plaintext data input by the user, mapping the plaintext X to the ciphertext M. The ciphertext M is the encrypted data in the above embodiments. Then, the client sends the ciphertext M to the adapted large language model, which can decrypt the ciphertext M to generate the encrypted output N. The encrypted output N is the first decryption result obtained in the above embodiments. After that, the client receives the streaming return result, conducts a risk assessment on the encrypted output N, and based on the pre-established sensitive word list, deletes and / or replaces sensitive fragments in the encrypted output N to obtain the second decryption result. Finally, the client truncates the message as needed and uses the target mapping dictionary D to decrypt the second decryption result to obtain the plaintext reply, which is the target return result in the above embodiments.
[0190] In a possible embodiment, the encryption method based on the large language model provided in this embodiment can be applied in different scenarios: in the intelligent customer service scenario of an enterprise, customers can safely ask various questions without worrying about the leakage of business secrets or personal information; in academic research cooperation, researchers can use the large language model for data analysis and discussion without worrying about the theft of sensitive research data; in the daily use of intelligent assistants, users can obtain accurate and secure services without worrying about the interference of bad information.
[0191] From the above description, it can be seen that in the actual application scenario, when the user interacts with the large language model, whether asking questions, requesting analysis, or seeking creative inspiration, the input content will be processed by the word list confusion encryption technology on the client side to ensure that the user's sensitive information and privacy data will not be transmitted in plaintext within the model, greatly reducing the risk of data privacy leakage.
[0192] The technical solution of the present disclosure uses multiple hash encryptions as the mapping method for vocabulary obfuscation, which has stronger anti-cracking ability compared with the linear mapping of traditional technologies, making it difficult for attackers to infer the corresponding relationship between the original vocabulary and the obfuscated vocabulary by analyzing the structure of the obfuscated vocabulary, and minimizing the hash collision problem generated by vocabulary mapping as much as possible; the technical solution of the present disclosure supports the dynamic update of the obfuscated vocabulary. When new vocabulary is added or old vocabulary is deleted, the obfuscated vocabulary can be recalculated by updating the mapping algorithm and parameters, and the mapping relationship can be quickly reconstructed. Compared with the traditional double hash mapping algorithm, the technical solution of the present disclosure uses a concurrent design when constructing the vocabulary, effectively solving the problems of poor performance and long time consumption brought by multiple hash calculations, and can cope with more frequent vocabulary updates; compared with the simple random shuffling vocabulary obfuscation method, the technical solution of the present disclosure can keep the mapping relationship of most vocabulary unchanged when the vocabulary is expanded and changed through consistent hash mapping, to a certain extent, ensuring the cost and fitting stability of model fine-tuning and update, which enables the obfuscated vocabulary to better adapt to the continuous development and changes of large language models; in the process of encryption and decryption, the technical solution of the present disclosure supports directly encrypting and decrypting quickly through the obfuscated vocabulary without re-executing the encryption calculation, with very high encryption and decryption efficiency, and is adapted to the real-time interaction scenario of large language models; the result of the obfuscation calculation proposed by the technical solution of the present disclosure is predictable and reproducible, so the obfuscated vocabulary can be more easily verified and managed, further improving the security of the large language model encryption method.
[0193] In the case of dividing each functional module according to the corresponding functions, the embodiment of the present disclosure provides an encryption device based on a large language model, and the encryption device based on the large language model can be a server or a chip applied to the server. Figure 7 It is a schematic block diagram of the functional modules of the encryption device based on the large language model provided by an exemplary embodiment of the present disclosure. As Figure 7 shown, the encryption device based on the large language model includes:
[0194] The first determination module 701 is configured to group the vocabulary in the original vocabulary of the large language model to obtain a plurality of target groups, and determine the target hash value of each vocabulary in the target group;
[0195] The second determination module 702 is configured to determine the first index of each vocabulary in the hash ring based on the target hash value; wherein, the first index in the hash ring is used to indicate the position of each vocabulary in the original vocabulary;
[0196] The third determination module 703 is configured to rearrange the vocabulary in the original vocabulary to obtain an obfuscated vocabulary, and determine the second index of each vocabulary in the original vocabulary in the obfuscated vocabulary; wherein, the second index is used to indicate the position of each vocabulary in the original vocabulary in the obfuscated vocabulary;
[0197] The first sending module 704 is configured to construct a target mapping dictionary according to the mapping relationship between the first index of each word in the original word list and the second index in the confusion word list, and encrypt and send the target mapping dictionary to the client.
[0198] In one embodiment, the first determining module 701 includes:
[0199] The first calculation unit is configured to calculate the first hash value of each word in each of the target groups through a hash algorithm;
[0200] The second calculation unit is configured to perform rounding processing on the first hash value to obtain the second hash value of the word;
[0201] The third calculation unit is configured to calculate the target hash value of the word based on the second hash value.
[0202] In one embodiment, the second determining module 702 includes:
[0203] The hash ring construction unit is configured to construct the hash ring based on the target interval range; wherein each node in the hash ring corresponds to a numerical value of a sub-range in the target interval range;
[0204] The first determining unit is configured to determine the first index of each word in the original word list in the hash ring based on the target hash value.
[0205] In one embodiment, the second determining module 702 includes:
[0206] The second determining unit is configured to sequentially determine the mapping nodes of the target hash values of each word in each of the target groups in the hash ring according to the order of the target groups, and determine the numerical value corresponding to the mapping node as the first index of the corresponding word.
[0207] In one embodiment, the device further includes:
[0208] The encryption mapping module is configured to, if the first indexes of different words in different target groups are the same, perform hash encryption mapping on any one of the different words corresponding to the same first index until the first indexes of different words in different target groups are different.
[0209] In one embodiment, the device further includes:
[0210] The first obtaining module is configured to obtain the fine-tuning corpus of the large language model;
[0211] A second acquisition module, configured to replace the vocabulary in the fine-tuning corpus based on the target mapping dictionary to obtain a confused fine-tuning corpus dataset;
[0212] An update module, configured to update the large language model based on the confused fine-tuning corpus dataset to obtain an updated large language model.
[0213] In one embodiment, the update module includes:
[0214] A third determination unit, configured to determine the position and parameters of the embedding layer of the large language model;
[0215] An input unit, configured to input the confused fine-tuning corpus dataset into the large language model while keeping the parameters of other network layers in the large language model except the embedding layer unchanged;
[0216] An acquisition unit, configured to determine the target parameters of the embedding layer in the large language model after inputting the confused fine-tuning corpus dataset to obtain the updated large language model.
[0217] In one embodiment, the apparatus further includes:
[0218] A second receiving module, configured to receive the encrypted data sent by the client; wherein, the encrypted data is obtained by the client encrypting the user's plaintext data;
[0219] A second decryption module, configured to decrypt the encrypted data through the large language model to obtain a first decryption result;
[0220] A third sending module, configured to send the first decryption result to the client.
[0221] In the case of dividing each function into corresponding function modules, an encryption apparatus based on a large language model provided by an embodiment of the present disclosure may be a client or a chip applied to the client. Figure 8 It is a schematic block diagram of function modules of an encryption apparatus based on a large language model provided by an exemplary embodiment of the present disclosure. As Figure 8 shown, the encryption apparatus based on the large language model includes:
[0222] A first receiving module 801, configured to receive the encrypted target mapping dictionary sent by the server; wherein, the encrypted target mapping dictionary is obtained by the server encrypting the target mapping dictionary through the encryption method based on the large language model described in any one of the above of the present disclosure;
[0223] The first decryption module 802 is configured to decrypt the encrypted target mapping dictionary with a private key to obtain the correspondence between the original vocabulary of the large language model and the words in the obfuscated vocabulary;
[0224] The encryption module 803 is configured to encrypt the user's plaintext data based on the correspondence to obtain encrypted data;
[0225] The second sending module 804 is configured to send the encrypted data to the server.
[0226] In one embodiment, the device further includes:
[0227] The third receiving module is configured to receive the first decryption result sent by the server;
[0228] The analysis module is configured to perform deletion and / or replacement processing on the sensitive segments in the first decryption result based on a pre-established list of sensitive words to obtain a second decryption result;
[0229] The third obtaining module is configured to decrypt the second decryption result based on the target mapping dictionary to obtain a target return result.
[0230] An embodiment of the present disclosure further provides an electronic device, including: at least one processor; a memory for storing executable instructions of the at least one processor; wherein, the at least one processor is configured to execute the instructions to implement the above method disclosed in the embodiments of the present disclosure.
[0231] Figure 9 It is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. As Figure 9 shown, the electronic device 900 includes at least one processor 901 and a memory 902 coupled to the processor 901. The processor 901 can execute the corresponding steps in the above method disclosed in the embodiments of the present disclosure.
[0232] The above-mentioned processor 901 can also be referred to as a central processing unit (CPU). It can be an integrated circuit chip with the ability to process signals. Each step in the above-mentioned method disclosed in the embodiments of the present disclosure can be completed by the integrated logic circuit in the hardware of the processor 901 or the instructions in the form of software. The above-mentioned processor 901 can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present disclosure can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module can be located in the memory 902, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and other mature storage media in the art. The processor 901 reads the information in the memory 902 and combines its hardware to complete the steps of the above-mentioned method.
[0233] In addition, when various operations / processes according to the present disclosure are implemented through software and / or firmware, a program constituting the software can be installed from a storage medium or a network into a computer system having a dedicated hardware structure, such as Figure 10 the computer system 1000 shown. When various programs are installed in the computer system, it can execute various functions, including the functions described above, etc. Figure 10 It is a block diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure.
[0234] The computer system 1000 is intended to represent various forms of digital electronic computer devices, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0235] As Figure 10As shown, computer system 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the computer system 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.
[0236] Multiple components in the computer system 1000 are connected to the I / O interface 1005, including: an input unit 1006, an output unit 1007, a storage unit 1008, and a communication unit 1009. The input unit 1006 can be any type of device that can input information into the computer system 1000. The input unit 1006 can receive input digital or character information, and generate key signal inputs related to user settings and / or function controls of the electronic device. The output unit 1007 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1008 can include but is not limited to a magnetic disk, an optical disk. The communication unit 1009 allows the computer system 1000 to exchange information / data with other devices through a network such as the Internet, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a BluetoothTM device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.
[0237] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1001 executes the various methods and processes described above. For example, in some embodiments, the above-described methods disclosed in the embodiments of the present disclosure can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 900 via the ROM 1002 and / or the communication unit 1009. In some embodiments, the computing unit 1001 can be configured to execute the above-described methods disclosed in the embodiments of the present disclosure by any other appropriate means (e.g., by means of firmware).
[0238] An embodiment of the present disclosure also provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above methods disclosed in the embodiments of the present disclosure.
[0239] The computer-readable storage medium in the embodiments of the present disclosure may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The above computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the above. More specifically, the above computer-readable storage medium may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0240] The above computer-readable medium may be included in the above electronic device; or may exist separately without being assembled into the electronic device.
[0241] Figure 11 A computer program product provided for an exemplary embodiment of the present disclosure, the computer program product 1100 includes a computer program 1101, wherein when the computer program 1101 is executed by a processor, the above methods disclosed in the embodiments of the present disclosure are implemented.
[0242] In the embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer.
[0243] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.
[0244] The modules, components, or units described in the embodiments of the present disclosure can be implemented in software or in hardware. Among them, the names of the modules, components, or units do not, in some cases, constitute a limitation on the modules, components, or units themselves.
[0245] The functions described above can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.
[0246] The above description is only some embodiments of the present disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.
[0247] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art should understand that the above examples are only for illustration and not for limiting the scope of the present disclosure. Those skilled in the art should understand that the above embodiments can be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.
Claims
1. An encryption method based on large language models, characterized in that, Applied to a server, the method includes: Group the words in the original vocabulary of the large language model to obtain multiple target groups, and determine the target hash value of each word in the target group; Determine the first index of each word in the hash ring based on the target hash value; wherein, the first index in the hash ring is used to indicate the position of each word in the original vocabulary; Rearrange the words in the original vocabulary to obtain a confused vocabulary, and determine the second index of each word in the original vocabulary in the confused vocabulary; wherein, the second index is used to indicate the position of each word in the original vocabulary in the confused vocabulary; Construct a target mapping dictionary according to the mapping relationship between the first index of each word in the original vocabulary and the second index in the confused vocabulary, and encrypt and send the target mapping dictionary to the client.
2. The method according to claim 1, wherein The determining the target hash value of each word in the target group includes: Calculate the first hash value of each word in each target group through a hash algorithm; Round the first hash value to obtain the second hash value of the word; Calculate the target hash value of the word based on the second hash value.
3. The method according to claim 2, wherein The determining the first index of each word in the hash ring based on the target hash value includes: Construct the hash ring based on the target interval range; wherein, each node in the hash ring corresponds to a numerical value in a sub-range of the target interval range; Determine the first index of each word in the original vocabulary in the hash ring based on the target hash value.
4. The method according to claim 3, wherein The determining the first index of each word in the original vocabulary in the hash ring based on the target hash value includes: In the order of the target groups, sequentially determine the mapping nodes of the target hash values of each word in each target group in the hash ring, and determine the numerical value corresponding to the mapping node as the first index of the corresponding word.
5. The method according to claim 4, wherein The method further includes: If the first indexes of different words in different target groups are the same, then perform hash encryption mapping on any one of the different words corresponding to the same first index until the first indexes of different words in different target groups are different.
6. The method according to claim 1, characterized in that, After constructing the target mapping dictionary according to the mapping relationship between the first index of each word in the original vocabulary and the second index in the confused vocabulary, the method further includes: Obtain the fine-tuning corpus of the large language model; Replace the words in the fine-tuning corpus based on the target mapping dictionary to obtain a confused fine-tuning corpus dataset; Update the large language model based on the confused fine-tuning corpus dataset to obtain an updated large language model.
7. The method according to claim 6, wherein The updating the large language model based on the confused fine-tuning corpus dataset to obtain an updated large language model includes: Determine the position and parameters of the embedding layer of the large language model; Input the confused fine-tuning corpus dataset into the large language model while keeping the parameters of other network layers except the embedding layer in the large language model unchanged. Determine the target parameters of the embedding layer in the large language model after fine-tuning the obfuscated corpus dataset of the input, and obtain the updated large language model.
8. The method according to claim 1, wherein, After encrypting and distributing the target mapping dictionary to the client, the method further includes: Receiving the encrypted data sent by the client; wherein, the encrypted data is obtained by the client encrypting the user's plaintext data; Decrypting the encrypted data through the large language model to obtain a first decryption result; Sending the first decryption result to the client.
9. An encryption method based on large language models, characterized in that, Applied to the client, the method includes: Receiving the encrypted target mapping dictionary sent by the server; wherein, the encrypted target mapping dictionary is obtained by the server encrypting the target mapping dictionary through the encryption method based on the large language model according to any one of claims 1 to 8 above; Decrypting the encrypted target mapping dictionary through the private key to obtain the correspondence between the words in the original vocabulary and the obfuscated vocabulary of the large language model; Encrypting the user's plaintext data based on the correspondence to obtain encrypted data; Sending the encrypted data to the server.
10. The method according to claim 9, characterized in that, After sending the encrypted data to the server, the method further includes: Receiving the first decryption result sent by the server; Based on the pre-established sensitive word list, performing deletion and / or replacement processing on the sensitive segments in the first decryption result to obtain a second decryption result; Decrypting the second decryption result based on the target mapping dictionary to obtain a target return result.
11. An encryption device based on a large language model, characterized in that, Disposed on the server, the device includes: A first determination module, configured to group the words in the original vocabulary of the large language model to obtain a plurality of target groups, and determine the target hash value of each word in the target group; A second determination module, configured to determine the first index of each word in the hash ring based on the target hash value; wherein, the first index in the hash ring is used to indicate the position of each word in the original vocabulary; A third determination module, configured to rearrange the words in the original vocabulary to obtain an obfuscated vocabulary, and determine the second index of each word in the original vocabulary in the obfuscated vocabulary; wherein, the second index is used to indicate the position of each word in the original vocabulary in the obfuscated vocabulary; A first sending module, configured to construct a target mapping dictionary according to the mapping relationship between the first index of each word in the original vocabulary and the second index in the obfuscated vocabulary, and encrypt and distribute the target mapping dictionary to the client.
12. An encryption device based on a large language model, characterized in that, Disposed on the client, the device includes: A first receiving module, configured to receive the encrypted target mapping dictionary sent by the server; wherein, the encrypted target mapping dictionary is obtained by the server encrypting the target mapping dictionary through the encryption method based on the large language model according to any one of claims 1 to 8 above; A first decryption module, configured to decrypt the encrypted target mapping dictionary through the private key to obtain the correspondence between the words in the original vocabulary and the obfuscated vocabulary of the large language model; An encryption module, configured to encrypt the user's plaintext data based on the correspondence to obtain encrypted data; A second sending module, configured to send the encrypted data to the server.
13. An electronic device, characterized in that, Comprising: At least one processor; A memory for storing executable instructions of the at least one processor; Wherein, the at least one processor is configured to execute the instructions to implement the method according to any one of claims 1-10.
14. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to any one of claims 1-10.
15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1-10 is implemented.