A large-model question-answering method and device capable of customizing personal privacy security protection

Abstract

The present invention relates to the technical field of AI large models, and provides a large model question-answering method and device for customizable personal privacy security protection. The method includes: pre-deploying a privacy protection model to the user's local device. The method includes: using the privacy protection model on the local device to identify the privacy information in the user input information and the category of each privacy information; according to the first information category that the user needs to hide, using the preset security word of the first information category to replace the first privacy information in the user input information to obtain a secure input information; transmitting the secure input information to the remote end, and using the general large model deployed at the remote end to output the secure answer corresponding to the secure input information. By deploying the privacy protection model locally, the present invention does not need to upload the user input information with privacy information to the network, improving the security of privacy protection. At the same time, the general large model deployed at the remote end has higher performance and can achieve better inference effects.

Description

Technical Field

[0001] The present invention relates to the field of AI big model technology, and in particular to a big model question-answering method and device capable of customizing personal privacy and security protection. Background Art

[0002] At present, large models have become human assistants, helping people to quickly complete various tasks. General large models such as ChatGPT and Wenxinyiyan are widely used in work and study. While improving people's work efficiency, they also bring about the problem of privacy leakage. It is not uncommon to hear news that user input information is passed into the model learning library. Large models can be divided into cloud large models and end-side large models. At present, general large models are generally deployed on the cloud and have sufficient computing power, so they generally have higher parameter quantities and better inspection effects, but they require us to transmit input information through the network, which will bring certain privacy leakage risks. Although the end-side large model is more secure, it is limited by hardware limitations. The parameters of the end-side large model are generally 1-2 orders of magnitude lower than those of the cloud-side large model, and the model effect is also much worse. Therefore, it is currently impossible to completely replace the cloud-side large model with the end-side large model. It can only be used in combination to reduce the risk of privacy leakage.

[0003] Therefore, it is difficult for existing technologies to achieve good reasoning effects while ensuring security.

[0004] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in the field of this technology. Summary of the invention

[0005] The technical problem to be solved by the present invention is that it is difficult for the existing technology to achieve good reasoning effect while ensuring security.

[0006] The present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a large-model question-answering method capable of customizing personal privacy and security protection, wherein a privacy protection model is pre-deployed on a user's local device, and the method comprises:

[0008] Using a privacy protection model on a local device, identifying private information and categories of private information in user input information;

[0009] According to the first information category that the user needs to hide, the first private information in the user input information is replaced with a preset security word corresponding to the first information category to obtain secure input information; wherein the first private information is private information belonging to the first information category, and the preset security word includes a hiding word and a category word;

[0010] The security input information is transmitted to a remote end, and a security answer corresponding to the security input information is output using a universal large model deployed at the remote end.

[0011] Preferably, the privacy protection model includes an embedding layer, a conversion layer, a first fully connected layer, and a second fully connected layer;

[0012] The output of the embedding layer is connected to the input of the transformation layer;

[0013] The output of the conversion layer is connected to the input of the first fully connected layer and the input of the second fully connected layer;

[0014] The embedding layer is used to convert the user input information into a vector, and the conversion layer is used to extract features from the vector;

[0015] The first fully connected layer is used to output a privacy vector according to the features output by the conversion layer, and the second fully connected layer is used to output a category vector according to the features output by the conversion layer; the privacy vector is used to represent whether each word in the user input information is private information, and the category vector is used to represent the category of each word in the user input information.

[0016] Preferably, the privacy protection model is pre-trained, and the loss function used in the training is: ;

[0017] in, ; ;in, and is the preset weight coefficient, N is the total number of samples, C is the number of information categories, The probability that the privacy protection model determines that the i-th sample belongs to private information, is the real privacy information label, The privacy protection model determines the probability that the i-th sample belongs to the j-th information category. is an indicator variable.

[0018] Preferably, the using of the privacy protection model on the local device to identify the privacy information and the category of each privacy information in the user input information specifically includes:

[0019] Inputting the user input information into the privacy protection model to obtain a privacy vector and a category vector; wherein, in the privacy vector, the value of the element corresponding to the private information is 1, and the value of the element corresponding to other information is 2, and in the category vector, each category corresponds to a unique numerical value;

[0020] The privacy vector is multiplied element by element by the category vector to obtain a privacy category vector; wherein the privacy category vector is used to represent the privacy information in the user input information and the category of each privacy information.

[0021] Preferably, the first privacy information in the user input information is replaced with a preset security word corresponding to the first information category according to the first information category that the user needs to hide to obtain the secure input information, specifically including:

[0022] Generate a replacement code according to the first information category; wherein the first bit in the replacement code is set to 0, and each of the other bits in the replacement code represents an information category, and when the represented information category belongs to the first information category, the value of the bit is set to 1, otherwise, the value of the bit is set to 0;

[0023] Access element i in the privacy vector in turn, determine whether the value of the i-th bit in the replacement code is 1, and if the value of the i-th bit in the replacement code is 1, replace the i-th word in the user input information with the preset security word corresponding to element i.

[0024] Preferably, after outputting the security answer corresponding to the security input information using the universal large model deployed at the remote end, the method further comprises:

[0025] Returning the security answer to the local device;

[0026] Replacing the preset security word in the security answer with the private information on the local device to obtain a user answer;

[0027] The user answer is returned to the user.

[0028] Preferably, replacing the preset security word in the security answer with the private information on the local device to obtain the user answer specifically includes:

[0029] Pre-replacing the first private information in the user input information with a preset security word corresponding to the first information category, and generating a mapping relationship between the private information and the preset security word for replacement;

[0030] After obtaining the security answer, the preset security word in the security answer is replaced with the corresponding privacy information according to the mapping relationship.

[0031] Preferably, the privacy protection model is quantized into INT8 format or INT4 format.

[0032] In a second aspect, the present invention further provides a large model question-answering device capable of customizing personal privacy and security protection, which is used to implement the large model question-answering method capable of customizing personal privacy and security protection described in the first aspect, and the device comprises:

[0033] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the large-model question-answering method with customizable personal privacy and security protection as described in the first aspect.

[0034] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method described in the first aspect.

[0035] In a fourth aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in the memory from a memory, and executing the method of the first aspect.

[0036] According to a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed on a computer or a processor, the computer or the processor executes the method according to the first aspect.

[0037] On the one hand, the present invention establishes a privacy protection model and a general large model, deploys the privacy protection model locally, deploys the general large model remotely, uses the privacy protection model locally to perform privacy security protection on user input information, and then uploads the privacy-protected input to the remote end to use the general large model for question-answering reasoning, thereby eliminating the need to upload user input information with privacy information to the network, thereby improving the security of privacy protection. At the same time, the general large model deployed remotely has higher performance, thereby being able to achieve better reasoning effects, thereby achieving good reasoning effects on the basis of ensuring privacy security. In addition, it supports users to customize privacy protection according to information categories, thereby accurately protecting the data that users need to protect, and by introducing category words into preset security words, reducing information loss in the privacy protection process, thereby improving the accuracy of subsequent question-answering reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1It is a flowchart of a first large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0040] Figure 2 It is a schematic diagram of a first large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0041] Figure 3 It is a structural schematic diagram of a privacy protection model in a large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0042] Figure 4 It is a flow chart of a second large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0043] Figure 5 It is a flowchart of a third large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0044] Figure 6 It is a flowchart of a fourth large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0045] Figure 7 It is a flowchart of a fifth large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0046] Figure 8 is a schematic diagram of a second large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0047] Fig. 9 It is a schematic diagram of a privacy protection model in a large-model question-answering method for customizable personal privacy security protection provided by an embodiment of the present invention;

[0048] Fig.10 is a schematic diagram of a third large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0049] Fig.11 is a schematic diagram of a fourth large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0050] Fig.12 is a schematic diagram of a fifth large-model question-answering method capable of customizing personal privacy security protection provided by an embodiment of the present invention;

[0051] Fig.13 It is a schematic diagram of the architecture of a large-model question-and-answer device that can be customized for personal privacy and security protection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0053] Unless the context requires otherwise, throughout the specification and claims, the term "including" is to be interpreted as open inclusion, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "examples", "specific examples" or "some examples" and the like are intended to indicate that specific features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner, that is, although they may be carried in the embodiments or examples of the above terms due to reasons such as the order and position of appearance, it is not limited to that they can be carried in combination by one embodiment or example.

[0054] In the description of the present invention, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more. In addition, for example, the same type of nouns may be described as two independent individuals by adding "A" and "B" at the end. In this case, the corresponding features defined as "A" and "B" are only used to distinguish the same type of individuals for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features.

[0055] In the description of the present invention, the expression "A and / or B" (where A and B are used to formally represent specific characteristic contents) is involved, and the corresponding expressions include the following three combinations: only A, only B, and a combination of A and B.

[0056] As used herein, "about," "substantially," or "approximately" includes the stated value and an average value that is within an acceptable range of deviation from the particular value as determined by one of ordinary skill in the art taking into account the measurements in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0057] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0058] Embodiment 1:

[0059] It is difficult for the existing technology to achieve good reasoning effects on the basis of ensuring security. After searching the existing technical literature, it was found that in the invention patent "A method for detecting and protecting personal information in a large language model" with patent application number 202310571582.8, a sensitive information protection method was adopted to shield sensitive enterprise information; in the invention patent "A privacy protection method and user terminal in the process of accessing a large model" with patent application number 202410382363.X, a security aspect was deployed in the user terminal to build a security terminal, and the privacy data was desensitized to achieve the protection of user privacy data; in the invention patent "A privacy protection user portrait analysis method based on a large language model" with patent application number 202410330981.X, the collected privacy information was used to generate a user personal portrait; in the invention patent "A document privacy security assessment method, device, equipment and medium" with patent application number 202310782089.0, the enterprise data was desensitized and restored after the large model answer was completed. In the invention patent “Personal Data Privacy Protection and Recovery Method Based on Large Language Model Service” with patent application number 202410355339.7, the data is desensitized to obtain answers. These disclosed patent inventions have several common points, namely, most of the current privacy protection systems are oriented towards enterprises, which not only have high deployment costs, but also lack flexibility and cannot meet the needs of individual users with poor hardware and changing privacy definitions. Moreover, the existing technologies all completely desensitize the data, that is, remove all privacy data. On the one hand, this method is difficult to perform privacy and security protection according to the needs of each user. On the other hand, it will also cause the information required by the user to be lost due to the desensitization operation, which directly affects the effect of subsequent questions and answers. In order to solve these problems, Example 1 of the present invention provides a large model question and answer method that can be customized for personal privacy and security protection, and the privacy protection model is pre-deployed on the user's local device, such as Figure 1 and Figure 2 As shown, the method includes:

[0060] In step 201, the privacy protection model on the local device is used to identify the privacy information and the categories of each privacy information in the user input information; the local device can be understood as a device for receiving user input information. In actual use, the local device can usually be a user's terminal, such as a mobile phone, computer, etc.

[0061] In step 202, according to the first information category that the user needs to hide, the first private information in the user input information is replaced with the preset security word corresponding to the first information category to obtain the secure input information; wherein, the first private information is the private information belonging to the first information category, and the preset security word includes a hidden word and a classifier; it should be noted that the private information is general, and is the information with user privacy obtained by those skilled in the art based on empirical analysis, and the user can choose whether to protect a certain category of private information, that is, the first information category is the private information that the user needs to protect securely. The hidden word can be a certain, * or a fixed mark, etc., and the classifier is used to characterize the information category to which the private information belongs. For example, in actual use, the information category can include time, place, person, etc., and the corresponding classifier can be time, place, person, that is, the corresponding preset security word can be a certain time, a certain place, a certain person, so as to protect the user's private information while minimizing the degree of information loss caused by privacy protection as much as possible.

[0062] In step 203, the security input information is transmitted to the remote end, and the security answer corresponding to the security input information is output using the universal large model deployed at the remote end. The remote end can be understood as a device on the network, which has higher performance and more sufficient computing power than the local device. The universal large model can be Wenxin Yiyan, Zhipu Qingyan, Qwen large model, etc.

[0063] On the one hand, this embodiment establishes a privacy protection model and a general large model, deploys the privacy protection model locally, and deploys the general large model remotely. The privacy protection model is used locally to perform privacy security protection on user input information, and then the privacy-protected input is uploaded to the remote end to perform question-answering reasoning using the general large model, thereby eliminating the need to upload user input information with privacy information to the network, thereby improving the security of privacy protection. At the same time, the general large model deployed remotely has higher performance, thereby achieving better reasoning effects, and thus achieving good reasoning effects on the basis of ensuring privacy security. In addition, it supports users to customize privacy protection according to information categories, thereby accurately protecting the data that users need to protect, and by introducing category words into preset security words, it reduces information loss during the privacy protection process, thereby improving the accuracy of subsequent question-answering reasoning.

[0064] In a preferred embodiment, Figure 3 As shown, the privacy protection model includes an embedding layer, a conversion layer, a first fully connected layer and a second fully connected layer.

[0065] The output of the embedding layer is connected to the input of the conversion layer; the output of the conversion layer is connected to the input of the first fully connected layer and the input of the second fully connected layer; the embedding layer is used to convert the user input information into a vector, and the conversion layer is used to extract features from the vector.

[0066] The first fully connected layer is used to output a privacy vector according to the features output by the conversion layer, and the second fully connected layer is used to output a category vector according to the features output by the conversion layer; the privacy vector is used to represent whether each word in the user input information is private information, and the category vector is used to represent the category of each word in the user input information.

[0067] In actual use, the embedding layer is also called the Embedding layer, and the conversion layer is also called the TransformerEncoder layer. In actual use, it includes multiple Transformer Encoder layers, and the inputs and outputs of the multiple TransformerEncoder layers are connected in series in sequence. It can be understood that: the above preferred implementation method improves the structure on the basis of the original BERT large model, and obtains the structure of the privacy protection model by setting two fully connected layers (i.e., the first fully connected layer and the second fully connected layer).

[0068] The privacy protection model is pre-trained, and the loss function used in the training is: .

[0069] in, ; .in, and is a preset weight coefficient, which is obtained by technicians in this field according to demand analysis. To determine the loss of privacy information, is the information category classification loss, N is the total number of samples, C is the number of information categories, The probability that the privacy protection model determines that the i-th sample belongs to private information, is the real private information label. When the i-th sample is private information, The value of is 1, otherwise, The value of is 0; The privacy protection model determines the probability that the i-th sample belongs to the j-th information category. is an indicator variable. When the i-th sample data belongs to the j-th information category, The value of is 1, otherwise, The value of is 0. and are obtained by pre-marking the i-th sample by a technician in this field, and represent the true result of the i-th sample. and It is output by the privacy protection model and is the result of the model's prediction of the i-th sample.

[0070] In one embodiment, the privacy protection model is quantized into INT8 format or INT4 format to reduce the demand for hardware storage and reasoning capabilities.

[0071] In a preferred embodiment, the privacy protection model on the local device is used to identify the privacy information and the category of each privacy information in the user input information, such as Figure 4 As shown, specifically including:

[0072] In step 301, the user input information is input into the privacy protection model to obtain a privacy vector and a category vector; wherein, in the privacy vector, the value of the element corresponding to the privacy information is 1, and the value of the element corresponding to other information is 2, and in the category vector, each category corresponds to a unique numerical value; and for the information category that needs to be paid attention to, the numerical value of each information category is greater than 0.

[0073] In step 302, the privacy vector is multiplied element by element with the category vector to obtain a privacy category vector; wherein the privacy category vector is used to represent the privacy information in the user input information and the category of each privacy information.

[0074] For example, assuming that the user input information contains the words ABCDEFGH, the first fully connected layer determines whether each word is private information, if it is private information, it is 1, if it is not private information, it is 0, and the privacy vector 10011100 is obtained. The second fully connected layer assigns elements according to the information category of the word. For example, assigning 0, 1, 2, 3, and 4 to the five categories of others, time, location, identity, and value respectively, can obtain the category vector 20030104. Since the information category of others is not concerned, it can be assigned 0. The two are multiplied element by element to obtain the privacy category vector 20030100, that is, 0-4 represents non-private information, time privacy information, location privacy information, identity privacy information, and value privacy information respectively.

[0075] According to the first information category that the user needs to hide, the first private information in the user input information is replaced with a preset security word corresponding to the first information category to obtain secure input information, such as Figure 5 As shown, specifically including:

[0076] In step 401, a replacement code is generated according to the first information category; wherein each bit in the replacement code represents an information category, and when the represented information category belongs to the first information category, the value of the bit is set to 1, otherwise, the value of the bit is set to 0; in actual use, the bits at preset positions in the replacement code can also be set to 0 to represent words that do not belong to any information category.

[0077] In step 402, element i in the privacy vector is accessed in sequence to determine whether the value of the i-th bit in the replacement code is 1. If the value of the i-th bit in the replacement code is 1, the i-th word in the user input information is replaced with the preset security word corresponding to element i, where i is the value of the element.

[0078] Still taking the privacy category vector 20030100 as an example, if the user sets the first information category to be protected to be time information and location information, if the information category includes 5 categories, the value corresponding to the time information is 1, and the value corresponding to the location information is 2, then the generated replacement code is 00011.

[0079] Since the first bit and the second bit of the replacement code are 1, the word F in the user input information corresponding to the value 1 in the privacy category vector 20030100 is replaced with the preset security word of the time information, such as "sometime", and the word A in the user input information corresponding to the value 2 in the privacy category vector 20030100 is replaced with the preset security word of the location information, such as "somewhere".

[0080] The above preferred implementation directly uses the vector output by the privacy protection model for calculation, thereby reducing the amount of calculation.

[0081] In some optional implementations, the security answer can be directly transmitted to the local device and then returned to the user by the local device. However, considering that there may be some privacy information in the security answer, it is easy to make the security answer not intuitive enough. In order to solve this problem, this embodiment also has a preferred implementation, that is, after the universal large model deployed at the remote end outputs the security answer corresponding to the security input information, Figure 6 As shown, the method also includes:

[0082] In step 501, the security answer is returned to the local device.

[0083] In step 502, the preset security word in the security answer is replaced with the private information on the local device to obtain a user answer.

[0084] In step 503, the user answer is returned to the user; wherein the user answer can be understood as a relatively intuitive answer obtained based on the combination of the security answer and the user input information.

[0085] The preset security word in the security answer is replaced with the private information on the local device to obtain the user answer, such as Figure 7 As shown, specifically including:

[0086] In step 601, the first private information in the user input information is replaced in advance with a preset security word corresponding to the first information category, and a mapping relationship between the private information and the preset security word used for replacement is generated.

[0087] In step 602, after obtaining the security answer, the preset security word in the security answer is replaced with the corresponding privacy information according to the mapping relationship. The mapping relationship is updated each time after receiving user input information.

[0088] In actual use, user input information and the output of the privacy protection model are also stored as the user's private privacy protection dataset. After the privacy protection model has been used for a period of time, the privacy protection model on the end side can be personalized and fine-tuned based on the stored private privacy protection dataset to improve the large model's ability to understand user needs.

[0089] Embodiment 2:

[0090] The present invention is based on the method described in Example 1, combined with specific application scenarios, and uses technical descriptions in related scenarios to illustrate the implementation process of the present invention in characteristic scenarios.

[0091] Currently, when using a large model, the input information needs to be transmitted through the network to the cloud-based large model for inference, and then the output result is returned. In this process, the private information in the input faces the risk of being stolen, stored, and used. Based on this problem, the current idea is to protect private information based on the collaboration between the end-side large model and the cloud-based large model. The end-side large model identifies and blurs the private content, and then transmits it to the cloud-based large model for inference, making full use of the high security of the end-side large model and the strong inference ability of the cloud-based large model. Some companies have launched a protection system for corporate confidential information to protect the company's important private information when employees use the large model. However, this system is usually designed only for enterprises. For individual users, there is still a lack of methods to protect personal privacy information when using the cloud-based large model.

[0092] As for the privacy protection of individual users, the current privacy protection methods have the following problems: First, the existing privacy protection solutions do not mention how to build a suitable data set to train the end-side privacy protection large model; second, the privacy protection needs of individual users are more flexible and changeable than those of enterprises, and users need to choose the content of privacy protection themselves; finally, the input of individual users varies from person to person, and a more personalized large model is needed to meet the unique needs of users.

[0093] To address the above issues, a big model question-answering method with customizable personal privacy and security protection is proposed. The main process includes training a privacy-preserving big model (i.e., privacy-preserving model), building a personalized prompt sentence generation system, collaborative reasoning of the big model on the end and cloud, and personalized fine-tuning of the big model on the end (i.e., general big model).

[0094] The traditional privacy protection model can only replace all the private information in the input, but cannot learn the specific category of private information. In the process of use by individual users, it is often necessary to replace only part of the private information, such as replacing a person's name and retaining other private information. In this case, the privacy protection model trained by the traditional method is less effective. This patent uses a multi-task learning method to split the privacy protection task into two tasks: private information identification and information category classification. The input sentence is divided into several words, and each word is judged to be private information and the information category of the word. Among them, the information category of the word is divided into five categories: other, time, place, identity, and value. That is, a paragraph is divided into words such as ABCDEFGH, and each word is judged to be private information. If it is private information, it is 1, and if it is not private information, it is 0, and the privacy vector 10011100 is obtained. The five categories of word information categories, other, time, place, identity, and value, are 0, 1, 2, 3, and 4 respectively, and the category vector 20030104 can be obtained. The two are multiplied element by element to obtain a vector 20030100, where 0-4 represent non-private information, time privacy information, location privacy information, identity privacy information, and numerical privacy information, respectively. The time privacy information, location privacy information, identity privacy information, and numerical privacy information are replaced by words such as a certain time, a certain place, a certain person, and so on according to the prompt word requirements.

[0095] In terms of specific implementation, the first step is to build the dataset. The training data of the large model needs to be composed of an input and an output. The input includes the original input sentence; the output includes the privacy information judgment result and the information category classification result of each word in the sentence. The privacy information judgment result is a vector of 0 or 1, and the information category classification result is a vector of 0-4. The data input is based on common Chinese datasets such as THUCNews. In model training, the BERT basic large model is used for training. The original BERT large model contains an Embedding layer, a multi-layer Transformer Encoder, and an output layer. By modifying the structure of the BERT large model, the final output layer is modified into two parallel fully connected layers, one responsible for privacy information judgment and the other responsible for information category classification, and the loss function is modified. The specific loss function is as follows:

[0096]

[0097]

[0098]

[0099] in, To determine the loss of privacy information, is the information category classification loss, and is the preset weight coefficient, N is the total number of samples, C is the number of information categories, The probability that the privacy protection model determines that the i-th sample belongs to private information, is the real private information label. When the i-th sample is private information, The value of is 1, otherwise, The value of is 0; The privacy protection model determines the probability that the i-th sample belongs to the j-th information category. is an indicator variable. When the i-th sample data belongs to the j-th information category, The value of is 1, otherwise, The value of is 0. and are obtained by pre-marking the i-th sample by a technician in this field, and represent the true result of the i-th sample. and It is output by the privacy protection model and is the result of the model's prediction of the i-th sample.

[0100] Combining the results of the two, different categories of private information are replaced separately to get the output.

[0101] The overall process is as follows:

[0102] The basic data input is based on common Chinese data sets such as THUCNews. After large-model assisted annotation and manual cleaning, the corresponding privacy information discrimination results and information category classification results in each user's input information are obtained. The input and output are combined to form a complete input-output pair to construct a privacy-preserving data set. The output layer of the BERT large model is modified, and the loss function is modified. Training is performed to obtain a privacy-preserving large model. Different categories of privacy information are replaced to obtain output (i.e., secure input information).

[0103] This patent uses a personalized prompt sentence generation system to solve the problem that individual users cannot freely choose the privacy content that needs to be protected. Unlike enterprises, individual users are more flexible in terms of the privacy content that needs to be protected. In different scenarios and different problems, the privacy content to be protected will change, and users need to be able to choose freely. The process of the personalized prompt sentence generation system is: first, allow users to freely generate protection requirements; then, generate standard prompt word sentences through the personalized prompt sentence generation system to guide the privacy protection model to perform directional reasoning; finally, combine the input sentence and the standardized prompt sentence to obtain an input sentence with personalized requirements, which can be used for reasoning by the end-side privacy protection model. The specific process is as follows:

[0104] The user inputs a question statement that requires reasoning (i.e., user input information); the user selects or freely inputs privacy protection requirements; the personalized prompt statement generation system generates a standard prompt statement based on user requirements; the question statement and the standardized prompt statement are combined to obtain an input statement (i.e., secure input information).

[0105] In the collaborative reasoning process of the cloud-side big model, the output statements of the end-side big model are mainly input into the cloud-side big model, and the results desired by the user are inferred. In this process, the statements can be mainly divided into three types: end-side big model input, end-side big model output / cloud-side big model input, and cloud-side big model output. Among them, the end-side big model input contains privacy information and can only be input into the end-side big model locally, while the end-side big model output / cloud-side big model input and cloud-side big model output do not contain privacy information and can be transmitted over the network. Through this process, the security of the end-side big model and the reasoning performance of the cloud-side big model are fully utilized, protecting privacy information while not affecting the model reasoning ability. The specific process is as follows:

[0106] The input sentence of the end-side big model is sent to the end-side privacy protection big model for reasoning to obtain the end-side big model output; the end-side big model output is used as the cloud-side big model input and sent to the cloud-side big model through the network; the cloud-side big model infers the cloud-side big model input to obtain the cloud-side big model output (i.e., the secure answer); the cloud-side big model output is returned to the user as the big model output result; the end-side big model input and the end-side big model output are saved locally as a private privacy protection data set.

[0107] After a user has used the privacy protection big model for a period of time, some private privacy protection data sets will be accumulated locally. These data are the user's real privacy data. Using these data to fine-tune the end-side privacy protection big model can obtain a more personalized privacy protection big model and improve the reasoning ability for the user's personal needs. The specific process is as follows:

[0108] Use the private privacy protection dataset to fine-tune the end-side privacy protection big model; convert the fine-tuned end-side privacy protection big model into a standard format; and use the fine-tuned end-side privacy protection big model to replace the previous generation privacy protection big model.

[0109] The following is an example of a specific application scenario. The specific implementation process of a large model question-answering method for customizable personal privacy security protection provided by this embodiment is as follows: Figure 8 As shown, the specific process can be divided into building a privacy-preserving public dataset, training and deploying a large privacy-preserving model on the edge (i.e., the privacy-preserving model), generating a personalized prompt sentence generation system, collaborative reasoning of the large model on the edge and cloud, and personalized fine-tuning of the large privacy-preserving model on the edge.

[0110] The training and deployment of the large end-side privacy protection model specifically includes: Fig. 9 As shown in the figure, the BERT basic model is used for training. The original BERT model contains an Embedding layer, a multi-layer Transformer Encoder, and an output layer. By modifying the structure of the BERT model, the final output layer is modified into two parallel fully connected layers, one for privacy information judgment and the other for information category classification. The loss function is also modified. The loss function used after the modification is: .

[0111] in, ; .

[0112] In the process of training and deploying the large model for edge privacy protection, due to the limited deployment conditions of individual users, a relatively small model can be selected as the basic large model for training. During the deployment process, the model can be quantized into INT8 or INT4 format according to the user's usage environment to reduce the demand for hardware storage and reasoning capabilities.

[0113] The generation of a personalized prompt sentence generation system specifically includes: when a user inputs a question sentence (i.e., user input information), the personalized prompt sentence generation system mainly converts it into a standard prompt sentence (i.e., secure input information) according to the user's needs, thereby enhancing the reasoning ability of the end-side privacy protection large model. It can not only protect a certain type of privacy information, but also protect a single noun. Fig.10 As shown, the sentence of the user input information is "On the morning of July 25, 2024, Li Hua met Zhang Wei in Beijing, and they planned to visit the World Expo in Shanghai together." The privacy protection requirement is that Li Hua and time should not appear. Therefore, the standard prompt word sentence generated is to replace the name Li Hua and all times in the text. By combining the original sentence and the standard prompt word sentence, the final end-side big model input "On the morning of July 25, 2024, Li Hua met Zhang Wei in Beijing, and they planned to visit the World Expo in Shanghai together. Replace the name Li Hua and all times in the text", and after the end-side big model input is inferred by the end-side big model, the end-side big model output "At a certain time, someone met Zhang Wei in Beijing, and they planned to visit the World Expo in Shanghai together" can be obtained. The process of converting the user's needs into a standard prompt sentence has been described in detail in Example 1, and will not be repeated here.

[0114] The end-cloud side big model collaborative reasoning specifically includes: because the end-side big model has insufficient reasoning capability and the cloud-side big model has the risk of privacy leakage, the end-cloud side big model collaborative reasoning is used in the reasoning process. Fig.11 As shown in the figure, the privacy-protected sentence (i.e., user input information) is inferred through the local privacy-protected big model to obtain a sentence without privacy information (i.e., secure input information). The sentence without privacy information is then transmitted over the network as the input of the cloud-based big model, and the cloud-based big model (i.e., the general cloud-based big model) performs inference to obtain the final result. At the same time, the input and output of the end-side privacy-protected big model are stored locally as the user's private privacy-protected dataset.

[0115] The personalized fine-tuning of the large privacy protection model on the end specifically includes: after a period of training, the large privacy protection model on the end can be personalized and fine-tuned according to the stored private privacy protection data set to improve the large model's ability to understand user needs. The specific operation process is as follows: Fig.12 shown.

[0116] This embodiment modifies the structure of the BERT big model, splits the privacy protection task into two relatively simple tasks: privacy information identification and information category classification, and trains them simultaneously, so that the privacy protection big model can learn the type of privacy information at the same time. In addition, by providing a personalized prompt sentence generation system, it can flexibly respond to various privacy protection needs of users, support users to customize their needs, and create a user-customized end-side privacy protection big model.

[0117] Embodiment 3:

[0118] like Fig.13 , is a schematic diagram of the architecture of a large model question-answering device with customizable personal privacy and security protection according to an embodiment of the present invention. The large model question-answering device with customizable personal privacy and security protection according to this embodiment includes one or more processors 21 and a memory 22. Fig.13 A processor 21 is taken as an example.

[0119] The processor 21 and the memory 22 may be connected via a bus or other means. Fig.13 The example of connecting through bus is taken in the following.

[0120] The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs and non-volatile computer executable programs, such as the large model question-answering method for customizable personal privacy security protection in Example 1. The processor 21 executes the large model question-answering method for customizable personal privacy security protection by running the non-volatile software programs and instructions stored in the memory 22.

[0121] The memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 22 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0122] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, the large-model question-answering method with customizable personal privacy security protection in the above-mentioned embodiment 1 is executed.

[0123] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0124] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A large-scale question-answering method with customizable personal privacy and security protection, characterized in that: Pre-deploy the privacy-preserving model to the user's local device by: Using a privacy protection model on a local device, identifying private information and categories of each private information in the user input information; According to the first information category that the user needs to hide, the first private information in the user input information is replaced with a preset security word corresponding to the first information category to obtain secure input information; wherein the first private information is private information belonging to the first information category, and the preset security word includes a hiding word and a category word; Transmitting the security input information to a remote end, and using a universal large model deployed at the remote end to output a security answer corresponding to the security input information; The privacy protection model includes an embedding layer, a conversion layer, a first fully connected layer, and a second fully connected layer; The output of the embedding layer is connected to the input of the conversion layer; The output of the conversion layer is connected to the input of the first fully connected layer and the input of the second fully connected layer; The embedding layer is used to convert the user input information into a vector, and the conversion layer is used to extract features from the vector; The first fully connected layer is used to output a privacy vector according to the features output by the conversion layer, and the second fully connected layer is used to output a category vector according to the features output by the conversion layer; the privacy vector is used to represent whether each word in the user input information is private information, and the category vector is used to represent the category of each word in the user input information.

2. The large-scale question-answering method with customizable personal privacy security protection according to claim 1 is characterized in that: The privacy protection model is pre-trained, and the loss function used in the training is: ; in, ; ;in, and is the preset weight coefficient, N is the total number of samples, C is the number of information categories, The probability that the privacy protection model determines that the i-th sample belongs to private information, is the real privacy information label, The privacy protection model determines the probability that the i-th sample belongs to the j-th information category. is an indicator variable.

3. The large-scale question-answering method with customizable personal privacy security protection according to claim 1 is characterized in that: The method of using the privacy protection model on the local device to identify the privacy information and the category of each privacy information in the user input information specifically includes: Inputting the user input information into the privacy protection model to obtain a privacy vector and a category vector; wherein, in the privacy vector, the value of the element corresponding to the private information is 1, and the value of the element corresponding to other information is 2, and in the category vector, each category corresponds to a unique numerical value; The privacy vector is multiplied element by element by the category vector to obtain a privacy category vector; wherein the privacy category vector is used to represent the privacy information in the user input information and the category of each privacy information.

4. The large-scale question-answering method with customizable personal privacy security protection according to claim 3 is characterized in that: The method of replacing the first private information in the user input information with a preset security word corresponding to the first information category according to the first information category that the user needs to hide, to obtain the security input information, specifically includes: Generate a replacement code according to the first information category; wherein the first bit in the replacement code is set to 0, and each of the other bits in the replacement code represents an information category, and when the represented information category belongs to the first information category, the value of the bit is set to 1, otherwise, the value of the bit is set to 0; Access element i in the privacy vector in turn, determine whether the value of the i-th bit in the replacement code is 1, and if the value of the i-th bit in the replacement code is 1, replace the i-th word in the user input information with the preset security word corresponding to element i.

5. The large-scale question-answering method with customizable personal privacy security protection according to claim 1 is characterized in that: After outputting the security answer corresponding to the security input information using the universal large model deployed at the remote end, the method further includes: Returning the security answer to the local device; Replacing the preset security word in the security answer with the private information on the local device to obtain a user answer; The user answer is returned to the user.

6. The large-scale question-answering method with customizable personal privacy security protection according to claim 5 is characterized in that: The step of replacing the preset security word in the security answer with the private information on the local device to obtain the user answer specifically includes: Pre-replacing the first private information in the user input information with a preset security word corresponding to the first information category, and generating a mapping relationship between the private information and the preset security word for replacement; After obtaining the security answer, the preset security word in the security answer is replaced with the corresponding privacy information according to the mapping relationship.

7. The large-scale question-answering method with customizable personal privacy security protection according to any one of claims 1 to 6, characterized in that: The privacy protection model is quantized into INT8 format or INT4 format.

8. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the large-model question-and-answer method for customizable personal privacy and security protection as described in any one of claims 1-7.

9. A large-scale question-answering device that can be customized to protect personal privacy and security, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the large-model question-answering method with customizable personal privacy and security protection as described in any one of claims 1-7.

Images