Application method and device of large model
By text filtering and knowledge editing of the big model, the toxicity problem in the model is eliminated, and the security risks of the big model outputting harmful information is solved, achieving the effect of reducing the output of sensitive information and improving the ability to resist attacks.
Patent Information
- Application Number
- CN202411404579.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-03-04
- Estimated Expiration
- 2044-10-09
AI Technical Summary
The big model has toxic problems during the content generation process, and outputs harmful information such as false information, hateful speech, privacy leaked content, etc., which brings safety hazards to users.
The model is toxic to eliminate the model through text filtering and knowledge editing, the area containing harmful information in the model parameters is positioned and the input content is filtered, and the harmful input is refused to reply.
On the premise of maintaining the original capabilities of the model, the probability of the model outputting sensitive information is reduced, the model's ability to resist attacks is improved, and the security of the large model is improved.
Smart Images
Figure CN118916903B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing, and in particular to a large model application method and device. Background Art
[0002] Currently, large language models are widely used in various business scenarios, which can greatly improve work efficiency. However, the application of large models also brings a series of security issues.
[0003] The harmful data that the big model itself is exposed to during the training process, or the prompt injection attacks on the big model, system prompt word attacks, etc., all pose challenges to the content generation security of the big model. The above security risks may cause the model to output harmful information such as false information, hate speech, privacy leakage content or other harmful content, thereby posing a security risk to users, which is defined as the toxicity of the big model.
[0004] Therefore, how to detoxify large models, reduce the probability of the model outputting sensitive information while maintaining the original capabilities of the model, and improve the model's ability to resist attacks is an urgent problem that needs to be solved. Summary of the invention
[0005] The embodiment of the present invention provides a large model application method and device for eliminating toxicity of the model through text filtering and knowledge editing. By locating the area containing harmful information in the model parameters and modifying this area, the harmful content contained therein is accurately and effectively removed, and then the input content of the large model is filtered, and harmful input is refused to respond. Under the premise of maintaining the original capabilities of the model, the probability of the model outputting sensitive information can be reduced, and the model's ability to resist attacks can be improved.
[0006] In a first aspect, an embodiment of the present invention provides an application method of a large model, including:
[0007] Obtaining an input request text and determining whether the request text contains sensitive information;
[0008] If the request text contains sensitive information, a first identifier is added to the request text and input into the first model to obtain an output rejection text, wherein the first model is edited based on the knowledge of the large model, and the first identifier is used to indicate that the first model refuses to generate a response to the request text;
[0009] If the request text does not contain sensitive information, the request text is input into the first model to obtain an output response text.
[0010] In the above technical solution, the user inputs the request text based on the user terminal. After obtaining the request text, the server determines whether the request text contains sensitive information to perform a preliminary screening of the request text. A first identifier is added to the request text containing sensitive information to instruct the first model to refuse to generate a response to the request text, thereby reducing model consumption. In addition, the first model is obtained based on the knowledge editing of the large model, and the model's ability to detoxify private information is improved while ensuring that the general ability of the model is less reduced. After the request text that does not contain sensitive information is input into the first model, the response text output by the first model can be obtained. Security protection is implemented on both the input side and the model side itself. On the premise of maintaining the original capabilities of the model, the probability of the model outputting sensitive information can be reduced, and the model's ability to resist attacks can be improved, thereby improving the security capabilities of the large model in practical applications.
[0011] Optionally, determining whether the request text contains sensitive information includes:
[0012] The request text is input into the second model to obtain a detection result output by the second model; the detection result is used to indicate whether the request text contains sensitive information.
[0013] Optionally, before obtaining the request data, the method further includes:
[0014] Constructing a first request text and a second request text respectively, wherein the first request text contains sensitive information, and the second request text does not contain sensitive information;
[0015] Inputting the first request text into the large model to obtain a first response text, and determining a first output text output by each hidden layer in the large model when generating the first response text;
[0016] Inputting the second request text into the large model to obtain a second response text, and determining a second output text output by each hidden layer in the large model when generating the second response text;
[0017] Determine the sensitive layer in the large model according to the Euclidean distance between the first output text and the second output text output by each hidden layer;
[0018] The parameters of the sensitive layer in the large model are optimized based on the first request text, the first response text, the second request text and the second response text to obtain the first model.
[0019] In the above technical solution, a first request text and a second request text are constructed respectively. The first request text contains sensitive information, and the second request text does not contain sensitive information. The large model contains at least one hidden layer. When the request text is input into the large model, it passes through at least one hidden layer in sequence, and finally obtains a response text. The sensitive layer of the model is located by comparing the difference between the output texts of each hidden layer of the first model of the first request text and the second request text. And according to the first request text, the first response text, the second request text and the second response text, the parameters of the sensitive layer in the large model are optimized to obtain the first model. It is realized that the model parameters are edited to improve the model's ability to detoxify sensitive information while ensuring that the general ability of the model is less reduced.
[0020] Optionally, determining the sensitive layer in the large model according to the Euclidean distance between the first output text and the second output text output by each hidden layer includes:
[0021] For any hidden layer, calculating the Euclidean distance and the growth value of the Euclidean distance between the first output text and the second output text output by the hidden layer;
[0022] The hidden layer corresponding to the maximum Euclidean distance and the hidden layer corresponding to the maximum growth value of the Euclidean distance are used as sensitive layers in the large model.
[0023] In the above technical solution, the hidden layer corresponding to the maximum Euclidean distance indicates that the sensitive information of the model has accumulated to the maximum value. The hidden layer corresponding to the maximum growth value of the Euclidean distance indicates that sensitive information begins to appear in the generated content of the large model.
[0024] Optionally, optimizing parameters of a sensitive layer in the large model based on the first request text, the first response text, the second request text, and the second response text to obtain the first model includes:
[0025] Vectorizing the first response text to obtain a vectorized first response text, inputting the first request text into the optimized large model to obtain a third response text, and calculating a cross entropy loss between the vectorized first response text and the third response text;
[0026] Inputting the second request text into the optimized large model to obtain a fourth response text, and calculating the information divergence between the second response text and the fourth response text;
[0027] Calculate the change ratio of the parameters of the sensitive layer in the optimized large model;
[0028] The cross entropy loss, the information divergence and the change ratio are weightedly summed to obtain the loss function of the large model, and the parameters of the sensitive layer in the large model are iterated based on the loss function to obtain the first model.
[0029] In the above technical solution, the cross entropy loss between the vectorized first response text and the third response text can be understood as the editing loss, which is used to measure the detoxification ability of the large model. The information divergence between the second response text and the fourth response text can be understood as the general ability loss, which is used to use general knowledge to ensure the general ability of the model. The change ratio of the parameters of the sensitive layer in the optimized large model can be understood as the offset loss, which measures the performance change between the edited model and the original model.
[0030] In a second aspect, an embodiment of the present invention provides an application device of a large model, including:
[0031] An acquisition module is used to acquire an input request text and determine whether the request text contains sensitive information;
[0032] a processing module, configured to add a first identifier to the request text if the request text contains sensitive information, and input the first identifier into a first model to obtain an output rejection text, wherein the first model is edited based on the knowledge of a large model, and the first identifier is used to indicate that the first model refuses to generate a response to the request text;
[0033] If the request text does not contain sensitive information, the request text is input into the first model to obtain an output response text.
[0034] Optionally, the processing module is specifically used for:
[0035] The request text is input into the second model to obtain a detection result output by the second model; the detection result is used to indicate whether the request text contains sensitive information.
[0036] Optionally, the processing module is further used for:
[0037] Constructing a first request text and a second request text respectively, wherein the first request text contains sensitive information, and the second request text does not contain sensitive information;
[0038] Inputting the first request text into the large model to obtain a first response text, and determining a first output text output by each hidden layer in the large model when generating the first response text;
[0039] Inputting the second request text into the large model to obtain a second response text, and determining a second output text output by each hidden layer in the large model when generating the second response text;
[0040] Determine the sensitive layer in the large model according to the Euclidean distance between the first output text and the second output text output by each hidden layer;
[0041] The parameters of the sensitive layer in the large model are optimized based on the first request text, the first response text, the second request text and the second response text to obtain the first model.
[0042] Optionally, the processing module is specifically used for:
[0043] For any hidden layer, calculating the Euclidean distance and the growth value of the Euclidean distance between the first output text and the second output text output by the hidden layer;
[0044] The hidden layer corresponding to the maximum Euclidean distance and the hidden layer corresponding to the maximum growth value of the Euclidean distance are used as sensitive layers in the large model.
[0045] Optionally, the processing module is specifically used for:
[0046] Vectorizing the first response text to obtain a vectorized first response text, inputting the first request text into the optimized large model to obtain a third response text, and calculating a cross entropy loss between the vectorized first response text and the third response text;
[0047] Inputting the second request text into the optimized large model to obtain a fourth response text, and calculating the information divergence between the second response text and the fourth response text;
[0048] Calculate the change ratio of the parameters of the sensitive layer in the optimized large model;
[0049] The cross entropy loss, the information divergence and the change ratio are weightedly summed to obtain the loss function of the large model, and the parameters of the sensitive layer in the large model are iterated based on the loss function to obtain the first model.
[0050] In a third aspect, an embodiment of the present invention further provides a computer device, including:
[0051] A memory for storing program instructions;
[0052] The processor is used to call the program instructions stored in the memory and execute the above-mentioned large model application method according to the obtained program.
[0053] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned large model application method.
[0054] In a fifth aspect, an embodiment of the present invention further provides a computer program product, wherein the computer program product includes an executable program, and the executable program is executed by a processor to implement the above-mentioned large model application method. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0056] Figure 1 A schematic diagram of a system architecture provided for an embodiment of the present invention;
[0057] Figure 2 A schematic diagram of a flow chart of a large model application method provided by an embodiment of the present invention;
[0058] Figure 3 A schematic structural diagram of a large model application device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0060] The application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions protected by the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It is known to those of ordinary skill in the art that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. The terms "first" and "second" in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.
[0061] Before introducing an application method of a large model provided in an embodiment of the present application, in order to facilitate understanding, the following terms and background technologies involved in the embodiment of the present application are first introduced.
[0062] Currently, large language models are widely used in a variety of business scenarios, such as intelligent conversational customer service, text generation and interpretation, etc., which greatly improves work efficiency; however, the application of large models also brings a series of security issues, including internal risks such as possible problems with training data and external risks such as prompt injection attacks. These security risks may cause the model to output harmful information such as false information, hate speech, privacy leakage or other harmful content, thereby posing a security risk to users. We define this as large model toxicity.
[0063] Large model detoxification can reduce the probability of the model outputting harmful content and improve the model's ability to resist attacks while maintaining the model's original capabilities.
[0064] Generative Models: Generative models are a class of statistical models that can represent and capture the distribution of data. Generative models are widely used in various tasks, including natural language generation, image generation (such as GANs), speech synthesis, unsupervised learning, etc.
[0065] Transformer neural network architecture: The advantages of Transformer are its parallelism, flexibility, and ability to capture long-distance dependencies. Due to its excellent performance and universality, it has become a standard in the field of NLP. Modern models such as BERT and GPT are designed based on the Transformer architecture.
[0066] Figure 1 A system architecture applicable to an embodiment of the present invention is exemplarily shown. The system architecture includes a server 100 . The server 100 may include a processor 110 , a communication interface 120 , and a memory 130 .
[0067] The communication interface 120 is used to transmit data with the terminal device.
[0068] The processor 110 is the control center of the server 100, and uses various interfaces and routes to connect various parts of the entire server 100, and executes various functions of the server 100 and processes data by running or executing software programs and / or modules stored in the memory 130, and calling data stored in the memory 130. Optionally, the processor 110 may include one or more processing units.
[0069] The memory 130 can be used to store software programs and modules. The processor 110 executes various functional applications and data processing by running the software programs and modules stored in the memory 130. The memory 130 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function, etc.; the data storage area can store data created according to business processing, etc. In addition, the memory 130 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0070] It should be noted that the above Figure 1 The structure shown is only an example and is not limited to this embodiment of the present invention.
[0071] Based on the above description, Figure 2 A schematic flow chart of a large model application method provided by an embodiment of the present invention is exemplarily shown, and the flow chart can be executed by a large model application device.
[0072] like Figure 2 As shown in the figure, the process specifically includes:
[0073] Step 210, obtaining the input request text, and determining whether the request text contains sensitive information.
[0074] In the embodiment of the present invention, the user can enter the request text on the client of the terminal device and send it to the server. The terminal device can be a mobile phone, a tablet computer, a laptop computer, a netbook, a car device, a business intelligent terminal (including: videophone, conference desktop intelligent terminal, etc.), a personal digital assistant (personal digital assistant, PDA), an augmented reality (augmented reality, AR)\virtual reality (virtual reality, VR) device, etc. The embodiment of the present application does not limit the specific form of the terminal device. The client can be an APP, a small program, a web page, etc. The embodiment of the present application does not limit the specific form of the client.
[0075] After obtaining the request text, the server will determine whether the request text contains sensitive information. Sensitive information may include harmful content such as false information, hate speech, and privacy leakage. Specifically, the request text is input into the second model to obtain the detection result output by the second model. The detection result is used to indicate whether the request text contains sensitive information. The second model can be any model with text classification capabilities, such as the RoBERTa text classification model, and the second model is not specifically limited here. Exemplarily, based on the RoBERTa model, fine-tuning training is performed according to sensitive text data and non-sensitive text data to obtain the second model.
[0076] Step 220: If the request text contains sensitive information, a first identifier is added to the request text and input into the first model to obtain an output rejection text. The first model is edited based on the large model knowledge, and the first identifier is used to indicate that the first model refuses to generate a response to the request text.
[0077] In an embodiment of the present invention, when the request text contains sensitive information, the server adds a first identifier to the request text and inputs it into the first model. Since the first identifier is used to indicate that the first model refuses to generate a response to the request text, that is, the first model refuses to respond to the request text with the first identifier, the first model will output a rejection text, such as "The text contains sensitive information, please correct it." The content of the rejection text is not specifically limited here.
[0078] The first model is obtained based on the knowledge editing of the large model, that is, first locate the sensitive layer in the large model, then edit the parameters of the sensitive layer in the large model based on the loss function, and finally obtain the first model. It can be understood that when judging whether the request text contains sensitive information, misjudgment may occur. Therefore, knowledge editing of the large model can improve the model's ability to detoxify sensitive information while ensuring that the general ability of the model is not reduced. The specific knowledge editing process includes:
[0079] First, a first request text and a second request text are constructed respectively, wherein the first request text contains sensitive information, and the second request text does not contain sensitive information.
[0080] Then, the first request text is input into the large model to obtain the first response text, and the first output text output by each hidden layer in the large model when the first response text is generated is determined. It can be understood that the large model includes at least one hidden layer. When the request text is input into the large model, it passes through at least one hidden layer in sequence and finally obtains the response text. Exemplarily, the large model includes 5 hidden layers. After the first request text is input into the large model, the first output text output by each hidden layer in the 5 hidden layers is determined, that is, there are 5 first output texts.
[0081] Then the second request text is input into the large model to obtain the second response text, and the second output text output by each hidden layer in the large model when the second response text is generated is determined. Exemplarily, continuing the above example, after the second request text is input into the large model, the second output text output by each hidden layer in the five hidden layers is determined, that is, there are five second output texts. There is a one-to-one correspondence between the five first output texts and the five second output texts.
[0082] Then, the sensitive layer in the large model is determined according to the Euclidean distance between the first output text and the second output text output by each hidden layer. Specifically, for any hidden layer, the Euclidean distance between the first output text and the second output text output by the hidden layer and the growth value of the Euclidean distance are calculated. Exemplarily, for the second hidden layer, the Euclidean distance between the first output text and the second output text output by the second hidden layer and the growth value of the Euclidean distance between the first output text and the second output text output by the second hidden layer are calculated. Finally, the hidden layer corresponding to the maximum Euclidean distance and the hidden layer corresponding to the maximum growth value of the Euclidean distance are used as the sensitive layer in the large model. Among them, the hidden layer corresponding to the maximum growth value of the Euclidean distance indicates that sensitive information begins to appear in the generated content of the large model; the hidden layer corresponding to the maximum Euclidean distance indicates that the sensitive information of the model has accumulated to the maximum value.
[0083] Finally, the parameters of the sensitive layer in the large model are optimized based on the first request text, the first response text, the second request text, and the second response text to obtain the first model. Specifically, the first response text is vectorized to obtain the vectorized first response text, and the first request text is input into the optimized large model to obtain the third response text, and the cross entropy loss between the vectorized first response text and the third response text is calculated. Among them, the cross entropy loss can be understood as the editing loss, which is used to measure the detoxification ability of the large model. Then, the second request text is input into the optimized large model to obtain the fourth response text, and the information divergence between the second response text and the fourth response text is calculated. Among them, the information divergence can be understood as the general ability loss, which is used to use general knowledge to ensure the general ability of the model. Then calculate the change ratio of the parameters of the sensitive layer in the optimized large model. Among them, the change ratio of the parameters can be understood as the offset loss, which measures the performance change between the edited model and the original model. Finally, the cross entropy loss, information divergence and change ratio are weighted and summed to obtain the loss function of the large model. Among them, the weights corresponding to the cross entropy loss, information divergence and change ratio can be values preset according to experience, and are not specifically limited here. However, it is understandable that the sum of the weights of the three should be 100%. Based on the obtained loss function, the parameters of the sensitive layer in the large model are iteratively optimized to finally obtain the first model.
[0084] Step 230: If the request text does not contain sensitive information, the request text is input into the first model to obtain an output response text.
[0085] In the embodiment of the present invention, when the request text does not contain sensitive information, the request text is directly input into the first model to obtain the response text output by the first model. It is understandable that there may be a situation where the request text contains sensitive information but it is not detected. However, since the first model is edited based on the knowledge of the large model, the large model's ability to detoxify sensitive information is improved while ensuring that the general ability of the model is not reduced. Therefore, the response text output by the first model will not contain sensitive information.
[0086] In an embodiment of the present invention, the knowledge of the large model is first edited, that is, the output text of each hidden layer of the large model when replying to request texts including sensitive information and request texts not containing sensitive information is used to locate the sensitive layer in the large model, and the parameters of the sensitive layer are adjusted to obtain the first model. This achieves separate adjustments to the areas where the model generates sensitive information without changing most of the model parameters and ensuring the efficiency of fine-tuning, thereby effectively improving the detoxification ability of the model. In addition, for the request text input into the first model, it is first screened through the second model, and then enters the first model after knowledge editing to generate a reply. Security protection is performed on both the input side and the model side itself, thereby improving the security capabilities of the large model in practical applications.
[0087] Based on the same technical concept, Figure 3 The schematic diagram of the structure of a large model application device provided by an embodiment of the present invention is exemplarily shown, and the device can execute the process of the large model application method.
[0088] like Figure 3 As shown, the device specifically includes:
[0089] The acquisition module 310 is used to acquire the input request text and determine whether the request text contains sensitive information;
[0090] A processing module 320 is used for adding a first identifier to the request text if the request text contains sensitive information, and inputting the first identifier into a first model to obtain an output rejection text, wherein the first model is edited based on the knowledge of a large model, and the first identifier is used to indicate that the first model refuses to generate a response to the request text;
[0091] If the request text does not contain sensitive information, the request text is input into the first model to obtain an output response text.
[0092] Optionally, the processing module 320 is specifically used for:
[0093] The request text is input into the second model to obtain a detection result output by the second model; the detection result is used to indicate whether the request text contains sensitive information.
[0094] Optionally, the processing module 320 is further used for:
[0095] Constructing a first request text and a second request text respectively, wherein the first request text contains sensitive information, and the second request text does not contain sensitive information;
[0096] Inputting the first request text into the large model to obtain a first response text, and determining a first output text output by each hidden layer in the large model when generating the first response text;
[0097] Inputting the second request text into the large model to obtain a second response text, and determining a second output text output by each hidden layer in the large model when generating the second response text;
[0098] Determine the sensitive layer in the large model according to the Euclidean distance between the first output text and the second output text output by each hidden layer;
[0099] The parameters of the sensitive layer in the large model are optimized based on the first request text, the first response text, the second request text and the second response text to obtain the first model.
[0100] Optionally, the processing module 320 is specifically used for:
[0101] For any hidden layer, calculating the Euclidean distance and the growth value of the Euclidean distance between the first output text and the second output text output by the hidden layer;
[0102] The hidden layer corresponding to the maximum Euclidean distance and the hidden layer corresponding to the maximum growth value of the Euclidean distance are used as sensitive layers in the large model.
[0103] Optionally, the processing module 320 is specifically used for:
[0104] Vectorizing the first response text to obtain a vectorized first response text, inputting the first request text into the optimized large model to obtain a third response text, and calculating a cross entropy loss between the vectorized first response text and the third response text;
[0105] Inputting the second request text into the optimized large model to obtain a fourth response text, and calculating the information divergence between the second response text and the fourth response text;
[0106] Calculate the change ratio of the parameters of the sensitive layer in the optimized large model;
[0107] The cross entropy loss, the information divergence and the change ratio are weightedly summed to obtain the loss function of the large model, and the parameters of the sensitive layer in the large model are iterated based on the loss function to obtain the first model.
[0108] Based on the same technical concept, an embodiment of the present invention further provides a computer device, including:
[0109] A memory for storing program instructions;
[0110] The processor is used to call the program instructions stored in the memory and execute the application method of the above-mentioned large model according to the obtained program.
[0111] Based on the same technical concept, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned large model application method.
[0112] Based on the same technical concept, an embodiment of the present invention further provides a computer program product, characterized in that the computer program product includes an executable program, and the executable program is executed by a processor to implement the above-mentioned large model application method.
[0113] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0114] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0117] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A large model application method, characterized in that: include: Obtaining an input request text and determining whether the request text contains sensitive information; If the request text contains sensitive information, a first identifier is added to the request text and input into the first model to obtain an output rejection text, wherein the first model is edited based on the knowledge of the large model, and the first identifier is used to indicate that the first model refuses to generate a response to the request text; If the request text does not contain sensitive information, input the request text into the first model to obtain an output response text; Before obtaining the request data, the method further includes: Constructing a first request text and a second request text respectively, wherein the first request text contains sensitive information, and the second request text does not contain sensitive information; Inputting the first request text into the large model to obtain a first response text, and determining a first output text output by each hidden layer in the large model when generating the first response text; Inputting the second request text into the large model to obtain a second response text, and determining a second output text output by each hidden layer in the large model when generating the second response text; Determine the sensitive layer in the large model according to the Euclidean distance between the first output text and the second output text output by each hidden layer; Optimizing the parameters of the sensitive layer in the large model based on the first request text, the first response text, the second request text, and the second response text to obtain the first model; Optimizing the parameters of the sensitive layer in the large model based on the first request text, the first response text, the second request text, and the second response text to obtain the first model includes: Vectorizing the first response text to obtain a vectorized first response text, inputting the first request text into the optimized large model to obtain a third response text, and calculating a cross entropy loss between the vectorized first response text and the third response text; Inputting the second request text into the optimized large model to obtain a fourth response text, and calculating the information divergence between the second response text and the fourth response text; Calculate the change ratio of the parameters of the sensitive layer in the optimized large model; The cross entropy loss, the information divergence and the change ratio are weightedly summed to obtain the loss function of the large model, and the parameters of the sensitive layer in the large model are iterated based on the loss function to obtain the first model.
2. The method according to claim 1, characterized in that Determining whether the request text contains sensitive information includes: The request text is input into the second model to obtain a detection result output by the second model; the detection result is used to indicate whether the request text contains sensitive information.
3. The method according to claim 1, characterized in that Determining the sensitive layer in the large model according to the Euclidean distance between the first output text and the second output text output by each hidden layer includes: For any hidden layer, calculating the Euclidean distance and the growth value of the Euclidean distance between the first output text and the second output text output by the hidden layer; The hidden layer corresponding to the maximum Euclidean distance and the hidden layer corresponding to the maximum growth value of the Euclidean distance are used as sensitive layers in the large model.
4. A large model application device, characterized in that: include: An acquisition module is used to acquire an input request text and determine whether the request text contains sensitive information; a processing module, configured to add a first identifier to the request text if the request text contains sensitive information, and input the first identifier into a first model to obtain an output rejection text, wherein the first model is edited based on the knowledge of a large model, and the first identifier is used to indicate that the first model refuses to generate a response to the request text; If the request text does not contain sensitive information, input the request text into the first model to obtain an output response text; Optionally, the processing module is further used for: Constructing a first request text and a second request text respectively, wherein the first request text contains sensitive information, and the second request text does not contain sensitive information; Inputting the first request text into the large model to obtain a first response text, and determining a first output text output by each hidden layer in the large model when generating the first response text; Inputting the second request text into the large model to obtain a second response text, and determining a second output text output by each hidden layer in the large model when generating the second response text; Determine the sensitive layer in the large model according to the Euclidean distance between the first output text and the second output text output by each hidden layer; Optimizing the parameters of the sensitive layer in the large model based on the first request text, the first response text, the second request text, and the second response text to obtain the first model; Optionally, the processing module is specifically used for: Vectorizing the first response text to obtain a vectorized first response text, inputting the first request text into the optimized large model to obtain a third response text, and calculating a cross entropy loss between the vectorized first response text and the third response text; Inputting the second request text into the optimized large model to obtain a fourth response text, and calculating the information divergence between the second response text and the fourth response text; Calculate the change ratio of the parameters of the sensitive layer in the optimized large model; The cross entropy loss, the information divergence and the change ratio are weightedly summed to obtain the loss function of the large model, and the parameters of the sensitive layer in the large model are iterated based on the loss function to obtain the first model.
5. A computer device, characterized in that: include: A memory for storing program instructions; A processor, configured to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 3 according to the obtained program.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method according to any one of claims 1 to 3.
7. A computer program product, characterized in that The computer program product comprises an executable program, which is executed by a processor to implement the method according to any one of claims 1 to 3.
Citation Information
Patent Citations
Construction method of self-evolutionary learning condition management question and answer robot
CN117390159A