A data security analysis method based on large language model and defense algorithm

Abstract

The present invention relates to the field of secure call technology, and specifically discloses a data security analysis method based on a large language model and a defense algorithm, comprising: when an intelligent agent receives a user question input by a front-end, generating a current prompt word according to the user question; inputting the current prompt word into the large language model, the large language model calculating an intermediate result according to the current prompt word, perturbing the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then generating a current reasoning trajectory according to the perturbed intermediate result by the large language model; determining whether there is a final answer in the current reasoning trajectory; if there is, outputting the final answer to the front-end; if not, calling a query tool according to the current reasoning trajectory to obtain target data, and then generating a current observation result based on the target data; updating the current prompt word according to the current reasoning trajectory and the current observation result through the ReAct reasoning framework, and reasoning again according to the updated prompt word until there is a final answer in the reasoning trajectory generated by the large language model, and finally outputting the final answer to the front-end. The present invention enables banks to use the large language model capability to generate query content and database searches while protecting private domain data from leaving the domain, thereby performing data security analysis.

Description

Technical Field

[0001] The present invention relates to the field of secure call technology, and more specifically, to a data security analysis method based on a large language model and a defense algorithm. Background Art

[0002] Data security and privacy are paramount in many industries, particularly banking. To prevent sensitive data leaks, banks often strictly restrict data flow, particularly preventing it from leaving their domain. In this context, banks cannot rely on cloud-based machine learning models and big data processing systems, preferring local solutions. Consequently, banks often employ locally deployed models and big data processing frameworks, leveraging high-performance computing equipment and dedicated servers to meet their analytical and predictive needs.

[0003] However, this presents a common problem: if resources are limited, it's impossible to deploy a high-performance large language model, hindering the benefits it offers. High-performance models typically have large parameter sizes, and to ensure data security and compliance, they must be deployed locally, which creates cost implications. Furthermore, using an API to call an external large language model, while cost-effective, means the original data will be out of the domain, posing a risk of data leakage. Summary of the Invention

[0004] In response to the above-mentioned drawbacks in the existing technology, the present invention provides a data security analysis method based on a large language model and a defense algorithm, which enables banks to use the large language model capabilities to generate query content and perform database searches, thereby performing data security analysis, while protecting private domain data from leaving the domain.

[0005] As a first aspect of the present invention, a data security analysis method based on a large language model and a defense algorithm is provided, which is implemented by a data security analysis system. The data security analysis system includes a data source, a front end, and an intelligent agent; the intelligent agent includes a query tool and a ReAct reasoning framework, wherein the ReAct reasoning framework includes an inference phase, an action phase, and an observation phase, the inference phase involves a large language model, and the large language model is divided into a model head, a model middle, and a model tail. The data source, front end, query tool, model head, model tail, action phase, and observation phase are all located on the client, and the model middle is located on the server. The data security analysis method based on the large language model and the defense algorithm includes the following steps:

[0006] Step S1: When the agent receives a user question input from the front-end, it embeds the user question into the corresponding prompt word template to generate the current prompt word, and then uses the current prompt word as the input of the reasoning stage in the ReAct reasoning framework;

[0007] Step S2: The large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result using a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current inference trajectory based on the perturbed intermediate result; wherein the defense algorithm is a differential privacy method, an adversarial training method, or a mutual information method;

[0008] Step S3: Determine whether there is a final answer with the keyword in the current reasoning trajectory through the ReAct reasoning framework; if there is a final answer with the keyword, directly output the final answer to the front end; if there is no final answer with the keyword, call the corresponding query tool according to the current reasoning trajectory through the action phase, enter the query content in the corresponding query tool to obtain the target data in the data source, and then execute step S4;

[0009] Step S4: generating a current observation result according to the target data during the observation phase;

[0010] Step S5: The current reasoning trajectory and the current observation result are spliced ​​after the current prompt word through the ReAct reasoning framework to update the prompt word, and then return to step S2 so that the large language model can reason again based on the updated prompt word until the final answer with the keyword exists in the reasoning trajectory generated by the large language model, and finally the final answer is output to the front end.

[0011] Furthermore, the large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory based on the perturbed intermediate result, further comprising the following steps:

[0012] After the current prompt word is input into the model head, the model head can generate a first intermediate result according to the current prompt word; wherein the client can perturb the first intermediate result using the differential privacy method to obtain a perturbed first intermediate result;

[0013] After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result;

[0014] The second intermediate result is sent to the client, and the client can perturb the second intermediate result through the differential privacy method to obtain a perturbed second intermediate result; wherein the model tail can generate a final result based on the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0015] Furthermore, the large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory based on the perturbed intermediate result, further comprising the following steps:

[0016] After the current prompt word is input into the model head, the model head can generate a first intermediate result based on the current prompt word; wherein the client can perturb the first intermediate result through the adversarial training method to obtain a perturbed first intermediate result;

[0017] After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result;

[0018] The second intermediate result is sent to the client, and the client can perturb the second intermediate result through the adversarial training method to obtain a perturbed second intermediate result; wherein the model tail can generate a final result based on the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0019] Furthermore, the large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory based on the perturbed intermediate result, further comprising the following steps:

[0020] After the current prompt word is input into the model head, the model head can generate a first intermediate result based on the current prompt word; wherein the client can perturb the first intermediate result using the mutual information method to obtain a perturbed first intermediate result;

[0021] After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result;

[0022] The second intermediate result is sent to the client, and the client can perturb the second intermediate result through the mutual information method to obtain a perturbed second intermediate result; wherein the model tail can generate a final result based on the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0023] Furthermore, the model head includes an embedding layer and n1 encoders / decoders, the model middle includes n2 encoders / decoders, and the model tail includes n3 encoders / decoders and an output layer.

[0024] Furthermore, the method further includes the following step: displaying the final answer through a front-end interface.

[0025] The data security analysis method based on a large language model and a defense algorithm provided by the present invention has the following advantages: it can use the large language model capabilities to generate query content and perform database searches, thereby performing data security analysis, while protecting data privacy security on the client side and protecting the large language model parameters from being shared on the server side. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention, but do not constitute a limitation of the present invention.

[0027] Figure 1 This is a flowchart of the data security analysis method based on a large language model and defense algorithm provided by the present invention.

[0028] Figure 2 This is a framework diagram of the data security analysis system provided by the present invention.

[0029] Figure 3 Schematic diagram of segmentation of a large language model with differential privacy provided by the present invention.

[0030] Figure 4 Schematic diagram of the segmentation of a large language model with adversarial training provided by the present invention.

[0031] Figure 5 A schematic diagram of segmenting a large language model with the mutual information method provided by the present invention.

[0032] Figure 6 This is the operation flow chart of the ReAct agent provided by the present invention. DETAILED DESCRIPTION

[0033] To further illustrate the technical means and effectiveness of the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of the data security analysis method based on a large language model and defense algorithm proposed by the present invention. Obviously, the described embodiments are only a portion of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0034] In this embodiment, a data security analysis method based on a large language model and a defense algorithm is provided, which is implemented through a data security analysis system. Figure 2 As shown, the data security analysis system includes a data source, a front end and an intelligent agent; the intelligent agent includes a query tool and a ReAct reasoning framework, wherein the ReAct reasoning framework includes a reasoning stage, an action stage and an observation stage, wherein the reasoning stage involves a large language model, and the large language model is divided into a model head, a model middle and a model tail, wherein the data source, front end, query tool, model head, model tail, action stage and observation stage are all located on the client side, and the model middle is located on the server side; wherein, as Figure 1 As shown, the data security analysis method based on the large language model and defense algorithm includes the following steps:

[0035] Step S1: When the agent receives a user question input from the front-end, it embeds the user question into the corresponding prompt word template to generate the current prompt word, and then uses the current prompt word as the input of the reasoning stage in the ReAct reasoning framework;

[0036] Step S2: The large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result using a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current inference trajectory based on the perturbed intermediate result; wherein the defense algorithm is a differential privacy method, an adversarial training method, or a mutual information method;

[0037] Step S3: Determine whether there is a final answer with the keyword in the current reasoning trajectory through the ReAct reasoning framework; if there is a final answer with the keyword, directly output the final answer to the front end; if there is no final answer with the keyword, call the corresponding query tool according to the current reasoning trajectory through the action phase, enter the query content in the corresponding query tool to obtain the target data in the data source, and then execute step S4;

[0038] Step S4: generating a current observation result according to the target data during the observation phase;

[0039] Step S5: The current reasoning trajectory and the current observation result are spliced ​​after the current prompt word through the ReAct reasoning framework to update the prompt word, and then return to step S2 so that the large language model can reason again based on the updated prompt word until the final answer with the keyword exists in the reasoning trajectory generated by the large language model, and finally the final answer is output to the front end.

[0040] Specifically, a user question like "Please help me find the total revenue of our Shanghai branch last month" is sent from the front-end interface to an intelligent agent powered by a large language model. The agent then forms different reasoning frameworks based on different prompts. In this scenario, the ReAct reasoning framework is used to alternately generate reasoning trajectories and task-specific actions, creating greater synergy between the two and improving task performance and explainability. The ReAct reasoning framework allows the large language model to interact with external query tools to obtain additional information, resulting in more reliable and practical responses.

[0041] It's important to note that when a user question enters the agent, it's embedded in a corresponding prompt template to form prompts. The prompts that aid the agent's thinking in this scenario essentially assign the LLM the role of a data analysis assistant, detailing the assistant's SQL generation capabilities based on the user question and table structure, the tools available, and the formatting requirements for answering questions. The assistant can assist with various tasks, including answering questions, providing summaries, and other types of analysis. The assistant can utilize a variety of tools and must utilize them appropriately based on the task's requirements. Tasks may need to be broken down into subtasks. When answering questions, the assistant should follow a specific format, such as "Thought," "Action," and "Action Input." Alternatively, if the assistant feels it has sufficient information, it can respond with "Thought," "Answer."

[0042] The above ReAct prompt words will be sent to the model head on the client, and then the model head sends the intermediate result H1 of the model inference calculation to the middle part of the model on the server. After the server completes a large number of major calculations, the middle part of the model sends the intermediate result H2 of the large language model inference calculation back to the model tail on the client. The model tail outputs the final result based on the large language model inference result H2. At this time, the final result is the inference trajectory.

[0043] If the agent feels that there is not enough information to answer the user's question and needs to further call the query tool to obtain additional information to answer the user's question, a reasoning trajectory will be generated. The reasoning trajectory includes: the model's reasoning explanation of what to do next to answer the user's question, the name of the query tool that needs to be called in the action, and the query content that needs to be entered in the query tool.

[0044] It should be noted that the front-end interface includes login, communication, and data analysis functions. Users log in to the data security analysis system by entering their username and password, and then enter the main interface. The main interface includes a navigation bar, a chat window, and a data analysis module. Users enter questions in the chat window and communicate with the data security analysis system in real time. The data security analysis system calls the back-end to perform data analysis and present the results in text or charts. The user settings page allows users to modify their personal information. The entire interface design is simple and intuitive, allowing users to efficiently obtain the required information and data analysis results.

[0045] Preferably, Figure 3 As shown, the large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result through the defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory based on the perturbed intermediate result, further comprising the following steps:

[0046] After the current prompt word X is input into the model head, the model head can generate a first intermediate result H1 according to the current prompt word X; wherein the client can perturb the first intermediate result H1 using the differential privacy method to obtain a perturbed first intermediate result H1';

[0047] After the disturbed first intermediate result H1' is input into the middle part of the model, the middle part of the model can generate a second intermediate result H2 according to the disturbed first intermediate result H1';

[0048] The second intermediate result H2 is sent to the client, and the client can perturb the second intermediate result H2 through the differential privacy method to obtain a perturbed second intermediate result H2'; wherein the model tail can generate a final result based on the perturbed second intermediate result H2', wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0049] It should be noted that differential privacy (DP) is a commonly used method to defend against model inversion attacks. In this scenario, it is the intermediate result H 1、H2 adds random perturbation, namely differential privacy noise (Noise), and generates H1' with added noise on the client. 、 H2', performs privacy protection on the input data X, preventing the attacker from obtaining the information through the intermediate result H 1、 H2 performs an inversion attack to reconstruct the private data X input into the current model.

[0050] Preferably, Figure 4 As shown, the large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result through the defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory based on the perturbed intermediate result, further comprising the following steps:

[0051] After the current prompt word X is input into the model head, the model head can generate a first intermediate result H1 according to the current prompt word; wherein the client can perturb the first intermediate result H1 through the adversarial training method to obtain a perturbed first intermediate result H1';

[0052] After the disturbed first intermediate result H1' is input into the middle part of the model, the middle part of the model can generate a second intermediate result H2 according to the disturbed first intermediate result H1';

[0053] The second intermediate result H2 is sent to the client, and the client can perturb the second intermediate result H2 through the adversarial training method to obtain a perturbed second intermediate result H2'; wherein the model tail can generate a final result based on the perturbed second intermediate result H2', wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0054] It should be noted that adversarial training is also a widely used strategy for defending against model reverse attacks in the inference phase of large language models. 1、 H2 is disturbed. Another attack model (Adversary Model) is used to perform an inverse attack, attempting to obtain the disturbed intermediate result H1' 、The original input data X is recovered or inferred from H2'. The goal of the attack model is to successfully perform the inversion attack as much as possible to expose the security weaknesses of the large model. The attack success of the attack model is added as a regular term in the loss function to the training process of the adversarial model. This means that a well-trained adversarial model can help the large language model resist potential inversion attacks. The implementation method in the current scenario is: the intermediate result H 1、 H2 will pass through an adversarial model that has been trained in advance using the above method. The trained model will 1、 H2 is disturbed to generate the disturbed intermediate result H1' 、 H2', protects the original input data X from being reconstructed by the attacker.

[0055] Preferably, Figure 5 As shown, the large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result through the defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory based on the perturbed intermediate result, further comprising the following steps:

[0056] After the current prompt word X is input into the model head, the model head can generate a first intermediate result H1 according to the current prompt word; wherein, the client can perturb the first intermediate result H1 by using the mutual information method to obtain a perturbed first intermediate result H1';

[0057] After the disturbed first intermediate result H1' is input into the middle part of the model, the middle part of the model can generate a second intermediate result H2 according to the disturbed first intermediate result H1';

[0058] The second intermediate result H2 is sent to the client, and the client can perturb the second intermediate result H2 through the mutual information method to obtain a perturbed second intermediate result H2'; wherein the model tail can generate a final result based on the perturbed second intermediate result H2', wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0059] It should be noted that in order to prevent attackers from recovering the original data X from the intermediate results, the mutual information method is to minimize the intermediate results H 1、The amount of information carried by H2 while retaining the necessary parts for reasoning. Minimizing the information dependency between the intermediate results and the original input data X can also increase the difficulty for the attacker to infer the original input X through the intermediate results. In this scenario, we pre-train a mutual information defense model, that is, an encoder-decoder structure model (MIDmodel), and use it to transmit the intermediate variable H 1、 H2 is disturbed to generate H1' 、 H2' reduces the information correlation between the intermediate results and the original input data X, thereby improving the system's privacy protection capabilities. During data transmission, even if the intermediate variables are intercepted or inferred, it is difficult to restore the original data. For example: H1 is input into the mutual information defense model. The encoder part of this defense model first maps the intermediate result H1 to the bottleneck layer M (the number of neurons is usually far less than the input or output). The purpose is to ignore unnecessary details in H1, compress information, and discard redundancy or noise. The decoder part decodes the bottleneck layer M to reconstruct the intermediate result H1'. At this time, the information correlation between H1' and the original input data X is significantly reduced, protecting the original input data X.

[0060] It should be noted that the data transmitted in the large language model is not the original data, but the result of protecting the intermediate results calculated by the large language model through the defense algorithm. In this scenario, it is presented in the form of tensors. It is the basic data structure for internal calculations of the large language model, rather than the original data input by the client.

[0061] It should be noted that the large language model can be: Bert, Roberta, GPT2, Llama, Baichuan, Gemma, Falcon, Mamba, ChatGLM, XLnet, Mistral, etc.

[0062] It's important to note that the query tool is responsible for MySQL queries. The agent enters the query, and the MySQL query tool connects to the client's local MySQL database. The MySQL query tool uses the received query to execute the query, retrieving the target data from the data source. The MySQL database contains banking information.

[0063] It should be noted that ReAct (Reasoning and Acting) is a framework that combines reasoning and acting, with the goal of improving the performance and interpretability of large language models (LLMs) in solving complex tasks; Figure 6 As shown in the figure, the ReAct agent's operation process in this scenario is as follows:

[0064] 1. Define the task and goal: The agent's task is to generate SQL queries based on user questions and invoke a query tool to retrieve target data from the MySQL database. The goal is to accurately and efficiently respond to user queries.

[0065] 2. Generate reasoning trajectory: The agent generates a reasoning trajectory, simulating the thought process to solve the problem. The reasoning trajectory includes steps such as analyzing the user problem, planning tool calls, constructing SQL queries, and handling possible exceptions.

[0066] 3. Action Execution: The agent executes specific actions based on the generated reasoning trajectory. First, the user question is translated into an SQL query. Then, the agent executes the query using interactive tools with a local MySQL database to retrieve the required data.

[0067] 4. Alternating Reasoning and Action: The agent performs new reasoning after each action. Based on the results of database queries, the agent generates a new reasoning trajectory to ensure the accuracy of subsequent actions. For example, if a query returns empty results or an error, the agent analyzes the cause and attempts to adjust the query.

[0068] 5. Context Update: After each action, the agent updates the context, including the executed SQL query, query results, and newly generated reasoning trajectory. Context updates help the agent track the current task status and provide information for subsequent reasoning and actions.

[0069] 6. Generate Task Solution Trajectory: Through the above steps, the agent generates a complete task solution trajectory, including actions, query results, and reasoning. This not only demonstrates how the agent solves the problem but also provides a transparent decision-making process, making it easier for users to understand and evaluate the model's behavior.

[0070] This approach enables the agent to dynamically generate SQL queries and retrieve target data from a MySQL database, improving accuracy and efficiency when performing tasks. This approach is particularly suitable for tasks that require interacting with a database and generating dynamic queries based on user input.

[0071] Specifically, the model head includes an embedding layer and the first n1 encoders / decoders (the first few layers close to the input layer), the middle of the model includes n2 encoders / decoders, and the tail of the model includes the last n3 encoders / decoders and the final output layer.

[0072] It's important to note that in a vertical federated learning scenario, a complete Large Language Model (LLM) is divided into three sections: the model head, model tail, and model middle. In this scenario, the client acts as the data provider, retaining the smaller model head and model tail. The server acts as the model provider, retaining the larger model middle. This isolates the raw data input by the server and client, preventing inversion attacks that could reveal the client's private data. In this scenario, only the reasoning capabilities of the large language model are utilized.

[0073] Preferably, the method further includes the following steps: displaying the final answer through a front-end interface for the user to view.

[0074] The data security analysis method based on a large language model and a defense algorithm provided by the present invention can use the large language model capabilities to generate query content and perform database searches, thereby performing data security analysis, while protecting data privacy security on the client side and protecting the large language model parameters from being shared on the server side.

[0075] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Any technician familiar with the present profession can make slight changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A data security analysis method based on a large language model and a defense algorithm, implemented by a data security analysis system, characterized in that: The data security analysis system includes a data source, a front end, and an intelligent agent; the intelligent agent includes a query tool and a ReAct reasoning framework, wherein the ReAct reasoning framework includes a reasoning phase, an action phase, and an observation phase. The reasoning phase involves a large language model, and the large language model is divided into a model head, a model middle, and a model tail. The data source, front end, query tool, model head, model tail, action phase, and observation phase are all located on the client, and the model middle is located on the server. The data security analysis method based on the large language model and the defense algorithm includes the following steps: Step S1: When the agent receives a user question input from the front-end, it embeds the user question into the corresponding prompt word template to generate the current prompt word, and then uses the current prompt word as the input of the reasoning stage in the ReAct reasoning framework; Step S2: The large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result using a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current inference trajectory based on the perturbed intermediate result; wherein the defense algorithm is a differential privacy method, an adversarial training method, or a mutual information method; Step S3: Determine whether there is a final answer with the keyword in the current reasoning trajectory through the ReAct reasoning framework; if there is a final answer with the keyword, directly output the final answer to the front end; if there is no final answer with the keyword, call the corresponding query tool according to the current reasoning trajectory through the action phase, enter the query content in the corresponding query tool to obtain the target data in the data source, and then execute step S4; Step S4: generating a current observation result according to the target data during the observation phase; Step S5: The current reasoning trajectory and the current observation result are spliced ​​after the current prompt word through the ReAct reasoning framework to update the prompt word, and then return to step S2 so that the large language model can reason again based on the updated prompt word until the final answer with the keyword exists in the reasoning trajectory generated by the large language model, and finally the final answer is output to the front end.

2. The data security analysis method based on a large language model and a defense algorithm according to claim 1 is characterized in that: The large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result using a defense algorithm to obtain a perturbed intermediate result, and then generates a current reasoning trajectory based on the perturbed intermediate result, further comprising the following steps: After the current prompt word is input into the model head, the model head can generate a first intermediate result according to the current prompt word; wherein the client can perturb the first intermediate result using the differential privacy method to obtain a perturbed first intermediate result; After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result; The second intermediate result is sent to the client, and the client can perturb the second intermediate result through the differential privacy method to obtain a perturbed second intermediate result; wherein the model tail can generate a final result based on the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

3. The data security analysis method based on a large language model and a defense algorithm according to claim 1 is characterized in that: The large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result using a defense algorithm to obtain a perturbed intermediate result, and then generates a current reasoning trajectory based on the perturbed intermediate result, further comprising the following steps: After the current prompt word is input into the model head, the model head can generate a first intermediate result based on the current prompt word; wherein the client can perturb the first intermediate result through the adversarial training method to obtain a perturbed first intermediate result; After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result; The second intermediate result is sent to the client, and the client can perturb the second intermediate result through the adversarial training method to obtain a perturbed second intermediate result; wherein the model tail can generate a final result based on the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

4. The data security analysis method based on a large language model and a defense algorithm according to claim 1 is characterized in that: The large language model calculates an intermediate result based on the current prompt word, perturbs the intermediate result using a defense algorithm to obtain a perturbed intermediate result, and then generates a current reasoning trajectory based on the perturbed intermediate result, further comprising the following steps: After the current prompt word is input into the model head, the model head can generate a first intermediate result based on the current prompt word; wherein the client can perturb the first intermediate result using the mutual information method to obtain a perturbed first intermediate result; After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result; The second intermediate result is sent to the client, and the client can perturb the second intermediate result through the mutual information method to obtain a perturbed second intermediate result; wherein the model tail can generate a final result based on the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

5. The data security analysis method based on a large language model and a defense algorithm according to claim 1 is characterized in that: The model head includes an embedding layer and n1 encoders / decoders, the model middle includes n2 encoders / decoders, and the model tail includes n3 encoders / decoders and an output layer.

6. The data security analysis method based on a large language model and a defense algorithm according to claim 1 is characterized in that: The method further includes the following steps: displaying the final answer through a front-end interface.

Images