Data security analysis method based on large language model and defense algorithm

Abstract

The invention relates to the technical field of security calling, and particularly discloses a data security analysis method based on a large language model and a defense algorithm, which comprises the following steps: when an intelligent agent receives a user question input by a front end, generating a current cue word according to the user question; the current cue word is input into a large language model, the large language model calculates an intermediate result according to the current cue word, the intermediate result is disturbed through a defense algorithm to obtain a disturbed intermediate result, and then the large language model generates a current reasoning track according to the disturbed intermediate result; judging whether a final answer exists in the current reasoning track or not; if yes, outputting a final answer to the front end; if not, calling a query tool according to the current reasoning track to obtain target data, and then generating a current observation result according to the target data; and updating the current cue word according to the current reasoning track and the current observation result through a ReAct reasoning framework, performing reasoning again according to the updated cue word until a final answer exists in the reasoning track generated by the large language model, and finally outputting the final answer to the front end. According to the method, under the condition that private domain data is protected from being out of the domain, a bank can use the large language model capability to carry out query content generation and database search, and then data security analysis is carried out.

Description

Technical Field

[0001] The present invention relates to the field of secure call technology, and more specifically, to a data security analysis method based on a large language model and a defense algorithm. Background Art

[0002] In many industries, especially the banking industry, data security and privacy protection are crucial. In order to prevent sensitive data leakage, banks usually strictly restrict the flow of data, especially to prevent data from leaving the domain. In this context, banks cannot use machine learning models and big data processing systems that rely on the cloud, but tend to use localized solutions. Therefore, the banking industry usually adopts locally deployed models and big data processing frameworks, using high-performance computing equipment and dedicated servers to meet its analysis and prediction needs.

[0003] But this brings a common problem: if resources are limited, it will be impossible to deploy a high-performance large language model, and thus you will not be able to enjoy the convenience brought by the large language model. Because models with good performance generally have large parameter scales, they need to be deployed locally under the condition of data application security compliance, which brings cost issues. If you use an API to call an out-of-domain large language model, although the cost is lower, the original data will be out of the domain, and there is a risk of data leakage. Summary of the invention

[0004] In view of the above-mentioned drawbacks existing in the prior art, the present invention provides a data security analysis method based on a large language model and a defense algorithm, so that banks can use the large language model capabilities to generate query content and perform database searches, thereby performing data security analysis, while protecting private domain data from leaving the domain.

[0005] As a first aspect of the present invention, a data security analysis method based on a large language model and a defense algorithm is provided, which is implemented by a data security analysis system, wherein the data security analysis system includes a data source, a front end and an agent; the agent includes a query tool and a ReAct reasoning framework, wherein the ReAct reasoning framework includes a reasoning stage, an action stage and an observation stage, wherein the reasoning stage involves a large language model, and the large language model is divided into a model head, a model middle and a model tail, wherein the data source, the front end, the query tool, the model head, the model tail, the action stage and the observation stage are all located on the client, and the model middle is located on the server; wherein the data security analysis method based on a large language model and a defense algorithm includes the following steps: Step S1: When the agent receives a user question input by the front end, the agent embeds the user question into a corresponding prompt word template to generate a current prompt word, and then uses the current prompt word as an input for the reasoning stage in the ReAct reasoning framework; Step S2: the large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result; wherein the defense algorithm is a differential privacy method, an adversarial training method, or a mutual information method; Step S3: judging whether there is a final answer with keywords in the current reasoning track through the ReAct reasoning framework; if there is a final answer with keywords, directly outputting the final answer to the front end; if there is no final answer with keywords, calling a corresponding query tool according to the current reasoning track through the action stage, and inputting query content in the corresponding query tool to obtain target data in the data source, and then executing step S4; Step S4: generating a current observation result according to the target data through the observation phase; Step S5: splicing the current reasoning trajectory and the current observation result after the current prompt word through the ReAct reasoning framework to update the prompt word, and then returning to step S2 so that the large language model can reason again according to the updated prompt word until the final answer with the keyword exists in the reasoning trajectory generated by the large language model, and finally outputting the final answer to the front end.

[0006] Furthermore, the large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result, further comprising the following steps: After the current prompt word is input into the model head, the model head can generate a first intermediate result according to the current prompt word; wherein the client can perturb the first intermediate result by using the differential privacy method to obtain the perturbed first intermediate result; After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result; The second intermediate result is sent to the client, and the client can perturb the second intermediate result by using the differential privacy method to obtain the perturbed second intermediate result; wherein the model tail can generate a final result based on the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0007] Furthermore, the large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result, further comprising the following steps: After the current prompt word is input into the model head, the model head can generate a first intermediate result according to the current prompt word; wherein the client can perturb the first intermediate result through the adversarial training method to obtain the perturbed first intermediate result; After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result; The second intermediate result is sent to the client, and the client can perturb the second intermediate result through the adversarial training method to obtain the perturbed second intermediate result; wherein the model tail can generate a final result based on the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0008] Furthermore, the large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result, further comprising the following steps: After the current prompt word is input into the model head, the model head can generate a first intermediate result according to the current prompt word; wherein the client can perturb the first intermediate result by using the mutual information method to obtain a perturbed first intermediate result; After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result; The second intermediate result is sent to the client, and the client can perturb the second intermediate result by the mutual information method to obtain the perturbed second intermediate result; wherein the model tail can generate a final result according to the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0009] Furthermore, the model head includes an embedding layer and n1 encoders / decoders, the model middle includes n2 encoders / decoders, and the model tail includes n3 encoders / decoders and an output layer.

[0010] Furthermore, the method also includes the following step: displaying the final answer through a front-end interface.

[0011] The data security analysis method based on a large language model and a defense algorithm provided by the present invention has the following advantages: it can realize the use of the large language model capability to generate query content and perform database searches, thereby performing data security analysis, while protecting data privacy security on the client side and protecting large language model parameters from being shared on the server side. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention, but do not constitute a limitation of the present invention.

[0013] Figure 1 A flow chart of the data security analysis method based on a large language model and a defense algorithm provided by the present invention.

[0014] Figure 2 This is a framework diagram of the data security analysis system provided by the present invention.

[0015] Figure 3 A schematic diagram of segmenting a large language model with differential privacy provided by the present invention.

[0016] Figure 4 A schematic diagram of segmentation of a large language model with adversarial training provided by the present invention.

[0017] Figure 5 A schematic diagram of segmentation of a large language model with the mutual information method provided by the present invention.

[0018] Figure 6 This is the operation flow chart of the ReAct agent provided by the present invention. DETAILED DESCRIPTION

[0019] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation method, structure, characteristics and effects of the data security analysis method based on the large language model and defense algorithm proposed by the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0020] In this embodiment, a data security analysis method based on a large language model and a defense algorithm is provided, which is implemented by a data security analysis system, such as Figure 2 As shown, the data security analysis system includes a data source, a front end and an intelligent agent; the intelligent agent includes a query tool and a ReAct reasoning framework, wherein the ReAct reasoning framework includes a reasoning stage, an action stage and an observation stage, the reasoning stage involves a large language model, and the large language model is divided into a model head, a model middle and a model tail, the data source, the front end, the query tool, the model head, the model tail, the action stage and the observation stage are all located on the client, and the model middle is located on the server; wherein, as Figure 1 As shown, the data security analysis method based on the large language model and the defense algorithm includes the following steps: Step S1: When the agent receives a user question input by the front end, the agent embeds the user question into a corresponding prompt word template to generate a current prompt word, and then uses the current prompt word as an input for the reasoning stage in the ReAct reasoning framework; Step S2: the large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result; wherein the defense algorithm is a differential privacy method, an adversarial training method, or a mutual information method; Step S3: judging whether there is a final answer with keywords in the current reasoning track through the ReAct reasoning framework; if there is a final answer with keywords, directly outputting the final answer to the front end; if there is no final answer with keywords, calling a corresponding query tool according to the current reasoning track through the action stage, and inputting query content in the corresponding query tool to obtain target data in the data source, and then executing step S4; Step S4: generating a current observation result according to the target data through the observation phase; Step S5: splicing the current reasoning trajectory and the current observation result after the current prompt word through the ReAct reasoning framework to update the prompt word, and then returning to step S2 so that the large language model can reason again according to the updated prompt word until the final answer with the keyword exists in the reasoning trajectory generated by the large language model, and finally outputting the final answer to the front end.

[0021] Specifically, user questions such as "Help me see what the total revenue of the Shanghai branch was last month" will be sent from the front-end interface to the agent driven by the large language model; the agent will form different reasoning frameworks according to different prompt words. In the current scenario, the ReAct reasoning framework is used to alternately generate reasoning trajectories and task-specific actions, so that there is greater synergy between the two, improving the performance and explainability of solving tasks. The ReAct reasoning framework allows the large language model to interact with external query tools to obtain additional information, thereby giving more reliable and practical responses.

[0022] It should be noted that when a user question enters the agent, the user question is embedded in the corresponding prompt word template to form a prompt word. The prompt word that helps the agent think in this scenario is roughly as follows: LLM is given the role of a data analysis assistant, and the assistant's function of generating SQL based on user questions and table structures, the tools that can be used, and the format requirements for answering questions are described in detail. The assistant can help complete a variety of tasks, including answering questions, providing summaries, and other types of analysis. The assistant can use a variety of tools and needs to use these tools reasonably according to task requirements. It may be necessary to break down the task into subtasks to complete. When answering questions, the assistant should respond to user questions in a specific format, such as: Thuoght, Action, and Action Input; or if it feels that there is enough information, answer the question with: Thought, Answer.

[0023] The above ReAct prompt words will be sent to the model head on the client side, and then the model head sends the intermediate result H1 of the model reasoning calculation to the middle part of the model on the server side. After a large number of main calculations are completed on the server side, the middle part of the model sends the intermediate result H2 of the large language model reasoning calculation back to the model tail on the client side. The model tail outputs the final result based on the large language model reasoning result H2. At this time, the final result is the reasoning trajectory.

[0024] If the agent feels that there is not enough information to answer the user's question and needs to further call the query tool to obtain additional information to answer the user's question, a reasoning trajectory will be generated. The reasoning trajectory includes: the model's reasoning explanation of what to do next to answer the user's question, the name of the query tool that needs to be called in the action, and the query content that needs to be entered in the query tool.

[0025] It should be noted that the front-end interface includes login, communication and data analysis functions. Users log in to the data security analysis system by entering their username and password and enter the main interface. The main interface includes a navigation bar, a chat window and a data analysis module. Users enter questions in the chat window and communicate with the data security analysis system in real time. The data security analysis system calls the back-end to perform data analysis and displays the analysis results in the form of text or charts. The user settings page allows users to modify their personal information. The entire interface design is simple and intuitive, which allows users to efficiently obtain the required information and data analysis results.

[0026] Preferably, Figure 3 As shown, the large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result, further comprising the following steps: After the current prompt word X is input into the model head, the model head can generate a first intermediate result H1 according to the current prompt word X; wherein the client can perturb the first intermediate result H1 by using the differential privacy method to obtain a perturbed first intermediate result H1'; After the disturbed first intermediate result H1' is input into the middle part of the model, the middle part of the model can generate a second intermediate result H2 according to the disturbed first intermediate result H1'; The second intermediate result H2 is sent to the client, and the client can perturb the second intermediate result H2 by using the differential privacy method to obtain a perturbed second intermediate result H2'; wherein the model tail can generate a final result based on the perturbed second intermediate result H2', wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0027] It should be noted that differential privacy (DP) is a commonly used method to defend against model inversion attacks. In this scenario, it is the intermediate result H 1、 H2 adds random disturbance, namely differential privacy noise (Noise), and generates H1' with added noise on the client. 、 H2', performs privacy protection on input data X, preventing attackers from obtaining the privacy information through the intermediate result H 1、 H2 performs an inversion attack and reconstructs the private data X input into the current model.

[0028] Preferably, Figure 4As shown, the large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result, further comprising the following steps: After the current prompt word X is input into the model head, the model head can generate a first intermediate result H1 according to the current prompt word; wherein the client can perturb the first intermediate result H1 through the adversarial training method to obtain a perturbed first intermediate result H1'; After the disturbed first intermediate result H1' is input into the middle part of the model, the middle part of the model can generate a second intermediate result H2 according to the disturbed first intermediate result H1'; The second intermediate result H2 is sent to the client, and the client can perturb the second intermediate result H2 through the adversarial training method to obtain a perturbed second intermediate result H2'; wherein the model tail can generate a final result based on the perturbed second intermediate result H2', wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0029] It should be noted that adversarial training is also a widely used strategy for defending against model reverse attacks in the inference phase of large language models. 1、 H2 is disturbed. Another attack model (Adversary Model) is used to perform an inversion attack, trying to obtain the disturbed intermediate result H1' 、 H2' to recover or infer the original input data X. The goal of the attack model is to successfully perform the inversion attack as much as possible to expose the security weaknesses of the large model. The attack success of the attack model is added as a regular term in the loss function to the training process of the adversarial model. This means that a well-trained adversarial model can help the large language model resist potential inversion attacks. The implementation method in the current scenario is: the intermediate result H 1、 H2 will be passed through an adversarial model that has been trained in advance using the above method. The trained model will be used to analyze the intermediate result H 1、 H2 is disturbed to generate the disturbed intermediate result H1' 、 H2', protects the original input data X from being reconstructed by the attacker.

[0030] Preferably, Figure 5As shown, the large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result, further comprising the following steps: After the current prompt word X is input into the model head, the model head can generate a first intermediate result H1 according to the current prompt word; wherein the client can perturb the first intermediate result H1 by the mutual information method to obtain a perturbed first intermediate result H1'; After the disturbed first intermediate result H1' is input into the middle part of the model, the middle part of the model can generate a second intermediate result H2 according to the disturbed first intermediate result H1'; The second intermediate result H2 is sent to the client, and the client can perturb the second intermediate result H2 by the mutual information method to obtain a perturbed second intermediate result H2'; wherein the model tail can generate a final result based on the perturbed second intermediate result H2', wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

[0031] It should be noted that in order to prevent attackers from recovering the original data X from the intermediate results, the mutual information method is to minimize the intermediate result H 1、 The amount of information carried by H2, while retaining the necessary part for reasoning. Minimizing the information dependency between the intermediate results and the original input data X can also increase the difficulty for the attacker to infer the original input X through the intermediate results. In this scenario, we pre-train a mutual information defense model, that is, a model with an encoder-decoder structure (MIDmodel), and use it to transmit the intermediate variable H 1、 H2 is disturbed to generate H1' 、 H2', reduces the information correlation between the intermediate result and the original input data X, thereby improving the privacy protection ability of the system. When transmitting data, even if the intermediate variables are intercepted or inferred, it is difficult to restore the original data. For example: H1 inputs the mutual information defense model. The encoder part of this defense model first maps the intermediate result H1 to the bottleneck layer M (the number of neurons is usually much less than the input or output). The purpose is to ignore unnecessary details in H1, compress information, and discard redundancy or noise. The decoder part decodes the bottleneck layer M to reconstruct the intermediate result H1'. At this time, the information correlation between H1' and the original input data X is significantly reduced, protecting the original input data X.

[0032] It should be noted that what is transmitted in the middle of the large language model is not the original data, but the result of protecting the intermediate results calculated by the large language model through the defense algorithm. In this scenario, it is presented in the form of tensor, which is the basic data structure for internal calculations of the large language model, rather than the original data input by the client.

[0033] It should be noted that the large language model can be: Bert, Roberta, GPT2, Llama, Baichuan, Gemma, Falcon, Mamba, ChatGLM, XLnet, Mistral, etc.

[0034] It should be noted that the query tool is responsible for MySQL query. The agent inputs the query content, and the MySQL query tool connects to the local MySQL database of the client. The MySQL query tool uses the received query content to execute the query to obtain the target data in the data source. The MySQL database contains banking business information.

[0035] It should be noted that ReAct (Reasoning and Acting) is a framework that combines reasoning and acting, aiming to improve the performance and interpretability of large language models (LLMs) in solving complex tasks; Figure 6 As shown in the figure, the operation process of the ReAct agent in this scenario is as follows: 1. Define the task and goal: The task of the agent is to generate the corresponding SQL query statement according to the user's question, and call the query tool to obtain the target data from the MySQL database. The goal is to respond to the user's query request accurately and effectively.

[0036] 2. Generate reasoning trajectory: The agent generates reasoning trajectory to simulate the thinking process to solve the problem. The reasoning trajectory includes steps such as analyzing user problems, planning tool calls, building SQL queries, and handling possible exceptions.

[0037] 3. Execute actions: The agent performs specific actions based on the generated reasoning trajectory. First, the user question is converted into an SQL query. Then, the agent executes the query operation through the interactive tool with the local MySQL database to obtain the required data.

[0038] 4. Alternating reasoning and action: The agent performs new reasoning after each action. Based on the results of querying the database, the agent generates new reasoning trajectories to ensure the accuracy of subsequent actions. For example, if the query result is empty or an error is reported, the agent analyzes the cause and tries to adjust the query.

[0039] 5. Update context: After each action is executed, the agent will update the context information, including the executed SQL query, query results, and newly generated reasoning trajectory. Context updates help the agent track the current task status and provide information for subsequent reasoning and actions.

[0040] 6. Generate task solution trajectory: Through the above steps, the agent generates a complete task solution trajectory, including actions, query results, and reasoning process. This not only shows how the agent solves the problem, but also provides a transparent decision-making process, making it easier for users to understand and evaluate the behavior of the model.

[0041] With this approach, the agent is able to dynamically generate SQL queries and fetch target data from a MySQL database, improving accuracy and efficiency when performing tasks. This approach is particularly useful for tasks that require interacting with a database and generating dynamic queries based on user input.

[0042] Specifically, the model head includes an embedding layer and the first n1 encoders / decoders (the first few layers close to the input layer), the middle of the model includes n2 encoders / decoders, and the tail of the model includes the last n3 encoders / decoders and the last output layer.

[0043] It should be noted that a complete large language model (LLM) is divided into three parts: the model head, the model tail, and the model middle. Among them, the client in the current scenario acts as the data party, retaining the smaller model head and model tail, and the server in the current scenario acts as the model provider, retaining the larger model middle. In this way, the original data input by the server and the client can be isolated to prevent them from performing inversion attacks to obtain the client's private data. In the current scenario, only the reasoning capability of the large language model is applied.

[0044] Preferably, the method further comprises the following step: displaying the final answer through a front-end interface for the user to view.

[0045] The data security analysis method based on a large language model and a defense algorithm provided by the present invention can use the large language model capabilities to generate query content and search the database, thereby performing data security analysis, while protecting data privacy security on the client side and protecting large language model parameters from being shared on the server side.

[0046] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. A data security analysis method based on a large language model and a defense algorithm, implemented by a data security analysis system, characterized in that: The data security analysis system includes a data source, a front end and an intelligent agent; the intelligent agent includes a query tool and a ReAct reasoning framework, wherein the ReAct reasoning framework includes a reasoning stage, an action stage and an observation stage, the reasoning stage involves a large language model, and the large language model is divided into a model head, a model middle and a model tail, the data source, the front end, the query tool, the model head, the model tail, the action stage and the observation stage are all located on the client, and the model middle is located on the server; wherein the data security analysis method based on the large language model and the defense algorithm includes the following steps: Step S1: When the agent receives a user question input by the front end, the agent embeds the user question into a corresponding prompt word template to generate a current prompt word, and then uses the current prompt word as an input for the reasoning stage in the ReAct reasoning framework; Step S2: the large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result; wherein the defense algorithm is a differential privacy method, an adversarial training method, or a mutual information method; Step S3: judging whether there is a final answer with keywords in the current reasoning track through the ReAct reasoning framework; if there is a final answer with keywords, directly outputting the final answer to the front end; if there is no final answer with keywords, calling a corresponding query tool according to the current reasoning track through the action stage, and inputting query content in the corresponding query tool to obtain target data in the data source, and then executing step S4; Step S4: generating a current observation result according to the target data through the observation phase; Step S5: splicing the current reasoning trajectory and the current observation result after the current prompt word through the ReAct reasoning framework to update the prompt word, and then returning to step S2 so that the large language model can reason again according to the updated prompt word until the final answer with the keyword exists in the reasoning trajectory generated by the large language model, and finally outputting the final answer to the front end.

2. The data security analysis method based on a large language model and a defense algorithm according to claim 1 is characterized in that: The large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result, further comprising the following steps: After the current prompt word is input into the model head, the model head can generate a first intermediate result according to the current prompt word; wherein the client can perturb the first intermediate result by using the differential privacy method to obtain the perturbed first intermediate result; After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result; The second intermediate result is sent to the client, and the client can perturb the second intermediate result by using the differential privacy method to obtain the perturbed second intermediate result; wherein the model tail can generate a final result based on the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

3. The data security analysis method based on a large language model and a defense algorithm according to claim 1 is characterized in that: The large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result, further comprising the following steps: After the current prompt word is input into the model head, the model head can generate a first intermediate result according to the current prompt word; wherein the client can perturb the first intermediate result through the adversarial training method to obtain the perturbed first intermediate result; After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result; The second intermediate result is sent to the client, and the client can perturb the second intermediate result through the adversarial training method to obtain the perturbed second intermediate result; wherein the model tail can generate a final result based on the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

4. The data security analysis method based on a large language model and a defense algorithm according to claim 1 is characterized in that: The large language model calculates an intermediate result according to the current prompt word, perturbs the intermediate result through a defense algorithm to obtain a perturbed intermediate result, and then the large language model generates a current reasoning trajectory according to the perturbed intermediate result, further comprising the following steps: After the current prompt word is input into the model head, the model head can generate a first intermediate result according to the current prompt word; wherein the client can perturb the first intermediate result by the mutual information method to obtain a perturbed first intermediate result; After the disturbed first intermediate result is input into the middle part of the model, the middle part of the model can generate a second intermediate result according to the disturbed first intermediate result; The second intermediate result is sent to the client, and the client can perturb the second intermediate result by the mutual information method to obtain the perturbed second intermediate result; wherein the model tail can generate a final result according to the perturbed second intermediate result, wherein the final result is the current reasoning trajectory, and the current reasoning trajectory includes the name of the query tool to be called, the query content to be entered in the query tool, and the final answer.

5. The data security analysis method based on a large language model and a defense algorithm according to claim 1 is characterized in that: The model head includes an embedding layer and n1 encoders / decoders, the model middle includes n2 encoders / decoders, and the model tail includes n3 encoders / decoders and an output layer.

6. The data security analysis method based on a large language model and a defense algorithm according to claim 1 is characterized in that: The method also includes the following steps: displaying the final answer through a front-end interface.

Images