A sensitive information leakage detection method for large model applications

Abstract

The application relates to the technical field of large-scale pre-training language model application, and particularly provides a sensitive information leakage detection method and device for large model application, which comprises the following steps: converting a to-be-detected text into a graph structure; taking the graph structure as the input of a pre-trained graph neural network to obtain the sensitive information detection result of the to-be-detected text output by the pre-trained graph neural network. The technical scheme provided by the application can ensure that the content generated by a large model does not contain unauthorized sensitive data, and can prevent the leakage of sensitive data such as personal privacy information and confidential information in natural language generation, a dialogue system or other large-scale model applications.

Description

Technical Field

[0001] This invention relates to the field of large-scale pre-trained language model application technology, specifically to a method and apparatus for detecting sensitive information leakage in large-scale model applications. Background Technology

[0002] With the widespread adoption of large-scale deep learning models, especially language generation models (such as the GPT series and BERT), more and more enterprises and organizations are deploying these models in production environments for tasks such as text generation, question answering, and dialogue systems. However, the training process of these models often uses a large number of public datasets and data that may contain sensitive information. When the model is applied, the generated content may inadvertently reveal the sensitive information contained in the training process.

[0003] Existing technical solutions mainly include the following categories:

[0004] 1) Data audit-based detection methods: This approach reduces the risk of model leakage during generation by marking sensitive information in the training data. However, this method relies on a complete audit of the training data and cannot effectively address implicit information within the data.

[0005] 2) Detection methods based on generated results analysis: This approach uses techniques such as keyword matching and privacy dictionary retrieval to filter the generated text content and detect whether it contains sensitive information. However, this method has limited accuracy and generalization ability, and is easily affected by variations in the input text, the diversity and complexity of the generated content.

[0006] 3) Protection methods based on adversarial training: This method reduces the probability of the model generating sensitive information by introducing adversarial training. However, this method has a large computational cost and fails to fundamentally solve the problem of information leakage during the generation process.

[0007] Although some methods exist for detecting and protecting sensitive information, most of them suffer from incomplete coverage, insufficient accuracy, and high computational overhead, failing to completely eliminate the risk of sensitive information leakage in large-scale model applications. Summary of the Invention

[0008] To overcome the above-mentioned shortcomings, this invention proposes a sensitive information leakage detection method and device for large-scale model applications.

[0009] Firstly, a sensitive information leakage detection method for large-scale model applications is provided, the sensitive information leakage detection method for large-scale model applications includes:

[0010] Convert the text to be detected into a graph structure;

[0011] The graph structure is used as input to a pre-trained graph neural network to obtain the sensitive information detection result of the text to be detected, which is output by the pre-trained graph neural network.

[0012] Preferably, the training process of the pre-trained graph neural network includes:

[0013] Generate training text using a large language model;

[0014] Sensitive information detection methods are used to identify sensitive information in the training text;

[0015] The training text labeled with sensitive information is used as input to the deep learning model to obtain the training text with relabeled sensitive information output by the deep learning model.

[0016] The training text with re-annotated sensitive information is transformed into a graph structure, and the training data is constructed using the graph structure.

[0017] The initial graph neural network is trained using the training data to obtain the pre-trained graph neural network.

[0018] Preferably, the sensitive information includes explicit sensitive information and implicit sensitive information.

[0019] Furthermore, the method for identifying sensitive information in the training text using a sensitive information detection method includes:

[0020] The named entity recognition method in natural language processing is used to identify explicit sensitive information in the training text;

[0021] The training text is transformed into a graph structure, and the graph structure is used as input to a pre-constructed latent causal network to obtain the implicit sensitive information in the training text output by the pre-constructed latent causal network.

[0022] Furthermore, the deep learning model is trained using an incremental learning algorithm. During training, the model parameters are optimized using the following formula:

[0023]

[0024] In the above formula, θ t+1 Let θ be the model parameters at time t+1. t Let be the model parameters at time t, and η be the learning rate. Let F be the gradient of the loss function F with respect to the model parameters θ.

[0025] Furthermore, the loss function is as follows:

[0026]

[0027] In the above formula, m is the number of annotations in the training text labeled with sensitive information, and Loss is... classification (i) represents the recognition error of the i-th annotation in the training text.

[0028] Preferably, the mathematical model corresponding to the graph structure is as follows:

[0029] G = (V, E)

[0030] In the above formula, G is the graph structure, V is the entity in the text, and E is the edge representing the causal relationship between entities.

[0031] Secondly, a sensitive information leakage detection device for large-scale model applications is provided, the sensitive information leakage detection device for large-scale model applications comprising:

[0032] The conversion module is used to convert the text to be detected into a graph structure;

[0033] The analysis module is used to take the graph structure as input to a pre-trained graph neural network and obtain the sensitive information detection result of the text to be detected output by the pre-trained graph neural network.

[0034] Thirdly, a computer device is provided, comprising: one or more processors;

[0035] The processor is used to store one or more programs;

[0036] When the one or more programs are executed by the one or more processors, the sensitive information leakage detection method for large-scale application is implemented.

[0037] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed, the aforementioned sensitive information leakage detection method for large-scale model applications is implemented.

[0038] The above-described technical solutions of the present invention have at least one or more of the following beneficial effects:

[0039] This invention relates to the field of large-scale pre-trained language model application technology, specifically providing a method and apparatus for detecting sensitive information leakage in large-scale model applications. The method includes: converting the text to be detected into a graph structure; using the graph structure as input to a pre-trained graph neural network to obtain the sensitive information detection result of the text to be detected, output by the pre-trained graph neural network. The technical solution provided by this invention can ensure that the content generated by large models does not contain unauthorized sensitive data, preventing the leakage of sensitive data such as personal privacy information and confidential information in natural language generation, dialogue systems, or other large-scale model applications. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the main steps of the sensitive information leakage detection method for large-scale model applications according to an embodiment of the present invention. Detailed Implementation

[0041] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] As disclosed in the background section, with the widespread adoption of large-scale deep learning models, especially language generation models (such as the GPT series and BERT), more and more enterprises and organizations are deploying these models in production environments for tasks such as text generation, question answering, and dialogue systems. However, during the training process of these models, a large number of public datasets and data that may contain sensitive information are often used. When the model is applied, the generated content may inadvertently reveal the sensitive information contained during the training process.

[0044] Existing technical solutions mainly include the following categories:

[0045] 1) Data audit-based detection methods: This approach reduces the risk of model leakage during generation by marking sensitive information in the training data. However, this method relies on a complete audit of the training data and cannot effectively address implicit information within the data.

[0046] 2) Detection methods based on generated results analysis: This approach uses techniques such as keyword matching and privacy dictionary retrieval to filter the generated text content and detect whether it contains sensitive information. However, this method has limited accuracy and generalization ability, and is easily affected by variations in the input text, the diversity and complexity of the generated content.

[0047] 3) Protection methods based on adversarial training: This method reduces the probability of the model generating sensitive information by introducing adversarial training. However, this method has a large computational cost and fails to fundamentally solve the problem of information leakage during the generation process.

[0048] Although some methods exist for detecting and protecting sensitive information, most suffer from incomplete coverage, insufficient accuracy, and high computational costs, failing to completely eliminate the risk of sensitive information leakage in large-scale model applications. Specifically:

[0049] The existing solutions mainly have the following problems:

[0050] 1) Implicit information leakage in input data: Current sensitive information detection methods mainly rely on explicit sensitive information labeling and word matching. Since the training process of large models may contain a large amount of implicit information (e.g., implicit personal data, sensitive industry information, etc.), this information is not easily detected by traditional text auditing methods.

[0051] 2) Complexity of generated content: Existing detection methods based on generated content analysis often rely on pre-set sensitive information vocabularies, but these vocabularies fail to cover all possible forms of sensitive information, especially implicit information in generated content, which leads to the possibility of missed detections and false alarms.

[0052] 3) High cost of adversarial training: Although adversarial training helps reduce the probability of the model leaking sensitive information, its computational overhead is huge and it fails to fundamentally solve the potential leakage risk in the model training and application process.

[0053] Therefore, existing technologies suffer from problems such as low accuracy, poor generalization ability, and high computational overhead in the detection of sensitive information leakage.

[0054] To address the aforementioned problems, this invention relates to the field of large-scale pre-trained language model application technology, specifically providing a method and apparatus for detecting sensitive information leakage in large-scale model applications. The method includes: converting the text to be detected into a graph structure; using the graph structure as input to a pre-trained graph neural network to obtain the sensitive information detection result of the text to be detected, output by the pre-trained graph neural network. The technical solution provided by this invention ensures that the content generated by large models does not contain unauthorized sensitive data, preventing the leakage of sensitive data such as personal privacy information and confidential information in natural language generation, dialogue systems, or other large-scale model applications. The solution is described in detail below.

[0055] Example 1

[0056] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a sensitive information leakage detection method for large-scale model applications according to an embodiment of the present invention. Figure 1 As shown, the sensitive information leakage detection method for large-scale model applications in this embodiment of the invention mainly includes the following steps:

[0057] Step S101: Convert the text to be detected into a graph structure;

[0058] Step S102: Use the graph structure as input to a pre-trained graph neural network to obtain the sensitive information detection result of the text to be detected output by the pre-trained graph neural network.

[0059] In this embodiment, the training process of the pre-trained graph neural network includes:

[0060] Generate training text using a large language model;

[0061] Sensitive information detection methods are used to identify sensitive information in the training text;

[0062] The training text labeled with sensitive information is used as input to the deep learning model to obtain the training text with relabeled sensitive information output by the deep learning model.

[0063] The training text with re-annotated sensitive information is transformed into a graph structure, and the training data is constructed using the graph structure.

[0064] The initial graph neural network is trained using the training data to obtain the pre-trained graph neural network.

[0065] In this embodiment, the sensitive information includes explicit sensitive information and implicit sensitive information.

[0066] In one implementation, the step of identifying sensitive information in the training text using a sensitive information detection method includes:

[0067] The named entity recognition method in natural language processing is used to identify explicit sensitive information in the training text;

[0068] The training text is transformed into a graph structure, and the graph structure is used as input to a pre-constructed latent causal network to obtain the implicit sensitive information in the training text output by the pre-constructed latent causal network.

[0069] In this way, the system can comprehensively analyze the risk of sensitive information leakage in text, whether it is explicit information or potential implicit information.

[0070] In one implementation, the deep learning model is trained using an incremental learning algorithm, and during training, the model parameters are optimized using the following formula:

[0071]

[0072] In the above formula, θ t+1 Let θ be the model parameters at time t+1. t Let be the model parameters at time t, and η be the learning rate. Let F be the gradient of the loss function F with respect to the model parameters θ.

[0073] In one implementation, the loss function is as follows:

[0074]

[0075] In the above formula, m is the number of annotations in the training text labeled with sensitive information, and Loss is... classification (i) represents the recognition error of the i-th annotation in the training text.

[0076] Specifically, the loss function can include the following aspects: 1) The model needs to correctly identify whether the text contains sensitive information. For example, if the model incorrectly labels some non-sensitive information as sensitive information or omits some sensitive information, it will lead to an increase in the loss value. 2) In the generated text, some information may only be sensitive in a specific context, so the goal of the model is to identify which information is sensitive based on the text context. 3) For different types of sensitive information (such as personal identity, financial data, medical information, etc.), the model needs to identify the sensitive information in each category and classify it.

[0077] Incremental learning optimizes the sensitive information filtering model, enabling rapid adaptation and accurate identification of new types of sensitive information. Through gradient updates and real-time feedback, the model continuously improves its filtering ability for potential sensitive information during text generation. The introduction of incremental learning ensures the model remains efficient and agile in dynamic and real-time environments, guaranteeing the efficient detection and processing of sensitive information.

[0078] In this embodiment, the mathematical model corresponding to the graph structure is as follows:

[0079] G = (V, E)

[0080] In the above formula, G is the graph structure, V is the entity in the text, and E is the edge representing the causal relationship between entities.

[0081] Example 2

[0082] Based on the same inventive concept, the present invention also provides a sensitive information leakage detection device for large-scale model applications, the sensitive information leakage detection device for large-scale model applications comprising:

[0083] The conversion module is used to convert the text to be detected into a graph structure;

[0084] The analysis module is used to take the graph structure as input to a pre-trained graph neural network and obtain the sensitive information detection result of the text to be detected output by the pre-trained graph neural network.

[0085] Preferably, the training process of the pre-trained graph neural network includes:

[0086] Generate training text using a large language model;

[0087] Sensitive information detection methods are used to identify sensitive information in the training text;

[0088] The training text labeled with sensitive information is used as input to the deep learning model to obtain the training text with relabeled sensitive information output by the deep learning model.

[0089] The training text with re-annotated sensitive information is transformed into a graph structure, and the training data is constructed using the graph structure.

[0090] The initial graph neural network is trained using the training data to obtain the pre-trained graph neural network.

[0091] Preferably, the sensitive information includes explicit sensitive information and implicit sensitive information.

[0092] Furthermore, the method for identifying sensitive information in the training text using a sensitive information detection method includes:

[0093] The named entity recognition method in natural language processing is used to identify explicit sensitive information in the training text;

[0094] The training text is transformed into a graph structure, and the graph structure is used as input to a pre-constructed latent causal network to obtain the implicit sensitive information in the training text output by the pre-constructed latent causal network.

[0095] Furthermore, the deep learning model is trained using an incremental learning algorithm. During training, the model parameters are optimized using the following formula:

[0096]

[0097] In the above formula, θ t+1 Let θ be the model parameters at time t+1. t Let be the model parameters at time t, and η be the learning rate. Let F be the gradient of the loss function F with respect to the model parameters θ.

[0098] Furthermore, the loss function is as follows:

[0099]

[0100] In the above formula, m is the number of annotations in the training text labeled with sensitive information, and Loss is... classification (i) represents the recognition error of the i-th annotation in the training text.

[0101] Preferably, the mathematical model corresponding to the graph structure is as follows:

[0102] G = (V, E)

[0103] In the above formula, G is the graph structure, V is the entity in the text, and E is the edge representing the causal relationship between entities.

[0104] Example 3

[0105] Based on the same inventive concept, this invention also provides a computer device, which includes a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to implement corresponding method flows or corresponding functions, thereby realizing the steps of a sensitive information leakage detection method for large-scale applications in the above embodiments.

[0106] Example 4

[0107] Based on the same inventive concept, this invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of a sensitive information leakage detection method for large-scale applications described in the above embodiments.

[0108] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0110] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0111] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting the leakage of sensitive information for large-scale model applications, characterized in that, The method includes: Convert the text to be detected into a graph structure; Using the graph structure as input to a pre-trained graph neural network, the sensitive information detection result of the text to be detected is obtained from the output of the pre-trained graph neural network. The training process of the pre-trained graph neural network includes: Generate training text using a large language model; Sensitive information detection methods are used to identify sensitive information in the training text; The training text labeled with sensitive information is used as input to the deep learning model to obtain the training text with relabeled sensitive information output by the deep learning model. The training text with re-annotated sensitive information is transformed into a graph structure, and the training data is constructed using the graph structure. The initial graph neural network is trained using the training data to obtain the pre-trained graph neural network; The sensitive information includes explicit sensitive information and implicit sensitive information; The method of identifying sensitive information in the training text using a sensitive information detection method includes: The named entity recognition method in natural language processing is used to identify explicit sensitive information in the training text; The training text is transformed into a graph structure, and the graph structure is used as the input of a pre-constructed latent causal network to obtain the implicit sensitive information in the training text output by the pre-constructed latent causal network. The deep learning model is trained using an incremental learning algorithm. During training, the model parameters are optimized using the following formula: In the above formula, θ t+1 Let θ be the model parameters at time t+1. t Let be the model parameters at time t, and η be the learning rate. Let F be the gradient of the loss function F with respect to the model parameters θ. The loss function is as follows: In the above formula, m is the number of annotations in the training text labeled with sensitive information, and Loss is... classification (i) represents the recognition error of the i-th annotation in the training text.

2. The method as described in claim 1, characterized in that, The mathematical model corresponding to the graph structure is as follows: G = (V, E) In the above formula, G is the graph structure, V is the entity in the text, and E is the edge representing the causal relationship between entities.

3. An apparatus for detecting sensitive information leakage in large-scale model applications based on the method described in any one of claims 1-2, characterized in that, The device includes: The conversion module is used to convert the text to be detected into a graph structure; The analysis module is used to take the graph structure as input to a pre-trained graph neural network and obtain the sensitive information detection result of the text to be detected output by the pre-trained graph neural network.

4. A computer device, characterized in that, include: One or more processors; The processor is used to execute one or more programs; When the one or more programs are executed by the one or more processors, the sensitive information leakage detection method for large-scale applications as described in any one of claims 1 to 2 is implemented.

5. A computer-readable storage medium, characterized in that, It contains a computer program, which, when executed, implements the sensitive information leakage detection method for large-scale model applications as described in any one of claims 1 to 2.

Images