Self-adaptive sensitive information extraction method and device

Abstract

The invention provides a self-adaptive sensitive information extraction method and device.The self-adaptive sensitive information extraction method comprises the steps that rich texts are obtained, and the rich texts are classified; the rich text comprises a code file and a non-code file; performing data processing on the non-code file, and converting the non-code file into a plain text file; selecting an information identification method according to the type of sensitive information to be identified, and performing sensitive information extraction on the code file and the plain text file to obtain target sensitive information; the information identification method comprises a large model code understanding method, a rule matching method and a multi-granularity associated sensitive information identification method. According to the method and the device, different information identification methods are designed according to different file types and sensitive information types, so that a more comprehensive identification result is obtained, and the problem that sensitive information extraction is limited in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to an adaptive sensitive information extraction method and device. Background Art

[0002] Informatization is the main trend of current economic and social development, and electronic files have gradually become the main form of data storage and transmission. With the continuous advancement of information technology, information security faces increasingly severe challenges. The leakage of sensitive information may cause varying degrees of losses to related individuals or organizations. Therefore, identifying sensitive information in various files is crucial to the healthy development of the Internet.

[0003] At present, the existing sensitive information identification solutions can be mainly divided into two categories: 1. Matching-based methods. This method uses sensitive word detection to find whether the text contains sensitive information. However, this requires the pre-construction of a sensitive word list, and the maintenance cost is high. 2. Deep learning-based methods. This method uses a deep learning model to automatically learn relevant features to identify potential sensitive information and its types in the text. However, this method often cannot classify the types of sensitive information in detail.

[0004] As the demand for detection increases, traditional methods have certain limitations. In practical applications, sensitive information often exists in various rich texts, not just plain text, and the types of sensitive information are also diverse. In addition, identifying the correlation between sensitive information, that is, which sensitive information belongs to the same individual, will help further analysis and obtain more comprehensive identification results.

[0005] There is currently no effective solution to the problem of limitations in sensitive information extraction in existing technologies. Summary of the invention

[0006] The present invention provides an adaptive sensitive information extraction method and device, which are used to solve the defect of the prior art that the sensitive information extraction has limitations.

[0007] In a first aspect, the present invention provides an adaptive sensitive information extraction method, comprising: Acquire rich text and classify the rich text; the rich text includes code files and non-code files; Performing data processing on the non-code file to convert the non-code file into a plain text file; According to the type of sensitive information to be identified, an information identification method is selected, and sensitive information is extracted from the code file and the plain text file to obtain the target sensitive information; the information identification method includes a large model code understanding method, a rule matching method and a multi-granularity associated sensitive information identification method.

[0008] According to an adaptive sensitive information extraction method provided by the present invention, data processing is performed on the non-code file to convert the non-code file into a plain text file, including: The text in the picture file in the non-code file is recognized by an optical character recognition method, and the text information in the non-code file is recognized by an open source library, so as to convert the non-code file into a plain text file.

[0009] According to an adaptive sensitive information extraction method provided by the present invention, an information identification method is selected according to the type of sensitive information to be identified, and sensitive information is extracted from the code file and the plain text file, including: Extracting sensitive information from the code file using a large model code understanding method; For the rule-sensitive information in the plain text file, a rule matching method is used to extract the sensitive information; For the non-regular sensitive information in the plain text file, a multi-granularity associated sensitive information identification method is used to extract the sensitive information.

[0010] According to an adaptive sensitive information extraction method provided by the present invention, sensitive information is extracted from the code file through a large model code understanding method, including: Call the pre-trained code to understand the large language model; The code file is used as input of the code understanding large language model, and target sensitive information in the code file is output.

[0011] According to an adaptive sensitive information extraction method provided by the present invention, the code understanding large language model is trained, including: Obtaining code training data, and manually marking sensitive information segments in the code training data; The sensitive information segment is used as a label and the code understanding large language model is trained and fine-tuned through the code training data.

[0012] According to an adaptive sensitive information extraction method provided by the present invention, a multi-granularity associated sensitive information identification method is used to extract sensitive information, including: Performing importance assessment and screening on the plain text files to obtain screened texts; Encoding the screened text through a language representation model and generating word representations through a bidirectional long short-term memory network; Generate a two-dimensional vocabulary representation for the plain text file based on the word representation; the two-dimensional vocabulary representation includes a plurality of word pairs; Capturing the interactions between word pairs of different distances in the two-dimensional vocabulary representation through two-dimensional dilated convolution to obtain a word pair grid representation; Determine the relationship distribution of each word pair in the word pair grid representation by a classifier to obtain a relationship distribution result; The relationship between word pairs is predicted based on the relationship distribution result, sensitive information clusters are determined, and sensitive information is extracted based on the sensitive information clusters.

[0013] According to an adaptive sensitive information extraction method provided by the present invention, importance evaluation and screening are performed on the plain text file to obtain the screened text, including: The trained extractive summarization model is called, the plain text file is taken as input, and the filtered text is output; the filtered text is the sentences and the preceding and following sentences containing sensitive information in the plain text file.

[0014] According to an adaptive sensitive information extraction method provided by the present invention, training the extractive summary model includes: Obtain open source sensitive information data and generate unstructured data through large language models to build training data sets; With the goal of minimizing the model loss of the extractive summarization model, the extractive summarization model is fine-tuned through a training dataset.

[0015] According to an adaptive sensitive information extraction method provided by the present invention, data enhancement is performed through a large language model, including: Given a task description, generate candidate prompts for the task description through a large language model; Evaluating and optimizing the candidate prompts to obtain optional prompts; The prompt learning template is input into the large language model, and unstructured data is generated according to the optional prompts.

[0016] In a second aspect, the present invention further provides an adaptive sensitive information extraction device, comprising: An acquisition module, used for acquiring rich text and classifying the rich text; the rich text includes code files and non-code files; A processing module, used for performing data processing on the non-code file and converting the non-code file into a plain text file; The identification module is used to select an information identification method according to the type of sensitive information to be identified, and extract sensitive information from the code file and the plain text file to obtain the target sensitive information; the information identification method includes a large model code understanding method, a rule matching method and a multi-granularity associated sensitive information identification method.

[0017] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the adaptive sensitive information extraction method as described in the first aspect above is implemented.

[0018] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the adaptive sensitive information extraction method as described in the first aspect above.

[0019] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the adaptive sensitive information extraction method as described in the first aspect above.

[0020] Compared with the prior art, the present invention has the following beneficial effects: The adaptive sensitive information extraction method provided by the present invention distinguishes the types of rich text and screens out code files and non-code files. In order to more fully understand and identify the sensitive information in the non-code files, data processing is performed on the non-code files, the text information in the pictures is understood, and the non-code files are converted into plain text files. Finally, according to the type of sensitive information to be identified and the type of file, the corresponding information identification method is selected, and the sensitive information is extracted from the above-mentioned code files and plain text files respectively to obtain the target sensitive information. In the above process, different information identification methods are designed according to different file types and sensitive information types, so as to obtain more comprehensive identification results, thereby solving the problem of limitations in sensitive information extraction in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 is a flow chart of the adaptive sensitive information extraction method provided by the present invention; Figure 2 is a flow chart of extracting sensitive information from rich text in an embodiment of the present invention; Figure 3 This is a schematic diagram of a prompt word structure of a large language model code understanding method in an embodiment of the present invention; Figure 4 is a flow chart for data enhancement of a large language model in an embodiment of the present invention; Figure 5 is a schematic diagram of a method for simultaneously modeling sensitive information, types, and associations between sensitive information in an embodiment of the present invention; Figure 6 is a flow chart of extracting sensitive information through a multi-granularity associated sensitive information identification method in an embodiment of the present invention; Figure 7 It is a structural block diagram of the adaptive sensitive information extraction device provided by the present invention; Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The present invention provides an adaptive sensitive information extraction method. Figure 1 is a flow chart of the adaptive sensitive information extraction method provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step S101, obtaining rich text and classifying the rich text; the rich text includes code files and non-code files.

[0025] Step S102, performing data processing on the non-code file to convert the non-code file into a plain text file.

[0026] Step S103, select an information identification method according to the type of sensitive information to be identified, and extract sensitive information from code files and plain text files to obtain target sensitive information; the information identification method includes a large model code understanding method, a rule matching method and a multi-granularity associated sensitive information identification method.

[0027] In this method, first, the rich text from which sensitive information needs to be extracted is obtained, and the type of rich text is distinguished to screen out code files and non-code files. Since non-code files may not only contain text content, but also image content. In order to more fully understand and identify the sensitive information in non-code files, in this method, data processing is performed on non-code files, the text information in the image is understood, and the non-code files are converted into plain text files. Finally, according to the type of sensitive information to be identified and the type of file, the corresponding information identification method is selected, and sensitive information is extracted from the above-mentioned code files and plain text files respectively to obtain the target sensitive information. In the above process, different information identification methods are designed according to different file types and sensitive information types, so as to obtain more comprehensive identification results, thereby solving the problem of limitations in sensitive information extraction in the prior art.

[0028] In some of the embodiments, step S102, data processing is performed on the non-code file to convert the non-code file into a plain text file, including: recognizing the text in the image file in the non-code file by an optical character recognition method (OCR), and recognizing the text information in the non-code file by an open source library to convert the non-code file into a plain text file. In this embodiment, converting the non-code file into a plain text file can fully understand the information content in the non-code file, prevent information omission, and thus improve the comprehensiveness and accuracy of sensitive information extraction.

[0029] Figure 2 is a flow chart of extracting sensitive information from rich text in an embodiment of the present invention. Figure 2 As shown, in some of the embodiments, step S103, according to the type of sensitive information to be identified, selects an information identification method, and extracts sensitive information from code files and plain text files, including: extracting sensitive information from code files through a large model code understanding method; for rule-based sensitive information in plain text files, using a rule matching method to extract sensitive information; for non-rule-based sensitive information in plain text files, using a multi-granularity associated sensitive information identification method to extract sensitive information.

[0030] Specifically, sensitive information is extracted from the code file through the large model code understanding method, including: calling a pre-trained code understanding large language model; using the code file as the input of the code understanding large language model, and outputting the target sensitive information in the code file.

[0031] In this implementation, the code comprehension large language model is trained, including: obtaining code training data and manually annotating sensitive information segments in the code training data; using the sensitive information segments as labels, and training and fine-tuning the code comprehension large language model through the code training data.

[0032] For example, a large language model such as Llama2 model that contains a large amount of code training data and has code understanding capabilities during pre-training is used to manually annotate some segments containing sensitive information, select multiple high-quality manually generated data as input examples to form a list of input-output pairs, add the code text to be extracted at the end of the prompt, and let the large language model determine the input distribution, output distribution, input-output mapping and format, such as Figure 3 As shown, Figure 3 It is a schematic diagram of a prompt word structure of a large language model code understanding method in an embodiment of the present invention.

[0033] For rule-sensitive information in plain text files, rule-sensitive information usually has an obvious pattern, such as regular expressions corresponding to IP, email, etc. Therefore, for rule-sensitive information, in this method, pre-defined templates and rules are used to extract sensitive information, which can improve the accuracy of sensitive information extraction.

[0034] A multi-granularity associated sensitive information identification method is used to extract sensitive information, including: evaluating and screening the importance of plain text files to obtain screened texts; encoding the screened texts through a language representation model, and generating word representations through a bidirectional long short-term memory network; generating a two-dimensional word table representation for plain text files based on the word representation; the two-dimensional word table representation contains several word pairs; capturing the interaction between word pairs of different distances in the two-dimensional word table representation through two-dimensional dilated convolution to obtain a word pair grid representation; determining the relationship distribution of each word pair in the word pair grid representation through a classifier to obtain a relationship distribution result; predicting the relationship between word pairs based on the relationship distribution result, determining sensitive information clusters, and extracting sensitive information based on the sensitive information clusters.

[0035] More specifically, the importance of the plain text file is evaluated and screened to obtain the screened text, including: calling the trained extractive summarization model, taking the plain text file as input, and outputting the screened text; the screened text is the sentences and the preceding and following sentences containing sensitive information in the plain text file.

[0036] Among them, the training of the extractive summary model includes: obtaining open source sensitive information data, generating unstructured data through a large language model, and constructing a training data set; with the goal of minimizing the model loss of the extractive summary model, the extractive summary model is fine-tuned through the training data set.

[0037] In this embodiment, data enhancement is performed through a large language model, including: given a task description, generating candidate prompts for the task description through a large language model; evaluating and optimizing the candidate prompts to obtain optional prompts; inputting the prompt learning template into the large language model, and generating unstructured data based on the optional prompts.

[0038] For example, in order to support the training and testing of large language models, and in view of the problems of insufficient data information, inconsistent data quality, relatively simple data structure, and insufficient data volume in open source data, a training data set is established by crawling open source sensitive information data and using large model data enhancement. Figure 4 As shown, Figure 4 This is a flowchart for data enhancement of a large language model in an embodiment of the present invention. Given a task description, a dynamic prompt and a self-checking mechanism are used to generate prompts, allowing the large language model to generate multiple candidate prompts at a time, and in the next round, it is required to evaluate the candidate prompts, select the best prompt and give reasons. In each call, the best prompt evaluated by humans is added to the example to improve the In-context capability. After obtaining the optional prompts, the unstructured data generated by the open source tool is integrated, and the large language model is used to achieve data enhancement.

[0039] In order to simultaneously identify sensitive information and its relevance, the task of extracting related sensitive information tuples is regarded as a word-pair relationship classification task, and a method of modeling related sensitive information tuples using a two-dimensional word pair table and three word pair relationships is proposed. The three word pair relationships are: Next-Neighboring-information- (NNI- ), indicating that two words are adjacent in the same information cluster, and the sub-relationship " " indicates sensitive information type; Tail-Head-information- (THI- ), indicating that a loop is formed to represent a cluster, and the sub-relationship " " indicates sensitive information type; None, indicating that the two sensitive information are not in the same cluster or do not belong to sensitive information.

[0040] Figure 5 is a schematic diagram of a method for simultaneously modeling sensitive information, types, and associations between sensitive information in an embodiment of the present invention, such as Figure 5As shown in the figure, the sample contains two clusters of related sensitive information, namely "{account: zhangs, password: 000}" and "{name: Wang Wu, phone number: 223800}". In order to avoid the sparsity of relationship instances, NNI (N) and THI (T) relationships are marked in the upper and lower triangle areas respectively. In the figure, according to the NNI relationship, (zhangs -> 000), (Wang -> Wu) and (Wu -> 223800) can be easily obtained. The THI relationship indicates the starting and ending positions of the related information clusters, and (000 -> zhangs) and (223800 -> Wang) are obtained. Combined with the sensitive information type sub-relationship, two related sensitive information clusters {account: zhangs, password: 000}, {name: Wang Wu, phone number: 223800} are decoded.

[0041] Figure 6 FIG. 1 is a flow chart of extracting sensitive information by using a multi-granularity associated sensitive information identification method in an embodiment of the present invention. Figure 6 As shown in the figure, first, the importance of sentences is evaluated, and sentences that directly contain sensitive information or contain contextual semantics are screened. Using the constructed training dataset, the original text is input, and the expected output is a sentence containing sensitive information and its preceding and following sentences, and the extractive summary model is trained to achieve the effect of data processing and prevent the subsequent calculation complexity from being too high. Then, the filtered sentences are encoded using the language representation model (Bidirectional Encoder Representations from Transformers, BERT), which has been proven to be one of the most advanced models for representation learning relationship classification. In order to further enhance context modeling, a bidirectional long short-term memory network (Long Short-Term Memory, LSTM) is used to generate the final word representation. The input of BERT consists of three parts, namely token embedding, position embedding, and segment embedding, which model word, position, and sentence information respectively. Inspired by BERT, the word pair grid representation is enriched using similar ideas. Then, the three embeddings are connected and the multilayer perceptron (Multilayer Perceptron, MLP) is used for dimensionality reduction to obtain the position region-aware representation of the word pair grid. The whole process can be formulated as:

[0042] Among them, C represents the word pair grid, the feature vector V represents the word information, and the feature vector E is obtained by using the conditional normalization layer. d Represents the relative position information between word pairs, feature vector E t Represents the area information used to distinguish the lower triangle area and the upper triangle area in the table.

[0043] Then, multiple two-dimensional dilated convolutions (DConv) with different dilation rates are used to capture the interactions between word pairs at different distances. The specific formula is as follows:

[0044] Among them, Q l represents the output of the dilated convolution with a dilation rate of 1, represents the GELU activation function, and C represents the word pair grid. Through the above process, the formula for word pair grid representation is as follows:

[0045] Among them, Q represents the word pair grid representation, Q l represents the output of the dilated convolution with a dilation rate of 1, Q 2 represents the output of the dilated convolution with a dilation rate of 2, Q 3 represents the output of the dilated convolution with a dilation rate of 3. The classifier is then used to calculate each word pair (x i ,x j ), the loss of the relationship distribution is calculated using the cross entropy loss function in the training phase, and the maximum probability relationship is taken as the predicted relationship between each word pair in the prediction phase. Finally, the relationship between word pairs is decoded. The goal of the model is to predict the relationship between word pairs, which can be considered as a directed word list. The purpose of decoding is to use the NNI relationship (Neural Network Intelligence) to find a specific path from one word to another in the graph. Each path corresponds to a sensitive information cluster. The THI relationship indicates the starting and ending positions of the path, assists in determining the boundaries of the cluster, and obtains the sensitive information cluster.

[0046] In addition, in this method, for the same sensitive file, the multi-granularity associated sensitive information identification method and the rule-based matching method are used to extract different types of sensitive information. The former can identify the association, that is, it can identify the sensitive information cluster, while the latter can only identify independent sensitive information. In order to perform information fusion, a threshold is set. If the distance between a piece of information and the cluster is within this threshold, it is added to the cluster. If it exceeds this threshold, it is clustered separately. Common sense knowledge is added, such as IP and port, account and password usually appear at the same time, to further improve the fusion effect.

[0047] The present invention also provides an adaptive sensitive information extraction device. The adaptive sensitive information extraction device provided by the present invention is described below. The adaptive sensitive information extraction device described below and the adaptive sensitive information extraction method described above can be referenced to each other. Figure 7 is a structural block diagram of the adaptive sensitive information extraction device provided by the present invention, such as Figure 7 As shown, the device comprises: The acquisition module 701 is used to acquire rich text and classify the rich text; the rich text includes code files and non-code files; The processing module 702 is used to perform data processing on the non-code file and convert the non-code file into a plain text file; The identification module 703 is used to select an information identification method according to the type of sensitive information to be identified, and extract sensitive information from code files and plain text files to obtain target sensitive information; the information identification method includes a large model code understanding method, a rule matching method and a multi-granularity associated sensitive information identification method.

[0048] When the present device is in use, first, the acquisition module 701 acquires the rich text that needs to be subjected to sensitive information extraction, and distinguishes the type of the rich text, and screens out the code file and the non-code file. Since the non-code file may not only contain text content, but also may include image content. In order to more fully understand and identify the sensitive information in the non-code file, when the present device is used, the processing module 702 performs data processing on the non-code file, understands the text information in the image, and converts the non-code file into a plain text file. Finally, the identification module 703 selects the corresponding information identification method according to the type of sensitive information to be identified and the type of file, and extracts sensitive information from the above-mentioned code file and plain text file respectively to obtain the target sensitive information. In the above process, different information identification methods are designed according to different file types and sensitive information types, so as to obtain more comprehensive identification results, and solve the problem of limitations in sensitive information extraction existing in the prior art.

[0049] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802 and the memory 803 communicate with each other through the communication bus 804. The processor 801 may call the logic instructions in the memory 803 to execute the adaptive sensitive information extraction method, which includes: Get rich text and classify the rich text; rich text includes code files and non-code files; Perform data processing on non-code files and convert them into plain text files; According to the type of sensitive information to be identified, an information identification method is selected, and sensitive information is extracted from code files and plain text files to obtain the target sensitive information; information identification methods include large model code understanding methods, rule matching methods, and multi-granularity associated sensitive information identification methods.

[0050] In addition, the logic instructions in the above-mentioned memory 803 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0051] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the adaptive sensitive information extraction method provided by the above methods, the method includes: Get rich text and classify the rich text; rich text includes code files and non-code files; Perform data processing on non-code files and convert them into plain text files; According to the type of sensitive information to be identified, an information identification method is selected, and sensitive information is extracted from code files and plain text files to obtain the target sensitive information; information identification methods include large model code understanding methods, rule matching methods, and multi-granularity associated sensitive information identification methods.

[0052] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the adaptive sensitive information extraction method provided by the above methods, the method comprising: Get rich text and classify the rich text; rich text includes code files and non-code files; Perform data processing on non-code files and convert them into plain text files; According to the type of sensitive information to be identified, an information identification method is selected, and sensitive information is extracted from code files and plain text files to obtain the target sensitive information; information identification methods include large model code understanding methods, rule matching methods, and multi-granularity associated sensitive information identification methods.

[0053] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0054] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive sensitive information extraction method, characterized in that: include: Obtaining rich text and classifying the rich text; The rich text includes code files and non-code files; Performing data processing on the non-code file to convert the non-code file into a plain text file; Selecting an information identification method according to the type of sensitive information to be identified, and extracting sensitive information from the code file and the plain text file to obtain target sensitive information; The information identification method includes a large model code understanding method, a rule matching method and a multi-granularity associated sensitive information identification method.

2. The adaptive sensitive information extraction method according to claim 1, characterized in that: Performing data processing on the non-code file to convert the non-code file into a plain text file includes: The text in the picture file in the non-code file is recognized by an optical character recognition method, and the text information in the non-code file is recognized by an open source library, so as to convert the non-code file into a plain text file.

3. The adaptive sensitive information extraction method according to claim 1, characterized in that: According to the type of sensitive information to be identified, an information identification method is selected, and sensitive information is extracted from the code file and the plain text file, including: Extracting sensitive information from the code file using a large model code understanding method; For the rule-sensitive information in the plain text file, a rule matching method is used to extract the sensitive information; For the non-regular sensitive information in the plain text file, a multi-granularity associated sensitive information identification method is used to extract the sensitive information.

4. The adaptive sensitive information extraction method according to claim 3, characterized in that: Extracting sensitive information from the code file using a large model code understanding method includes: Call the pre-trained code to understand the large language model; The code file is used as input of the code understanding large language model, and target sensitive information in the code file is output.

5. The adaptive sensitive information extraction method according to claim 4, characterized in that: The code understanding large language model is trained, including: Obtaining code training data, and manually marking sensitive information segments in the code training data; The sensitive information segment is used as a label and the code understanding large language model is trained and fine-tuned through the code training data.

6. The adaptive sensitive information extraction method according to claim 3, characterized in that: A multi-granularity associated sensitive information identification method is used to extract sensitive information, including: Performing importance assessment and screening on the plain text files to obtain screened texts; Encoding the screened text through a language representation model and generating word representations through a bidirectional long short-term memory network; Generate a two-dimensional vocabulary representation for the plain text file based on the word representation; the two-dimensional vocabulary representation includes a plurality of word pairs; Capturing the interactions between word pairs of different distances in the two-dimensional vocabulary representation through two-dimensional dilated convolution to obtain a word pair grid representation; Determine the relationship distribution of each word pair in the word pair grid representation by a classifier to obtain a relationship distribution result; The relationship between word pairs is predicted based on the relationship distribution result, sensitive information clusters are determined, and sensitive information is extracted based on the sensitive information clusters.

7. The adaptive sensitive information extraction method according to claim 6, characterized in that: The plain text files are evaluated and screened according to their importance to obtain screened texts, including: The trained extractive summarization model is called, the plain text file is taken as input, and the filtered text is output; the filtered text is the sentences and the preceding and following sentences containing sensitive information in the plain text file.

8. The adaptive sensitive information extraction method according to claim 7, characterized in that: Training the extractive summarization model includes: Obtain open source sensitive information data and generate unstructured data through large language models to build training data sets; With the goal of minimizing the model loss of the extractive summarization model, the extractive summarization model is fine-tuned through a training dataset.

9. The adaptive sensitive information extraction method according to claim 8, characterized in that: Data augmentation through large language models, including: Given a task description, generate candidate prompts for the task description through a large language model; Evaluating and optimizing the candidate prompts to obtain optional prompts; The prompt learning template is input into the large language model, and unstructured data is generated according to the optional prompts.

10. An adaptive sensitive information extraction device, characterized in that: include: An acquisition module, used for acquiring rich text and classifying the rich text; The rich text includes code files and non-code files; A processing module, used for performing data processing on the non-code file and converting the non-code file into a plain text file; An identification module, used to select an information identification method according to the type of sensitive information to be identified, and extract sensitive information from the code file and the plain text file to obtain target sensitive information; The information identification method includes a large model code understanding method, a rule matching method and a multi-granularity associated sensitive information identification method.

Images