Sensitive data identification method, system, device and medium based on CRF

Abstract

This invention discloses a sensitive data identification method, system, device, and medium based on CRF, belonging to the field of data identification technology. The technical problem it addresses is how to quickly and efficiently identify sensitive information in data. The method includes: constructing a named entity model based on a bidirectional neural network model and a CRF model, wherein a matrix is ​​defined in the loss layer of the bidirectional neural network; performing sensitive data identification through a pre-trained named entity model; the pre-training process includes: taking the data text containing sensitive information as input, inputting it into the bidirectional neural network model for feature extraction, constructing a named entity library and a corresponding label knowledge base; using the output of the bidirectional neural network, the label of the input text, and the input of the matrix as parameters, inputting these parameters into the CRF model to calculate the loss; iterating multiple times to train, optimizing the model parameters, expanding the named entity library and label knowledge base, and obtaining the pre-trained named entity model.

Description

Technical Field

[0001] This invention relates to the field of data identification technology, specifically to a sensitive data identification method, system, device, and medium based on CRF. Background Technology

[0002] In the context of big data, data types are diverse, complex, and massive, and processing speeds are relatively fast—characteristics not found in traditional information data. Especially with the development of information technologies such as the internet, big data has been widely applied across various industries, fully leveraging its advantages. However, as the value of data is explored more deeply, sensitive information within the data also faces the risk of leakage and misuse. Therefore, strengthening the identification and protection of sensitive data has become an urgent issue to be addressed in the field of data security.

[0003] Sensitive data identification involves discovering and identifying sensitive information in data, including identity information, location information, contact information, etc. This information involves the privacy of personal information. Therefore, if we can accurately identify the sensitive information contained in massive amounts of big data, it is of great significance for building a data security shield for big data.

[0004] Named Entity Recognition (NER), also known as proper name recognition, refers to the identification of entities with specific meanings in text. NER is a crucial foundational tool in applications such as information extraction, question answering systems, syntactic analysis, and machine translation, playing a vital role in the practical application of natural language processing technology. Generally, the task of NER is to identify entity classes in the text to be processed, specifically numbers, personal names, organization names, place names, and time-related categories. Based on NER algorithms, deep machine learning can be applied to expand its knowledge base with more entity classes. Typically, NER algorithms involve two main processes: entity boundary identification and entity category determination. The entity boundaries are identified based on the entity classes in its knowledge base, and the entity category is determined based on the boundary features.

[0005] Entities refer to instances of a concept. Entity recognition is the process of extracting the desired entity type from a text. NER is a sequence labeling problem, therefore, the data labeling process follows the methods of sequence labeling problems, primarily using BIO and BIOES. Taking BIOES as an example, B - Begin, I - Intermediate, E - End, S - Single, and O - Other (marking irrelevant characters). The main process is as follows... Figure 1 As shown.

[0006] How to quickly and efficiently identify sensitive information in data is a technical problem that needs to be solved. Summary of the Invention

[0007] The technical objective of this invention is to address the above-mentioned shortcomings by providing a CRF-based sensitive data identification method, system, device, and medium to solve the technical problem of how to quickly and efficiently identify sensitive information in data.

[0008] In a first aspect, the CRF-based sensitive data identification method of the present invention includes the following steps:

[0009] A named entity model is constructed based on a bidirectional neural network model and a CRF model. The loss layer of the bidirectional neural network defines a matrix, which is used as a decision table. The shape of the matrix is ​​[batch_size, num_steps, num_tags], where batch_size is the number of data batches, num_steps is the length of the text sentence, and num_tags is the number of tags, i.e., the total number of tags for named entity recognition.

[0010] Sensitive data identification using a pre-trained named entity model;

[0011] The pre-training of the named entity model includes:

[0012] The data text containing sensitive information is used as input to this document. The input text is fed into a bidirectional neural network model for feature extraction, outputting features and feature probabilities. Based on the features, corresponding named entities are generated, and a named entity library and a corresponding label knowledge base are constructed. The label is the tag of the named entity.

[0013] The output of the bidirectional neural network, the label of the input text, and the input of the matrix are used as parameters, and these parameters are input into the CRF model to calculate the loss.

[0014] Based on the pre-training process of the recognition model, multiple iterations of training are performed to optimize the model parameters, expand the named entity library and label knowledge base, and obtain the pre-trained named entity model.

[0015] Preferably, the parameterized representation of the CRF model is as follows:

[0016] ,

[0017] ,

[0018] in, Represents the state transition function. This represents the weights corresponding to the state transition function. Represents the emission function. This represents the weights corresponding to the emission function, when , , as well as When known, it is possible to calculate .

[0019] Preferably, the loss includes univariate loss and binary loss, and the formula for calculating univariate loss is as follows:

[0020] ,

[0021] in, This represents the probability that the i-th word in the input sentence is assigned a certain label;

[0022] The binary loss is the loss value for transforming the entire sequence corresponding to the input sentence from one label to the next, expressed as:

[0023] .

[0024] In a second aspect, the present invention provides a CRF-based sensitive data identification system for identifying sensitive data using the CRF-based sensitive data identification method as described in any one of the first aspects, the system comprising:

[0025] The model building module is used to construct a named entity model based on a bidirectional neural network model and a CRF model. A matrix is ​​defined in the loss layer of the bidirectional neural network, which serves as a decision table. The shape of the matrix is...

[0026] [batch_size, num_steps, num_tags], where batch_size is the number of data batches, num_steps is the length of the text sentences, and num_tags is the number of tags, i.e., the total number of tags for named entity recognition;

[0027] The model pre-training module is used to pre-train the constructed named entity model and expand the named entity library and label knowledge base.

[0028] The identification module is used to identify and discover sensitive data through a pre-trained named entity model.

[0029] The model pre-training module is used to perform pre-training through the following steps:

[0030] The data text containing sensitive information is used as input text. The input text is fed into a bidirectional neural network model for feature extraction, outputting features and feature probabilities. Based on the features, corresponding named entities are generated, and a named entity library and a corresponding label knowledge base are constructed. The label is the tag of the named entity.

[0031] The output of the bidirectional neural network, the label of the input text, and the input of the matrix are used as parameters, and these parameters are input into the CRF model to calculate the loss.

[0032] Based on the pre-training process of the recognition model, multiple iterations of training are performed to optimize the model parameters, expand the named entity library and label knowledge base, and obtain the pre-trained named entity model.

[0033] Preferably, the parameterized representation of the CRF model is as follows:

[0034] ,

[0035] ,

[0036] in, Represents the state transition function. This represents the weights corresponding to the state transition function. Represents the emission function. This represents the weights corresponding to the emission function, when , , as well as When known, it is possible to calculate .

[0037] Preferably, the loss includes univariate loss and binary loss, and the formula for calculating univariate loss is as follows:

[0038] ,

[0039] in, This represents the probability that the i-th word in the input sentence is assigned a certain label;

[0040] The binary loss is the loss value for transforming the entire sequence corresponding to the input sentence from one label to the next, expressed as:

[0041] .

[0042] Thirdly, the apparatus of the present invention includes: at least one memory and at least one processor;

[0043] The at least one memory is used to store a machine-readable program;

[0044] The at least one processor is configured to invoke the machine-readable program to execute any of the methods described in the first aspect.

[0045] Fourthly, the medium of the present invention is a computer-readable medium, characterized in that the computer-readable medium stores computer instructions, which, when executed by a processor, cause the processor to perform any of the methods described in the first aspect.

[0046] The sensitive data identification method, system, device, and medium based on CRF of the present invention have the following advantages: a named entity model is constructed based on a bidirectional neural network model and a CRF model. During the model pre-training process, feature extraction is performed through the bidirectional neural network model. A label knowledge base that is bound to the data entities is generated through feature extraction. Based on the named entity model and the label knowledge base, it is possible to identify defined sensitive data. It is also possible to continuously expand its entity knowledge base and label knowledge base by performing named entity machine training on undefined sensitive data, thereby enabling efficient and accurate identification of sensitive information. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] The present invention will be further described below with reference to the accompanying drawings.

[0049] Figure 1 Here is a flowchart of NER;

[0050] Figure 2 This is a flowchart of the sensitive data identification based on CRF in Example 1. Detailed Implementation

[0051] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0052] This invention provides a CRF-based sensitive data identification method, system, device, and medium to solve the technical problem of how to quickly and efficiently identify sensitive information in data.

[0053] Example 1:

[0054] This invention presents a sensitive data identification method based on CRF. A named entity model is constructed using a bidirectional neural network model and a CRF model. The loss layer of the bidirectional neural network defines a matrix that serves as a decision table. The matrix has the shape [batch_size, num_steps, num_tags], where batch_size is the number of data batches, num_steps is the length of the text sentence, and num_tags is the number of tags, i.e., the total number of tags for named entity identification. Sensitive data identification is performed using a pre-trained named entity model.

[0055] The pre-training of the named entity model includes:

[0056] (1) Take the data text containing sensitive information as the input text, input the input text into the bidirectional neural network model for feature extraction, output the features and feature probabilities, and generate corresponding named entities based on the features, construct a named entity library and a corresponding label knowledge base, where the label is the tag of the named entity;

[0057] (2) The output of the bidirectional neural network, the label of the input text, and the input of the matrix are used as parameters, and the parameters are input into the CRF model to calculate the loss;

[0058] (3) Based on the pre-training process of the recognition model, perform multiple iterations of training, optimize model parameters, expand the named entity library and label knowledge base, and obtain the pre-trained named entity model.

[0059] In this embodiment, a named entity model is constructed based on a bidirectional neural network model and a CRF model.

[0060] Conditional Random Fields (CRFs) are conditional probability models that solve for one set of variables given another set of variables. Their core principle is to infer the conditional distribution of unknown variables based on observable variables. Their parameterized representation is as follows:

[0061] ,

[0062] ,

[0063] in, Represents the state transition function. This represents the weights corresponding to the state transition function. Represents the emission function. This represents the weights corresponding to the emission function, when , , as well as When known, it is possible to calculate .

[0064] Sensitive data identification based on the named entity algorithm model involves word segmentation and sequence labeling. During model pre-training, data text containing sensitive information (ID card number, phone number, home address) is processed through a neural network (bidirectional LSTM) to extract features, yielding the feature output. During training, this feature and its corresponding label are input into a conditional random field to calculate the loss.

[0065] Named entity recognition is a decision problem, and a decision table is used to describe the consequences using loss. In this embodiment, a matrix is ​​defined in the loss layer of the bidirectional neural network model to act as a decision table. The shape of the matrix is ​​[batch_size, num_steps, num_tags], where batch_size is the number of data batches, num_steps is the length of the text sentence, and num_tags is the number of tags, i.e., the total number of tags for named entity recognition.

[0066] By using the output of the last layer of the bidirectional neural network model, the labels of the input text sentences (i.e., the labels of each character), and the input matrix as parameters, the loss can be calculated. The sentence labels are the named entity labels identified in the named entity model, which are related to feature identifiers.

[0067] The losses obtained above include univariate loss and binary loss, representing the loss due to the transformation between sequences and the loss value of the transformation matrix, respectively. In the named entity algorithm, the univariate loss is the probability that the i-th word in the input sentence will be assigned a certain tag. Binary loss: The loss value of the entire sequence when it is transformed from one label to the next.

[0068] One-yuan loss is represented as:

[0069] ,

[0070] Binary loss is represented as:

[0071] .

[0072] During pre-training, taking "Xiaoming's contact number is 16452861111" as an example, the text is first labeled: "Xiaoming": B-person, "his contact number is": O, "16452861111": E-tel. Based on a large number of such training parameters, the neural network continuously learns to obtain the probability that a certain subscript text is labeled with a certain type of label. By expanding the label knowledge base and continuously expanding the types of sensitive information, and continuously optimizing it during the training process, the discovery and identification of sensitive data can be achieved.

[0073] By training a neural network and expanding the named entity category library, we can obtain the probability that a certain text will be identified as a certain named entity when predicting text information.

[0074] CRF is a probabilistic graphical model that calculates the probability distribution of one random sequence given another random sequence. This embodiment implements a method for identifying and discovering sensitive data information using the named entity algorithm based on this.

[0075] In this embodiment, the sensitive data recognition training process involves inputting the recognition text into the named entity recognition neural network. First, it undergoes layer-by-layer feature extraction, and then progressively calculates the loss value of sequence transformation to obtain the final feature output. Through continuous training, the output features and corresponding labels are fed as input into a conditional random field (CRF) to calculate the loss and obtain a feature probability distribution model. In the training of sensitive data recognition, the sensitive data text and label are continuously trained, so that the named entity of the sensitive data can be identified using the CRF.

[0076] In this embodiment, a named entity algorithm model is implemented based on the CRF algorithm. The named entity algorithm model is continuously trained using machine learning with custom or pre-defined named entities. A label knowledge base, bound to the data entities, is generated through feature extraction. Based on the named entity algorithm model and the label knowledge base, it is possible to identify defined sensitive data, and it is also possible to continuously expand the entity knowledge base and label knowledge base by training named entities on undefined sensitive data.

[0077] Machine learning training is performed on the named entity algorithm model. Through the initial training, pre-set sensitive data named entities can be generated based on common sensitive data. Through continuous learning and feature extraction of the neural network, a corresponding label knowledge base is formed.

[0078] Through neural network learning and feature extraction, each sensitive data named entity is bound to its corresponding label. Then, the named entity algorithm model has the computational ability to quickly identify, scan, and label the bound sensitive data entity types.

[0079] Based on a pre-built sensitive data named entity library and label knowledge base, the named entity algorithm model in this invention has the ability to efficiently, accurately and quickly identify common sensitive data.

[0080] By expanding custom named entities and training them with neural network machine learning, the named entity algorithm model in this invention can expand the scope of sensitive data identification. The named entity library and label knowledge base in the model are continuously expanded. Faced with unknown sensitive data identification scenarios, the model extracts features of unknown sensitive data through neural network learning, generates corresponding named entities, and further expands the label knowledge base through neural network training. This enables the named entity algorithm model to quickly learn, record, and identify unknown sensitive data.

[0081] The sensitive data identification method based on the named entity algorithm model can eliminate the impact of different sensitive data types on identification efficiency. In this method, each type of sensitive data undergoes iterative neural network feature extraction to generate named entities. Therefore, for different types of sensitive data in the text, the identification method assigns a unique label, thus achieving the universality of the sensitive data identification method for sensitive data identification.

[0082] Example 2:

[0083] The present invention relates to a sensitive data identification system based on CRF, comprising a model building module, a model pre-training module, and an identification module.

[0084] The model building module is used to construct named entity models based on bidirectional neural network models and CRF models. A matrix is ​​defined in the loss layer of the bidirectional neural network, which serves as a decision table. The shape of the matrix is...

[0085] [batch_size, num_steps, num_tags], where batch_size is the number of data batches, num_steps is the length of the text sentences, and num_tags is the number of tags, i.e., the total number of tags for named entity recognition.

[0086] The model pre-training module is used to pre-train the constructed named entity model and expand the named entity library and label knowledge base.

[0087] The identification module is used to identify and discover sensitive data using a pre-trained named entity model.

[0088] The model pre-training module is used to perform pre-training through the following steps:

[0089] (1) Take the data text containing sensitive information as the input text, input the input text into the bidirectional neural network model for feature extraction, output the features and feature probabilities, and generate corresponding named entities based on the features, construct a named entity library and a corresponding label knowledge base, where the label is the tag of the named entity;

[0090] (2) The output of the bidirectional neural network, the label of the input text, and the input of the matrix are used as parameters, and the parameters are input into the CRF model to calculate the loss;

[0091] (3) Based on the pre-training process of the recognition model, perform multiple iterations of training, optimize model parameters, expand the named entity library and label knowledge base, and obtain the pre-trained named entity model.

[0092] The parameterized representation of the CRF model is as follows:

[0093] ,

[0094] ,

[0095] in, Represents the state transition function. This represents the weights corresponding to the state transition function. Represents the emission function. This represents the weights corresponding to the emission function, when , , as well as When known, it is possible to calculate .

[0096] Losses include univariate losses and binary losses. The formula for calculating univariate losses is as follows:

[0097] ,

[0098] in, This represents the probability that the i-th word in the input sentence is assigned a certain label;

[0099] The binary loss is the loss value for transforming the entire sequence corresponding to the input sentence from one label to the next, expressed as:

[0100] .

[0101] The system in this embodiment can execute the method disclosed in Embodiment 1, which enables efficient and accurate identification of sensitive information.

[0102] Implementation: 3:

[0103] The apparatus of the present invention includes: at least one memory and at least one processor;

[0104] The at least one memory is used to store a machine-readable program;

[0105] The at least one processor is configured to invoke the machine-readable program to execute the method disclosed in Embodiment 1.

[0106] Implementation: 4:

[0107] The medium of the present invention is a computer-readable medium storing computer instructions that, when executed by a processor, cause the processor to perform the method disclosed in Embodiment 1. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above, and causing the computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0108] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0109] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0110] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0111] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0112] It should be noted that not all steps and modules in the above processes and system structure diagrams are mandatory; some steps or modules can be omitted as needed. The execution order of each step is not fixed and can be adjusted as required. The system structure described in the above embodiments can be a physical structure or a logical structure. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0113] In the above embodiments, the hardware units can be implemented mechanically or electrically. For example, a hardware unit may include permanent dedicated circuitry or logic (such as a dedicated processor, FPGA, or ASIC) to perform the corresponding operation. The hardware unit may also include programmable logic or circuitry (such as a general-purpose processor or other programmable processor), which can be temporarily configured by software to perform the corresponding operation. The specific implementation method (mechanical, dedicated permanent circuitry, or temporarily configured circuitry) can be determined based on cost and time considerations.

[0114] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the means in the different embodiments described above, and these embodiments are also within the protection scope of the present invention.

Claims

1. A sensitive data identification method based on CRF, characterized in that... Includes the following steps: A named entity model is constructed based on a bidirectional neural network model and a CRF model. The loss layer of the bidirectional neural network defines a matrix, which is used as a decision table. The shape of the matrix is ​​[batch_size, num_steps, num_tags], where batch_size is the number of data batches, num_steps is the length of the text sentence, and num_tags is the number of tags, i.e., the total number of tags for named entity recognition. Sensitive data identification using a pre-trained named entity model; The pre-training of the named entity model includes: The data text containing sensitive information is used as input text. The input text is fed into a bidirectional neural network model for feature extraction, outputting features and feature probabilities. Based on the features, corresponding named entities are generated, and a named entity library and a corresponding label knowledge base are constructed. The label is the tag of the named entity. The output of the bidirectional neural network, the label of the input text, and the input of the matrix are used as parameters, and these parameters are input into the CRF model to calculate the loss. Based on the pre-training process of the named entity model, multiple iterations of training are performed to optimize model parameters, expand the named entity library and label knowledge base, and obtain the pre-trained named entity model.

2. The sensitive data identification method based on CRF according to claim 1, characterized in that... The parameterized representation of the CRF model is as follows: ， ， in, Represents the state transition function. This represents the weights corresponding to the state transition function. Represents the emission function. This represents the weights corresponding to the emission function, when , , as well as When known, it is possible to calculate .

3. The sensitive data identification method based on CRF according to claim 1, characterized in that... The losses include univariate losses and binary losses. The formula for calculating univariate losses is as follows: ， in, This represents the probability that the i-th word in the input sentence is assigned a certain label; The binary loss is the loss value for transforming the entire sequence corresponding to the input sentence from one label to the next, expressed as: 。 4. A sensitive data identification system based on CRF, characterized in that... The system is used for sensitive data identification using the CRF-based sensitive data identification method as described in any one of claims 1-3, the system comprising: The model building module is used to construct a named entity model based on a bidirectional neural network model and a CRF model. A matrix is ​​defined in the loss layer of the bidirectional neural network, which serves as a decision table. The shape of the matrix is... [batch_size, num_steps, num_tags], where batch_size is the number of data batches, num_steps is the length of the text sentences, and num_tags is the number of tags, i.e., the total number of tags for named entity recognition; The model pre-training module is used to pre-train the constructed named entity model and expand the named entity library and label knowledge base. The identification module is used to identify and discover sensitive data through a pre-trained named entity model. The model pre-training module is used to perform pre-training through the following steps: The data text containing sensitive information is used as input text. The input text is fed into a bidirectional neural network model for feature extraction, outputting features and feature probabilities. Based on the features, corresponding named entities are generated, and a named entity library and a corresponding label knowledge base are constructed. The label is the tag of the named entity. The output of the bidirectional neural network, the label of the input text, and the input of the matrix are used as parameters, and these parameters are input into the CRF model to calculate the loss. Based on the pre-training process of the named entity model, multiple iterations of training are performed to optimize model parameters, expand the named entity library and label knowledge base, and obtain the pre-trained named entity model.

5. The sensitive data identification system based on CRF according to claim 4, characterized in that... The parameterized representation of the CRF model is as follows: ， ， in, Represents the state transition function. This represents the weights corresponding to the state transition function. Represents the emission function. This represents the weights corresponding to the emission function, when , , as well as When known, it is possible to calculate .

6. The sensitive data identification system based on CRF according to claim 4, characterized in that... The losses include univariate losses and binary losses. The formula for calculating univariate losses is as follows: ， in, This represents the probability that the i-th word in the input sentence is assigned a certain label; The binary loss is the loss value for transforming the entire sequence corresponding to the input sentence from one label to the next, expressed as: 。 7. An apparatus, characterized in that, include: At least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to invoke the machine-readable program to perform the method according to any one of claims 1 to 3.

8. A medium, which is a computer-readable medium, characterized in that, The computer-readable medium stores computer instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1 to 3.

Images