Sensitive information identification method and system based on ERNIE model, storage medium and equipment

Abstract

The embodiment of the invention discloses a sensitive information identification method based on an ERNIE model, and the method comprises the steps: obtaining a power grid sensitive text data set, carrying out the word segmentation of the power grid sensitive text data set, and determining a word list; processing the power grid sensitive text data set by using an ERNIE model to obtain a dynamic vector corresponding to each sentence and a label matched with the dynamic vector; obtaining context semantic information of each sentence, and determining a corresponding weight according to the importance of each segmented word in the word list on sensitive classification of the sentences; determining a tag sequence corresponding to the power grid sensitive text data set according to the dependency relationship between tags and the weight of each segmented word; when an ERNIE model objective function corresponding to the tag sequence meets a threshold condition, training of the ERNIE model is completed; and determining an identification result of the sensitive information according to the trained ERNIE model. According to the invention, through the ERNIE model, the accuracy and efficiency of sensitive information identification in professional data in the power grid field can be significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of power system communication security technology, and in particular to a sensitive information identification method, system, storage medium and device based on an ERNIE model. Background Art

[0002] In the field of natural language processing, identifying sensitive information in the power grid field is a crucial task. Its core purpose is to accurately identify whether a document contains sensitive information in the power grid field. This identification process is extremely important for protecting information security in the power grid field, preventing information leakage, and maintaining power safety.

[0003] At present, extensive and in-depth research has been carried out on the task of sensitive information identification, and the accuracy of sensitive information identification has been improved to a certain extent. However, there are still significant gaps and challenges in sensitive information identification in the power sector. Traditional language models often cannot effectively deal with the problem of polysemy when constructing word vectors, which leads to the fact that in the process of text feature extraction, the model may not be able to accurately understand the specific meaning of words in different contexts. In addition, traditional models also show obvious deficiencies in extracting entity relationships in long-distance text information, which further limits the accuracy and reliability of sensitive information identification.

[0004] Especially in special fields such as power grids, due to the complexity and professionalism of the text, traditional sensitive information recognition methods often find it difficult to achieve ideal recognition results. Summary of the invention

[0005] Based on this, it is necessary to propose a sensitive information identification method based on the ERNIE model to address the above problems.

[0006] A sensitive information identification method based on the ERNIE model, the method comprising the following steps:

[0007] Obtain the sensitive text data set of the power grid and perform word segmentation on it to determine the vocabulary;

[0008] The power grid sensitive text dataset is processed using the ERNIE model to obtain a dynamic vector corresponding to each sentence and a label matching the dynamic vector;

[0009] Obtaining contextual semantic information of each sentence, and determining the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence;

[0010] According to the dependency relationship between labels and the weight of each word segmentation, the label sequence corresponding to the power grid sensitive text dataset is determined;

[0011] When the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed;

[0012] The identification result of sensitive information is determined based on the ERNIE model after training.

[0013] In the above scheme, the ERNIE model is used to process the sensitive text dataset in the power grid field to obtain the dynamic vector corresponding to each sentence, which specifically includes:

[0014] Add a preceding context marker [CLS] and a following context marker [SEP] at the beginning and end of each sentence respectively;

[0015] Determine the context information of the preceding tag [CLS] and the following tag [SEP] in each sentence according to the ERNIE model:

[0016]

[0017] Among them, Q, K, and V represent the query matrix, key matrix, and value matrix in the attention mechanism respectively, and K T represents the transpose of the K matrix, Normalization factor, d k is the dimension of the K matrix, and softmax represents the normalization function.

[0018] The context information is integrated through multi-stage knowledge masking to obtain a semantic representation that fuses phrase level and entity level, and is converted into a corresponding vector according to the vocabulary.

[0019] In the above scheme, the ERNIE model is used to process the power grid sensitive text dataset to obtain the dynamic vector corresponding to each sentence and the label matching the dynamic vector, which specifically includes:

[0020] According to the grid-sensitive text dataset, label constraints are performed on the dynamic vector corresponding to each sentence:

[0021] The BiLSTM layer is used to extract the text features of each sentence, and two LSTM networks for processing long dependencies are constructed to obtain forward text information and reverse text information:

[0022]

[0023] H={H1,……,H t ,……,H T}

[0024] Among them, h t represents the hidden layer representation at time t in the BiLSTM layer, H1,H t ,HT Represent the representation of the context in the BiLSTM layer at the first moment, time t, and time T respectively;

[0025] The attention mechanism is used to assign corresponding weights to the hidden vector of each time step output by the BiLSTM layer, and the weighted sum is integrated into the sentence features to determine the label matching the dynamic vector.

[0026] In the above scheme, the BiLSTM layer includes an input unit, a forgetting unit, an updating unit and an output unit:

[0027] Determine the input of the current sequence position corresponding to each sentence through the input unit:

[0028] i t =σ(W i ×[h t-1 ,x t ]+b i )

[0029]

[0030] Among them, i t represents the position representation at time t in the BiLSTM layer, W i represents the parameter matrix at position i, x t represents the input at time t, b i represents the offset of position i, h t-1 represents the hidden layer representation at time t-1 in the BiLSTM layer, represents the context representation of the input sequence at time t in the BiLSTM layer, b C The bias matrix representing the context, C t-1 represents the context representation obtained by the BiLSTM layer at time t-1, C t Represents the context representation obtained by the BiLSTM layer at time t;

[0031] According to the forgetting unit, the forgetting information is processed:

[0032] f t =σ(W f ×[h t-1 ,x t ]+b f )

[0033] The update unit updates the unit state after processing the forgotten information:

[0034]

[0035] The output value of the BiLSTM layer is determined by the output unit:

[0036] o t =σ(W o ×[h t-1 ,x t ]+b o )

[0037] Z t =o t ×tanh(C t )

[0038] Among them, f t represents the output of the forgetting unit of the BiLSTM layer, o t represents the representation of the output layer of the BiLSTM layer, Z t represents the final output of the BiLSTM layer, b o Represents the bias of the output layer of the BiLSTM layer.

[0039] In the above scheme, the attention mechanism is used to assign corresponding weights to the hidden vectors of each time step output by the BiLSTM layer, and the weighted sum is integrated into the sentence features, specifically including:

[0040] Initialize W g , for each H t Multiply by W g , and according to the tanh function, we get u t :

[0041] u t =tanh(W g ·H t +b g )

[0042] The weight value a corresponding to each hidden layer is obtained by presetting the weight t :

[0043]

[0044] The vectors of each hidden layer are weighted and summed to obtain the corresponding sentence feature R:

[0045]

[0046] Among them, α t represents the attention weight at the moment, H t Represents the contextual hidden layer representation at time t.

[0047] In the above scheme, the label sequence corresponding to the power grid sensitive text data set is determined according to the dependency relationship between labels and the weight of each word segmentation, which specifically includes:

[0048] Through the fully connected tanh layer, the output feature dimension of the node is reduced to a low-dimensional space with the same number of entity categories;

[0049] e i =tanh(W e z i )

[0050] The input tensor is label-constrained through the CRF layer, and the label sequence with the highest score is taken as the final result:

[0051]

[0052] In the above scheme, the input tensor is label-constrained through the CRF layer, and the label sequence with the highest score is taken as the final result, which specifically includes:

[0053] Given a known labeled path y, we apply the softmax function to all labeled paths to obtain their conditional probabilities:

[0054]

[0055] Determine the label sequence with the highest probability score:

[0056]

[0057] The objective function of the model is obtained according to the label sequence:

[0058]

[0059] Among them, y represents the true label of the input text, represents the predicted label of the CRF layer, and E represents the context text input sequence of the CRF layer.

[0060] In the above solution, the identification result of sensitive information is determined based on the trained ERNIE model, specifically including:

[0061] The power grid sensitive text dataset is processed using the ERNIE model to obtain a dynamic vector corresponding to each sentence and a label matching the dynamic vector;

[0062] Obtaining contextual semantic information of each sentence, and determining the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence;

[0063] According to the dependency relationship between labels and the weight of each word segmentation, the label sequence corresponding to the power grid sensitive text dataset is determined;

[0064] A tag with the highest probability score in the tag sequence is determined. When the tag is associated with sensitive information, sensitive information exists in the text data in the power grid field.

[0065] The present application also proposes a sensitive information identification system based on the ERNIE model, the system comprising: a data processing unit, a model training unit and an identification unit;

[0066] The data processing unit acquires a sensitive text data set of a power grid and performs word segmentation processing on it to determine a vocabulary;

[0067] The model training unit processes the power grid sensitive text data set using the ERNIE model to obtain a dynamic vector corresponding to each sentence and a label matching the dynamic vector; obtains contextual semantic information of each sentence, and determines the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence; determines the label sequence corresponding to the power grid sensitive text data set according to the dependency relationship between the labels and the weight of each word; when the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed;

[0068] The identification unit is used to determine the identification result of sensitive information according to the trained ERNIE model.

[0069] The present application also proposes a readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the following steps:

[0070] Obtain the sensitive text data set of the power grid and perform word segmentation on it to determine the vocabulary;

[0071] The power grid sensitive text dataset is processed using the ERNIE model to obtain a dynamic vector corresponding to each sentence and a label matching the dynamic vector;

[0072] Obtaining contextual semantic information of each sentence, and determining the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence;

[0073] According to the dependency relationship between labels and the weight of each word segmentation, the label sequence corresponding to the power grid sensitive text dataset is determined;

[0074] When the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed;

[0075] The identification result of sensitive information is determined based on the ERNIE model after training.

[0076] The present application also proposes a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor in the following steps:

[0077] Obtain the sensitive text data set of the power grid and perform word segmentation on it to determine the vocabulary;

[0078] The power grid sensitive text dataset is processed using the ERNIE model to obtain a dynamic vector corresponding to each sentence and a label matching the dynamic vector;

[0079] Obtaining contextual semantic information of each sentence, and determining the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence;

[0080] According to the dependency relationship between labels and the weight of each word segmentation, the label sequence corresponding to the power grid sensitive text dataset is determined;

[0081] When the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed;

[0082] The identification result of sensitive information is determined based on the ERNIE model after training.

[0083] The embodiment of the present invention has the following beneficial effects: firstly, a sensitive text data set of a power grid is obtained and segmented to determine a vocabulary; the sensitive text data set of the power grid is processed by using the ERNIE model to obtain the dynamic vector corresponding to each sentence and the label matching the dynamic vector; the contextual semantic information of each sentence is obtained, and the corresponding weight is determined according to the importance of each segmentation in the vocabulary to the sensitive classification of the sentence; the label sequence corresponding to the sensitive text data set of the power grid is determined according to the dependency relationship between the labels and the weight of each segmentation; when the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed; the recognition result of sensitive information is determined according to the ERNIE model after training. The present invention combines the ERNIE model, which can significantly improve the accuracy and efficiency of sensitive information recognition in professional data in the power grid field. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0085] in:

[0086] Figure 1 The figure is a flowchart of a sensitive information identification method based on the ERNIE model in one embodiment. DETAILED DESCRIPTION

[0087] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0088] In the following description, a large number of specific details are given in order to provide a more thorough understanding of the present invention; however, it is obvious to those skilled in the art that the present invention can be implemented without one or more of these details; in other examples, in order to avoid confusion with the present invention, some technical features known in the art are not described, and it should be understood that the present invention can be implemented in different forms and should not be interpreted as limited to the embodiments presented here; on the contrary, providing these embodiments will make the disclosure thorough and complete and fully convey the scope of the present invention to those skilled in the art.

[0089] The purpose of the terms used herein is only to describe specific embodiments and is not intended to be limiting of the present invention. When used herein, the singular forms "a", "an" and "said / the" are also intended to include the plural forms, unless the context clearly indicates otherwise, and it should also be understood that the terms "consisting of" and / or "comprising", when used in this specification, determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0090] In order to thoroughly understand the present invention, a detailed structure will be presented in the following description to illustrate the technical solution proposed by the present invention; optional embodiments of the present invention are described in detail as follows, but in addition to these detailed descriptions, the present invention may also have other implementation methods.

[0091] To facilitate understanding, the relevant terms involved in this application are first introduced below.

[0092] (1) Embedding is a method of mapping high-dimensional data to low-dimensional space. It is usually used to convert discrete, non-continuous data into continuous vector representations for computer processing. This technology captures the key features of the data, making it more efficient in processing, analysis, and machine learning tasks;

[0093] (2) Word embedding, which is the most common type of embedding in the text field. It converts words (or phrases) in the text into low-dimensional dense vectors to capture the semantic and contextual information of the words.

[0094] (3) ERNIE model. ERNIE not only focuses on the representation of words, but also on the representation of phrases, entities, and sentences. This multi-granular knowledge modeling helps to capture the rich structure in the language and improve the performance of the model in natural language processing tasks. During the pre-training process, ERNIE adopts a continuous semantic integration strategy. By integrating entity recognition and semantic relationship mining into the pre-training task, ERNIE can observe the contextual information of the entire sentence, thereby better understanding the deep meaning and semantic relationship of the sentence.

[0095] like Figure 1 As shown, in one embodiment, a sensitive information identification method based on the ERNIE model is provided. The sensitive information identification method based on the ERNIE model includes steps S101 to S106, which are described in detail as follows:

[0096] S101, obtaining a sensitive text data set of a power grid and performing word segmentation processing on it to determine a vocabulary;

[0097] Preferably, obtaining a sensitive text data set for a power grid includes: searching for data involving sensitive information in the power grid field from Weibo, Baidu Tieba, Zhihu, Baidu Search, etc., collecting and cleaning the collected data, deleting special tags, special characters, etc., manually annotating the cleaned data involving sensitive information in the power grid field, and constructing a sensitive text data set for a power grid. Collecting a sensitive text data set for a power grid ensures that the model can learn sensitive information features related to the power grid.

[0098] Word segmentation for this text dataset is the process of breaking down the text into smaller units (such as words or phrases), which helps the model understand the text content more carefully. The quality of word segmentation directly affects the results of subsequent steps. Therefore, you can manually select a word segmentation tool suitable for Chinese.

[0099] Through the above word segmentation processing, a vocabulary containing all the words that have appeared can be constructed. This vocabulary provides a limited vocabulary range for the model, which helps to reduce the amount of calculation and improve processing efficiency.

[0100] S102, using the ERNIE model to process the power grid sensitive text data set to obtain a dynamic vector corresponding to each sentence and a label matching the dynamic vector;

[0101] Specifically, the above dynamic vector is the dynamic word vector representation of the sentence.

[0102] In some embodiments, the ERNIE model is used to process the sensitive text dataset in the power grid field to obtain the dynamic vector corresponding to each sentence, specifically including:

[0103] Add a preceding context marker [CLS] and a following context marker [SEP] at the beginning and end of each sentence respectively;

[0104] Determine the context information of the previous tag [CLS] and the following tag [SEP] in each sentence according to the ERNIE model:

[0105]

[0106] Among them, Q, K, and V represent the query matrix, key matrix, and value matrix in the attention mechanism respectively, and K T represents the transpose of the K matrix, Normalization factor, d k is the dimension of the K matrix, softmax represents the normalization function, which ensures that the sum of all weighted values ​​is 1, so that the output of the model can be interpreted as probability or importance weight.

[0107] The context information is integrated through multi-stage knowledge masking to obtain a fusion of phrase-level and entity-level semantic representations, which are converted into corresponding vectors according to the vocabulary.

[0108] Specifically, each character in the sentence is converted into a corresponding vector according to the vocabulary. The final Embedding result is formed by adding the position vector, sentence vector and word embedding vector. The ERNIE model integrates phrase- and entity-level knowledge into the language representation through phrase- and entity-level masking to obtain a dynamic word vector representation. The contextual information of each token in the sentence is captured through the self-attention mechanism, and the result is input into the feedforward neural network to generate the corresponding word vector representation.

[0109] Among them, ERNIE is an advanced pre-trained language model that can capture the deep semantic information of text. By using the ERNIE model to process the power grid sensitive text dataset, we can obtain the dynamic vector representation of each sentence, which contains the semantic information of the sentence; at the same time, the ERNIE model can also generate labels that match the dynamic vectors for each sentence. These labels are used to represent the sensitive category of the sentence, such as "sensitive" and "non-sensitive".

[0110] In some embodiments, the ERNIE model is used to process the power grid sensitive text dataset to obtain the dynamic vector corresponding to each sentence and the label matching the dynamic vector, specifically including:

[0111] According to the grid-sensitive text dataset, label constraints are imposed on the dynamic vector corresponding to each sentence:

[0112] The BiLSTM layer is used to extract the text features of each sentence, and two LSTM networks for processing long dependencies are constructed to obtain forward text information and reverse text information:

[0113]

[0114] H={H1,……,H t ,……,H T}

[0115] Among them, h t represents the hidden layer representation at time t in the BiLSTM layer, H1,H t ,H T Represent the representation of the context in the BiLSTM layer at the first moment, time t, and time T respectively;

[0116] The attention mechanism is used to assign corresponding weights to the hidden vector of each time step output by the BiLSTM layer, and its weighted sum is integrated into the sentence features to determine the label that matches the dynamic vector.

[0117] The hidden vector of each time step output is:

[0118] H={H1,……,H t ,……,H T}

[0119] Corresponding weights are assigned to the hidden layer vectors of each time step, and the weighted sum is combined with the extracted text features, forward text information, and reverse text information of the sentence to obtain the sentence features of each sentence.

[0120] In some embodiments, a BiLSTM layer includes an input unit, a forget unit, an update unit, and an output unit:

[0121] Determine the input of the current sequence position corresponding to each sentence through the input unit:

[0122] i t =σ(W i ×[h t-1 ,x t ]+b i )

[0123]

[0124] Among them, i t represents the position representation at time t in the BiLSTM layer, W i represents the parameter matrix at position i, x t represents the input at time t, b irepresents the offset of position i, h t-1 represents the hidden layer representation at time t-1 in the BiLSTM layer, represents the context representation of the input sequence at time t in the BiLSTM layer, b C The bias matrix representing the context, C t-1 represents the context representation obtained by the BiLSTM layer at time t-1, C t Represents the context representation obtained by the BiLSTM layer at time t;

[0125] Process forgotten information according to the forgetting unit:

[0126] f t =σ(W f ×[h t-1 ,x t ]+b f )

[0127] Update the cell state after processing the forgotten information by updating the cell:

[0128]

[0129] The output value of the BiLSTM layer is determined by the output unit:

[0130] o t =σ(W o ×[h t-1 ,x t ]+b o )

[0131] Z t =o t ×tanh(C t )

[0132] Among them, f t represents the output of the forgetting unit of the BiLSTM layer, o t represents the representation of the output layer of the BiLSTM layer, Z t represents the final output of the BiLSTM layer, b o Represents the bias of the output layer of the BiLSTM layer

[0133] In some embodiments, an attention mechanism is used to assign corresponding weights to the hidden vectors of each time step output by the BiLSTM layer, and the weighted sum is integrated into the sentence features, specifically including:

[0134] Initialize W g , for each H t Multiply by W g , and according to the tanh function, we get u t :

[0135] u t =tanh(W g ·H t +b g )

[0136] The weight value a corresponding to each hidden layer is obtained by presetting the weight t :

[0137]

[0138] Perform weighted summation on the vectors of each hidden layer to obtain the corresponding sentence feature R:

[0139]

[0140] Among them, α t represents the attention weight at the moment, H t Represents the contextual hidden layer representation at time t.

[0141] S103, obtaining contextual semantic information of each sentence, and determining the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence;

[0142] Understanding the contextual semantic information of each sentence is crucial to accurately identifying sensitive information. This helps the model distinguish the different meanings of the same words in different contexts. The corresponding weights are determined based on the importance of each word in the vocabulary to the sensitive classification of the sentence. This helps the model pay more attention to key information when identifying sensitive information, thereby improving recognition accuracy.

[0143] Specifically, the weight is determined based on whether a word contains the project keyword, and the contextual semantic information is obtained by the BiLSTM layer.

[0144] S104, determining a label sequence corresponding to the power grid sensitive text data set according to the dependency relationship between the labels and the weight of each word segmentation;

[0145] Specifically, dependency means that when labeling data, each item will have multiple sensitive information, and if the keywords of the included items are the same, they will be given the same label.

[0146] In some embodiments, according to the dependency relationship between the labels and the weights of each word segmentation, determining the label sequence corresponding to the power grid sensitive text dataset specifically includes:

[0147] Through the fully connected tanh layer, the output feature dimension of the node is reduced to a low-dimensional space with the same number of entity categories;

[0148] e i =tanh(W e z i )

[0149] The input tensor is label-constrained through the CRF layer, and the label sequence with the highest score is taken as the final result:

[0150]

[0151] In some embodiments, the input tensor is subjected to label constraints through the CRF layer, and the label sequence with the highest score is taken as the final result, specifically including:

[0152] Given a known labeled path y, we apply the softmax function to all labeled paths to obtain their conditional probabilities:

[0153]

[0154] Determine the label sequence with the highest probability score:

[0155]

[0156] The objective function of the model is obtained based on the label sequence:

[0157]

[0158] Among them, y represents the true label of the input text, represents the predicted label of the CRF layer, and E represents the context text input sequence of the CRF layer.

[0159] S105. When the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed;

[0160] The ERNIE model can make good use of the vocabulary, grammatical structure and semantic information in the training data, and integrate phrase and entity-level knowledge into the semantic representation through a multi-stage knowledge masking strategy, thereby improving the quality of semantic representation.

[0161] Among them, the threshold condition is when the loss of the ERNIE model reaches a value close to 0.

[0162] Furthermore, the ERNIE model can be fine-tuned through manually annotated sensitive text datasets in the power grid field, so that the ERNIE model can represent entity words in the power grid field more accurately and the sensitive information recognition rate of the entire model can be higher.

[0163] S106. Determine the identification result of sensitive information according to the trained ERNIE model.

[0164] In some embodiments, the identification result of sensitive information is determined according to the trained ERNIE model, specifically including:

[0165] The ERNIE model is used to process the power grid sensitive text dataset to obtain the dynamic vector corresponding to each sentence and the label matching the dynamic vector;

[0166] Obtain the contextual semantic information of each sentence, and determine the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence;

[0167] According to the dependency relationship between labels and the weight of each word segmentation, the label sequence corresponding to the power grid sensitive text dataset is determined;

[0168] The tag with the highest probability score in the tag sequence is determined. When the tag is associated with sensitive information, sensitive information exists in the text data in the power grid field.

[0169] The above-mentioned ERNIE model after training is used to identify sensitive information in new power grid text. The model will generate a corresponding label sequence based on the input text to determine whether the text contains sensitive information. It can be seen that the sensitive information identification method based on the ERNIE model proposed in this application can be applied to security monitoring, data protection and other scenarios in the power grid field, helping enterprises and institutions to promptly discover and deal with the risk of sensitive information leakage.

[0170] This application also proposes a sensitive information identification system based on the ERNIE model, the system comprising: a data processing unit, a model training unit and an identification unit;

[0171] The data processing unit obtains the sensitive text data set of the power grid and performs word segmentation processing on it to determine the vocabulary;

[0172] The model training unit uses the ERNIE model to process the sensitive text data set of the power grid, obtains the dynamic vector corresponding to each sentence and the label matching the dynamic vector; obtains the contextual semantic information of each sentence, and determines the corresponding weight according to the importance of each word in the vocabulary to the sensitive classification of the sentence; determines the label sequence corresponding to the sensitive text data set of the power grid according to the dependency relationship between the labels and the weight of each word; when the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed;

[0173] The recognition unit is used to determine the recognition result of sensitive information according to the trained ERNIE model.

[0174] The present application also proposes a readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the following steps:

[0175] Obtain sensitive text data sets of power grids and perform word segmentation on them to determine the vocabulary;

[0176] The ERNIE model is used to process the power grid sensitive text dataset to obtain the dynamic vector corresponding to each sentence and the label matching the dynamic vector;

[0177] Obtain the contextual semantic information of each sentence, and determine the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence;

[0178] According to the dependency relationship between labels and the weight of each word segmentation, the label sequence corresponding to the power grid sensitive text dataset is determined;

[0179] When the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed;

[0180] The identification result of sensitive information is determined based on the ERNIE model after training.

[0181] The present application also proposes a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor in the following steps:

[0182] Obtain the sensitive text data set of the power grid and perform word segmentation on it to determine the vocabulary;

[0183] The ERNIE model is used to process the power grid sensitive text dataset to obtain the dynamic vector corresponding to each sentence and the label matching the dynamic vector;

[0184] Obtain the contextual semantic information of each sentence, and determine the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence;

[0185] According to the dependency relationship between labels and the weight of each word segmentation, the label sequence corresponding to the power grid sensitive text dataset is determined;

[0186] When the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed;

[0187] The identification result of sensitive information is determined based on the ERNIE model after training.

[0188] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0189] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0190] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present application. It should be pointed out that, for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. The above disclosure is only a preferred embodiment of the present invention, and of course it cannot be used to limit the scope of the rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present invention.

Claims

1. A sensitive information identification method based on the ERNIE model, characterized in that: The method comprises: Obtain sensitive text data sets of power grids and perform word segmentation on them to determine the vocabulary; The power grid sensitive text dataset is processed using the ERNIE model to obtain a dynamic vector corresponding to each sentence and a label matching the dynamic vector; Obtaining contextual semantic information of each sentence, and determining the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence; According to the dependency relationship between labels and the weight of each word segmentation, the label sequence corresponding to the power grid sensitive text dataset is determined; When the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed; The identification result of sensitive information is determined based on the ERNIE model after training.

2. The sensitive information identification method based on the ERNIE model according to claim 1 is characterized in that: The ERNIE model is used to process the sensitive text dataset in the power grid field to obtain the dynamic vector corresponding to each sentence, specifically including: Add a preceding context marker [CLS] and a following context marker [SEP] at the beginning and end of each sentence respectively; Determine the context information of the preceding tag [CLS] and the following tag [SEP] in each sentence according to the ERNIE model: Among them, Q, K, and V represent the query matrix, key matrix, and value matrix in the attention mechanism respectively, and K T represents the transpose of the K matrix, represents the normalization factor, d k represents the dimension of K matrix, and softmax represents the normalization function; The context information is integrated through multi-stage knowledge masking to obtain a semantic representation that fuses phrase level and entity level, and is converted into a corresponding vector according to the vocabulary.

3. The sensitive information identification method based on the ERNIE model according to claim 1 is characterized in that: The method of processing the power grid sensitive text dataset using the ERNIE model to obtain a dynamic vector corresponding to each sentence and a label matching the dynamic vector specifically includes: According to the grid-sensitive text dataset, label constraints are performed on the dynamic vector corresponding to each sentence: The BiLSTM layer is used to extract the text features of each sentence, and two LSTM networks for processing long dependencies are constructed to obtain forward text information and reverse text information: H={H1,……,H t ,……,H T } Among them, h t represents the hidden layer representation at time t in the BiLSTM layer, H1,H t ,H T Represent the representation of the context in the BiLSTM layer at the first moment, time t, and time T respectively; The attention mechanism is used to assign corresponding weights to the hidden vector of each time step output by the BiLSTM layer, and the weighted sum is integrated into the sentence features to determine the label matching the dynamic vector.

4. The sensitive information identification method based on the ERNIE model according to claim 3 is characterized in that: The BiLSTM layer includes an input unit, a forgetting unit, an updating unit, and an output unit: Determine the input of the current sequence position corresponding to each sentence through the input unit: i t =σ(W i ×[h t-1 ,x t ]+b i ) Among them, i t represents the position representation at time t in the BiLSTM layer, W i represents the parameter matrix at position i, x t represents the input at time t, b i represents the offset of position i, h t-1 represents the hidden layer representation at time t-1 in the BiLSTM layer, represents the context representation of the input sequence at time t in the BiLSTM layer, b C The bias matrix representing the context, C t-1 represents the context representation obtained by the BiLSTM layer at time t-1, C t Represents the context representation obtained by the BiLSTM layer at time t; According to the forgetting unit, the forgetting information is processed: f t =σ(W f ×[h t-1 ,x t ]+b f ) The update unit updates the unit state after processing the forgotten information: The output value of the BiLSTM layer is determined by the output unit: the t =σ(W o ×[h t-1 ,x t ]+b o ) From t =o t ×tanh(C t ) Among them, f t represents the output of the forgetting unit of the BiLSTM layer, o t represents the representation of the output layer of the BiLSTM layer, Z t represents the final output of the BiLSTM layer, b o Represents the bias of the output layer of the BiLSTM layer.

5. The sensitive information identification method based on the ERNIE model according to claim 3 is characterized in that: The attention mechanism is used to assign corresponding weights to the hidden vectors of each time step output by the BiLSTM layer, and the weighted sum is integrated into the sentence features, specifically including: Initialize W g , for each H t Multiply by W g , and according to the tanh function, we get u t : u t =tanh(W g ·H t +b g ) The weight value a corresponding to each hidden layer is obtained by presetting the weight t : The vectors of each hidden layer are weighted and summed to obtain the corresponding sentence feature R: Among them, α t represents the attention weight at the moment, H t Represents the contextual hidden layer representation at time t.

6. The sensitive information identification method based on the ERNIE model according to claim 1 is characterized in that: Determining the label sequence corresponding to the power grid sensitive text data set according to the dependency relationship between the labels and the weights of each word segmentation specifically includes: Through the fully connected tanh layer, the output feature dimension of the node is reduced to a low-dimensional space with the same number of entity categories; e i =tanh(W e z i ) The input tensor is label-constrained through the CRF layer, and the label sequence with the highest score is taken as the final result:

7. The sensitive information identification method based on the ERNIE model according to claim 6 is characterized in that: The CRF layer is used to constrain the input tensor by labels, and the label sequence with the highest score is taken as the final result, which specifically includes: Given a known labeled path y, we apply the softmax function to all labeled paths to obtain their conditional probabilities: Determine the label sequence with the highest probability score: The objective function of the model is obtained according to the label sequence: Among them, y represents the true label of the input text, represents the predicted label of the CRF layer, and E represents the context text input sequence of the CRF layer.

8. The sensitive information identification method based on the ERNIE model according to claim 1 is characterized in that: The step of determining the identification result of sensitive information based on the trained ERNIE model specifically includes: The power grid sensitive text dataset is processed using the ERNIE model to obtain a dynamic vector corresponding to each sentence and a label matching the dynamic vector; Obtaining contextual semantic information of each sentence, and determining the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence; According to the dependency relationship between labels and the weight of each word segmentation, the label sequence corresponding to the power grid sensitive text dataset is determined; A tag with the highest probability score in the tag sequence is determined. When the tag is associated with sensitive information, sensitive information exists in the text data in the power grid field.

9. A sensitive information identification system based on the ERNIE model, characterized in that: The system comprises: a data processing unit, a model training unit and a recognition unit; The data processing unit acquires a sensitive text data set of a power grid and performs word segmentation processing on it to determine a vocabulary; The model training unit processes the power grid sensitive text data set using the ERNIE model to obtain a dynamic vector corresponding to each sentence and a label matching the dynamic vector; obtains contextual semantic information of each sentence, and determines the corresponding weight of each word in the vocabulary according to its importance to the sensitive classification of the sentence; determines the label sequence corresponding to the power grid sensitive text data set according to the dependency relationship between the labels and the weight of each word; when the ERNIE model objective function corresponding to the label sequence meets the threshold condition, the ERNIE model training is completed; The identification unit is used to determine the identification result of sensitive information according to the trained ERNIE model.

10. A readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 8.

Images