Method for training sensitive entity recognition model, sensitive entity recognition method, computing device, readable storage medium and program product

Abstract

Embodiments of the present application provide a sensitive entity recognition model training method, a sensitive entity recognition method, a computing device, a computer readable storage medium and a computer program product. The sensitive entity recognition model training method comprises: performing word segmentation processing on unstructured text and generating a sample label sequence based on a preset rule annotation; inputting a word segmentation sequence into a bidirectional encoding layer to generate a semantic vector, so as to determine an emission score of each label corresponding to the word segmentation; inputting the emission score into a conditional random field layer to obtain a transition score between adjacent labels, and combining the emission score and the transition score to determine a joint probability of each label corresponding to the word segmentation sequence; outputting a predicted label sequence based on the joint probability; and finally training the bidirectional encoding layer and the conditional random field layer with the objective of minimizing the difference between the sample and the predicted label sequence. The technical solution provided by the embodiments of the present application realizes accurate sensitive entity recognition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method for a sensitive entity recognition model, a sensitive entity recognition method, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the rapid development of information technology and the deepening of digital transformation, government and enterprise units generate massive amounts of unstructured text data in their daily operations. This data often contains a large amount of sensitive entity information, such as names, ID numbers, bank card numbers, phone numbers, and email addresses. Accurate identification and rapid labeling of these sensitive entities are key technical means to ensure data security and prevent privacy leaks.

[0003] Therefore, how to accurately identify sensitive entities has become a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This application provides a training method for a sensitive entity recognition model, a sensitive entity recognition method, a computing device, a computer-readable storage medium, and a computer program product.

[0005] In a first aspect, embodiments of this application provide a training method for a sensitive entity recognition model, the sensitive entity recognition model comprising a bidirectional coding layer and a conditional random field layer; the method includes: Obtain unstructured text data containing multiple sensitive entities as training samples; The unstructured text data is segmented to obtain a segmentation sequence; Based on preset labeling rules, sensitive entities contained in the word segmentation sequence are labeled to generate a sample label sequence. The labeling rules include the correspondence between multiple labels and sensitive entities. The word segmentation sequence is input into a bidirectional coding layer to extract the semantic information of each word in the word segmentation sequence, generate a semantic vector, and determine the emission score of each tag corresponding to each word based on the semantic vector. The emission score is input into a conditional random field layer, and a label transition matrix is ​​constructed using the conditional random field layer. The label transition matrix is ​​used to record the transition scores between the corresponding labels of adjacent words in the word segmentation sequence. Based on the emission score and the transition score, the joint probability of each label corresponding to each word in the word segmentation sequence is determined. Based on the joint probability, the predicted label sequence corresponding to the word segmentation sequence is output. With the goal of minimizing the difference between the sample label sequence and the predicted label sequence, the bidirectional coding layer and the conditional random field layer are trained to obtain a trained sensitive entity recognition model, which is used to identify sensitive entities from the text to be recognized.

[0006] Secondly, this application provides a sensitive entity identification method, including: Obtain the text to be recognized; The text to be identified is input into a sensitive entity recognition model, which identifies at least one sensitive entity from the text. The sensitive entity recognition model includes a bidirectional coding layer and a conditional random field layer. The model is trained as follows: unstructured text data containing multiple sensitive entities is acquired as training samples; the unstructured text data is segmented into words to obtain a segmented sequence; sensitive entities in the segmented sequence are labeled according to preset labeling rules to generate a sample label sequence, wherein the labeling rules include the correspondence between multiple labels and sensitive entities; the segmented sequence is input into the bidirectional coding layer to utilize the bidirectional coding layer to extract... Semantic information of each word in the segmented sequence is extracted to generate a semantic vector, and the emission score of each label corresponding to each word is determined based on the semantic vector. The emission score is input into a conditional random field layer, and a label transition matrix is ​​constructed using the conditional random field layer. The label transition matrix is ​​used to record the transition scores between the labels corresponding to adjacent words in the segmented sequence. Based on the emission score and the transition score, the joint probability of each label corresponding to each word in the segmented sequence is determined. Based on the joint probability, the predicted label sequence corresponding to the segmented sequence is output. With the goal of minimizing the difference between the sample label sequence and the predicted label sequence, the bidirectional coding layer and the conditional random field layer are trained to obtain a trained sensitive entity recognition model.

[0007] Thirdly, this application provides a computing device, including a processing component and a storage component; The storage component stores a computer program; the computer program is invoked and executed by the processing component to implement the training method for the sensitive entity recognition model provided in the first aspect above, or to implement the sensitive entity recognition method provided in the second aspect above.

[0008] Fourthly, this application provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processing component, it implements the training method for the sensitive entity recognition model provided in the first aspect, or the sensitive entity recognition method provided in the second aspect.

[0009] Fifthly, this application provides a computer program product, including a computer program or instructions, which, when executed by a processing component, implements the training method for the sensitive entity recognition model provided in the first aspect, or implements the sensitive entity recognition method provided in the second aspect.

[0010] In this embodiment, firstly, semantic information is extracted and semantic vectors are generated using a bidirectional coding layer. Secondly, a label transition matrix is ​​constructed by introducing a conditional random field layer. The emission score of the word segment itself is combined with the transition scores between adjacent labels to calculate the joint probability. Based on the joint probability, the predicted label sequence corresponding to the word segment sequence is output. The bidirectional coding layer and the conditional random field layer are trained with the goal of minimizing the difference between the sample label sequence and the predicted label sequence to obtain a trained sensitive entity recognition model. This embodiment effectively overcomes the problem that traditional rule-based or word-by-word classification methods struggle to accurately identify complex and diverse sensitive entities in unstructured text by combining bidirectional semantic feature extraction with label sequence dependency modeling. It significantly improves the overall accuracy and stability of sensitive entity recognition, achieving accurate identification of sensitive entities.

[0011] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart of an embodiment of a training method for a sensitive entity recognition model provided in this application; Figure 2 A flowchart of an embodiment of a sensitive entity identification method provided in this application; Figure 3 A schematic diagram of the structure of an embodiment of a training device for a sensitive entity recognition model provided in this application; Figure 4 A schematic diagram of the structure of an embodiment of a sensitive entity identification device provided in this application; Figure 5 A schematic diagram of the structure of the computing device provided in this application is shown. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] It should be noted that, in the cases involving user information in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards.

[0015] With the rapid development of information technology and the deepening of digital transformation, government and enterprise units generate massive amounts of unstructured text data in their daily business processes. This data often contains a large amount of sensitive entity information, such as names, ID numbers, bank card numbers, phone numbers, and email addresses. Accurate identification and rapid labeling of these sensitive entities are key technical means to ensure data security and prevent privacy leaks.

[0016] In related technologies, sensitive entity recognition schemes typically employ matching methods based on regular expressions or predefined rules. However, in practical applications, on the one hand, regular expressions can only recognize strings with specific formats and struggle to handle the phenomenon of "different meanings for the same word." For example, the same sequence of numbers may represent a bank card number or simply a regular number in different contexts, making traditional methods prone to false positives. On the other hand, sensitive features in unstructured text exhibit diverse and random forms, with sensitive entities often fragmented and embedded within long texts. To cover constantly changing expressions, frequent manual updates and maintenance of complex rule bases are required, leading to low recognition efficiency and a high likelihood of missed detections.

[0017] To address the technical problem of accurately identifying sensitive entities, this application provides a solution. The basic idea is to construct a deep learning model integrating a bidirectional coding layer and a conditional random field (CRF) layer. First, the bidirectional coding layer extracts semantic information and generates semantic vectors. Second, a label transition matrix is ​​constructed by introducing a CRF layer. The emission score of the word segment itself is combined with the transition scores between adjacent labels to calculate the joint probability. Based on the joint probability, a predicted label sequence corresponding to the word segment sequence is output. The bidirectional coding layer and the CRF layer are trained with the goal of minimizing the difference between the sample label sequence and the predicted label sequence to obtain a trained sensitive entity recognition model. This application, through a training method combining bidirectional semantic feature extraction and label sequence dependency modeling, effectively overcomes the problem that traditional rule-based or word-by-word classification methods struggle to accurately identify complex and diverse sensitive entities in unstructured text. It significantly improves the overall accuracy and stability of sensitive entity recognition, achieving accurate identification of sensitive entities.

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The implementation details of the technical solutions in the embodiments of this application are described in detail below.

[0020] Figure 1 A flowchart of an embodiment of a training method for a sensitive entity recognition model provided in this application. Figure 1 The training method for the sensitive entity recognition model shown may include the following steps: 101: Obtain unstructured text data containing multiple sensitive entities as training samples.

[0021] Unstructured text can include, but is not limited to, various electronic documents, email content, instant messaging records, and text descriptions extracted from business system logs.

[0022] Sensitive entities refer to entities that need to be identified under specific scenarios or regulations. These may include personal privacy, confidential information, or protected critical information items. For example, sensitive entities may include names, ID numbers, email addresses, phone numbers, and bank card numbers.

[0023] In the embodiments of this application, raw traffic data transmitted via network protocols can be parsed according to actual application needs (e.g., based on different business types in the industry and the storage characteristics of equipment assets). By performing feature extraction, redundancy cleaning, content restoration, and persistent recording on the parsed traffic data, a complete data ledger is formed in the database, thereby obtaining unstructured text data to be further filtered.

[0024] To ensure the effectiveness of training samples and address issues such as repeated optimization of sensitive rules in practical applications, this embodiment of the application sets specific admission rules for the selection of training samples, and uses unstructured text that meets these admission rules as training samples. For example, the admission rules may include the number of sensitive entities contained in the unstructured text being greater than a preset number. For instance, the number of sensitive entities in a single unstructured text data point can be determined first, and if the number of sensitive entities is three or more, the unstructured text data can be used as a training sample.

[0025] Furthermore, to improve the generalization ability of the sensitive entity recognition model and avoid overfitting to a single sensitive entity, training samples can be selected based on the distribution of sensitive features in unstructured text data. For example, if the sensitive features in a single unstructured text dataset are completely identical (e.g., all sensitive entities in the unstructured text dataset are names), the number of samples containing such identical sensitive features can be limited to less than 10. Conversely, if the sensitive features in the unstructured text data exhibit a diverse distribution (e.g., the sensitive entities in the unstructured text data include names, ID numbers, and phone numbers), the number of samples containing such unstructured texts with different sensitive features can be limited to more than 10 to ensure the diversity and coverage of features within the sample space. Finally, the selected unstructured text data can be standardized and preprocessed according to the data security industry classification and grading standards, including null value handling and outlier removal. When the number of standardized texts detected reaches a preset scale (e.g., more than 500), it is formally determined as the training sample set for model iteration.

[0026] 102: Perform word segmentation on unstructured text data to obtain a word segmentation sequence.

[0027] 103: Based on preset labeling rules, sensitive entities contained in the word segmentation sequence are labeled to generate a sample label sequence. The labeling rules include the correspondence between multiple labels and sensitive entities.

[0028] In an embodiment of the present application, continuous text strings in unstructured text data can be segmented into the smallest discrete units with semantic orientation (referred to as terms or Tokens) according to specific linguistic rules, model dictionaries, etc., so as to obtain a word segmentation sequence. The analysis sequence may include a set of terms with a sequential order generated by word segmentation processing.

[0029] In order to meet the processing requirements of the deep learning model for sentence-level sequence tasks, when generating the word segmentation sequence, a special identification bit "[CLS]" can be inserted at the starting position of the sequence, and a special identification bit "[SEP]" can be inserted at the end of the sequence or between sentences. These identification bits and the characters, words, or characters split from the original text together form a single unit in the sequence. For example, for the text "Scholar Li Si", the word segmentation sequence after word segmentation processing can be presented as "[CLS]", "学", "者", "李", "四", "[SEP]".

[0030] Based on the generated word segmentation sequence, the embodiment of the present application can further label sensitive entities in the word segmentation sequence based on a preset label rule, so as to generate a sample label sequence.

[0031] Among them, the label rule is a set of predefined mapping criteria for defining the physical boundaries and semantic classifications of sensitive information in the text. In an embodiment of the present application, in order to accurately capture the entity range, the IOB annotation method is preferably adopted. This annotation method is an annotation rule for identifying entity boundaries by combining a position prefix and an entity category. Among them, the prefix "B (Begin)" is used to annotate the content at the starting position of the sensitive entity, the prefix "I (Inside)" is used to annotate the content at the middle and ending positions of the sensitive entity, and "O (Outside)" is used to annotate non-sensitive content that does not belong to any sensitive type.

[0032] In a possible implementation manner, the label rule can define multiple groups of mapping labels for sensitive entities with different sensitive features. For example, the NAME label is used to map names, the CARD label is used to map ID numbers or bank card numbers, the EMAIL label is used to map emails, and the PHONE label is used to map phone numbers. The sample label sequence refers to a symbol sequence that is generated by mapping the word segmentation sequence one by one through the above rules and is strictly corresponding to the word segmentation sequence in terms of position. For example, if "李" and "四" in the word segmentation sequence are recognized as name entities, according to the annotation rule, the label corresponding to "李" is "B-NAME", the label corresponding to "四" is "I-NAME", and the other non-sensitive word segmentation units in the sequence (including [CLS] and [SEP]) are uniformly labeled as "O". Through this one-to-one mapping method, the text data can be converted into machine-readable annotation data, laying a foundation for the model to capture the boundary information of sensitive features.

[0033] 104: Input the word segmentation sequence into the bidirectional coding layer, use the bidirectional coding layer to extract the semantic information of each word in the word segmentation sequence to generate a semantic vector, and determine the emission score of each tag corresponding to each word based on the semantic vector.

[0034] The bidirectional encoding layer can refer to a neural network architecture capable of extracting deep features from the contextual information of each word in the input sequence from both forward (left-to-right) and backward (right-to-left) dimensions. In the embodiments of this application, the bidirectional encoding layer preferably employs a pre-trained language model (such as BERT) based on a multi-layer transformer architecture, which can capture long-distance semantic dependencies through its internal self-attention mechanism. Semantic information refers to the semantic connotation carried by a word in a specific textual context and its association features with other words. Specifically, this semantic information can include the part-of-speech features, syntactic functions, and role positioning of the word in a specific sensitive data context. For example, in the text "Zhang San's account", the bidirectional encoding layer can identify the close combination between "Zhang" and "San", as well as their semantic association with the word "account".

[0035] In the embodiments of this application, the bidirectional encoding layer can convert each word segment into a numerical representation with a fixed dimension (such as 768 or 1024 dimensions) through multiple nonlinear transformations to generate a semantic vector, thereby achieving a quantitative description of abstract semantics. Subsequently, the generated semantic vector can be processed by a linear mapping layer within the model to determine the emission score of each tag corresponding to each word. The emission score refers to the original probability score or confidence value of classifying the current word as a specific sensitive tag (such as a name tag, phone number tag, or non-sensitive content tag) based solely on its own semantic features, without considering temporal transition constraints between tag sequences.

[0036] For example, for a specific number segment in a word segmentation sequence, after the bidirectional encoding layer extracts its semantic vector, the linear layer calculates that the score for this segmentation corresponding to the tag "B-CARD (starting with a bank card number)" might be 0.85, while the score corresponding to the tag "O (non-sensitive)" might be 0.10. These emission scores constitute the static mapping strength between the segmentation and the tag, providing the core data foundation for subsequently combining the logical transition rules between tags to determine the final globally optimal tag sequence. Through this process, the model achieves the transformation from discrete character information to a deep semantic probability space, significantly improving its ability to represent sensitive entities in complex contexts.

[0037] 105: Input the emission score into the conditional random field layer, and use the conditional random field layer to construct the label transition matrix. The label transition matrix is ​​used to record the transition scores between the corresponding labels of adjacent segments in the word segmentation sequence. Based on the emission score and the transition score, determine the joint probability of each label corresponding to each segment in the word segmentation sequence; output the predicted label sequence corresponding to the word segmentation sequence based on the joint probability.

[0038] The label transition matrix refers to a set of numerical values ​​used to quantify the state transition patterns between adjacent labels in a word segmentation sequence. It records the transition scores between the corresponding labels of adjacent word segments. The transition score represents the logical rationality and probabilistic tendency of transitioning from one label state to another. For example, this transition score reflects the structured constraints in sequence labeling tasks. For instance, according to preset grammatical or logical rules, the transition score of a word labeled "B-NAME (starting with name)" followed by a word labeled "I-NAME (middle or end of name)" will be much higher than that of a word labeled "B-CARD (starting with bank card number)".

[0039] After obtaining the transition score, the joint probability of each tag corresponding to each segment in the segmented sequence can be determined by summing and normalizing the scores across the entire sequence path based on the emission score and transition score. The joint probability is a statistical value used to measure the global confidence of the prediction path, which comprehensively considers the semantic features of the segment itself (represented by the emission score) and the temporal logical features of the tag sequence (represented by the transition score).

[0040] Subsequently, based on the joint probability, the label combination path with the highest confidence can be selected from all possible label combination paths, and the predicted label sequence corresponding to the segmented sequence can be output. The predicted label sequence refers to a set of final annotation symbols that correspond to the positions of the original segmented sequence and can accurately describe the sensitive types to which each component in the unstructured text belongs.

[0041] 106: With the goal of minimizing the difference between the sample label sequence and the predicted label sequence, a bidirectional coding layer and a conditional random field layer are trained to obtain a trained sensitive entity recognition model. The sensitive entity recognition model is used to identify sensitive entities from the text to be recognized.

[0042] In embodiments of this application, a loss function can be used to determine the difference between the sample label sequence and the predicted label sequence. This loss function may include cross-entropy loss or negative log-likelihood loss optimized for sequence labeling tasks, which mathematically reflects the degree to which the predicted label sequence deviates from the sample label sequence.

[0043] In the embodiments of this application, the gradient information of the loss relative to the parameters of the bidirectional coding layer and the transition matrix parameters in the conditional random field layer can be calculated using the backpropagation algorithm. Subsequently, the weight coefficients and bias vectors inside the sensitive entity recognition model can be iteratively updated multiple times using a preset optimization algorithm (such as the Adam optimizer or stochastic gradient descent algorithm), so that the value of the loss function gradually decreases and tends to converge during the training process.

[0044] Through this supervised learning approach, the bidirectional encoding layer can continuously improve its ability to extract deep semantic features, while the conditional random field layer simultaneously optimizes its modeling accuracy of the transition patterns between labels, thereby achieving collaborative optimization of the two functional layers in the feature space and logical constraint space.

[0045] When the difference decreases to a preset threshold range, or when the training iterations reach a preset number of cycles, the trained sensitive entity recognition model can be obtained.

[0046] A trained sensitive entity recognition model refers to a set of computational logic whose internal parameters have reached an optimal distribution and which possesses the ability to perform high-confidence inference on unknown data. This trained sensitive entity recognition model can be deployed in a real-world production environment to identify sensitive entities from text to be recognized. The text to be recognized refers to raw, unstructured data generated in real-world business scenarios that has not yet been system-identified or manually verified. This data can include real-time captured network packet payloads, unstructured documents in enterprise storage servers, and text logs generated from various business interactions. By inputting this text into the trained sensitive entity recognition model, the model can automatically locate and extract sensitive entity information from the text based on learned semantic patterns and label transfer logic.

[0047] In this embodiment, firstly, semantic information is extracted and semantic vectors are generated using a bidirectional coding layer. Secondly, a label transition matrix is ​​constructed by introducing a conditional random field layer. The emission score of the word segment itself is combined with the transition scores between adjacent labels to calculate the joint probability. Based on the joint probability, the predicted label sequence corresponding to the word segment sequence is output. The bidirectional coding layer and the conditional random field layer are trained with the goal of minimizing the difference between the sample label sequence and the predicted label sequence to obtain a trained sensitive entity recognition model. This embodiment effectively overcomes the problem that traditional rule-based or word-by-word classification methods struggle to accurately identify complex and diverse sensitive entities in unstructured text by combining bidirectional semantic feature extraction with label sequence dependency modeling. It significantly improves the overall accuracy and stability of sensitive entity recognition, achieving accurate identification of sensitive entities.

[0048] In this embodiment, the semantic information of each word in the word segmentation sequence is extracted using a bidirectional coding layer to generate a semantic vector. The specific implementation steps are as follows: Using the training parameters corresponding to the current training round of the bidirectional coding layer, calculate the query matrix, key matrix, and value matrix for each word segmentation. For any word segment, the query matrix of the word segment and the key matrix after the transpose of each word segment are multiplied together to obtain the association score matrix. The association score matrix is ​​then scaled. The association score matrix is ​​used to record the association scores of any word segment with other words. The scaled association score matrix is ​​normalized to obtain the attention weight matrix; The value matrix is ​​weighted and summed based on the attention weight matrix to output a semantic vector containing contextual information.

[0049] In the embodiments of this application, during the process of extracting the semantic information of each word in the word segmentation sequence and generating semantic vectors in the bidirectional coding layer, the self-attention mechanism can be used to achieve in-depth mining of the internal correlation features of the text.

[0050] In the embodiments of this application, the query matrix, key matrix, and value matrix of each word segment can be calculated using the training parameters corresponding to the current training round of the bidirectional coding layer. The training parameters can be a set of numerical weights continuously updated through backpropagation during the deep learning model training process, used to perform linear or non-linear mapping of input features. The query matrix (Q) represents the numerical matrix of the actively addressed features of the current word segment, used to characterize "what kind of context the word needs to find" when calculating associations; the key matrix (K) represents the numerical matrix of the addressed features of the word segment, used to characterize "what kind of matching information the word can provide to other words"; the value matrix (V) refers to the numerical matrix carrying the original semantic information of the word segment, used to participate in the final feature aggregation after determining the association strength.

[0051] In embodiments of this application, an inner product operation can be performed on the query matrix and the transposed key matrix to obtain an association score matrix, which is then scaled. In one possible implementation, the inner product operation is performed globally across the segmentation sequence; that is, the query matrix of a segment is inner-producted with the transposed key matrices of all segments (including itself) in the segmentation sequence to extract semantic vectors containing contextual information. The bidirectional encoding layer is used to identify the degree of association between any two segments in the sequence. For the first segment in the sequence... Each word segment is used to generate a query vector. , sequentially with the first to the second in the sequence The key vectors generated by word segmentation The dot product (inner product) calculation is performed on the transpose. When calculating the semantic representation of the th word segment, not only the inner product of this word segment itself (i.e., ) is calculated, but also the inner products of it with other word segments in the word segment sequence (such as , where ). This calculation method is used to quantify the "attention" degree of each word segment in the sequence to all other word segments. Through the inner product operation at the matrix level, the finally generated correlation score matrix is a square matrix. In this correlation score matrix, the value in the th row and the th column represents the semantic correlation score between the th word segment and the th word segment in the sequence. This mechanism ensures that the model can capture long-distance dependencies in unstructured text, enabling the model to, when dealing with the sensitive entity "bank card number", allow the word segment "number" to extract key semantic features from the digital word segments at a relatively far position through the inner product operation, thereby generating accurate semantic vectors.

[0052] Through scaling processing, a specific adjustment coefficient is introduced to adjust the numerical magnitude, avoiding entering the saturation region of the activation function due to excessive values after the inner product operation, thereby ensuring the stability of gradient propagation.

[0053] After obtaining the scaled correlation score matrix, it is normalized to obtain the attention weight matrix. The above normalization process uses a specific algorithm (such as the Softmax function) to map the original correlation scores to the interval (0, 1), making the sum of each row of scores equal to 1, thus transforming it into a probability distribution form to obtain the attention weight matrix. Each element of the attention weight matrix is a probability score (ranging from 0 to 1, and the sum of elements in each row is 1), and this probability score clearly specifies the information weight to be absorbed from the word segments at the corresponding positions in the sequence when calculating the semantic vector of the current word segment, thereby achieving differential weighted aggregation of different context information.

[0054] Finally, the embodiment of the present application performs weighted summation on the value matrix based on the attention weight matrix and outputs a semantic vector containing context information. Through the weighted summation operation, the output semantic vector is no longer an isolated word segment representation, but a dynamic vector that deeply integrates context features. For the word segment "Li", the finally generated semantic vector will contain the feature information of the immediately following word segment "Si", enabling the model to accurately capture the boundary and connotation of the sensitive entity from a global perspective.

[0055] In this embodiment, the specific steps for scaling the correlation score matrix are as follows: Get the dimension values ​​of the key matrix; Calculate the square root of the dimension value as the scaling factor; Divide the elements in the correlation score matrix by the scaling factor to scale the correlation score matrix.

[0056] In the embodiments of this application, the dimension value of the key matrix is ​​obtained. This dimension value refers to the total number of elements used to represent each key vector in the feature space of the bidirectional coding layer, reflecting the depth and complexity of the word segmentation feature description. For example, in a model configuration with a hidden layer size of 768, the dimension value corresponding to each key vector is 768.

[0057] Subsequently, the arithmetic square root of the dimension value can be calculated and used as a scaling factor. This scaling factor can be used to adjust the magnitude of the inner product operation result. In one possible implementation, if the dimension value is... Then the scaling factor can be Based on this, each element in the correlation score matrix can be divided by a scaling factor to complete the scaling process of the correlation score matrix.

[0058] As the dimensionality increases, the result of the inner product operation between the query matrix and the key matrix expands significantly in magnitude. Directly normalizing excessively large association scores (e.g., using Softmax transformation) can lead to a polarized output probability distribution (i.e., exhibiting extremely large weights or weights approaching zero), resulting in extremely small gradients during backpropagation and causing the vanishing gradient problem. Dividing by the square root of the dimensionality effectively controls the range of association score variance, making the normalized attention weight distribution smoother. This ensures that the sensitive entity recognition model can maintain efficient and stable parameter updates when processing long texts or complex semantic features, ultimately improving the model's accuracy in recognizing various sensitive entities.

[0059] In this embodiment, before inputting the segmented word sequence into the bidirectional encoding layer of the sensitive entity recognition model to perform feature extraction, the discrete segmented word symbols can first be converted into vectors that the model can directly compute through multi-dimensional feature mapping and fusion. In some embodiments, before inputting the segmented word sequence into the bidirectional encoding layer of the sensitive entity recognition model, the method may further include: Match the corresponding word vector based on the content of a single word in the word segmentation sequence; Generate positional codes based on the position of individual words in the word segmentation sequence; Generate sentence segment codes based on the sentence to which a single word belongs; Word vectors, positional encodings, and sentence segment encodings are superimposed and transmitted as input data to the bidirectional encoding layer.

[0060] In an embodiment of the present application, first, corresponding word vectors can be matched according to the content of individual word segments in the word segmentation sequence. Among them, a word vector (Word Embedding) can refer to a feature representation generated after mapping discrete word segments to a continuous, high-dimensional, and dense numerical vector space, which carries the static semantic attributes of the word segments themselves. For example, in the word segmentation sequence, the two word segments "Li" and "Si" will be respectively converted into predefined vectors according to their indexes in the preset word list, enabling the computer to perform quantization processing on the abstract character content.

[0061] Considering that the transformer architecture adopted by the bidirectional encoding layer is position-independent and cannot automatically capture the sequential relationship between word segments, embodiments of the present application can further generate position encoding (Position Encoding) according to the position of an individual word segment in the word segmentation sequence, so as to use the position encoding to represent a numerical sequence of the absolute or relative position information of the word segment in the entire word segmentation sequence. By introducing position encoding, the sensitive entity recognition model can recognize the order of word segments in a sentence, for example, can distinguish a sensitive entity at the beginning of the sentence from a sensitive entity at the end of the sentence, thereby perceiving the semantic features of the text. At the same time, in order to avoid the situation where word segments span sentences or paragraphs, segment encoding (Segment Encoding) can also be introduced to use the segment encoding to indicate that the word segment belongs to a specific semantic segment or sentence in the macro context. Specifically, if there is no symbol segmentation in a sentence, it is represented by 0. If there is a comma segmentation in a sentence, the part before the comma is represented by 0, and the part after the comma is represented by 1.

[0062] Finally, the above-generated word vectors, position encoding, and segment encoding can be superimposed and transmitted to the bidirectional encoding layer as input data. In a possible implementation, the above three-dimensional vectors can be superimposed and fused in the way of element-wise addition to generate a unified representation vector integrating various information such as content meaning, spatial position, and segment attribution.

[0063] By superimposing the above-generated word vectors, position encoding, and segment encoding, the originally isolated word segments can be transformed into composite inputs containing rich environmental information, enabling the bidirectional encoding layer to simultaneously perceive sensitive entities from three dimensions of semantics, position, and context structure in subsequent calculation processes, and quickly improving the accuracy of the sensitive entity recognition model in recognizing sensitive entities.

[0064] In some embodiments, the implementation steps of determining the emission scores of each word segment corresponding to each label according to the semantic vector are as follows: Perform a linear transformation on the semantic vector through a linear layer to obtain a first transformation result; The first transformation result is adjusted using the attention weight matrix and bias vector to obtain the second transformation result; The second transformation result is mapped to a probability value using an activation function, and the probability value is used as the emission score of the word segmentation belonging to a specific tag.

[0065] Here, a linear layer can refer to a neural processing unit that uses a weight matrix to perform spatial dimension transformation and feature projection on the input vector. In this embodiment, the linear layer can be used to map a high-dimensional semantic vector to a feature space equal to the total number of preset labels (such as the number of labels including names, ID numbers, and non-sensitive items). The first transformation result can be the original score sequence representing the initial feature intensity of each label, generated after the linear mapping.

[0066] After obtaining the first transformation result, the attention weight matrix and bias vector can be used to adjust the first transformation result to obtain the second transformation result. In one embodiment of this application, the first intermediate transformation result can be obtained by matrix multiplication (or weighted aggregation) of the attention weights and the first transformation result. Then, the first intermediate transformation result and the bias vector can be added to obtain the second transformation result.

[0067] The bias vector is a parameter automatically learned during model training. Its role is to provide compensation or offset for the linear transformation result, thereby enhancing the model's ability to fit different label distributions. By combining the attention weight matrix calculated above, the sensitive entity recognition model can perform fine-grained weighted correction on the first transformation result based on the contribution of word segmentation in the global context, thus obtaining the second transformation result. This second transformation result can be a set of candidate label scores with higher confidence after incorporating weight preferences and system biases.

[0068] After obtaining the second transformation result, an activation function can be used to map the result into probability values, which are then used as emission scores for word segments belonging to specific tags. Activation functions can introduce non-linear features and compress values ​​to a specific range (e.g., between 0 and 1). In this embodiment, the Softmax function is preferred. Through the operation of this activation function, each score in the second transformation result can be transformed into a statistically significant probability distribution, thereby determining the likelihood that the current word segment will be identified as a specific sensitive entity.

[0069] In some embodiments, the specific steps for constructing the label transition matrix using a conditional random field layer are as follows: Initialize the parameters in the transition matrix, where each parameter represents the probability of transitioning from the first label to the second label.

[0070] In some embodiments, the method may further include: During training, the parameters in the transition matrix are adjusted using the backpropagation algorithm to learn the dependencies between adjacent labels in the label sequence; Based on the adjusted parameters, the transfer score is determined for the transfer of the label of the previous segment to the label of the current segment in the segmentation sequence.

[0071] In the embodiments of this application, the parameters in the transition matrix can first be initialized. The label transition matrix is ​​used to store the probability values ​​of a single word segment mapping to all possible labels. The parameters in the transition matrix are initialized, and through model learning, the values ​​of the transition matrix parameters are gradually updated, which are the probability values, representing the probability of a single word segment transitioning from the first label to the second label.

[0072] The size of the transition matrix can be determined by the total number of categories in the preset label set. For example, if it contains 5 sensitive entities such as name and phone number, as well as non-sensitive items, the transition matrix is ​​a square matrix containing the start and end states.

[0073] After the sensitive entity recognition model enters the training phase, the parameters in the transition matrix can be adjusted using the backpropagation algorithm. Through iterative updates of the parameters, the sensitive entity recognition model can automatically learn and solidify the dependencies between adjacent labels in the label sequence. This dependency refers to the inherent constraints and co-occurrence features between labels in the annotation logic of unstructured text. For example, under the IOB annotation system, the sensitive entity recognition model can learn the linguistic logic rule that "B-NAME (starting with name)" is followed by "I-NAME (middle or end of name)" with a very high probability, rather than directly appearing as "I-PHONE (middle or end of phone number)".

[0074] Based on the adjusted parameters, the transfer score of the label of the previous segment in the segmentation sequence to the label of the current segment can be determined.

[0075] The transition score can refer to the dynamic weight score assigned to a specific adjacent label path based on the optimized transition matrix during actual inference or calculation of joint probabilities.

[0076] By calculating transition scores, sensitive entity recognition models can make global judgments based not only on the semantic features of individual word segments but also on the overall structural rationality of the sequence when identifying sensitive entities. For example, through the constraints of transition scores, erroneous recognition results that are semantically similar but have logically inconsistent label sequences can be effectively filtered out, thereby significantly improving the rigor and accuracy of defining the boundaries of sensitive entities in complex texts.

[0077] In some embodiments, the specific steps for determining the joint probability of each tag corresponding to each segment in the segmentation sequence based on the emission score and the transfer score are as follows: Calculate the sum of the emission scores corresponding to all words in the word segmentation sequence, and use it as the total emission score; Calculate the sum of the transition scores corresponding to all adjacent words in the word segmentation sequence, and use it as the total transition score; Add the total launch score to the total transfer score, perform an exponential operation on the sum, and divide the result by a normalization factor to obtain the joint probability.

[0078] In the embodiments of this application, the sum of the emission scores corresponding to all words in the word segmentation sequence can be calculated and used as the total emission score. The total emission score refers to the cumulative value of the original scores of all word segmentation units determined to be corresponding tags based on their own semantic features on a specific candidate tag path. The total emission score can reflect the overall confidence of the bidirectional coding layer in that path that each word belongs to a specific sensitive entity category. In one possible implementation, for a specific candidate tag path corresponding to the word segmentation sequence, the scores of each position word under the specified tag on that path can be extracted sequentially from the emission score matrix, and these scores can be summed one by one to obtain the total emission score.

[0079] Furthermore, embodiments of this application can also calculate the sum of transition scores corresponding to all adjacent word segments in the word segmentation sequence as the total transition score. The total transition score refers to the cumulative value of all state transition feature scores between adjacent tags provided by the conditional random field layer within the same candidate tag path. This score characterizes the logical rationality of the entire tag sequence. For example, a tag sequence that conforms to the naming convention of "name" will have a significantly higher total transition score than a logically disordered sequence. In one possible implementation, the transition scores between adjacent tags can also be sequentially searched from the tag transition matrix maintained by the conditional random field layer according to the tag arrangement order of the candidate tag path, and all adjacent transition scores of the entire sequence can be summed to obtain the total transition score.

[0080] Subsequently, in this embodiment of the application, the total launch score and the total transfer score are added together to obtain the total score of the candidate path, and an exponential operation is performed on the total score. Finally, the result is divided by a normalization factor to obtain the joint probability.

[0081] The joint probability refers to the conditional probability of a specific tag sequence appearing given a word segmentation sequence, reflecting the relative credibility of that path among all possible paths. Through exponential operations, potentially negative scores can be transformed into non-negative real values, thus widening the gap between scores from different paths.

[0082] The normalization factor (also known as the partition function) refers to the sum of the scores of all possible tag paths corresponding to the word segmentation sequence after exponential operation. By dividing the exponential result of the current path by this normalization factor, the final output joint probability distribution can be made to be within the interval (0, 1) and the sum of the probabilities of all paths is 1.

[0083] In some embodiments, the specific steps for outputting the predicted tag sequence corresponding to the word segmentation sequence based on the joint probability are as follows: Use the Viterbi algorithm to perform dynamic programming search on all possible tag paths of the word segmentation sequence; Calculate the joint probability value of each candidate tag path; Compare the joint probability values of all candidate tag paths, and determine the tag path with the maximum joint probability value as the predicted tag sequence.

[0084] Among them, the Viterbi algorithm is a computational method used to find the most likely hidden state sequence that generates the observed sequence under the given observed sequence and model parameters. By performing dynamic programming search based on the Viterbi algorithm, complex problems can be decomposed into multiple decision-making stages with overlapping sub-problems, and the optimal sub-structure characteristics of each stage can be used for recursive solution to find the global optimal search strategy.

[0085] In this dynamic search process, the joint probability value of each candidate tag path can be calculated synchronously. Among them, the candidate tag path is any potential annotation sequence composed of tags in the preset tag set and having the same length as the input word segmentation sequence. For example, for the sequence composed of the words "Zhang", "San", "Electric", and "Phone", the candidate tag paths can include various permutation schemes such as [B-NAME, I-NAME, O, O], [O, O, B-PHONE, I-PHONE], or [O, O, O, O] with all non-sensitive tags combined by the IOB annotation rule. Since the number of combinations of tag sequences grows exponentially with the text length, in this embodiment, the Viterbi algorithm only retains the optimal sub-path scores of the current tag state at each word segmentation position, thus significantly reducing the computational amount.

[0086] After obtaining the joint probability value of each candidate tag path, the joint probability values of all candidate tag paths obtained during the search process can be compared, and the tag path with the maximum joint probability value can be determined as the predicted tag sequence.

[0087] Through this dynamic programming process, the interference of local optimal solutions can be effectively avoided. Even when the text is ambiguous or the input is incomplete, the sensitive entities in the unstructured text can be accurately identified based on the global logical constraints.

[0088] To make the technical solution of this invention clearer and more complete, an embodiment of this application is described in detail below. Obviously, the described embodiment is only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0089] In this embodiment, the training dataset is automatically collected from actual business traffic (such as HTTP, MySQL protocols, etc.) or stored data, containing unstructured text with multiple sensitive entities. Specifically, unstructured text data containing multiple sensitive entities can be obtained as training samples, for example, the following unstructured text example: "Scholar Li Si's ID number is 110...4".

[0090] The unstructured text data is processed using the built-in features of the BERT model. The word segmentation algorithm performs word segmentation processing to obtain the following word segmentation sequence: [CLS], the identity card number of scholar Li Si is 1,1,0,...,4,[SEP] [CLS] is used at the beginning of the sequence, and [SEP] is used at the end of the sequence or as a middle split, both of which are treated as a single token.

[0091] Sensitive entities contained in the segmented sequence are labeled based on preset labeling rules to generate a sample label sequence. This embodiment of the application uses the IOB labeling method, with the following rules: B-Sensitive Type: Marks the beginning of the content of sensitive entities; I-Sensitive Type: Marks the content in the middle and at the end of sensitive entities; O: Mark non-sensitive content.

[0092] Example of a sample tag sequence corresponding to the above word segmentation sequence: O,O,O,B-NAME,I-NAME,O,O,O,O,O,O,B-CARD,I-CARD,I-CARD,...,I-CARD,O In this system, names are mapped using the NAME tag (e.g., "Li Si" corresponds to both B-NAME and I-NAME), ID card numbers are mapped using the CARD tag, email addresses are mapped using the EMAIL tag, phone numbers are mapped using the PHONE tag, and bank card numbers are mapped using the BANK tag. The tags for [CLS] and [SEP] are both 'O'.

[0093] Before inputting the word segmentation sequence into the bidirectional encoding layer, the corresponding word vector (Embedding_token) is matched based on the content of each individual word in the sequence, a positional code (Embedding_position) is generated based on the position of each individual word in the sequence, and a segmental code (Embedding_segment) is generated based on the sentence to which each individual word belongs. These three are then combined as input data. The word segmentation sequence is input into a bidirectional encoding layer, which extracts the contextual semantic information of each word segment to generate a semantic vector. The specific process is as follows: The bidirectional coding layer initializes three independent weight matrices. , , (dimensions are respectively) , , ,in The default value is 768, and the number of attention heads h=12. = = / h = 64.

[0094] Calculate the query matrix Q, key matrix K, and value matrix V: ; ; .

[0095] Calculate the correlation score matrix: Fractional matrix = ; Scaling the fractional matrix: Scaled fraction matrix = ( ) / .

[0096] Softmax normalization is applied to each row of the scaled score matrix to obtain the attention weight matrix: ; in, For any token in the token sequence, From Start moving from left to right to the next token in the token sequence.

[0097] The value matrix V is weighted and summed based on the attention weight matrix. After multiple layers of Transformer encoding, each token outputs a semantic vector containing contextual information. This forms the sequence H, as shown in the formula: ; Where n is the number of tokens, and each h_i incorporates the semantic context of the entire sentence.

[0098] The emission score of each tag corresponding to each word is determined based on the semantic vector: ; Where W is the weight matrix, b is the bias vector, and softmax is the activation function.

[0099] The launch score is input into the Conditional Random Field (CRF) layer, and the label transition matrix A is constructed using the CRF layer. (L is the total number of labels), and the initial parameters are 0. During training, the transition matrix parameters are adjusted through backpropagation to learn the dependencies between adjacent labels (e.g., B-CARD is followed by I-CARD with a high probability).

[0100] Determine the joint probability based on the launch score and transfer score: Where Z is the normalization factor, which ensures that the sum of the probabilities of all sequences is 1; It is an exponential function; For the total score of the launch, To transfer the total score.

[0101] Based on the joint probability output, the label sequence is predicted, and dynamic programming search is performed using the Viterbi algorithm: ; Obtain the predicted label sequence with the highest joint probability .

[0102] With the goal of minimizing the difference between the sample label sequence and the predicted label sequence, an end-to-end joint optimization is performed on the bidirectional coding layer and the conditional random field layer.

[0103] After training, the resulting sensitive entity recognition model can accurately and completely identify various deformed sensitive entities (such as long number fragments, homonyms, etc.) in real business scenarios.

[0104] Figure 2 A flowchart of one embodiment of a sensitive entity identification method provided in this application. Figure 2 The sensitive entity identification method shown may include the following steps: 201: Obtain the text to be recognized; 202: The text to be identified is input into the sensitive entity recognition model, so that the sensitive entity recognition model can identify at least one sensitive entity from the text; wherein, the sensitive entity recognition model includes a bidirectional encoding layer and a conditional random field layer; the sensitive entity recognition model is trained in the following way: unstructured text data containing multiple sensitive entities is obtained as training samples; the unstructured text data is segmented into words to obtain a segmented sequence; the sensitive entities contained in the segmented sequence are labeled according to preset labeling rules to generate a sample label sequence, the labeling rules including the correspondence between multiple labels and sensitive entities; the segmented sequence is input into the bidirectional encoding layer to utilize bidirectional... The encoding layer extracts semantic information from each word in the segmented sequence to generate semantic vectors, and determines the emission score of each label corresponding to each word based on the semantic vectors. The emission scores are input into the conditional random field layer, which is used to construct a label transition matrix. The label transition matrix is ​​used to record the transition scores between the labels corresponding to adjacent words in the segmented sequence. Based on the emission scores and transition scores, the joint probability of each label corresponding to each word in the segmented sequence is determined. Based on the joint probability, the predicted label sequence corresponding to the segmented sequence is output. With the goal of minimizing the difference between the sample label sequence and the predicted label sequence, the bidirectional encoding layer and the conditional random field layer are trained to obtain the trained sensitive entity recognition model.

[0105] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0106] To more clearly illustrate the training and application process of the sensitive entity recognition model provided in the embodiments of this application, the following describes a possible implementation process of the embodiments of this application in conjunction with specific application scenarios.

[0107] In the embodiments of this application, unstructured text data containing multiple sensitive entities is first obtained as training samples by monitoring business system logs or network traffic. For example, the original unstructured text obtained is: "User Mr. Zhang has China Construction Bank card number 6227001234567890, phone number 13812345678, and email address zhang321@xx.com". This original unstructured text contains multiple sensitive entities such as bank card number, mobile phone number, and email address. To enable the sensitive entity recognition model to process this data, the WordPiece word segmentation algorithm can be called to perform word segmentation processing, dividing the text into a word sequence (token sequence): "[CLS]", "user", "Zhang", "Xian", "Sheng", "in", "construction", "bank", "card", "number", "62", "27", "001", "23", "45", "67", "89", "0", ", "telephone", "is", "138", "123", "45", "67", "8", ", "electronic", "email", "for", "zhang321", "@", "xx", ".", "com", "[SEP]".

[0108] The word segmentation sequence includes special identifiers “[CLS]”, “[SEP]”, and sub-word units (such as “62”, “27”, “001”, etc.) after long numbers are segmented.

[0109] In addition, these segmented words can be mapped to unique numerical codes (tokenIDs) based on a preset dictionary table (vocab.txt): [101, 4500, 2787, 2476, 1044, 4495, 1762, 2456, 6392, 7213, 6121, 1305, 1384, 8356, 8149, 9263, 8133, 8208, 8369, 8426, 121, 8024, 4510, 6413, 3221, 9209, 8604, 8208, 8369, 129, 8024, 4510, 2094, 6934, 5056, 711, 9998, 8688, 8709, 8148, 137, 8584, 119, 8134, 102].

[0110] It can also generate a mask sequence to indicate the validity of each index position in the sequence, where a value of 1 indicates a valid word segment and 0 indicates an invalid padding position: [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0].

[0111] In the annotation phase, this embodiment uses the IOB annotation method to map labels to the segmented word sequences. The labeling rules can be as follows: “O”: 0, non-sensitive data; “B-CARD”: 1. Maps to the first part of the ID card number; “I-CARD”: 2, maps to the middle and end of the ID card number; “B-PHONE”: 3, maps to the first part of the phone number; “I-PHONE”: 4, which maps to the middle and end of the phone number; “B-EMAIL”: 5, maps to the beginning of an email address; “I-EMAIL”: 6, maps the middle and end content of the email; “B-BANK”: 7, maps to the beginning of the bank card information; “I-BANK”: 8, which maps the middle and end of the bank card information; “B-NAME”: 9, maps to the beginning of the name; “I-NAME”: 10 Maps the middle and end of the name.

[0112] For the above word segmentation sequence, the following tag mapping index values ​​can be generated according to the tag rules: [0: "O", 1: "O", 2: "O", 3: "O", 4: "O", 5: "O", 6: "O", 7: "O", 8: "O", 9: "O", 10: "O", 11: "O", 12: "B-BANK", 13: "I-BANK", 14: "I-BANK", 15: "I-BANK", 16: "I-BANK", 17: "I-BANK", 18: "I-BANK", 19: "I-BANK", 20: "O", 21: "O", 22: "O", 23: "O", 24: "B-PHONE", 25: "I-PHONE", 26: "I-PHONE", 27: "I-PHONE", 28: "I-PHONE", 29: "O", 30: "O", 31: "O", 32: "O", 33: "O", 34: "O", 35: "B-EMAIL", 36: "I-EMAIL", 37: "I-EMAIL", 38: "I-EMAIL", 39: "I-EMAIL" ] The bank card number's initial segment "62" is marked as index 7 (B-BANK), and subsequent segments such as "27" and "001" are marked as index 8 (I-BANK).

[0113] During this process, the following positional encoding values ​​can also be generated to ensure that the segmented sub-words can be accurately mapped back to the tag logic of the original words. Positional encoding values: [None, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, None ] Subsequently, the preprocessed tokenID, mask sequence, and label index are input into a sensitive entity recognition model consisting of a bidirectional coding layer (BERT) and a conditional random field layer (CRF). During training, the BERT layer extracts semantic association features between word segments through a self-attention mechanism and outputs the emission score of each sensitive label corresponding to each word segment; the CRF layer applies logical constraints to the predicted sequence by learning the transition probabilities between labels (e.g., "B-BANK" is highly likely to be followed by "I-BANK"). The sensitive entity recognition model aims to minimize the difference between the predicted label and the sample label, completing parameter iteration and saving.

[0114] In the application (inference) phase of the sensitive entity recognition model, a new set of unstructured text to be recognized is input: "My Alipay account is 15645678901, ID number is 510101198703189012". The trained model uses its internal semantic extraction capabilities and the Viterbi algorithm to perform global path search and outputs the predicted label index value "[[0,0, 0, 0, 0, 0, 0, 0, 0, 3, 4, 4, 4, 4, 4, 0, 0, 0, 0, 0, 1, 2, 2, 2, 2, 2, 2, 2, 0 ] ]". Specifically, the sensitive entity recognition model identified the word segmentation sequence fragment corresponding to index “3, 4, 4, 4, 4” as a mobile phone number, and the word segmentation sequence fragment corresponding to index “1, 2, 2, 2, 2, 2, 2, 2” as an ID card number.

[0115] Finally, by combining the above label mapping relationship, the prediction results can be restored to obtain the sensitive label restoration results: [ 0,0,0,0,0,0,0,0,0,“156”,“456”,“78”,“90”,“1”,0,0,0,0,0,“510”,“101”,“19”,“870”,“31”,“89”,“01”,“2”, 0] ].

[0116] Therefore, it can be accurately determined that "15645678901" in the text is a mobile phone number and "510101198703189012" is an ID card number.

[0117] Figure 3This is a schematic diagram of the structure of an embodiment of a training device for a sensitive entity recognition model provided in this application. Figure 3 The training apparatus for the sensitive entity recognition model shown may include: The first acquisition module 301 is used to acquire unstructured text data containing multiple sensitive entities as training samples; The word segmentation module 302 is used to perform word segmentation processing on unstructured text data to obtain a word segmentation sequence; The annotation module 303 is used to annotate sensitive entities contained in the word segmentation sequence based on preset labeling rules, and generate a sample label sequence. The labeling rules include the correspondence between multiple labels and sensitive entities. The first input module 304 is used to input the word segmentation sequence into the bidirectional coding layer, so as to extract the semantic information of each word in the word segmentation sequence using the bidirectional coding layer, generate a semantic vector, and determine the emission score of each tag corresponding to each word based on the semantic vector. The second output module 305 is used to input the emission score into the conditional random field layer, construct a label transition matrix using the conditional random field layer, and record the transition scores between the corresponding labels of adjacent words in the word segmentation sequence. Based on the emission score and the transition score, the joint probability of each label corresponding to each word in the word segmentation sequence is determined; and the predicted label sequence corresponding to the word segmentation sequence is output based on the joint probability. Training module 306 is used to train the bidirectional coding layer and the conditional random field layer with the goal of minimizing the difference between the sample label sequence and the predicted label sequence, so as to obtain the trained sensitive entity recognition model. The sensitive entity recognition model is used to identify sensitive entities from the text to be recognized.

[0118] Figure 3 The training device for the sensitive entity recognition model can perform Figure 1 The implementation principle and technical effects of the training method for the sensitive entity recognition model in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit of the training device for the sensitive entity recognition model in the above embodiments perform operations have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0119] Figure 4 This is a schematic diagram of one embodiment of a sensitive entity identification device provided in this application. Figure 4 The sensitive entity recognition device shown may include: The second acquisition module 401 is used to acquire the text to be recognized; The recognition module 402 is used to input the text to be recognized into the sensitive entity recognition model, so that the sensitive entity recognition model can identify at least one sensitive entity from the text. The sensitive entity recognition model includes a bidirectional encoding layer and a conditional random field layer. The sensitive entity recognition model is trained as follows: unstructured text data containing multiple sensitive entities is acquired as training samples; the unstructured text data is segmented into words to obtain a segmented sequence; the sensitive entities contained in the segmented sequence are labeled according to preset labeling rules to generate a sample label sequence, the labeling rules including the correspondence between multiple labels and sensitive entities; the segmented sequence is input into the bidirectional encoding layer to facilitate... A bidirectional coding layer is used to extract semantic information from each word in the segmented sequence to generate semantic vectors. The emission score of each label corresponding to each word is determined based on the semantic vectors. The emission score is input into a conditional random field layer to construct a label transition matrix. The label transition matrix is ​​used to record the transition scores between the labels corresponding to adjacent words in the segmented sequence. Based on the emission score and the transition score, the joint probability of each label corresponding to each word in the segmented sequence is determined. The predicted label sequence corresponding to the segmented sequence is output based on the joint probability. The bidirectional coding layer and the conditional random field layer are trained with the goal of minimizing the difference between the sample label sequence and the predicted label sequence to obtain the trained sensitive entity recognition model.

[0120] Figure 4 The aforementioned sensitive entity identification device can perform Figure 2 The implementation principle and technical effects of the sensitive entity recognition method described in the illustrated embodiments will not be repeated here. The specific methods by which each module and unit of the sensitive entity recognition device in the above embodiments performs its operations have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0121] It should be noted that some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear in this document, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should also be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0122] Figure 5 This is a schematic diagram of the structure of one embodiment of a computing device provided in this application. Figure 5 As shown, in practice, the computing device may include a storage component 501 and a processing component 502.

[0123] Storage component 501 is used to store computer programs and can be configured to store various other data to support operation on a computing device. Examples of this data include instructions for any application or method used to operate on the computing device, data structures, contact data, phone book data, messages, pictures, videos, etc.

[0124] Processing component 502, coupled to storage component 501, is used to execute computer programs in storage component 501 for implementing, etc. Figure 1 The training method for the sensitive entity recognition model shown, or its implementation as follows: Figure 2 The method for identifying sensitive entities is shown.

[0125] Furthermore, such as Figure 5 As shown, the computing device may also include other components such as a communication component 503, a display component 504, a power supply component 505, and an audio component 506. Figure 5 The diagram only shows some components and does not mean that the device includes only these components. Figure 5 The components shown. Additionally... Figure 5 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product form of the computing device. The computing device in this embodiment can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT (Internet of Things) device, or a server-side device such as a conventional server, cloud server, or server array. If the computing device in this embodiment is implemented as a terminal device such as a desktop computer, laptop computer, or smartphone, it may include... Figure 5 The components within the dashed box; if the computing device in this embodiment is implemented as a conventional server, cloud server, or server array, etc., then it may not include... Figure 5 The component within the dashed box.

[0126] The processing component described above includes one or more processors to execute computer instructions to complete all or part of the steps in the method described above. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the method described above.

[0127] The aforementioned storage components can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0128] The aforementioned communication component is configured to facilitate wired or wireless communication between the device housing the communication component and other devices. The device housing the communication component can access wireless networks based on communication standards, such as mobile communication networks, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0129] The aforementioned display components may include a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0130] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0131] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0132] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile components, or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, digital video disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium.

[0133] Accordingly, this application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above method embodiments.

[0134] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0135] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0136] Finally, it should be noted that the above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A training method for a sensitive entity recognition model, characterized in that, The sensitive entity recognition model includes a bidirectional coding layer and a conditional random field layer; the method includes: Obtain unstructured text data containing multiple sensitive entities as training samples; The unstructured text data is segmented to obtain a segmentation sequence; Based on preset labeling rules, sensitive entities contained in the word segmentation sequence are labeled to generate a sample label sequence. The labeling rules include the correspondence between multiple labels and sensitive entities. The word segmentation sequence is input into the bidirectional coding layer to extract the semantic information of each word in the word segmentation sequence, generate a semantic vector, and determine the emission score of each tag corresponding to each word based on the semantic vector. The emission score is input into the conditional random field layer, and a label transition matrix is ​​constructed using the conditional random field layer. The label transition matrix is ​​used to record the transition scores between the corresponding labels of adjacent words in the word segmentation sequence. Based on the emission score and the transition score, the joint probability of each label corresponding to each word in the word segmentation sequence is determined. Based on the joint probability, the predicted label sequence corresponding to the word segmentation sequence is output. With the goal of minimizing the difference between the sample label sequence and the predicted label sequence, the bidirectional coding layer and the conditional random field layer are trained to obtain a trained sensitive entity recognition model, which is used to identify sensitive entities from the text to be recognized.

2. The method according to claim 1, characterized in that, The step of extracting semantic information from each word in the word segmentation sequence using the bidirectional coding layer to generate a semantic vector includes: Using the training parameters corresponding to the current training round of the bidirectional coding layer, calculate the query matrix, key matrix, and value matrix for each word segmentation. For any word segment, the query matrix of the word segment and the key matrix after the transpose of each word segment are multiplied together to obtain the association score matrix. The association score matrix is ​​then scaled. The association score matrix is ​​used to record the association scores of any word segment with other words. The scaled association score matrix is ​​normalized to obtain the attention weight matrix; The value matrix is ​​weighted and summed based on the attention weight matrix to output the semantic vector containing contextual information.

3. The method according to claim 2, characterized in that, The scaling process for the correlation score matrix includes: Obtain the dimension values ​​of the key matrix; Calculate the square root of the dimension value as the scaling factor; Divide the elements in the correlation score matrix by the scaling factor to scale the correlation score matrix.

4. The method according to claim 2, characterized in that, Before inputting the word segmentation sequence into the bidirectional coding layer, the method further includes: Match the corresponding word vector based on the content of a single word in the word segmentation sequence; A positional code is generated based on the position of the individual word in the word segmentation sequence; Generate a sentence segment code based on the sentence to which the individual word belongs; The word vectors, position codes, and sentence segment codes are superimposed and transmitted as input data to the bidirectional coding layer.

5. The method according to claim 3, characterized in that, The step of determining the emission score of each tag corresponding to each word based on the semantic vector includes: The semantic vector is linearly transformed by a linear layer to obtain the first transformation result; The first transformation result is adjusted using the attention weight matrix and bias vector to obtain the second transformation result; The second transformation result is mapped to a probability value using an activation function, and the probability value is determined as the emission score of the word segmentation belonging to a specific tag.

6. The method according to claim 1, characterized in that, The construction of the label transition matrix using the conditional random field layer includes: Initialize the parameters in the transition matrix, where the parameters represent the probability values ​​of transitioning from the first label to the second label; The method further includes: During training, the parameters in the transition matrix are adjusted using the backpropagation algorithm to learn the dependencies between adjacent labels in the label sequence; Based on the adjusted parameters, the transfer score of the label of the previous segmented word in the segmentation sequence is determined.

7. The method according to claim 6, characterized in that, The step of determining the joint probability of each tag corresponding to each word in the word segmentation sequence based on the emission score and the transition score includes: The sum of the emission scores corresponding to all words in the word segmentation sequence is calculated as the total emission score. The sum of the transition scores corresponding to all adjacent words in the word segmentation sequence is used as the total transition score. The total launch score is added to the total transfer score, the result is exponentially calculated, and the result is divided by a normalization factor to obtain the joint probability.

8. The method according to claim 7, characterized in that, The step of outputting the predicted label sequence corresponding to the word segmentation sequence based on the joint probability includes: The Viterbi algorithm is used to perform dynamic programming search on all possible tag paths of the segmented sequence to determine the joint probability value of each candidate tag path; The joint probability values ​​of all candidate label paths are compared, and the label path with the highest joint probability value is determined as the predicted label sequence.

9. The method according to claim 1, characterized in that, The process of labeling sensitive entities contained in the word segmentation sequence based on preset labeling rules to generate sample label sequences includes: Using the IOB tagging method, the word segments corresponding to the beginning position of the sensitive entity in the word segmentation sequence are tagged as Class B tags; the word segments corresponding to the middle and end positions of the sensitive entity in the word segmentation sequence are tagged as Class I tags; and the word segments in the word segmentation sequence that do not belong to the sensitive entity are tagged as Class O tags.

10. A method for sensitive entity recognition, characterized in that, include: Obtain the text to be recognized; The text to be identified is input into a sensitive entity recognition model, which identifies at least one sensitive entity from the text. The sensitive entity recognition model includes a bidirectional coding layer and a conditional random field layer. The model is trained by: acquiring unstructured text data containing multiple sensitive entities as training samples; performing word segmentation on the unstructured text data to obtain a word segmentation sequence; and labeling the sensitive entities contained in the word segmentation sequence based on preset labeling rules to generate a sample label sequence. The labeling rules include the correspondence between multiple labels and sensitive entities. The word segmentation sequence is input into a bidirectional coding layer to extract semantic information of each word in the word segmentation sequence, thereby generating a semantic vector. The emission score of each label corresponding to each word is determined based on the semantic vector. The emission score is input into a conditional random field layer to construct a label transition matrix. This label transition matrix records the transition scores between labels corresponding to adjacent words in the word segmentation sequence. Based on the emission score and the transition score, the joint probability of each label corresponding to each word in the word segmentation sequence is determined. A predicted label sequence corresponding to the word segmentation sequence is output based on the joint probability. The bidirectional coding layer and the conditional random field layer are trained with the goal of minimizing the difference between the sample label sequence and the predicted label sequence to obtain a trained sensitive entity recognition model.

11. A computing device, characterized in that, This includes processing components and storage components; The storage component stores a computer program; the computer program is invoked and executed by the processing component to implement the training method of the sensitive entity recognition model as described in any one of claims 1 to 9, or to implement the sensitive entity recognition method as described in claim 10.

12. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processing component, implements the training method for the sensitive entity recognition model as described in any one of claims 1 to 9, or the sensitive entity recognition method as described in claim 10.

13. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processing component, implement the training method for the sensitive entity recognition model as described in any one of claims 1 to 9, or implement the sensitive entity recognition method as described in claim 10.

Images