Sensitive-information identification method and apparatus, and electronic device and storage medium

Abstract

Provided in the present description are a sensitive-information identification method and apparatus, and an electronic device and a storage medium. The method comprises: extracting an entity sequence from streaming data subjected to identification, wherein the entity sequence comprises at least one entity; inputting the entity sequence into a pre-trained classification model, wherein the classification model comprises a machine learning model obtained by means of performing training on the basis of sample entity sequences labeled to indicate whether sensitive information is included therein; and acquiring a first identification result output by the classification model, wherein the first identification result is used for indicating whether the entity sequence includes sensitive information. In the technical solution of the present description, whether sensitive information is involved is no longer merely determined on the basis of a single entity, whether a data stream is a sensitive data stream is instead identified by means of determining whether an entity sequence composed of a plurality of entities in the data stream includes sensitive information, thereby improving the accuracy of identifying sensitive information in the data stream.

Description

Sensitive information identification method, device, electronic device and storage medium Technical Field

[0001] One or more embodiments of the present specification relate to the field of stream data technology, and in particular, to a sensitive information identification method, device, electronic device, and storage medium. Background Art

[0002] Streaming data carries data of various types and structures, including sensitive information. Sensitive information typically includes, but is not limited to, personal identity information, organizational information, and address information. Leakage or unauthorized access to this sensitive information can lead to serious security and privacy issues. To improve network data security, it is necessary to effectively identify sensitive information carried in streaming data.

[0003] In related technologies, whether stream data carries sensitive information is usually determined only from the perspective of a single entity, resulting in inaccurate recognition results.

[0004] Summary of the Invention

[0005] This application provides a sensitive information identification method, the method comprising:

[0006] Extracting an entity sequence from the stream data to be identified, wherein the entity sequence includes at least one entity;

[0007] Inputting the entity sequence into a pre-trained classification model, wherein the classification model includes a machine learning model trained based on sample entity sequences that are labeled as to whether they contain sensitive information;

[0008] A first recognition result output by the classification model is obtained, where the first recognition result is used to indicate whether the entity sequence contains sensitive information.

[0009] Optionally, extracting the entity sequence from the stream data to be identified includes:

[0010] Converting the stream data to be identified into text data;

[0011] Inputting the text data into a bidirectional long short-term memory-conditional random field BiLSTM-CRF model to obtain entities included in the text data output by the bidirectional long short-term memory-conditional random field model;

[0012] The entities included in the text data are converted into the entity sequence.

[0013] Optionally, the method further includes:

[0014] If concept drift occurs in the classification model, marking the entity sequence based on a second recognition result, wherein the second recognition result is used to indicate whether the entity sequence contains sensitive information, and the credibility of the second recognition result is higher than the credibility of the first recognition result;

[0015] The labeled entity sequence is used as a training sample to retrain the classification model.

[0016] Optionally, the method further includes:

[0017] Identify the entity sequence based on a preset rule to obtain the second identification result;

[0018] If the second recognition result is inconsistent with the first recognition result, it is determined that concept drift occurs in the classification model.

[0019] Optionally, if the second recognition result is inconsistent with the first recognition result, determining that concept drift occurs in the classification model includes:

[0020] For multiple identified entity sequences, if a proportion of a first-category entity sequence in the multiple entity sequences reaches a first threshold, it is determined that concept drift occurs in the classification model, wherein a first recognition result of the first-category entity sequence is inconsistent with a second recognition result of the first-category entity sequence.

[0021] Optionally, marking the entity sequence based on the second recognition result includes:

[0022] The first-category entity sequence is marked based on the second recognition result of the first-category entity sequence.

[0023] Optionally, the preset rule includes at least one of the following:

[0024] A first identification rule, wherein the first identification rule is that an entity sequence carrying a preset keyword contains sensitive information;

[0025] A second identification rule, wherein the second identification rule is that the entity sequence belonging to the preset IP address contains sensitive information;

[0026] A third identification rule, wherein the third identification rule is that an entity sequence belonging to a preset pattern contains sensitive information.

[0027] The present application also provides a sensitive information identification device, the device comprising:

[0028] An extraction unit, configured to extract an entity sequence from the stream data to be identified, wherein the entity sequence includes at least one entity;

[0029] An input unit, inputting the entity sequence into a pre-trained classification model, wherein the classification model includes a machine learning model trained based on sample entity sequences marked as containing sensitive information;

[0030] An acquisition unit is used to acquire a first recognition result output by the classification model, where the first recognition result is used to indicate whether the entity sequence contains sensitive information.

[0031] The present application also provides an electronic device, comprising a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus;

[0032] The memory stores machine-readable instructions, and the processor executes the above method by calling the machine-readable instructions.

[0033] The present application also provides a machine-readable storage medium, which stores machine-readable instructions. When the machine-readable instructions are called and executed by a processor, the above method is implemented.

[0034] Through the above method, the present application extracts an entity sequence including at least one entity from the stream data to be identified, then inputs the entity sequence into a pre-trained classification model, and then determines whether the entity sequence contains sensitive information based on the first recognition result output by the classification model. Accordingly, compared to the method of determining whether sensitive information is determined only from the perspective of a single entity, the technical solution of this specification determines whether an entity sequence composed of multiple entities in a data stream is sensitive information, thereby identifying whether the data stream is a sensitive data stream, thereby improving the accuracy of identifying sensitive information in the data stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0036] FIG1 is a flow chart of a sensitive information identification method according to an exemplary embodiment;

[0037] FIG2 is a flow chart of another sensitive information identification method according to an exemplary embodiment;

[0038] FIG3 is a schematic diagram of a sensitive information identification system according to an exemplary embodiment;

[0039] FIG4 is a schematic diagram showing a method of identifying an entity sequence according to a preset rule according to an exemplary embodiment;

[0040] FIG5 is a hardware structure diagram of an electronic device in which a sensitive information identification device is located, showing an exemplary embodiment;

[0041] FIG6 is a block diagram of a sensitive information identification device according to an exemplary embodiment. DETAILED DESCRIPTION

[0042] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative work should fall within the scope of protection of this specification.

[0043] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0044] Streaming data carries data of various types and structures, including sensitive information. Sensitive information typically includes, but is not limited to, personal identity information, organizational information, and address information. Leakage or unauthorized access to this sensitive information can lead to serious security and privacy issues. To improve network data security, it is necessary to effectively identify sensitive information carried in streaming data.

[0045] Related technologies typically determine whether stream data contains sensitive information from the perspective of a single entity, resulting in inaccurate recognition results. For example, stream data containing personal sensitive information, such as "Zhang San zhangsan@mail.com," contains two entities: "Zhang San" and "zhangsan@mail.com." One entity is a name, and the other is an email address. While these two entities alone would not be considered to contain sensitive information, the stream data containing "Zhang San zhangsan@mail.com" does contain sensitive information.

[0046] In view of this, this specification aims to propose a technical solution for identifying whether stream data contains sensitive information.

[0047] During implementation, an entity sequence is first extracted from the stream data to be identified, wherein the entity sequence includes at least one entity; then, the entity sequence is input into a pre-trained classification model, wherein the classification model includes a machine learning model trained based on sample entity sequences that are marked as containing sensitive information; further, a first recognition result output by the classification model is obtained, wherein the first recognition result is used to indicate whether the entity sequence contains sensitive information.

[0048] For example, the stream data to be identified contains the text "Zhang San's ID card is 511002197****46614, and his email address is zhangsan@mail.com." The entities extracted from the stream data to be identified are "Zhang San," "511002197****46614," and "zhangsan@mail.com," resulting in an entity sequence ["Zhang San," "511002197****46614," "zhangsan@mail.com"]. The entity sequence ["Zhang San," "511002197****46614," "zhangsan@mail.com"] is then input into a pre-trained classification model, which is a machine learning model trained on sample entity sequences labeled as containing sensitive information. Finally, the classification model outputs recognition result 1 for the entity sequence ["Zhang San," "511002197****46614," "zhangsan@mail.com"]. This recognition result indicates that the entity sequence contains sensitive information.

[0049] In one embodiment, the output result may be the probability value of each classification result "90% probability of 1, 10% probability of 0", so the classification result is 1, which means that the entity sequence contains sensitive information, and the classification result is 0 which means that the entity sequence does not contain sensitive information.

[0050] As can be seen, in the technical solution of this specification, an entity sequence including at least one entity is extracted from the stream data to be identified, the entity sequence is then input into a pre-trained classification model, and then a determination is made as to whether the entity sequence contains sensitive information based on the first recognition result output by the classification model. Therefore, compared to the method of determining whether sensitive information is present only from the perspective of a single entity, the technical solution of this specification determines whether an entity sequence consisting of multiple entities in a data stream is sensitive information, thereby identifying whether the data stream is a sensitive data stream, thereby improving the accuracy of identifying sensitive information in the data stream.

[0051] The present application is described below through specific embodiments in combination with specific application scenarios.

[0052] Please refer to FIG. 1. FIG. 1 is a schematic flow diagram of a sensitive information recognition method shown in an exemplary embodiment. The method may perform the following steps:

[0053] Step 102: Extract an entity sequence from the streaming data to be recognized, where the entity sequence includes at least one entity.

[0054] For example, extract the entity sequence ["Zhang San", "511002197****46614", "zhangsan@mail.com", "Building 1, Unit 2, Zhongying Garden", "XX Technology Software Development Department"] from the streaming data to be recognized. Here, "Zhang San", "511002197****46614", "zhangsan@mail.com", "Building 1, Unit 2, Zhongying Garden", and "XX Technology Software Development Department" are entities, representing name, ID number, email, address, and organization respectively.

[0055] Among them, streaming data refers to data streams generated in a continuous, high-speed, and uninterrupted manner. Different from traditional batch data, streaming data is generated in real time and is transmitted and processed item by item continuously. Streaming data usually comes from various sources such as sensors, devices, log files, social media, etc. Streaming data has real-time and flexibility, and can be used in scenarios such as quick response, real-time analysis, and instant decision-making. In the field of natural language processing, an entity usually refers to something with real-world existence or conceptual attributes. An entity can be a specific person, place, organization, date, time, or an abstract concept, event, product, etc. Entity recognition can identify key information in the text, such as person names, place names, organization names, date and time, etc., thus helping technicians understand the meaning of the text for more in-depth text analysis and information extraction.

[0056] In this specification, one or more entity sequences can be extracted from the streaming data to be recognized. For example, if it is stipulated that each entity sequence contains at most three entities, then the first entity sequence ["Zhang San", "511002197****46614"] and the second entity sequence ["zhangsan@mail.com", "Building 1, Unit 2, Zhongying Garden", "XX Technology Software Development Department"] can be extracted from the streaming data to be recognized, or other combinations, such as the first entity sequence ["Zhang San", "zhangsan@mail.com"] and the second entity sequence ["511002197****46614", "Building 1, Unit 2, Zhongying Garden", "XX Technology Software Development Department"]. The specific combination form and the maximum number of entities contained in each entity sequence can be customized according to actual needs. This specification does not limit the number and specific form of the entity sequences extracted from the streaming data to be recognized.

[0057] Step 104: Input the entity sequence into a pre-trained classification model, which includes a machine learning model trained based on sample entity sequences marked with whether they contain sensitive information.

[0058] For example, input the entity sequence ["Zhang San", "511002197****46614", "zhangsan@mail.com", "Building 1, Unit 2, Zhongying Garden", "XX Technology Software Development Department"] into the pre-trained classification model. This classification model is a machine learning model trained based on sample entity sequences marked with whether they contain sensitive information (such as the sample entity sequence ["Zhang San", "511002197****46614", "zhangsan@mail.com"], and the label of this sample entity sequence is 1, meaning that this sample entity sequence contains sensitive information).

[0059] Among them, the classification model can specifically be a binary classification model. A binary classification model refers to a machine learning model used to divide the input data into one of two categories. Binary classification models are usually used to solve binary classification problems, such as determining whether an email is spam, predicting stock price increases or decreases, identifying whether a specific object is contained in an image, etc. In this specification, the binary classification model is used to identify whether a data stream contains sensitive information. For the training method of the binary classification model, various machine learning algorithms can be specifically used, including logistic regression, support vector machines, decision trees, etc. These algorithms will learn the features and patterns of the input data during the training process and classify and predict new data based on these features and patterns. In this specification, marking the sample entity sequence for sensitive information can be marking it as 1 to represent that the sample entity sequence contains sensitive information, marking it as 0 to represent that the sample entity sequence does not contain sensitive information, or marking it as "yes" to represent that the sample entity sequence contains sensitive information, marking it as "no" to represent that the sample entity sequence does not contain sensitive information. Regarding the specific form of marking the sample entity sequence, this specification does not limit it.

[0060] In a preferred embodiment, in order to improve the accuracy of classification model construction, the XGBoost (eXtreme Gradient Boosting) algorithm can be used, which is an improved algorithm based on GBDT (Gradient Boosting Decision Tree). Compared with the traditional GBDT, XGBoost has made some improvements and innovations in the algorithm to improve the accuracy and efficiency of the model. Specifically, XGBoost introduces a regularization term to prevent overfitting by controlling the complexity of the model; it uses second-order derivative information to more accurately estimate the residual; it supports a variety of different loss functions to adapt to different application scenarios, etc. In addition, in XGBoost, each decision tree is constructed by gradient boosting, and the generation of each tree is based on the residuals of all previous trees to gradually optimize the predictive ability of the model.

[0061] Step 106: Obtain a first recognition result output by the classification model, where the first recognition result is used to indicate whether the entity sequence contains sensitive information.

[0062] For example, the recognition result of the classification model output for the input entity sequence ["Zhang San","511002197****46614","zhangsan@mail.com","Zhongying Garden 1 Building 2 Unit","XXX Technology Software Development Department"] is 1. This recognition result indicates that the entity sequence ["Zhang San","511002197****46614","zhangsan@mail.com","Zhongying Garden 1 Building 2 Unit","XXX Technology Software Development Department"] contains sensitive information.

[0063] Among them, when the classification model outputs the recognition result, it usually outputs a probability value belonging to a certain category or a binary label. The probability value indicates the possibility that the input data belongs to the category. The higher the probability value, the greater the possibility that the model believes the input data belongs to the category. For example, the model can output a probability value between 0 and 1, such as 0.75, which means that the probability that the input data belongs to the category is 75%. The binary label indicates which category the input data is divided into. The probability value output by the model can usually be compared with a threshold. If the probability value is greater than or equal to the threshold, the label of the positive category (for example, 1) is output, otherwise the label of the negative category (for example, 0) is output. The specific form of the classification model output recognition result depends on the specific problem and application scenario. If a more fine-grained result is required, a probability output can be used; if only a simple classification result is required, a binary label output can be used. This is not limited in this specification.

[0064] In one embodiment shown, the extracting of entity sequences from the stream data to be identified includes: converting the stream data to be identified into text data; inputting the text data into a bidirectional long short-term memory-conditional random field BiLSTM-CRF model to obtain entities included in the text data output by the bidirectional long short-term memory-conditional random field model; and converting the entities included in the text data into the entity sequence.

[0065] For example, a traffic packet for transmitting stream data to be identified is obtained; text data is extracted from the traffic packet, such as "Zhang San's ID card is 511002197****46614, his email address is zhangsan@mail.com, his address is Unit 2, Building 1, Zhongying Garden, and his organizational relationship is in the XX Technology Software Development Department". Then, entity recognition needs to be performed on the text data. The text data can be input into a BiLSTM-CRF (Bidirectional Long Short-Term Memory Conditional Random Field) model to obtain the entities "Zhang San", "511002197****46614", "zhangsan@mail.com", "Unit 2, Building 1, Zhongying Garden", and "XX Technology Software Development Department" included in the text data output by the BiLSTM-CRF model. The entities included in the text data are then converted into an entity sequence ["Zhang San", "511002197****46614", "zhangsan@mail.com", "Unit 2, Building 1, Zhongying Garden", and "XX Technology Software Development Department"].

[0066] Network packet capture tools can be used to capture network traffic packets. For example, to capture HTTP protocol traffic packets, filters must be set to capture only HTTP protocol traffic packets, extract the textual information in the text layer of the traffic packets, and filter data containing non-entity objects in other protocol layers to convert the flow data in network traffic into valid textual information. The BiLSTM-CRF model is a deep learning model for sequence labeling tasks. It combines the BiLSTM (Bidirectional Long Short-Term Memory) and CRF (Conditional Random Field) components. By jointly training the BiLSTM and CRF, it can automatically capture the dependencies between contextual information and word-position tagging annotations, resulting in accurate and coherent labeling results.

[0067] In the process of converting the entities included in the text data into an entity sequence, multiple permutation and combination forms of entity sequences can be generated from the entities according to the maximum number of entities that the entity sequence can contain. For example, if it is stipulated that the maximum number of entities that the entity sequence can contain is 3, then the entities "Zhang San", "511002197****46614", "zhangsan@mail.com", "Building 1, Unit 2, Zhongying Garden", and "Moumou Technology Software Development Department" can be converted into two entity sequences, such as ["Zhang San", "511002197****46614"] and ["zhangsan@mail.com", "Building 1, Unit 2, Zhongying Garden", "Moumou Technology Software Development Department"], or ["Zhang San", "511002197****46614", "zhangsan@mail.com"] and ["Building 1, Unit 2, Zhongying Garden", "Moumou Technology Software Development Department"] and other multiple combination forms. Subsequently, sensitive information recognition is performed on these two entity sequences respectively. If any one of them is recognized as containing sensitive information, it is considered that the streaming data containing the entities "Zhang San", "511002197****46614", "zhangsan@mail.com", "Building 1, Unit 2, Zhongying Garden", and "Moumou Technology Software Development Department" contains sensitive information. This specification does not limit the maximum number of entities that the entity sequence can contain, nor the permutation and combination rules for generating entity sequences from entities.

[0068] In an illustrated embodiment, the method further includes: if the classification model undergoes concept drift, marking the entity sequence based on the second recognition result, where the second recognition result is used to indicate whether the entity sequence contains sensitive information, and the credibility of the second recognition result is higher than that of the first recognition result; using the marked entity sequence as a training sample to retrain the classification model.

[0069] For example, in the case of concept drift in a pre-trained classification model, it is necessary to identify the entity sequences ["Zhang San", "511002197****46614", "zhangsan@mail.com"] and ["Building 1, Unit 2, Zhongying Garden", "XX Technology Software Development Department"] input into the pre-trained classification model with higher credibility. For example, identify them based on expert experience, and the recognition results are all 1, which means that the entity sequences ["Zhang San", "511002197****46614", "zhangsan@mail.com"] and ["Building 1, Unit 2, Zhongying Garden", "XX Technology Software Development Department"] both contain sensitive information. Then, according to the recognition results, mark the entity sequences to obtain the entity sequences ["Zhang San", "511002197****46614", "zhangsan@mail.com"] and ["Building 1, Unit 2, Zhongying Garden", "XX Technology Software Development Department"] with all marks being 1, and then use the marked entity sequences as training samples to retrain the classification model.

[0070] Please refer to FIG. 2. FIG. 2 is a schematic flow diagram of another sensitive information recognition method shown in an exemplary embodiment. As shown in FIG. 2, first receive the streaming data to be recognized, convert the streaming data to be recognized into text, identify entities in the text, convert the entities into entity sequences, and then input the entity sequences into a pre-trained classification model. The classification model can be adaptively adjusted after concept drift, and finally obtain the recognition result of whether the entity sequence contains sensitive information output by the classification model.

[0071] Among them, in predictive analysis and machine learning, concept drift refers to the phenomenon that the statistical characteristics of the target variable change in an unpredictable manner over time. Over time, the classification model built based on machine learning for detecting sensitive information faces problems such as a decline in prediction ability, poor robustness, and many false positives and false negatives after a certain stage, resulting in a decrease in the prediction accuracy of the classification model. Therefore, it is necessary to identify the occurrence of concept drift in the classification model and retrain it based on new samples.

[0072] It is necessary to identify and mark the entity sequences input into the pre-trained classification model with higher credibility to obtain new samples. The methods of identification with higher credibility include, but are not limited to, simple and direct keyword matching of entity sequences, expert manual experience discrimination, etc. This specification does not limit the method of obtaining new entity sequence samples after concept drift in the pre-trained classification model.

[0073] In one of the illustrated embodiments, the method further includes: identifying the entity sequence based on a preset rule to obtain the second identification result; if the second identification result is inconsistent with the first identification result, determining that the classification model has concept drift.

[0074] For example, based on expert manual experience, identify the entity sequences ["Zhang San", "511002197****46614", "zhangsan@mail.com"] and ["Building 1, Unit 2 of Zhongying Garden", "XX Technology Software Development Department"] input to the pre-trained classification model, and the obtained identification results are both 1. However, if the identification results of the pre-trained classification model for the entity sequences ["Zhang San", "511002197****46614", "zhangsan@mail.com"] and ["Building 1, Unit 2 of Zhongying Garden", "XX Technology Software Development Department"] are 1 and 0 respectively, it is determined that the pre-trained classification model has concept drift.

[0075] Among them, the preset rule represents a relatively accurate discrimination rule, which is used to identify the concept drift of the classification model in this embodiment. The preset rule can be expert manual experience judgment or comparison of keyword information in the entity sequence. The specific form of the preset rule is not limited in this specification.

[0076] In this embodiment, the first identification result of the entity sequence output by the pre-trained classification model can include the identification results of one or more entity sequences, and the second identification result of the entity sequence output by the preset rule can include the identification results of one or more entity sequences. The inconsistency between the second identification result and the first identification result can mean that the second identification result of an entity sequence is inconsistent with the first identification result of this entity sequence, or it can also mean that when the proportion of the difference results between the second identification results of several entity sequences and the first identification results of these several entity sequences in all the results reaches a certain degree, it is considered that the two identification results are inconsistent.

[0077] In one of the illustrated embodiments, the step of "if the second identification result is inconsistent with the first identification result, determining that the classification model has concept drift" includes: for multiple identified entity sequences, if the proportion of the first type of entity sequences in the multiple entity sequences reaches the first threshold, determining that the classification model has concept drift, where the first identification result of the first type of entity sequences is inconsistent with the second identification result of the first type of entity sequences.

[0078] For example, the first threshold is 33%. For the three identified entity sequences ["Zhang San", "511002197****46614", "zhangsan@mail.com"], ["Building 1, Unit 2, Zhongying Garden", "XX Technology Software Development Department"], ["Sunny", "Female"], the first recognition results output by the classification model are 1, 1, 1 respectively, that is, all three entity sequences contain sensitive information. The second recognition results obtained by the preset rules for these three entity sequences are 1, 1, 0 respectively, that is, the first two entity sequences contain sensitive information, while the last entity sequence does not contain sensitive information. It can be seen that for the entity sequence ["Sunny", "Female"], the first recognition result obtained by the classification model and the second recognition result obtained by the preset rules are inconsistent. Then, the proportion of the entity sequence ["Sunny", "Female"] as the first type of entity sequence in the three identified entity sequences is 33%, reaching the preset first threshold. At this time, it can be determined that the classification model has undergone concept drift.

[0079] Among them, in addition to determining that the classification model has undergone concept drift when the proportion of the entity sequence with inconsistent recognition results in the identified entity sequences reaches the first threshold, it can also be determined that the classification model has undergone concept drift when the proportion of the entity sequences with consistent first and second recognition results in the identified entity sequences reaches the first threshold. In addition, not only can the difference result be set in the form of a percentage, but the difference result can also be converted into the form of a confidence interval for discrimination, or a warning of possible concept drift can be given before determining that concept drift has occurred. If the confidence level of the error rate reaches 95%, it is considered that the classification model may have undergone concept drift; if the confidence level of the error rate reaches 99%, it is determined that the classification model has undergone concept drift. Regarding the probability proportion and specific form of the probability of concept drift in the difference result, this specification does not limit it.

[0080] In an illustrated embodiment, the marking of the entity sequence based on the second recognition result includes: marking the first type of entity sequence based on the second recognition result of the first type of entity sequence.

[0081] For example, please refer to Figure 3, which is a schematic diagram of a sensitive information identification system shown in an exemplary embodiment. As shown in Figure 3, after the classification model outputs the recognition result of the entity sequence (i.e., the first recognition result), the preset rules will perform a second recognition on the entity sequence that has been recognized by the classification model to obtain a second recognition result. If the first recognition result and the second recognition result are compared and a conclusion is drawn that the classification model has undergone concept drift, the entity sequence data set input to the classification model that has undergone concept drift will be marked based on the second recognition result obtained from the data set constructed by the preset rules. Then, based on the marked entity sequence data set and other data sets, such as industry standard data sets, the classification model will be actively learned, so that the classification model can be adaptively adjusted based on the new training samples.

[0082] The preset rule dataset may include an expert experience dataset, i.e., an expert's manual rules for judging sensitive information. Other datasets may be other trustworthy datasets, such as public datasets, academic research datasets, etc. This specification does not limit the specific form of the preset rules and other datasets.

[0083] In this embodiment, in order to improve the adaptive training effect of the classification model and make the trained classification model more accurate in identifying entity sequences containing sensitive information, all first-category entity sequences whose first recognition results are inconsistent with the second recognition results can be marked according to the second recognition result.

[0084] In this embodiment, to reduce the labeling workload, it is also possible to only label entity sequences that were input into the classification model when concept drift was confirmed to have occurred and within a previous threshold. Specifically, entity sequences that were input into the pre-trained classification model after the difference ratio between the first recognition result and the second recognition result reaches a second threshold can be labeled, where the second threshold is less than the first threshold, such as the second threshold is 20% and the first threshold is 33%.

[0085] In one embodiment shown, the preset rules include at least one of the following: a first identification rule, wherein the first identification rule is that an entity sequence carrying a preset keyword contains sensitive information; a second identification rule, wherein the second identification rule is that an entity sequence belonging to a preset IP address contains sensitive information; and a third identification rule, wherein the third identification rule is that an entity sequence belonging to a preset pattern contains sensitive information.

[0086] For example, please refer to Figure 4, which is a schematic diagram of an exemplary embodiment showing a method of identifying an entity sequence according to preset rules. As shown in Figure 4, the entity sequence input is a model obtained by modeling the preset rules, and the model is obtained by modeling the preset rules; wherein the preset rules include a first identification rule, that is, judging whether the entity sequence contains sensitive information based on whether the identified entity sequence carries a preset keyword; a second identification rule, that is, judging whether the entity sequence contains sensitive information based on whether the identified entity sequence belongs to a preset IP address; and a third identification rule, that is, judging whether the entity sequence contains sensitive information based on whether the identified entity sequence belongs to a preset pattern. Then, the preset rule model outputs a second identification result on whether the entity sequence contains sensitive information. After determining the preset rules for the entity sequence, the preset rules are implemented in code. The implemented preset rules can identify whether the entity sequence contains sensitive information, thereby obtaining a second identification result on the entity sequence.

[0087] Among them, the identification of keywords can be achieved by pre-defining a list containing sensitive words, which are usually associated with sensitive information or sensitive topics. For example, in scenarios involving the financial industry, words such as bank, account, loan, and personal information may be defined as sensitive words, such as bank account name and password. In addition, to identify whether an entity sequence belongs to a preset IP address, a regular expression can be used for matching. Generally speaking, the characteristic of an illegal address in an IP address is that any three-digit segment is greater than or equal to 255. Among the entity sequences of the preset pattern, entity sequences that may contain sensitive information include but are not limited to: 1. The name of a sensitive organization or institution; 2. Address or location information involving privacy protection, such as residential address, office address, sensitive places, etc.; 3. Sensitive information such as personal ID number, passport number, etc.; 4. Bank card number or payment account; 5. Disease name or drug name involving privacy protection; 6. Sensitive information such as date of birth, family member information, salary income, etc. A considerable portion of the above information has a fixed pattern. For example, the first 6 digits of the 18-digit ID number are the address code, the 7th to 14th digits are the date of birth code, the 15th to 17th digits are the sequence code, and the 18th digit is the check code. Together, they constitute a person's unique identification code.

[0088] Corresponding to the above-mentioned embodiment of the sensitive information identification method, this specification also provides an embodiment of a sensitive information identification device.

[0089] Please refer to Figure 5, which is a hardware structure diagram of an electronic device in which a sensitive information identification device is located, shown in an exemplary embodiment. At the hardware level, the device includes a processor 502, an internal bus 504, a network interface 506, a memory 508, and a non-volatile memory 510, and of course may also include other required hardware. One or more embodiments of this specification can be implemented based on software, such as the processor 502 reading the corresponding computer program from the non-volatile memory 510 into the memory 508 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0090] Please refer to Figure 6, which is a block diagram of a sensitive information identification device according to an exemplary embodiment. The sensitive information identification device can be applied to the electronic device shown in Figure 5 to implement the technical solution of this specification. The device may include:

[0091] An extraction unit 602 is configured to extract an entity sequence from the stream data to be identified, wherein the entity sequence includes at least one entity;

[0092] An input unit 604 inputs the entity sequence into a pre-trained classification model, wherein the classification model includes a machine learning model trained based on sample entity sequences that are marked as containing sensitive information;

[0093] The acquisition unit 606 is configured to acquire a first recognition result output by the classification model, where the first recognition result is used to indicate whether the entity sequence contains sensitive information.

[0094] In this embodiment, the extraction unit includes:

[0095] A first conversion sub-unit is configured to determine that data whose update time is greater than the first time and less than or equal to the third time is incremental data;

[0096] An input subunit, configured to input the text data into a bidirectional long short-term memory-conditional random field BiLSTM-CRF model to obtain entities included in the text data output by the bidirectional long short-term memory-conditional random field model;

[0097] The second conversion subunit is configured to convert the entities included in the text data into the entity sequence.

[0098] In this embodiment, the device further includes:

[0099] a marking unit, configured to mark the entity sequence based on a second recognition result when concept drift occurs in the classification model, wherein the second recognition result is used to indicate whether the entity sequence contains sensitive information, and the credibility of the second recognition result is higher than that of the first recognition result;

[0100] The training unit is used to use the labeled entity sequence as a training sample to retrain the classification model.

[0101] In this embodiment, the device further includes:

[0102] an identification unit, configured to identify the entity sequence based on a preset rule to obtain the second identification result;

[0103] A determining unit is configured to determine that concept drift occurs in the classification model when the second recognition result is inconsistent with the first recognition result.

[0104] In this embodiment, the determining unit includes:

[0105] A determination subunit is configured to determine, for a plurality of identified entity sequences, that concept drift has occurred in the classification model if a proportion of a first type of entity sequence in the plurality of entity sequences reaches a first threshold, wherein a first recognition result of the first type of entity sequence is inconsistent with a second recognition result of the first type of entity sequence.

[0106] In this embodiment, the marking unit includes:

[0107] The marking subunit is configured to mark the first-category entity sequence based on the second recognition result of the first-category entity sequence.

[0108] In this embodiment, the preset rules include at least one of the following:

[0109] A first identification rule, wherein the first identification rule is that an entity sequence carrying a preset keyword contains sensitive information;

[0110] A second identification rule, wherein the second identification rule is that the entity sequence belonging to the preset IP address contains sensitive information;

[0111] A third identification rule, wherein the third identification rule is that an entity sequence belonging to a preset pattern contains sensitive information.

[0112] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0113] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this specification. Ordinary technicians in this field can understand and implement it without paying any creative work.

[0114] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0115] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0116] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0117] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0118] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0119] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0120] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0121] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "an," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0122] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when..." or "when..." or "in response to determining."

[0123] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included in the scope of protection of one or more embodiments of this specification.

Claims

1. A sensitive information identification method, characterized in that: The method comprises: Extracting an entity sequence from the stream data to be identified, wherein the entity sequence includes at least one entity; Inputting the entity sequence into a pre-trained classification model, wherein the classification model includes a machine learning model trained based on sample entity sequences that are labeled as to whether they contain sensitive information; A first recognition result output by the classification model is obtained, where the first recognition result is used to indicate whether the entity sequence contains sensitive information.

2. The method according to claim 1, characterized in that The extracting of entity sequences from the stream data to be identified includes: Converting the stream data to be identified into text data; Inputting the text data into a bidirectional long short-term memory-conditional random field BiLSTM-CRF model to obtain entities included in the text data output by the bidirectional long short-term memory-conditional random field model; The entities included in the text data are converted into the entity sequence.

3. The method according to claim 1, characterized in that The method further comprises: If concept drift occurs in the classification model, marking the entity sequence based on a second recognition result, wherein the second recognition result is used to indicate whether the entity sequence contains sensitive information, and the credibility of the second recognition result is higher than the credibility of the first recognition result; The labeled entity sequence is used as a training sample to retrain the classification model.

4. The method according to claim 3, characterized in that The method further comprises: Identify the entity sequence based on a preset rule to obtain the second identification result; If the second recognition result is inconsistent with the first recognition result, it is determined that concept drift occurs in the classification model.

5. The method according to claim 4, characterized in that If the second recognition result is inconsistent with the first recognition result, determining that concept drift occurs in the classification model includes: For multiple identified entity sequences, if a proportion of a first-category entity sequence in the multiple entity sequences reaches a first threshold, it is determined that concept drift occurs in the classification model, wherein a first recognition result of the first-category entity sequence is inconsistent with a second recognition result of the first-category entity sequence.

6. The method according to claim 3, characterized in that The marking of the entity sequence based on the second recognition result includes: The first-category entity sequence is marked based on the second recognition result of the first-category entity sequence.

7. The method according to claim 4, characterized in that The preset rules include at least one of the following: A first identification rule, wherein the first identification rule is that an entity sequence carrying a preset keyword contains sensitive information; A second identification rule, wherein the second identification rule is that the entity sequence belonging to the preset IP address contains sensitive information; A third identification rule, wherein the third identification rule is that an entity sequence belonging to a preset pattern contains sensitive information.

8. A sensitive information identification device, characterized in that: The device comprises: An extraction unit, configured to extract an entity sequence from the stream data to be identified, wherein the entity sequence includes at least one entity; An input unit, inputting the entity sequence into a pre-trained classification model, wherein the classification model includes a machine learning model trained based on sample entity sequences marked as containing sensitive information; An acquisition unit is used to acquire a first recognition result output by the classification model, where the first recognition result is used to indicate whether the entity sequence contains sensitive information.

9. An electronic device comprising a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus; The memory stores machine-readable instructions, and the processor executes the method according to any one of claims 1 to 7 by calling the machine-readable instructions.

10. A machine-readable storage medium, wherein the machine-readable storage medium stores machine-readable instructions, wherein when the machine-readable instructions are called and executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images