Sensitive information identification method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a sensitive information identification method and device, electronic equipment and storage medium, relates to the technical field of artificial intelligence, in particular to the technical field of deep learning and information security. A specific implementation scheme is: obtaining entity information of a target webpage text; using a preset sensitive information identification model to identify and process the entity information to obtain an identification result of the entity information, the preset sensitive information identification model is obtained by performing knowledge distillation processing on a pre-training model, the pre-training model includes a pre-training network, a first text feature extraction network and a first classifier, and the preset sensitive information identification model includes a second text feature extraction network and a second classifier; and obtaining a sensitive information identification result of the target webpage text according to the identification result of the entity information.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to the fields of deep learning and information security. Background Technology

[0002] Web page text data, as a crucial carrier of information dissemination, has gradually become a major source of sensitive information leaks. Attackers use web crawlers and other attack methods to continuously access pages containing sensitive information, thereby collecting large amounts of sensitive data and posing a significant risk of sensitive information leakage. Therefore, in an era of information sharing and data-driven development, the identification of sensitive information in web page text is particularly important. Summary of the Invention

[0003] This disclosure provides methods, apparatus, electronic devices, and storage media for identifying sensitive information.

[0004] According to one aspect of this disclosure, a method for identifying sensitive information is provided, comprising:

[0005] Obtain entity information from the target webpage text;

[0006] The entity information is identified using a pre-set sensitive information identification model to obtain the identification result of the entity information. The pre-set sensitive information identification model is obtained by knowledge distillation of a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The pre-set sensitive information identification model includes a second text feature extraction network and a second classifier.

[0007] Based on the recognition results of the entity information, the sensitive information recognition results of the target webpage text are obtained.

[0008] According to another aspect of this disclosure, a method for training a sensitive information identification model is provided, comprising:

[0009] Obtain first training data and second training data, wherein the first training data is labeled data and the second training data is unlabeled data;

[0010] Based on the first training data, the pre-trained model to be trained is iteratively trained to obtain a pre-trained model that has been trained; the pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier;

[0011] Using the pre-trained model that has been trained, the second training data is processed for recognition to obtain soft labels for the second training data;

[0012] Based on the second training data and the soft labels of the second training data, the sensitive information recognition model to be trained is iteratively trained to obtain the sensitive information recognition model, which includes a second text feature extraction network and a second classifier.

[0013] According to another aspect of this disclosure, a sensitive information identification device is provided, comprising:

[0014] The acquisition unit is used to acquire entity information of the target webpage text;

[0015] The identification unit is used to identify the entity information using a preset sensitive information identification model to obtain the identification result of the entity information. The preset sensitive information identification model is obtained by knowledge distillation of a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network and a first classifier. The sensitive information identification model includes a second text feature extraction network and a second classifier.

[0016] The obtaining unit is used to obtain the sensitive information identification result of the target webpage text based on the identification result of the entity information.

[0017] According to another aspect of this disclosure, a training apparatus for a sensitive information identification model is provided, comprising:

[0018] A data acquisition unit is used to acquire first training data and second training data, wherein the first training data is labeled data and the second training data is unlabeled data;

[0019] The first training unit is used to iteratively train the pre-trained model to be trained based on the first training data to obtain the pre-trained model after training; the pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier.

[0020] The data recognition unit is used to perform recognition processing on the second training data using the pre-trained model that has been trained, so as to obtain soft labels for the second training data.

[0021] The second training unit is used to iteratively train the sensitive information recognition model to be trained based on the second training data and the soft labels of the second training data, so as to obtain the sensitive information recognition model. The preset sensitive information recognition model includes a second text feature extraction network and a second classifier.

[0022] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0023] At least one processor; and

[0024] A memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described above and any possible implementations.

[0026] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described above and any possible implementation thereof.

[0027] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the aspects and any possible implementations described above.

[0028] As can be seen from the above technical solution, on the one hand, the embodiments of this disclosure obtain entity information of the target webpage text, and then use a preset sensitive information recognition model to recognize the entity information to obtain the recognition result of the entity information. This enables the recognition result of sensitive information of the target webpage text to be obtained based on the recognition result of the entity information. The preset sensitive information recognition model is obtained by knowledge distillation of a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The preset sensitive information recognition model includes a second text feature extraction network and a second classifier. Since the sensitive information in the target webpage text can be identified using a simple sensitive information recognition model obtained through knowledge distillation, more accurate sensitive information can be obtained quickly, thereby improving the reliability of sensitive information recognition.

[0029] As can be seen from the above technical solution, on the other hand, the embodiments of this disclosure obtain first training data and second training data, and then iteratively train the pre-trained model to be trained based on the first training data to obtain a pre-trained model that has been trained. The pre-trained model is then used to identify the second training data to obtain soft labels for the second training data. Furthermore, the sensitive information recognition model to be trained is iteratively trained based on the second training data and the soft labels for the second training data to obtain a sensitive information recognition model. The first training data is labeled data, and the second training data is unlabeled data. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The sensitive information recognition model includes a second text feature extraction network and a second classifier. Since a more complex pre-trained model is trained using the labeled first training data, and then soft labels for training a simpler sensitive information recognition model are obtained using the trained pre-trained model and the unlabeled second training data, the training of the sensitive information recognition model is completed. This effectively reduces the amount of labeled training data required, requiring only a small amount of labeled data to ensure the model's recognition performance and accuracy, thereby optimizing the effectiveness of model training.

[0030] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0031] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0032] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0033] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0034] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0035] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0036] Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure;

[0037] Figure 6 This is a block diagram of an electronic device used to implement the sensitive information identification method or model training method of the embodiments of this disclosure. Detailed Implementation

[0038] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0039] Obviously, the described embodiments are only some, not all, of the embodiments disclosed herein. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0040] It should be noted that the terminal devices involved in the embodiments of this disclosure may include, but are not limited to, smart devices such as mobile phones, personal digital assistants (PDAs), wireless handheld devices, and tablet computers; the display devices may include, but are not limited to, personal computers, televisions, and other devices with display functions.

[0041] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0042] In an era of information sharing and data-driven development, identifying sensitive information in web page text data is particularly important to ensure information security.

[0043] Currently, methods for identifying sensitive information in related technologies include supervised learning and unsupervised learning. Specifically, supervised learning can involve using Naive Bayes to classify and identify sensitive data by using keywords from a high-frequency word set as attributes. Unsupervised learning, on the other hand, can be used for structured datasets, employing clustering and association rule algorithms to identify sensitive and non-sensitive attribute sets. Sensitive attributes are then classified, and the average sensitivity of each attribute class is quantified to define its sensitivity level, thereby achieving the classification and grading of sensitive information.

[0044] However, the identification methods for sensitive information in related technologies are still not ideal in terms of both effectiveness and speed.

[0045] Therefore, there is an urgent need to provide a method for identifying sensitive information that can identify sensitive information in web page text data based on knowledge distillation, thereby improving the reliability of sensitive information identification and detection.

[0046] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure, as shown below. Figure 1 As shown.

[0047] 101. Obtain entity information of the target webpage text.

[0048] 102. Using a preset sensitive information recognition model, the entity information is processed to obtain the recognition result of the entity information. The preset sensitive information recognition model is obtained by knowledge distillation of a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The preset sensitive information recognition model includes a second text feature extraction network and a second classifier.

[0049] 103. Based on the recognition results of the entity information, obtain the sensitive information recognition results of the target webpage text.

[0050] It should be noted that sensitive information may include personal privacy information or other information that requires special attention. Specifically, the specific text and type of sensitive information can be determined based on the actual application scenario.

[0051] It should be noted that some or all of the execution entities of 101 to 103 can be applications located on the local terminal, or they can be functional units such as plug-ins or software development kits (SDKs) set in applications located on the local terminal, or they can be processing engines located on the network-side server, or they can be distributed systems located on the network side, such as processing engines or distributed systems in information recognition platforms on the network side. This embodiment does not impose any special limitations on these.

[0052] It is understood that the application may be a native program installed on the local terminal, or it may be a web application of a browser on the local terminal. This embodiment does not limit this.

[0053] In this way, entity information of the target webpage text can be obtained, and then a preset sensitive information recognition model can be used to recognize and process the entity information to obtain the recognition result of the entity information. Based on the recognition result of the entity information, the sensitive information recognition result of the target webpage text can be obtained. The preset sensitive information recognition model is obtained by knowledge distillation of a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The preset sensitive information recognition model includes a second text feature extraction network and a second classifier. Since the sensitive information recognition model with a simple structure obtained by knowledge distillation can be used to identify sensitive information in the target webpage text, more accurate sensitive information can be obtained quickly, thereby improving the reliability of sensitive information recognition.

[0054] Moreover, since sensitive information in target webpage text can be effectively identified through a simple sensitive information identification model, the hardware resources used for identification and processing can be reduced, and the hardware computing efficiency can be improved, thereby increasing resource utilization and processing speed.

[0055] Optionally, in one possible implementation of this embodiment, in step 101, the target webpage text is obtained, and the target webpage text is segmented to obtain entity information of the target webpage text.

[0056] In a specific implementation of this method, firstly, the target webpage text can be segmented to obtain word segments and character segments. Secondly, a preset mapping algorithm can be used to map the word segments and character segments of the target webpage text to obtain a sentence identifier sequence of the target webpage text. Thirdly, entity information of the target webpage text can be obtained based on the sentence identifier sequence.

[0057] In this way, the target webpage text can be segmented first to obtain word segments and character segments. Then, a preset mapping algorithm is used to map the word segments and character segments to obtain a sentence identifier sequence, thereby obtaining the entity information of the target webpage text. This facilitates the effective identification of the entity information of the target webpage text in subsequent steps, thus further improving the reliability of sensitive information identification.

[0058] In one specific implementation, the target webpage text can be segmented using preset text segmentation rules to obtain sentence segments, word segments, and character segments of the target webpage text.

[0059] Specifically, the preset text segmentation rules can include sentence-based segmentation, word-based segmentation, and character-based segmentation.

[0060] In this specific implementation process, the preset mapping algorithm may include performing mapping processing based on the preset mapping relationship between words and characters and numerical identifiers respectively.

[0061] In another specific implementation, after obtaining the word segmentation and character segmentation of the target webpage text, a preset mapping algorithm can be used to map the word segmentation and character segmentation of the target webpage text into numerical identifiers to obtain the sentence identifier sequence of the target webpage text.

[0062] Specifically, the sentence identifier sequence may include a sentence ID sequence.

[0063] In this implementation, the entities of the target webpage text can include text strings or blocks, such as sentences, words, characters, etc. The entity information of the target webpage text can include, but is not limited to, a sequence of sentence IDs.

[0064] Thus, in this implementation, by segmenting the target webpage text to obtain entity information, it is easier to effectively identify the entity information of the target webpage text in the future, thereby further improving the reliability of the identification of sensitive information.

[0065] Optionally, in one possible implementation of this embodiment, in step 102, the entity information can be input into the second text feature extraction network to obtain the contextual feature information of the entity information, and then the contextual feature information can be input into the second classifier to obtain the recognition result of the entity information.

[0066] In this implementation, the identification result of the entity information may include the entity's start position, end position, and the label with the highest probability among all entities based on the start and end positions.

[0067] In a specific implementation of this method, after obtaining the recognition result of entity information, in step 103, the sensitive information recognition result of the target webpage text can be obtained based on the entity start position, end position, and the tag with the highest probability for all entities based on the start and end positions.

[0068] Here, the results of sensitive information identification can include all sensitive information in the target webpage text.

[0069] For example, XXX is a name; YYYYYY is an address, and so on.

[0070] Thus, in this implementation, the input entity information can be identified by utilizing the second text feature extraction network and the second classifier, thereby achieving rapid and accurate identification of entity information and further improving the reliability of sensitive information identification.

[0071] It should be noted that the specific implementation process provided in this embodiment can be combined with various specific implementation processes provided in the foregoing implementation methods to realize the sensitive information identification method of this embodiment. Detailed descriptions can be found in the relevant content of the foregoing implementation methods, and will not be repeated here.

[0072] Optionally, in one possible implementation of this embodiment, the second text feature extraction network may include a BiLSTM-based network, and the second classifier may include a span-based classifier.

[0073] In this implementation, the pre-trained network in the pre-trained model may include, but is not limited to, an ERNIE-based pre-trained network and a BERT-based pre-trained network.

[0074] It is understandable that the first and second text feature extraction networks can have the same structure, meaning both can be based on BiLSTM. Similarly, the first and second classifiers can have the same structure. Both can be Span-based classifiers.

[0075] For example, the pre-trained model can be a BERT-BiLSTM-Span model, or an ERNIE-BiLSTM-Span model. The sensitive information recognition model can be a BiLSTM-Span model.

[0076] In one specific implementation of this approach, a sensitive information identification model can be obtained based on a pre-trained model using a data distillation algorithm.

[0077] Thus, in this implementation, a simpler and shorter sensitive information recognition model can be obtained by distilling a pre-trained model with a more complex structure. Using this sensitive information recognition model, entity information of target web page text can be quickly identified while achieving a recognition effect similar to that of the pre-trained model, thereby further improving the reliability of sensitive information recognition.

[0078] It should be noted that the specific implementation process provided in this embodiment can be combined with various specific implementation processes provided in the foregoing implementation methods to realize the sensitive information identification method of this embodiment. Detailed descriptions can be found in the relevant content of the foregoing implementation methods, and will not be repeated here.

[0079] In this embodiment, entity information of the target webpage text can be obtained, and then a preset sensitive information recognition model can be used to recognize and process the entity information to obtain the recognition result of the entity information. This allows the sensitive information recognition result of the target webpage text to be obtained based on the recognition result of the entity information. The preset sensitive information recognition model is obtained by knowledge distillation of a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The preset sensitive information recognition model includes a second text feature extraction network and a second classifier. Since the sensitive information recognition model with a simple structure obtained through knowledge distillation can be used to identify sensitive information in the target webpage text, more accurate sensitive information can be obtained quickly, thereby improving the reliability of sensitive information recognition.

[0080] Furthermore, by employing the technical solution provided in this embodiment, the entity information of the target webpage text can be obtained by segmenting the target webpage text, which facilitates the effective identification of the entity information of the target webpage text in the future, thereby further improving the reliability of the identification of sensitive information.

[0081] Furthermore, by employing the technical solution provided in this embodiment, the target webpage text can be segmented first to obtain word segments and character segments. Then, a preset mapping algorithm is used to map the word segments and character segments to obtain a sentence identifier sequence, thereby obtaining the entity information of the target webpage text. This facilitates the effective identification of the entity information of the target webpage text, further improving the reliability of sensitive information identification.

[0082] Furthermore, by employing the technical solution provided in this embodiment, the input entity information can be identified by utilizing a second text feature extraction network and a second classifier, thereby achieving rapid and accurate identification of entity information and further improving the reliability of sensitive information identification.

[0083] Furthermore, by adopting the technical solution provided in this embodiment, a simpler and shorter sensitive information recognition model can be obtained by distilling a pre-trained model with a more complex structure. Using this sensitive information recognition model, the entity information of the target web page text can be identified quickly while achieving a recognition effect similar to that of the pre-trained model, thereby further improving the reliability of sensitive information recognition.

[0084] Figure 2 This is a schematic diagram based on the second embodiment of the present disclosure, as shown below. Figure 2 As shown.

[0085] 201. Obtain first training data and second training data, wherein the first training data is labeled data and the second training data is unlabeled data.

[0086] 202. Based on the first training data, iteratively train the pre-trained model to be trained to obtain a pre-trained model that has been trained; the pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier.

[0087] 203. Using the pre-trained model that has been trained, identify the second training data to obtain soft labels for the second training data.

[0088] 204. Based on the second training data and the soft labels of the second training data, the sensitive information recognition model to be trained is iteratively trained to obtain the sensitive information recognition model, which includes a second text feature extraction network and a second classifier.

[0089] It should be noted that some or all of the execution entities in 201 to 204 can be applications located on the local terminal, or they can be functional units such as plug-ins or software development kits (SDKs) set in applications located on the local terminal, or they can be processing engines located on network-side servers, or they can be distributed systems located on the network side, such as processing engines or distributed systems in network-side model training platforms, etc. This embodiment does not impose any special limitations on this.

[0090] It is understood that the application may be a native program installed on the local terminal, or it may be a web application of a browser on the local terminal. This embodiment does not limit this.

[0091] In this way, a more complex pre-trained model can be trained using the first training data with labels. Then, using the trained pre-trained model and the second training data without labels, soft labels can be obtained to train a more simple sensitive information recognition model. This effectively reduces the amount of training data that needs to be labeled. Only a small amount of labeled data is needed to ensure the recognition performance and accuracy of the model, thereby optimizing the effectiveness of model training.

[0092] Optionally, in one possible implementation of this embodiment, in step 201, during the process of acquiring the first training data, sample webpage text can be acquired, and then the sample webpage text can be segmented to obtain word segmentation and character segmentation of the sample webpage text, so that the word segmentation and character segmentation of the sample webpage text can be annotated using preset annotation rules to obtain annotation tags for the sample webpage text.

[0093] Here, the preset annotation rules may include, but are not limited to, BIO annotation rules.

[0094] For example, if sensitive information contains only a single word, it is marked with "B"; if sensitive information consists of two or more words, the entity begins with "B" and the middle and end with "I", for example, name tags are "B-NAME" and "I-NAME". Non-sensitive information is marked with "O".

[0095] Understandably, the specific sensitive information to be labeled should be determined based on the actual business scenario. For example, in scenarios involving personal privacy, it is necessary to label three sensitive entities: name, address, and organization.

[0096] In a specific implementation of this method, firstly, the sample webpage text can be segmented into sentences, words, and characters to obtain processed text data, namely sentence entities, word entities, and character entities. Then, using BIO annotation rules, the processed text data is annotated to obtain annotation tags. Secondly, each character entity in the segmented text data is mapped to a number to obtain a sentence ID sequence, which contains entity information and annotation tag IDs. Thirdly, based on the start and end positions of each character entity in the sentence ID sequence and its annotation tag ID, a sentence entity start position ID sequence and a sentence entity end position ID sequence are obtained.

[0097] It is understandable that, here, the sample webpage text can be the first sample webpage text, and the entity information can be the first entity information.

[0098] In another specific implementation of this approach, the second training data is unlabeled data. Specifically, the process of acquiring the second training data involves obtaining sample webpage text, which can then be segmented to obtain entity information.

[0099] It is understandable that, here, the sample webpage text of the second training data can be the second sample webpage text, and the entity information of the second training data can be the second entity information.

[0100] In this way, the training data can be constructed so that the constructed training data can be used to train the application model, thereby ensuring the recognition performance of the trained model.

[0101] Optionally, in one possible implementation of this embodiment, in step 202, the first training data can be input into the pre-trained network to obtain word vector features corresponding to the first training data. Then, the word vector features can be input into the first text feature extraction network to obtain contextual feature information corresponding to the word vector features. The contextual feature information can be input into the first classifier to obtain the predicted label of the sample entity information, so that the pre-trained model to be trained can be iteratively trained according to the labeled label of the sample entity information and the predicted label of the sample entity information to obtain the pre-trained model after training.

[0102] In this implementation, the first training data includes sample entity information of the sample webpage text and annotation tags of the sample entity information.

[0103] In a specific implementation of this method, firstly, the sample entity information of the first training data can be input into the pre-trained network to obtain the corresponding word vector features. Secondly, the word vector features can be input into the first text feature extraction network to obtain the contextual feature information corresponding to the word vector features. Thirdly, the contextual feature information is input into the first classifier to obtain the predicted label of the sample entity information. Finally, the pre-trained model to be trained can be iteratively updated and trained according to the labeled label and the predicted label of the sample entity information until the training termination condition is met, resulting in a pre-trained model that has been trained.

[0104] In this implementation, the pre-trained model can be a deep neural network model built based on a pre-trained network, a first text feature extraction network, and a first classifier, i.e., a pre-trained large model.

[0105] Here, the pre-trained large model has already learned general semantic representations from large-scale predictions. Therefore, by using the pre-trained large model, good recognition results can be obtained with only a small amount of sample data and annotations, as well as a small amount of training resources and time.

[0106] For example, the pre-trained model can be a deep neural network model built on BERT-BiLSTM-Span, or it can be a deep neural network model built on ERNIE-BiLSTM-Span.

[0107] Understandably, in practical applications, the sensitive information, or sensitive entities, involved in different industries may differ. Therefore, pre-trained models can be trained according to the specific business application scenarios.

[0108] Thus, in this implementation, a pre-trained model that has learned the relevant features of the first training data can be obtained by training the pre-trained model, which includes a pre-trained network, a first text feature extraction network, and a first classifier, based on the first training data, thereby ensuring the learning effect of the pre-trained model.

[0109] It should be noted that the specific implementation process for acquiring sample data provided in this implementation method can be combined with various specific implementation processes for training pre-trained models provided in the aforementioned implementation methods to realize the training method of the sensitive information identification model in this embodiment. Detailed descriptions can be found in the relevant content of the aforementioned implementation methods, and will not be repeated here.

[0110] Optionally, in one possible implementation of this embodiment, in step 204, the second training data can be input into the second text feature extraction network to obtain contextual feature information of the second training data. Then, the contextual feature information of the second training data can be input into the second classifier to obtain the predicted label of the second training data. This enables iterative training of the sensitive information recognition model to be trained based on the soft label and the predicted label of the second training data to obtain the sensitive information recognition model.

[0111] In this implementation, the soft label of the second training data can be the predicted soft label obtained by the pre-trained model through the recognition processing of the unlabeled second training data.

[0112] Specifically, here, a soft label can be a label that carries probability, that is, a label that is given a degree of uncertainty by probability.

[0113] For example, for a given word, the probability of it being a name is 0.8, and the probability of it being a place name is 0.2.

[0114] In a specific implementation of this method, during the iterative training of the sensitive information recognition model to be trained, based on the second training data and its soft labels, firstly, the second text feature extraction network can be used to extract the contextual feature information of the second training data. Secondly, the second classifier is used to classify the contextual feature information of the second training data to obtain the predicted labels of the second training data. Finally, based on the soft labels and predicted labels of the second training data, the sensitive information recognition model to be trained is iteratively trained until the training termination condition is met, resulting in a completed sensitive information recognition model.

[0115] In this implementation, the second training data may include second entity information. The second entity information may include a sequence of sentence IDs. The soft labels of the second training data may include a sequence of sentence entity start position IDs, a sequence of sentence entity end position IDs, and the probability corresponding to each label.

[0116] Here, training termination conditions may include, but are not limited to, model convergence, reaching a threshold of training iterations, etc.

[0117] In another specific implementation of this approach, the sentence ID sequence, the sentence entity start position ID sequence, the sentence entity end position ID sequence, and the probability corresponding to each label can be input into the sensitive information recognition model to be trained. The model is then trained using the SGD stochastic gradient descent method, and the parameters are updated and optimized through multiple rounds of iteration until convergence, thus obtaining the trained sensitive information recognition model.

[0118] It is understandable that sensitive information identification models can include models based on the BiLSTM-Span structure, models based on the BiLSTM-CRF structure, etc.

[0119] Thus, in this implementation, the sensitive information recognition model, which includes a second text feature extraction network and a second classifier, can be trained based on the second training data and the soft labels of the second training data. This allows the model to learn more about the second training data and the soft labels, ensuring the learning effect of the sensitive information recognition model and improving its performance.

[0120] It should be noted that the specific implementation process for iteratively training the sensitive information recognition model to be trained provided in this implementation method can be combined with various specific implementation processes provided in the aforementioned implementation methods to realize the training method of the sensitive information recognition model in this embodiment. Detailed descriptions can be found in the relevant content of the aforementioned implementation methods, and will not be repeated here.

[0121] In this embodiment, a pre-trained model with a more complex structure can be trained using labeled first training data. Then, using the pre-trained model and unlabeled second training data, soft labels can be obtained to train a sensitive information recognition model with a simpler structure, thereby completing the training of the sensitive information recognition model. This effectively reduces the amount of training data that needs to be labeled. Only a small amount of labeled data is needed to ensure the recognition performance and accuracy of the model, thus optimizing the effectiveness of model training.

[0122] Furthermore, by adopting the technical solution provided in this embodiment, a pre-trained model that has learned the relevant features of the first training data can be obtained by training the pre-trained model, which includes a pre-trained network, a first text feature extraction network, and a first classifier, based on the first training data, thereby ensuring the learning effect of the pre-trained model.

[0123] Furthermore, by adopting the technical solution provided in this embodiment, the sensitive information recognition model, which includes a second text feature extraction network and a second classifier, can be trained based on the second training data and the soft labels of the second training data. This allows the model to learn the relevant information of the second training data and the soft labels of the second training data more effectively, ensuring the learning effect of the sensitive information recognition model and thus improving its performance.

[0124] Figure 3 This is a schematic diagram based on the third embodiment of the present disclosure, as shown below. Figure 3 As shown.

[0125] To better illustrate the solution in this embodiment, a detailed description is provided below in conjunction with the application scenario.

[0126] 301. Obtain the original webpage text.

[0127] 302. Preprocess the original webpage text to obtain the first training data.

[0128] In an embodiment, preprocessing may include at least one of sentence segmentation, word segmentation and character segmentation, annotation and labeling, and entity number mapping.

[0129] In this embodiment, firstly, the original webpage text can be segmented into sentences, words, and characters to obtain processed text data, namely sentence entities, word entities, and character entities. Then, using BIO annotation rules, the processed text data is annotated to obtain annotation tags. Secondly, each character entity in the segmented text data is mapped to a number to obtain a sentence ID sequence, which contains entity information and annotation tag IDs. Thirdly, based on the start and end positions of each character entity in the sentence ID sequence and its annotation tag ID, a sentence entity start position ID sequence and a sentence entity end position ID sequence are obtained.

[0130] Specifically, the data after being processed with labeled text can be used as the first training data. Alternatively, the segmented text data without labeled text can be used as the second training data.

[0131] Understandably, through steps 301 to 302, various types of data for model training can be obtained. Training data may be web page text data containing sensitive information.

[0132] 303. Based on the first training data, iteratively train the pre-trained model to be trained to obtain the pre-trained model after training.

[0133] In this implementation, the pre-trained model can be constructed using BERT-BiLSTM-Span or ERNIE-BiLSTM-Span structures to build a large deep neural network model.

[0134] For example, the BERT-BiLSTM-Span pre-trained model mainly consists of a BERT network, a BiLSTM network, and a Span classifier. The BERT network can be composed of multiple multi-head self-attention modules, feedforward network modules, normalization, and residual connections. Training data is processed through the BERT pre-trained network to obtain corresponding word vectors. These word vectors are then input into the BiLSTM network to extract textual context features. These textual context features are then input into the Span classifier to obtain the predicted start and end positions of entities in the webpage text, as well as the posterior probabilities of all entities formed based on the start and end positions. Furthermore, the category with the highest probability score can be used as the label for the corresponding entity, thus obtaining the predicted sequence and the labels for each entity in the sequence.

[0135] 304. Obtain the second training data.

[0136] In this embodiment, the second training data can be obtained from the aforementioned original web page text, and / or, real unlabeled samples from actual application scenarios can be used as the second training data.

[0137] Furthermore, the acquired second training data can be processed into sentence, word, and character segments to obtain sentence entities, word entities, and character entities. Entity number mapping can then be performed on these entities to obtain the processed second training data. The processed second training data may include sentence ID sequences, etc.

[0138] 305. Using the pre-trained model that has been trained, perform recognition processing on the second training data to obtain soft labels for the second training data.

[0139] 306. Based on the second training data and the soft labels of the second training data, train the sensitive information recognition model to be trained.

[0140] 307. Determine whether the model has converged.

[0141] In this embodiment, if not, execute step 308; if yes, execute step 309.

[0142] 308. Execute SGD to update network parameters.

[0143] In this embodiment, the SGD gradient descent method is used to update the network parameters, and steps 304 to 307 are executed in reverse order.

[0144] 309. Obtain the trained sensitive information recognition model.

[0145] It is understandable that a sensitive information recognition model can be trained through steps 303 to 309. In other words, a sensitive information recognition model can be trained using a knowledge distillation algorithm based on a pre-trained model and training data. This sensitive information recognition model can be a model based on a BiLSTM-Span structure.

[0146] It's important to note that sensitive information, or sensitive entities, differ across industries. Even entities with the same name may have different meanings. Training a general-purpose model would likely result in poor performance across various personalized application scenarios. However, training from scratch for personalized scenarios would require collecting and labeling a massive amount of training data, consuming significant resources and time. Pre-trained large-scale models can solve this problem. These models have already learned general semantic representations from large-scale datasets and possess powerful expressive capabilities. Therefore, in personalized applications, only a small amount of training data and labeling, along with minimal training resources and time, is needed to achieve good recognition results. Using pre-trained large-scale models for training specific models for each personalized scenario ensures high recognition performance and accuracy with minimal labeled data.

[0147] In this embodiment, a pre-trained large model is used to perform batch predictions on unlabeled second training data in the corresponding application scenario for sensitive information identification to obtain predicted soft labels. Then, based on the predicted soft labels and the second training data, a sensitive information identification model is trained, that is, a small model with a simple structure is trained.

[0148] It's important to note that when training small models, it's typically difficult to learn effective feature information and achieve robust model building capabilities when labeled data is limited. Therefore, this approach uses a pre-trained large model to perform batch predictions on a large number of real-world unlabeled samples from various application scenarios. The pre-trained large model demonstrates good recognition performance, and subsequent training of small models using the soft labels predicted by this model allows for more effective learning.

[0149] Furthermore, in practical applications, using probabilistic soft labels for model training produces models with significantly better performance and results than those using hard labels.

[0150] 310. Obtain the target webpage text to be identified.

[0151] 311. Perform segmentation processing on the target webpage text to obtain word segments and character segments of the target webpage text.

[0152] 312. Using a preset mapping algorithm, perform word segmentation and character segmentation mapping on the target webpage text to obtain the sentence identifier sequence of the target webpage text.

[0153] 313. Using the trained sensitive information recognition model, the sentence identifier sequence of the target webpage text is identified and processed to obtain the sensitive information recognition result.

[0154] In this implementation, the sentence identifier sequence of the target webpage text, i.e. the sentence ID sequence, is input into the trained sensitive information recognition model. The model can output the start position, end position, and the label of the maximum probability of all entities formed based on the start and end positions in the sentence, so as to obtain the sensitive information recognition result.

[0155] Specifically, the results of sensitive information identification may include, but are not limited to, sensitive information labels corresponding to entities in a sentence, i.e., labels with the highest probability.

[0156] In this embodiment, sensitive information in web page text can be identified by segmenting the text into sentences, words, characters, labeling, and modeling, and then utilizing a sensitive information identification model based on a knowledge distillation algorithm. This achieves both high processing speed and good recognition results. Furthermore, only a small number of manually labeled samples are needed during the training of the identification model. This effectively reduces the number of manually labeled samples, improves the efficiency of model training and the effectiveness of model recognition and detection, and thus enhances the reliability of sensitive information identification.

[0157] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0158] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0159] Figure 4 This is a schematic diagram based on the fourth embodiment of the present disclosure, as shown below. Figure 4 As shown. The sensitive information identification device 400 of this embodiment may include an acquisition unit 401, an identification unit 402, and an acquisition unit 403. The acquisition unit 401 is used to acquire entity information of the target webpage text; the identification unit 402 is used to perform identification processing on the entity information using a preset sensitive information identification model to obtain the identification result of the entity information. The preset sensitive information identification model is obtained by knowledge distillation processing of a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The preset sensitive information identification model includes a second text feature extraction network and a second classifier; the acquisition unit 403 is used to obtain the sensitive information identification result of the target webpage text based on the identification result of the entity information.

[0160] It should be noted that some or all of the sensitive information identification device in this embodiment may be an application located on a local terminal, or it may be a plugin or software development kit (SDK) or other functional unit set in an application located on a local terminal, or it may be a processing engine located on a network-side server, or it may be a distributed system located on the network side, such as a processing engine or distributed system in a network-side identification platform, etc. This embodiment does not impose any particular limitations on this.

[0161] It is understood that the application may be a native program installed on the local terminal, or it may be a web application of a browser on the local terminal. This embodiment does not limit this.

[0162] Optionally, in one possible implementation of this embodiment, the acquisition unit 401 may be used to acquire the target webpage text, perform segmentation processing on the target webpage text, and obtain entity information of the target webpage text.

[0163] Optionally, in one possible implementation of this embodiment, the acquisition unit 401 can also be used to segment the target webpage text to obtain word segments and character segments of the target webpage text, and use a preset mapping algorithm to map the word segments and character segments of the target webpage text to obtain the sentence identifier sequence of the target webpage text, and obtain the entity information of the target webpage text based on the sentence identifier sequence.

[0164] Optionally, in one possible implementation of this embodiment, the recognition unit 402 may be used to input the entity information into the second text feature extraction network to obtain the contextual feature information of the entity information, and input the contextual feature information into the second classifier to obtain the recognition result of the entity information.

[0165] Optionally, in one possible implementation of this embodiment, the second text feature extraction network includes a BiLSTM-based network; the second classifier includes a Span-based classifier.

[0166] In this embodiment, the acquisition unit obtains entity information of the target webpage text, and then the recognition unit uses a preset sensitive information recognition model to process the entity information to obtain the recognition result of the entity information. The preset sensitive information recognition model is obtained by knowledge distillation of a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The preset sensitive information recognition model also includes a second text feature extraction network and a second classifier, enabling the acquisition unit to obtain the sensitive information recognition result of the target webpage text based on the recognition result of the entity information. Since the sensitive information in the target webpage text can be identified using a simple sensitive information recognition model obtained through knowledge distillation, more accurate sensitive information can be obtained quickly, thereby improving the reliability of sensitive information recognition.

[0167] Furthermore, by employing the technical solution provided in this embodiment, the entity information of the target webpage text can be obtained by segmenting the target webpage text, which facilitates the effective identification of the entity information of the target webpage text in the future, thereby further improving the reliability of the identification of sensitive information.

[0168] Furthermore, by employing the technical solution provided in this embodiment, the target webpage text can be segmented first to obtain word segments and character segments. Then, a preset mapping algorithm is used to map the word segments and character segments to obtain a sentence identifier sequence, thereby obtaining the entity information of the target webpage text. This facilitates the effective identification of the entity information of the target webpage text, further improving the reliability of sensitive information identification.

[0169] Furthermore, by employing the technical solution provided in this embodiment, the input entity information can be identified by utilizing a second text feature extraction network and a second classifier, thereby achieving rapid and accurate identification of entity information and further improving the reliability of sensitive information identification.

[0170] Furthermore, by adopting the technical solution provided in this embodiment, a simpler and shorter sensitive information recognition model can be obtained by distilling a pre-trained model with a more complex structure. Using this sensitive information recognition model, the entity information of the target web page text can be identified quickly while achieving a recognition effect similar to that of the pre-trained model, thereby further improving the reliability of sensitive information recognition.

[0171] Figure 5 This is a schematic diagram based on the fifth embodiment of the present disclosure, as shown below. Figure 5 As shown. The training device 500 for the sensitive information recognition model in this embodiment may include a data acquisition unit 501, a first training unit 502, a data recognition unit 503, and a second training unit 504. The data acquisition unit 501 is used to acquire first training data and second training data, wherein the first training data is labeled data and the second training data is unlabeled data; the first training unit 502 is used to iteratively train a pre-trained model to be trained based on the first training data to obtain a pre-trained model; the pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier; the data recognition unit 503 is used to use the pre-trained model to perform recognition processing on the second training data to obtain soft labels for the second training data; the second training unit 504 is used to iteratively train the sensitive information recognition model to be trained based on the second training data and the soft labels of the second training data to obtain a sensitive information recognition model, wherein the sensitive information recognition model includes a second text feature extraction network and a second classifier.

[0172] It should be noted that the training device of the model in this embodiment may be part or all of an application located on a local terminal, or it may be a plugin or software development kit (SDK) or other functional unit set in an application located on a local terminal, or it may be a processing engine located on a network-side server, or it may be a distributed system located on the network side, such as a processing engine or distributed system in a network-side model training platform, etc. This embodiment does not impose any particular limitations on this.

[0173] It is understood that the application may be a native program installed on the local terminal, or it may be a web application of a browser on the local terminal. This embodiment does not limit this.

[0174] Optionally, in one possible implementation of this embodiment, the first training data includes sample entity information and labeled tags of sample webpage text. The first training unit 502 can be specifically used to input the first training data into the pre-training network to obtain word vector features corresponding to the first training data, input the word vector features into the first text feature extraction network to obtain context feature information corresponding to the word vector features, input the context feature information into the first classifier to obtain predicted tags of the sample entity information, and iteratively train the pre-training model to be trained based on the labeled tags and predicted tags of the sample entity information to obtain a pre-trained model that has been trained.

[0175] Optionally, in one possible implementation of this embodiment, the second training unit 504 may be used to input the second training data into the second text feature extraction network to obtain contextual feature information of the second training data, input the contextual feature information of the second training data into the second classifier to obtain the predicted label of the second training data, and iteratively train the sensitive information recognition model to be trained based on the soft label of the second training data and the predicted label of the second training data to obtain the sensitive information recognition model.

[0176] In this embodiment, a data acquisition unit can acquire first training data and second training data. The first training data is labeled data, and the second training data is unlabeled data. A first training unit can then iteratively train a pre-trained model based on the first training data to obtain a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. A data recognition unit uses the pre-trained model to process the second training data to obtain soft labels for the second training data. This allows a second training unit to iteratively train a sensitive information recognition model based on the second training data and its soft labels to obtain a sensitive information recognition model. The pre-trained sensitive information recognition model includes a second text feature extraction network and a second classifier. By utilizing labeled first training data and the pre-trained model and unlabeled second training data to obtain soft labels for training a simpler sensitive information recognition model during the acquisition of the more complex pre-trained model, the amount of labeled training data required is effectively reduced. Only a small amount of labeled data is needed to ensure the model's recognition performance and accuracy, thereby optimizing the effectiveness of model training.

[0177] Furthermore, by adopting the technical solution provided in this embodiment, a pre-trained model that has learned the relevant features of the first training data can be obtained by training the pre-trained model, which includes a pre-trained network, a first text feature extraction network, and a first classifier, based on the first training data, thereby ensuring the learning effect of the pre-trained model.

[0178] Furthermore, by adopting the technical solution provided in this embodiment, the sensitive information recognition model, which includes a second text feature extraction network and a second classifier, can be trained based on the second training data and the soft labels of the second training data. This allows the model to learn the relevant information of the second training data and the soft labels of the second training data more effectively, ensuring the learning effect of the sensitive information recognition model and thus improving its performance.

[0179] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0180] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0181] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0182] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0183] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0184] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as methods for identifying sensitive information and methods for training models. For example, in some embodiments, the methods for identifying sensitive information and methods for training models can be implemented as computer software programs, which are tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the methods for identifying sensitive information and methods for training models described above can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured in any other suitable manner (e.g., by means of firmware) to perform a method for identifying sensitive information or a method for training a model.

[0185] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0186] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0187] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0188] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0189] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0190] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

Claims

1. A training method for a sensitive information identification model, comprising: Obtain first training data and second training data, wherein the first training data is labeled data and the second training data is unlabeled data; Based on the first training data, the pre-trained model to be trained is iteratively trained to obtain a pre-trained model that has been trained; the pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier; Using the pre-trained model, the second training data is processed to obtain soft labels for the second training data. The soft labels for the second training data include a sequence of sentence entity start position IDs, a sequence of sentence entity end position IDs, and the probability corresponding to each label. The sequence of sentence entity start position IDs and the sequence of sentence entity end position IDs are obtained as follows: the sample webpage text in the second training data is processed into sentence, word, and character segments to obtain processed text data including sentence entities, word entities, and character entities; the processed text data is annotated using BIO annotation rules to obtain annotation labels; each character entity in the segmented and character-segmented text data is mapped to a number to obtain a sentence ID sequence; the sequence of sentence entity start position IDs and the sequence of sentence entity end position IDs are obtained based on the start and end positions of each character entity in the sentence ID sequence and the annotation label ID of the character entity. Based on the second training data and the soft labels of the second training data, the sensitive information recognition model to be trained is iteratively trained to obtain the sensitive information recognition model, which includes a second text feature extraction network and a second classifier.

2. The method according to claim 1, wherein, The first training data includes sample entity information and annotation labels for sample webpage text. The step of iteratively training the pre-trained model to be trained based on the first training data to obtain the trained pre-trained model includes: The first training data is input into the pre-trained network to obtain the word vector features corresponding to the first training data; The word vector features are input into the first text feature extraction network to obtain the context feature information corresponding to the word vector features; The contextual feature information is input into the first classifier to obtain the predicted label of the sample entity information; Based on the labeled and predicted labels of the sample entity information, the pre-trained model to be trained is iteratively trained to obtain the pre-trained model after training.

3. The method according to claim 1 or 2, wherein, The step of iteratively training the sensitive information recognition model to be trained based on the second training data and the soft labels of the second training data to obtain the sensitive information recognition model includes: The second training data is input into the second text feature extraction network to obtain the contextual feature information of the second training data; The contextual feature information of the second training data is input into the second classifier to obtain the predicted label of the second training data; Based on the soft labels and predicted labels of the second training data, the sensitive information recognition model to be trained is iteratively trained to obtain the sensitive information recognition model.

4. A method for identifying sensitive information, comprising: Obtain entity information from the target webpage text; The entity information is identified using a pre-defined sensitive information identification model to obtain the identification result of the entity information. The pre-defined sensitive information identification model is obtained by knowledge distillation of a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The pre-defined sensitive information identification model also includes a second text feature extraction network and a second classifier. The sensitive information identification model is the sensitive information identification model trained by the method described in any one of claims 1-3. Based on the recognition results of the entity information, the sensitive information recognition results of the target webpage text are obtained.

5. The method according to claim 4, wherein, The acquisition of entity information from the target webpage text includes: Obtain the target webpage text; The target webpage text is segmented to obtain entity information of the target webpage text.

6. The method according to claim 5, wherein, The segmentation process of the target webpage text to obtain entity information of the target webpage text includes: The target webpage text is segmented to obtain word segments and character segments. Using a preset mapping algorithm, the word segmentation and character segmentation of the target webpage text are mapped to obtain the sentence identifier sequence of the target webpage text; Based on the sentence identifier sequence, the entity information of the target webpage text is obtained.

7. The method according to any one of claims 4-6, wherein, The step of using a preset sensitive information recognition model to identify and process the entity information to obtain the identification result of the entity information includes: The entity information is input into the second text feature extraction network to obtain the contextual feature information of the entity information; The contextual feature information is input into the second classifier to obtain the recognition result of the entity information.

8. The method according to any one of claims 4-6, wherein, The second text feature extraction network includes a BiLSTM-based network; the second classifier includes a Span-based classifier.

9. A training device for a sensitive information identification model, comprising: A data acquisition unit is used to acquire first training data and second training data, wherein the first training data is labeled data and the second training data is unlabeled data; The first training unit is used to iteratively train the pre-trained model to be trained based on the first training data to obtain the pre-trained model after training; the pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The data recognition unit is used to process the second training data using a pre-trained model to obtain soft labels for the second training data. The soft labels for the second training data include a sequence of sentence entity start position IDs, a sequence of sentence entity end position IDs, and the probability corresponding to each label. The sequence of sentence entity start position IDs and the sequence of sentence entity end position IDs are obtained as follows: The sample webpage text in the second training data is processed into sentence, word, and character segments to obtain processed text data including sentence entities, word entities, and character entities; the processed text data is annotated using BIO annotation rules to obtain annotation labels; each character entity in the segmented and character-segmented text data is mapped to a number to obtain a sentence ID sequence; the sequence of sentence entity start position IDs and the sequence of sentence entity end position IDs are obtained based on the start and end positions of each character entity in the sentence ID sequence and the annotation label ID of the character entity. The second training unit is used to iteratively train the sensitive information recognition model to be trained based on the second training data and the soft labels of the second training data, so as to obtain the sensitive information recognition model, which includes a second text feature extraction network and a second classifier.

10. The apparatus according to claim 9, wherein, The first training data includes sample entity information and annotation labels for sample webpage text. The first training unit is specifically used for: The first training data is input into the pre-trained network to obtain the word vector features corresponding to the first training data; The word vector features are input into the first text feature extraction network to obtain the context feature information corresponding to the word vector features; The contextual feature information is input into the first classifier to obtain the predicted label of the sample entity information; Based on the labeled and predicted labels of the sample entity information, the pre-trained model to be trained is iteratively trained to obtain the pre-trained model after training.

11. The apparatus according to claim 9 or 10, wherein, The second training unit is specifically used for: The second training data is input into the second text feature extraction network to obtain the contextual feature information of the second training data; The contextual feature information of the second training data is input into the second classifier to obtain the predicted label of the second training data; as well as, Based on the soft labels and predicted labels of the second training data, the sensitive information recognition model to be trained is iteratively trained to obtain the sensitive information recognition model.

12. A device for identifying sensitive information, comprising: The acquisition unit is used to acquire entity information of the target webpage text; The identification unit is used to identify the entity information using a preset sensitive information identification model to obtain the identification result of the entity information. The preset sensitive information identification model is obtained by knowledge distillation of a pre-trained model. The pre-trained model includes a pre-trained network, a first text feature extraction network, and a first classifier. The preset sensitive information identification model also includes a second text feature extraction network and a second classifier. The sensitive information identification model is a sensitive information identification model trained by the device according to any one of claims 9-11. The obtaining unit is used to obtain the sensitive information identification result of the target webpage text based on the identification result of the entity information.

13. The apparatus according to claim 12, wherein, The acquisition unit is specifically used for: Obtain the target webpage text; The target webpage text is segmented to obtain entity information of the target webpage text.

14. The apparatus according to claim 13, wherein, The acquisition unit is further configured to: The target webpage text is segmented to obtain word segments and character segments. Using a preset mapping algorithm, the word segmentation and character segmentation of the target webpage text are mapped to obtain the sentence identifier sequence of the target webpage text; and Based on the sentence identifier sequence, the entity information of the target webpage text is obtained.

15. The apparatus according to any one of claims 12 to 14, wherein, The identification unit is specifically used for: The entity information is input into the second text feature extraction network to obtain the contextual feature information of the entity information; The contextual feature information is input into the second classifier to obtain the recognition result of the entity information.

16. The apparatus according to any one of claims 12 to 14, wherein, The second text feature extraction network includes a BiLSTM-based network; the second classifier includes a Span-based classifier.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images