A method and device for automatically labeling sensitive information based on unsupervised learning

Abstract

The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for automatic labeling of sensitive information based on unsupervised learning. The method for automatic labeling of sensitive information based on unsupervised learning includes: obtaining input data; performing word segmentation preprocessing on the input data to obtain preprocessed data; generating word embeddings on the preprocessed data using an autoencoder model; constructing a graph model based on the preprocessed data; performing graph-based statistical analysis on the graph model to obtain candidate sensitive information; screening candidate sensitive information based on an adaptive labeling rule generation mechanism to obtain sensitive information; determining sensitive information nodes based on the graph model according to the sensitive information; integrating word embeddings, analysis results and sensitive information nodes to obtain a hybrid model; and making sensitive information labeling decisions based on the hybrid model. The present invention is a method and system that can effectively, comprehensively and automatically label sensitive information.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for automatically labeling sensitive information based on unsupervised learning. Background Art

[0002] In today's era of information explosion, the secure processing and management of sensitive information has become a growing concern. Traditional sensitive information detection and annotation methods mostly rely on pre-defined rules or learning patterns from annotated data through supervised learning. However, these methods have some obvious limitations. First, pre-defined rules may not cover all types and forms of sensitive information, and need to be frequently updated to adapt to the ever-changing information environment. Second, supervised learning methods require a large amount of annotated data, which is not only costly but also unrealistic in many cases.

[0003] In addition, existing methods often only focus on information at a single level (single word or vocabulary), ignoring the complex relationships that may exist between different levels of information. For example, a word may be sensitive in one context, but completely harmless in another context. In addition, single-level methods also tend to ignore the intrinsic connection between vocabulary and single words in text data, thereby reducing the accuracy and comprehensiveness of sensitive information detection.

[0004] Therefore, there is an urgent need to develop a method and system that can effectively, comprehensively and automatically mark sensitive information. Summary of the invention

[0005] In order to solve the technical problem of ignoring the complex relationship between information at different levels in the prior art, the embodiment of the present invention provides a method and device for automatically labeling sensitive information based on unsupervised learning. The technical solution is as follows:

[0006] On the one hand, a method for automatic labeling of sensitive information based on unsupervised learning is provided, the method being implemented by a sensitive information automatic labeling device, the method comprising:

[0007] Acquire input data; perform word segmentation preprocessing on the input data to obtain preprocessed data; the word segmentation preprocessing includes word level preprocessing and word level preprocessing; the preprocessed data includes word level information, word level information, inter-word relationships, and inter-word relationships;

[0008] Generate word embeddings on the preprocessed data using an autoencoder model; the autoencoder model is used to generate high-dimensional word representations based on word-level information and vocabulary-level information;

[0009] A graph model is constructed according to the preprocessed data; the nodes of the graph model represent word-level information or vocabulary-level information from the preprocessed data; the edges of the graph model represent the relationship between words or the relationship between words;

[0010] Performing graph-based statistical analysis on the graph model to obtain candidate sensitive information;

[0011] Based on the adaptive annotation rule generation mechanism, the candidate sensitive information is screened to obtain sensitive information; based on the graph model, sensitive information nodes are determined according to the sensitive information; the adaptive annotation rule generation mechanism optimizes the annotation rules through iteration; the adaptive annotation rules are used to match sensitive information at the single word level and the vocabulary level;

[0012] A hybrid model is obtained by integrating the word embedding, the analysis results and the sensitive information nodes; sensitive information labeling decisions are made based on the hybrid model; and the hybrid model is a multi-layer perceptron.

[0013] The encoder of the autoencoder is used to convert the word-level information and the vocabulary-level information into a latent space vector;

[0014] The hybrid fusion layer of the autoencoder is used to fuse the latent space vector into a fused latent space vector; the hybrid fusion layer uses an attention mechanism to merge the word-level information and the vocabulary-level information

[0015] The decoder of the autoencoder is used to generate word embedding according to the fused latent space vector.

[0016] Wherein, the encoder adopts an adaptive learning rate adjustment mechanism; the adaptive learning rate adjustment mechanism dynamically adjusts the learning rate according to the complexity of the input data.

[0017] Wherein, the decoder comprises a decomposition conversion layer, an information reconstruction layer and a multimodal generation network;

[0018] The decomposition transformation layer is used to expand the fused latent space vector from the hybrid fusion layer;

[0019] The information reconstruction layer is used for nonlinear conversion of the convolutional transposition layer or the fully connected layer and outputs reconstructed single word level information and vocabulary level information;

[0020] The multimodal generative network is used to generate multimodal word embeddings according to the latent space vector.

[0021] Optionally, constructing a graph model according to the preprocessed data includes:

[0022] Traversing the preprocessed data, creating nodes according to the single-word level information and the word-level information in the preprocessed data; determining the single-word level information and the word-level information corresponding to the node as node attributes;

[0023] Determine the relationship between nodes according to the nodes and the node attributes, construct edges according to the relationship between nodes and calculate edge weights; the edge weights are determined according to the physical distance, cosine similarity and co-occurrence probability of the nodes;

[0024] A graph model is constructed according to the nodes, the edges, and the edge weights.

[0025] Optionally, performing a graph-based statistical analysis on the graph model to obtain candidate sensitive information includes:

[0026] Calculating according to the nodes of the graph model to obtain the number of edges directly connected to the nodes; obtaining the degree of the node according to the number of edges; and storing the degree of the node in a hash table;

[0027] According to the nodes of the graph model, the neighbor nodes of the nodes are obtained; according to the nodes and the neighbor nodes, the actual number of edges and the maximum number of edges are calculated; according to the actual number of edges and the maximum number of edges, the clustering coefficient of the nodes is calculated; and the clustering coefficient is stored in a hash table;

[0028] Traverse the hash table storing the degree of the node and the clustering coefficient of the node, mark the nodes of the graph model according to a preset threshold, and obtain candidate sensitive information nodes; according to the graph model and the candidate sensitive information nodes, obtain candidate sensitive information corresponding to the candidate sensitive information nodes.

[0029] Optionally, the screening of the candidate sensitive information based on the adaptive annotation rule generation mechanism to obtain the sensitive information includes:

[0030] In the initialization phase, a preliminary rule set is created; the preliminary rule set includes at least one regular expression and at least one keyword list; initialization is performed according to the preliminary rule set; a sensitive information hash table is created;

[0031] In the rule optimization stage, the preliminary rule set is evaluated by F1 score to obtain an F1 score result; the preliminary rule set is optimized according to the F1 score result to obtain an optimized rule set;

[0032] In the sensitive information determination stage, the candidate sensitive information is screened based on the optimized rule set to obtain screened sensitive information; the screened sensitive information is cross-validated and manually reviewed by experts to obtain sensitive information; and the sensitive information is stored in the sensitive information hash table.

[0033] On the other hand, a device for automatically labeling sensitive information based on unsupervised learning is provided, and the device is applied to a method for automatically labeling sensitive information based on unsupervised learning, and the device comprises:

[0034] A processing unit is used to obtain input data; perform word segmentation preprocessing on the input data to obtain preprocessed data; the word segmentation preprocessing includes word level preprocessing and word level preprocessing; the preprocessed data includes word level information, word level information, inter-word relations and inter-word relations;

[0035] A generating unit, configured to generate word embeddings on the preprocessed data using an autoencoder model; the autoencoder model is configured to generate high-dimensional word representations based on word-level information and vocabulary-level information;

[0036] A construction unit, configured to construct a graph model according to the preprocessed data; the nodes of the graph model represent word-level information or vocabulary-level information from the preprocessed data; the edges of the graph model represent the relationship between words or the relationship between words;

[0037] An analysis unit, configured to perform a graph-based statistical analysis on the graph model to obtain candidate sensitive information;

[0038] A determination unit is used to screen the candidate sensitive information based on an adaptive annotation rule generation mechanism to obtain sensitive information; based on the graph model, determine sensitive information nodes according to the sensitive information; the adaptive annotation rule generation mechanism optimizes the annotation rules through iteration; the adaptive annotation rules are used to match sensitive information at the single word level and the vocabulary level;

[0039] An integration unit is used to integrate the word embedding, the analysis result and the sensitive information node to obtain a hybrid model; and make sensitive information labeling decisions based on the hybrid model; the hybrid model is a multi-layer perceptron.

[0040] The encoder of the autoencoder is used to convert the word-level information and the vocabulary-level information into a latent space vector;

[0041] The hybrid fusion layer of the autoencoder is used to fuse the latent space vector into a fused latent space vector; the hybrid fusion layer uses an attention mechanism to merge the single word level information and the vocabulary level information;

[0042] The decoder of the autoencoder is used to generate word embedding according to the fused latent space vector.

[0043] Wherein, the encoder adopts an adaptive learning rate adjustment mechanism; the adaptive learning rate adjustment mechanism dynamically adjusts the learning rate according to the complexity of the input data.

[0044] Wherein, the decoder comprises a decomposition conversion layer, an information reconstruction layer and a multimodal generation network;

[0045] The decomposition transformation layer is used to expand the fused latent space vector from the hybrid fusion layer;

[0046] The information reconstruction layer is used for nonlinear conversion of the convolutional transposition layer or the fully connected layer and outputs reconstructed single word level information and vocabulary level information;

[0047] The multimodal generative network is used to generate multimodal word embeddings according to the latent space vector.

[0048] Optionally, the construction unit is further used to:

[0049] Traversing the preprocessed data, creating nodes according to the single-word level information and the word-level information in the preprocessed data; determining the single-word level information and the word-level information corresponding to the node as node attributes;

[0050] Determine the relationship between nodes according to the nodes and the node attributes, construct edges according to the relationship between nodes and calculate edge weights; the edge weights are determined according to the physical distance, cosine similarity and co-occurrence probability of the nodes;

[0051] A graph model is constructed according to the nodes, the edges, and the edge weights.

[0052] Optionally, the analysis unit is further used for:

[0053] Calculating according to the nodes of the graph model to obtain the number of edges directly connected to the nodes; obtaining the degree of the node according to the number of edges; and storing the degree of the node in a hash table;

[0054] According to the nodes of the graph model, the neighbor nodes of the nodes are obtained; according to the nodes and the neighbor nodes, the actual number of edges and the maximum number of edges are calculated; according to the actual number of edges and the maximum number of edges, the clustering coefficient of the nodes is calculated; and the clustering coefficient is stored in a hash table;

[0055] Traverse the hash table storing the degree of the node and the clustering coefficient of the node, mark the nodes of the graph model according to a preset threshold, and obtain candidate sensitive information nodes; according to the graph model and the candidate sensitive information nodes, obtain candidate sensitive information corresponding to the candidate sensitive information nodes.

[0056] Optionally, the determining unit is further configured to:

[0057] In the initialization phase, a preliminary rule set is created; the preliminary rule set includes at least one regular expression and at least one keyword list; initialization is performed according to the preliminary rule set; a sensitive information hash table is created;

[0058] In the rule optimization stage, the preliminary rule set is evaluated by F1 score to obtain an F1 score result; the preliminary rule set is optimized according to the F1 score result to obtain an optimized rule set;

[0059] In the sensitive information determination stage, the candidate sensitive information is screened based on the optimized rule set to obtain screened sensitive information; the screened sensitive information is cross-validated and manually reviewed by experts to obtain sensitive information; and the sensitive information is stored in the sensitive information hash table.

[0060] On the other hand, a sensitive information automatic labeling device is provided, which includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned sensitive information automatic labeling methods based on unsupervised learning is implemented.

[0061] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement any of the above-mentioned methods for automatic labeling of sensitive information based on unsupervised learning.

[0062] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0063] The present invention proposes a method for automatic labeling of sensitive information based on unsupervised learning. By fusing word-level and vocabulary-level information, the generated high-dimensional word representation captures the semantic and structural features of the text more comprehensively. Constructing a graph model and performing graph-based statistical analysis can take into account the relationship between words or vocabulary, so that the system can not only identify a single sensitive word, but also identify sensitive information composed of multiple words. Since it is based on unsupervised learning, this method does not rely on a large amount of labeled data, which reduces the implementation cost, and is also suitable for occasions where it is difficult to obtain a large amount of labeled data. The present invention is a method and system that can effectively, comprehensively and automatically label sensitive information. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0065] Figure 1 This is a flow chart of a method for automatically labeling sensitive information based on unsupervised learning provided by an embodiment of the present invention;

[0066] Figure 2 It is a block diagram of a sensitive information automatic labeling device based on unsupervised learning provided by an embodiment of the present invention;

[0067] Figure 3 It is a structural schematic diagram of a sensitive information automatic labeling device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0069] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0070] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0071] In the embodiments of the present invention, sometimes a subscript such as W1 may be expressed in a non-subscript form such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0072] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0073] The embodiment of the present invention provides a method for automatically labeling sensitive information based on unsupervised learning. The method can be implemented by an automatic labeling device for sensitive information, which can be a terminal or a server. Figure 1 The flowchart of the method for automatic labeling of sensitive information based on unsupervised learning is shown in the figure. The processing flow of the method may include the following steps:

[0074] S1. Obtain input data; perform word segmentation preprocessing on the input data to obtain preprocessed data; the word segmentation preprocessing includes word-level preprocessing and vocabulary-level preprocessing; the preprocessed data includes word-level information, vocabulary-level information, relationships between words, and relationships between words.

[0075] In a feasible implementation manner, the present invention lays a solid foundation for subsequent sensitive information labeling work in this step, and enhances the accuracy and sensitivity of potential sensitive information through precise analysis of text data.

[0076] Input data refers to the raw text that needs to be labeled with sensitive information. The raw text may come from a variety of channels, including but not limited to social media posts, emails, online forums, news articles, or user-generated comments. Raw text data is usually in unstructured or semi-structured format and may contain plain text, punctuation, or even HTML or XML tags. Therefore, before entering this step, the input data usually needs to undergo preliminary cleaning and standardization, such as denoising, removing redundant tags, or standardized encoding.

[0077] The single-word segmentation includes the following steps:

[0078] Character parsing: Parsing the cleaned and standardized input data into single characters, including Chinese characters, punctuation marks, and numbers.

[0079] Data standardization is further performed to remove redundant spaces, punctuation marks, and other non-text symbols in order to standardize the parsed characters.

[0080] Encoding conversion is carried out to encode the standardized characters through a predefined encoding format (such as UTF-8) for convenient subsequent processing.

[0081] Lexical-level word segmentation includes the following steps:

[0082] Initial word segmentation is performed on the text after standardization and encoding conversion using an unsupervised learning-based word segmentation algorithm for initial lexical-level word segmentation.

[0083] After receiving the text, standardization and encoding conversion are first required to ensure that all characters are converted into a unified standard format, such as UTF-8. This is to prevent potential problems caused by different encodings or character sets. After standardization and encoding conversion, an unsupervised learning-based word segmentation algorithm is used for initial word segmentation. The unsupervised learning algorithm here can be a dictionary-based method, or a statistical or deep learning model. The core advantage of unsupervised learning is that it does not rely on labeled data and is therefore more general.

[0084] Part-of-speech tagging is carried out based on the results of initial word segmentation, which helps with subsequent precise analysis.

[0085] Part-of-speech tagging is an optional processing step, but part-of-speech tagging can provide more precise subsequent analysis in many situations. Part-of-speech tagging means, based on initial word segmentation, attaching part-of-speech tags to each word, such as "noun", "verb", "adjective", etc. For Chinese, there are also some special parts of speech, such as "measure word". Part-of-speech tagging is usually completed using machine learning models such as conditional random fields (CRF), hidden Markov models (HMM), etc., and these models need to be trained and optimized for Chinese.

[0086] Lexical filtering is carried out to filter out words that are not relevant to subsequent analysis through a predefined dictionary or rules, such as common stop words.

[0087] Lexical filtering is to improve the efficiency and accuracy of subsequent analysis. Through a predefined dictionary or rules, words that are not relevant to the purpose can be filtered out. For example, in the context of sensitive information identification, some common stop words that do not contain sensitive information, such as "de", "shi", "zai", etc., may need to be filtered out.

[0088] Polysemy disambiguation is carried out through context information to ensure the accuracy of words.

[0089] There are many polysemous words in Chinese, and these words have different meanings in different contexts. For example, the word "行" has completely different meanings in "银行" (bank) and "行走" (walk). Therefore, polysemous word disambiguation has become a crucial step. This is usually achieved by examining the context of the polysemous word. Through context information, such as adjacent words before and after, or broader text fragments, the most likely meaning of the polysemous word in the current context can be determined.

[0090] After word segmentation at the single-character level and lexical level is completed, it is also necessary to fuse the word segmentation results at the single-character level and lexical level. In addition, the data formats of the word segmentation results at the single-character level and lexical level need to be unified to facilitate subsequent analysis.

[0091] Through this series of meticulous steps, preprocessed data suitable for subsequent analysis has been successfully generated.

[0092] The preprocessed data suitable for subsequent analysis mainly includes the following parts:

[0093] Word segmentation results: This includes word segmentation results at the single-character level (or character level) and lexical level. In short, the text is cut into smaller and more easily analyzable fragments.

[0094] Fused word segmentation data: The word segmentation results at the single-character level and lexical level are fused to form a more comprehensive and integrated text representation.

[0095] Data standardization: All word segmentation results will be converted or organized into a unified data format. This is mainly to facilitate subsequent automatic annotation and other analysis tasks.

[0096] Part-of-speech tags: If part-of-speech tagging is performed, each word (single character or multi-word phrase) will be followed by one or more part-of-speech tags, such as "noun", "verb", etc.

[0097] Metadata: In addition to the text content and part-of-speech tags, the preprocessed data may also contain other metadata, such as context relationships, syntactic structures, semantic roles, etc. These metadata help with subsequent sensitive information identification and annotation.

[0098] Filtered and optimized information: The relevant information generated in the steps of lexical filtering and polysemous word disambiguation will also be included. These information help reduce misjudgments and improve annotation accuracy.

[0099] Extra markers or tags: For example, in certain application scenarios, specific types of words (such as proper nouns, sensitive words, etc.) may also be specially marked or annotated.

[0100] Therefore, the preprocessed data is a multi-level and multi-dimensional information set, which includes not only the basic text content, but also rich additional information that is helpful for subsequent analysis and processing. These comprehensive data will greatly help improve the accuracy and effectiveness of subsequent automatic labeling of sensitive information.

[0101] S2. Generate word embeddings on the preprocessed data using an autoencoder model; the autoencoder model is used to generate high-dimensional word representations based on word-level information and vocabulary-level information.

[0102] In one possible implementation, word embedding is a technique that maps words or phrases from a vocabulary (i.e., the set of all independent words that appear in text data) to a high-dimensional vector space. These embeddings can capture the semantic content, contextual similarity, relationships, and even more abstract properties of words. This is a common method for simplifying text data so that it can be used in machine learning models.

[0103] The "high dimension" in high-dimensional word representation is usually relative to the original data. Text data usually appears in the form of natural language, which is not easy for computers to process. Through word embedding, a high-dimensional space (usually hundreds of dimensions) can be obtained, in which each dimension captures different characteristics or information of the word in some aspect (such as semantics, syntax, etc.). Therefore, a high-dimensional word representation is a vector containing multiple elements, each of which carries information about the characteristics of the word.

[0104] The main goal of this step is to generate a semantically rich representation of words and vocabulary. This representation not only captures the syntactic structure but also the semantic context, which is very useful for identifying potential sensitive information in subsequent steps.

[0105] Among them, the encoder of the autoencoder is used to convert word-level information and vocabulary-level information into latent space vectors;

[0106] The hybrid fusion layer of the autoencoder is used to fuse the latent space vector into a fused latent space vector; the hybrid fusion layer uses the attention mechanism to merge the single word level information and the vocabulary level information;

[0107] The decoder of the autoencoder is used to generate word embeddings based on the fused latent space vectors.

[0108] In one possible implementation, the autoencoder model consists of two main components: an encoder and a decoder. The autoencoder model is an unsupervised learning model. Unlike the traditional encoder-decoder architecture, this autoencoder uses a novel mechanism that allows it to input and learn from both character-level and word-level elements. In order to fuse this two-level information, the model uses an innovative design called a "hybrid fusion layer", which is crucial for generating high-dimensional word embeddings.

[0109] The autoencoder model consists of an encoder, a hybrid fusion layer, and a decoder.

[0110] The encoder takes in the character-level and word-level preprocessed data and converts them into initial latent space vectors. Two separate initial encoders may be used for each preprocessing level. The outputs of these initial encoders are then fed into the hybrid fusion layer.

[0111] The encoder receives character-level and word-level data from the preprocessed data. This data goes through a series of preprocessing steps, including tokenization, removal of punctuation and other unnecessary elements.

[0112] The encoder outputs two sets of latent space vectors, corresponding to character-level and word-level information respectively. These latent space vectors are generated by a series of hidden layers that use activation functions such as ReLU or Sigmoid for nonlinear transformation.

[0113] Among them, the encoder adopts an adaptive learning rate adjustment mechanism; the adaptive learning rate adjustment mechanism dynamically adjusts the learning rate according to the complexity of the input data.

[0114] In one possible implementation, an adaptive learning rate adjustment mechanism is designed to dynamically adjust the learning rate of the neural network based on the characteristics of the input data. This mechanism specifically takes into account the complexity of the data, including but not limited to vocabulary diversity and sentence length. Doing so helps the network better adapt to data of different types and complexities.

[0115] To quantify the complexity of data, a variety of indicators can be used, such as:

[0116] Character diversity: measured by counting the number of unique characters in the input data and dividing by the total number of words.

[0117] Sentence length: measured by calculating the average number of words per sentence in the input data.

[0118] These metrics can be used to calculate a "complexity score" that is used as a basis for adjusting the learning rate.

[0119] The complexity score S is a value between 0 and 1 that quantifies the complexity of the input data. This score combines several factors that affect complexity, such as vocabulary diversity and sentence length.

[0120] The calculation method includes calculating character diversity and sentence length:

[0121] Character diversity: Calculate the number of all different characters in the input data and divide it by the total number of characters. Assuming the number of all different characters is V and the total number of characters is N, the character diversity index is V / N.

[0122] Sentence length: Calculate the average number of characters in all sentences in the input data. Assume there are M sentences, and the number of characters in each sentence is len1, len2, ..., len M , then the sentence length index is (len1+len2+...+len M ) / M.

[0123] After calculating these two indicators, the complexity score can be obtained by weighted average. Assuming that the weights of character diversity and sentence length are w1 and w2 respectively, the calculation formula of the complexity score S is shown in the following formula (1):

[0124]

[0125] Dynamically adjust strategies

[0126] When the "complexity score" exceeds or falls below a certain threshold, the learning rate will be adjusted appropriately. For example, when the input data is very complex (high vocabulary diversity or long sentence length), the learning rate can be reduced to avoid overfitting. On the contrary, when the input data is relatively simple, the learning rate can be increased to accelerate model convergence.

[0127] Set a threshold that is used to determine whether the data complexity is high or low enough to decide whether the learning rate needs to be adjusted. For example, you can set the threshold to 0.5.

[0128] When the complexity score exceeds a threshold (for example, greater than 0.5), the learning rate will be reduced. This is to make the model more cautious in updating parameters to avoid overfitting.

[0129] When the complexity score is below a threshold (e.g., less than or equal to 0.5), the learning rate will be increased. This is done to speed up the convergence of the model.

[0130] The adjustment range of the learning rate can be controlled by a factor, which is called the learning rate adjustment factor. For example, the factor can be set to 0.1. Then when the learning rate needs to be reduced, the new learning rate will be 1-0.1=0.9 times the original learning rate; correspondingly, when the learning rate needs to be increased, the new learning rate will be 1+0.1=1.1 times the original learning rate.

[0131] The following is a sample Python code for the encoder:

[0132]

[0133]

[0134] Among them, the decoder includes a decomposition conversion layer, an information reconstruction layer and a multimodal generation network;

[0135] The decomposition transformation layer is used to expand the fused latent space vector from the hybrid fusion layer;

[0136] The information reconstruction layer is used for nonlinear transformation of the convolutional transposition layer or the fully connected layer and outputs the reconstructed single-word level information and vocabulary level information;

[0137] The multimodal generative network is used to generate multimodal word embeddings based on the latent space vectors.

[0138] In one possible implementation, the decoder receives the fused latent space vector from the hybrid fusion layer.

[0139] The decoder outputs a reconstruction of the original input, but more importantly, this fused latent space vector is now a high-dimensional word embedding.

[0140] The decoder may use a multimodal generative network that is able to reconstruct character-level and word-level inputs based on the fused latent space vectors.

[0141] The specific implementation of the decoder is as follows:

[0142] Decomposition conversion layer

[0143] Input: Accepts the fused latent space vector from the hybrid fusion layer.

[0144] Operation: Use fully connected layers and non-linear activation functions (such as ReLU) to expand the fused latent space vector.

[0145] Output: Get an expanded medium-sized tensor.

[0146] Information reconstruction layer

[0147] Input: Medium-sized tensors from factorization and transformation layers.

[0148] Operation: Use convolutional transpose layers (also called deconvolution) or fully connected layers to perform nonlinear transformations via Softmax or Sigmoid activation functions.

[0149] Output: Reconstructed character-level and word-level input, which are saved in two different tensors.

[0150] Multimodal Generative Network

[0151] Input: Two reconstructed tensors (character level and word level).

[0152] Operation: Further use Generative Adversarial Network (GAN) or Variational Autoencoder (VAE) architecture to generate multi-modal outputs as needed.

[0153] Output: The fused latent space vector is converted into a high-dimensional word embedding, which can be used for downstream tasks.

[0154] The following is an example code of the decoder, which is implemented using PYTHON language and tensorflow.

[0155]

[0156]

[0157]

[0158]

[0159] In this code example:

[0160] The DecompositionAndTransformationLayer receives the fused latent space vector and outputs an expanded tensor.

[0161] The InformationReconstructionLayer uses the expanded tensors to reconstruct character-level and word-level inputs.

[0162] The MultimodalGenerationNetwork uses a generative adversarial network (GAN) architecture, receives a reconstructed tensor and produces a multimodal output.

[0163] When training an autoencoder model (including encoder, hybrid fusion layer and decoder), the general goal is to minimize the reconstruction error and increase the multimodal generation capability through generative adversarial network (GAN).

[0164] Initialization: Initialize the encoder, hybrid fusion layer, and decoder models.

[0165] Choose appropriate loss functions such as Mean Squared Error (MSE) for reconstruction loss and Binary Cross-Entropy for the discriminator loss of Generative Adversarial Networks (GAN).

[0166] Select an optimization algorithm, such as the Adam optimizer.

[0167] Data preparation: Prepare input data, which includes preprocessed character-level and word-level data.

[0168] Training loop: In each training cycle (epoch), for each input sample, the following steps are performed:

[0169] Forward propagation: The input sample passes through the encoder and the hybrid fusion layer to generate a fused latent space vector.

[0170] Decoding: The fused latent space vector is decoded by a decoder to generate multimodal outputs and corresponding discriminator scores.

[0171] Calculate the loss: reconstruction loss, which calculates the error between the reconstructed output and the original input.

[0172] Discriminator loss, which calculates how well the discriminator classifies the generated output and the real data.

[0173] Backpropagation and optimization: Calculate the gradient of the loss function with respect to the model parameters and use the optimizer to update the parameters.

[0174] S3. Construct a graph model based on the preprocessed data; the nodes of the graph model represent word-level information or vocabulary-level information from the preprocessed data; the edges of the graph model represent the relationship between words or the relationship between words.

[0175] Optionally, a graph model is constructed based on the preprocessed data, including:

[0176] Traversing the preprocessed data, creating nodes according to the single word level information and the word level information in the preprocessed data; determining the single word level information and the word level information corresponding to the node as node attributes;

[0177] Determine the relationship between nodes based on the nodes and their attributes, build edges based on the relationship between nodes and calculate edge weights; edge weights are determined based on the physical distance, cosine similarity, and co-occurrence probability of the nodes;

[0178] Construct a graph model based on nodes, edges, and edge weights.

[0179] In one feasible implementation, each node in the graph model corresponds to a single word or a word in the preprocessed data. When creating these nodes, the high-dimensional word representations of these words or words (that is, the word embeddings generated by the autoencoder model) are used as node attributes. In this way, each node contains not only the text information of the original data, but also the contextual information obtained through unsupervised learning, which is crucial for subsequent analysis.

[0180] The edges in the graph model are used to represent the relationships between words or terms in the preprocessed data. These relationships can be based on a variety of factors, such as the distance between two words or terms in the original text, their similarity in the word embedding space, or the probability of their co-occurrence under a specific language model. The weight of the edge can be calculated by a linear or nonlinear combination of these factors.

[0181] In the process of building a graph model, edges are used to represent the relationships between words or vocabulary in the preprocessed data. The weight calculation of these relationships involves multiple factors, namely:

[0182] Distance in text, which is the physical distance between two words or terms in the original text, usually in the number of characters or words. For example, in the sentence "Apple is a fruit", the distance between "apple" and "fruit" is 3 words.

[0183] Similarity in word embedding space, which is the similarity between the word embedding vectors of two words or vocabulary. This is usually quantified by cosine similarity or Euclidean distance.

[0184] Co-occurrence probability, which is the probability that two words or terms co-occur in a particular corpus or language model. This can be calculated by counting the number of times they appear in the same context window (such as a sentence or a paragraph) and then dividing it by the total number of context windows.

[0185] The calculation of weights can be a linear or nonlinear combination, the specific form depends on the requirements of the task and the characteristics of the data.

[0186] Linear combination: In the simplest case, the weights of the various factors can be added directly. For example, assuming the distance weight in the text is w1, the similarity weight in the word embedding space is w2, and the co-occurrence probability weight is w3, then the total weight W of the edge can be calculated as W = w1 + w2 + w3.

[0187] Nonlinear combination: In more complex cases, a function f can be used for nonlinear combination, such as W = f(w1,w2,w3). Here f can be any appropriate nonlinear function, such as Sigmoid function or other custom neural network.

[0188] The construction process of the graph model includes the following steps:

[0189] Traverse the preprocessed data, create a node for each word and vocabulary, and store its corresponding word embedding as a node attribute.

[0190] For each node, find other nodes that are directly related to it. Specifically, this can be achieved by measuring cosine similarity, Euclidean distance in word embedding space, or through some predefined semantic rules. After performing the above calculations, select other nodes that are most directly related to the current node according to a set threshold or sorting mechanism. For example, you can select all nodes with a cosine similarity greater than 0.8, or all nodes with a Euclidean distance less than a certain value.

[0191] According to the mutual relationship found in the previous step, add an edge between each pair of related nodes and calculate the weight of the edge.

[0192] Integrate all created nodes and edges into a unified graph model.

[0193] In the graph model constructed in this way, each node not only carries the information of the word or vocabulary itself, but also reflects its relationship with the surrounding words or vocabulary, providing rich and structured information for subsequent graph-based statistical analysis.

[0194] In the above way, the constructed graph model will be a highly complex network with a large number of nodes and edges, which is suitable for the subsequent automatic labeling of sensitive information. Such a graph model can not only capture the characteristics of individual words or words themselves, but also capture their mutual relationship in a larger text environment, which is critical for accurately identifying sensitive information.

[0195] S4. Perform graph-based statistical analysis on the graph model to obtain candidate sensitive information.

[0196] Optionally, a graph-based statistical analysis is performed on the graph model to obtain candidate sensitive information, including:

[0197] Calculate the number of edges directly connected to the nodes based on the nodes of the graph model; obtain the degree of the node based on the number of edges; store the degree of the node in a hash table;

[0198] According to the nodes of the graph model, the neighbor nodes of the nodes are obtained; according to the nodes and the neighbor nodes, the actual number of edges and the maximum number of edges are calculated; according to the actual number of edges and the maximum number of edges, the clustering coefficient of the nodes is calculated; the clustering coefficient is stored in the hash table;

[0199] Traverse the hash table that stores the degree of the node and the clustering coefficient of the node, mark the nodes of the graph model according to the preset threshold, and obtain the candidate sensitive information nodes; according to the graph model and the candidate sensitive information nodes, obtain the candidate sensitive information corresponding to the candidate sensitive information nodes.

[0200] In a feasible implementation, after the graph model is built, the next key step is to perform statistical analysis based on the graph. The goal here is to identify possible candidate sensitive information by quantitatively analyzing the specific attributes of each node and edge in the graph.

[0201] The degree of a node refers to the number of edges directly connected to the node. In this application scenario, the degree of a node may be related to its possibility of being sensitive information. Specifically, a high-degree node (i.e., a node connected to multiple other nodes) may be an important or sensitive word.

[0202] The implementation method of statistically analyzing the degree of a node includes: for each node in the graph, traversing all edges to calculate the number of edges directly connected to the node, and storing the calculated degree in a data structure, such as a hash table, for subsequent analysis.

[0203] The clustering coefficient is used to quantify the degree of connection between a node and its neighboring nodes. If a node has a high degree of interconnectivity with its neighboring nodes, then the clustering coefficient of this node will be high, which may mean that the node is sensitive or important.

[0204] The implementation method of statistically analyzing the clustering coefficient includes: for each node in the graph, finding all neighbor nodes; calculating the number of edges that actually exist between all neighbor nodes; calculating the maximum number of edges that may exist between all neighbor nodes; the clustering coefficient is equal to the number of edges that actually exist divided by the maximum number of edges that may exist; and storing the calculated clustering coefficient in a data structure, such as a hash table.

[0205] After completing the above statistical analysis, the next step is to identify possible candidate sensitive information. One method is to set a threshold, for example, nodes with a degree greater than a specific value or a clustering coefficient greater than a specific value will be marked as candidate sensitive information.

[0206] The implementation method of identifying possible candidate sensitive information includes: traversing the data structure storing node degrees and clustering coefficients; applying predefined thresholds or standards to filter the degrees or clustering coefficients; marking nodes that meet the conditions as candidate sensitive information and storing them in a list or database for subsequent processing.

[0207] Through this series of detailed steps and implementation methods, not only can the importance of each node in the graph be quantified, but also possible candidate sensitive information can be effectively identified, thus providing strong support for subsequent adaptive labeling and final sensitive information labeling decisions.

[0208] S5. Based on the adaptive annotation rule generation mechanism, the candidate sensitive information is screened to obtain sensitive information; based on the graph model, the sensitive information nodes are determined according to the sensitive information; the adaptive annotation rule generation mechanism optimizes the annotation rules through iteration; the adaptive annotation rules are used to match sensitive information at the single word level and the vocabulary level.

[0209] Optionally, based on the adaptive annotation rule generation mechanism, the candidate sensitive information is screened to obtain sensitive information, including:

[0210] In the initialization phase, a preliminary rule set is created; the preliminary rule set includes at least one regular expression and at least one keyword list; initialization is performed according to the preliminary rule set; a sensitive information hash table is created;

[0211] In the rule optimization stage, the preliminary rule set is evaluated by F1 score to obtain the F1 score result; based on the F1 score result, the preliminary rule set is optimized to obtain the optimized rule set;

[0212] In the sensitive information determination stage, candidate sensitive information is screened based on the optimized rule set to obtain screened sensitive information; the screened sensitive information is cross-validated and reviewed by manual experts to obtain sensitive information; the sensitive information is stored in a sensitive information hash table.

[0213] In a feasible implementation, the implementation of this step includes an initialization phase, a rule optimization phase, and a sensitive information screening and determination phase.

[0214] During the initialization phase, the following steps need to be performed:

[0215] Initialize a rule set that can contain some basic sensitive information annotation rules. These rules can be regular expressions, keyword lists, or other complex pattern matching algorithms. At the same time, create an empty data structure (such as an array or hash table) to store the nodes that are ultimately determined to be sensitive information.

[0216] Initializing a rule set involves selecting or defining a set of basic sensitive information labeling rules. These rules may include three main elements:

[0217] Regular Expressions: These are expressions used to recognize specific text patterns. For example, a regular expression for recognizing a credit card number might be (?:\d{4}-){3}\d{4} or a regular expression for recognizing an email address might be [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}.

[0218] Keyword list: This is a list of words that are sensitive or need to be noted. For example, in some contexts, "password", "bank account", etc. may be considered sensitive keywords.

[0219] Sophisticated pattern matching algorithms: These can be more advanced natural language processing or text mining algorithms, such as methods based on semantic analysis to identify phrases or sentences that may be considered sensitive in a certain context.

[0220] In the initialization phase, the rule set is initialized by including a preliminary rule set consisting of at least one regular expression and at least one keyword list, and a hash table is created to store nodes that are ultimately determined to be sensitive information.

[0221] The following steps need to be performed during the rule optimization phase:

[0222] Use the preliminary set of rules built in the previous step to mark candidate sensitive information in the graph model. This step is applied to nodes that have been identified as candidate sensitive information.

[0223] Use evaluation mechanisms, such as F1 score or precision and recall, to evaluate the performance of the current rule set.

[0224] Based on the evaluation results, the rule set is optimized. The optimization method may include adding new rules, deleting poorly performing rules, or adjusting the weights of existing rules.

[0225] This phase will be repeated several times until the performance of the rule set reaches an acceptable standard or exceeds a predefined number of iterations.

[0226] During the sensitive information screening and identification phase, the following steps need to be performed:

[0227] Use the optimized rule set to re-mark candidate sensitive information in the graph model. The result of this marking should be more accurate than the initial one.

[0228] Nodes marked as candidate sensitive information are further verified, which may include comparison with other data sources, expert review, etc.

[0229] The nodes that pass the verification are finally determined to be sensitive information and stored in the data structure created previously.

[0230] S6. Integrate word embedding, analysis results and sensitive information nodes to obtain a hybrid model; make sensitive information labeling decisions based on the hybrid model; the hybrid model is a multi-layer perceptron.

[0231] In a feasible implementation, an appropriate machine learning algorithm is selected as the basic framework of the hybrid model, such as a support vector machine (SVM), a decision tree, or a neural network.

[0232] Neural networks are generally more suitable for processing multi-source high-dimensional data. Specifically, you can consider using a multi-layer feedforward neural network, also known as a multi-layer perceptron (MLP). This network can automatically learn high-level feature representations from raw features.

[0233] The word embedding generated by the autoencoder, the statistical analysis results based on the graph, and the sensitive information nodes determined by the adaptive labeling rule generation mechanism are input into the multi-layer perceptron as features.

[0234] The feature input of the multi-layer perceptron includes:

[0235] Word embeddings generated by autoencoders: These are high-dimensional word vectors that capture the semantic information of text data.

[0236] Statistical analysis results based on the graph: such as node degree, clustering coefficient, etc. These features can reflect the information in the graph structure.

[0237] Sensitive information nodes determined by the adaptive annotation rule generation mechanism: This is a binary vector indicating which nodes are considered sensitive information.

[0238] Output of a multilayer perceptron: A single node indicates whether the information is sensitive or not. A Sigmoid activation function can be used to output the probability.

[0239] The following is a detailed description of the training process of the multilayer perceptron:

[0240] Get sample data from an existing, labeled dataset to train a multi-layer perceptron.

[0241] The multi-layer perceptron is trained using optimization algorithms such as gradient descent. During the training process, the objective function is to minimize the error of the model in the task of labeling sensitive information. A part of the data set that has not been used for training is randomly selected as a validation set to evaluate the performance of the model.

[0242] The labeling ability of the multilayer perceptron is evaluated using performance indicators such as precision, recall, and F1 score.

[0243] After model training and validation are complete, the multi-layer perceptron is applied to new or unlabeled data. The model will comprehensively consider all input features and output a labeling decision that indicates which nodes (single words or vocabulary) are ultimately determined to be sensitive information.

[0244] The present invention proposes a method for automatic labeling of sensitive information based on unsupervised learning. By fusing word-level and vocabulary-level information, the generated high-dimensional word representation captures the semantic and structural features of the text more comprehensively. Constructing a graph model and performing graph-based statistical analysis can take into account the relationship between words or vocabulary, so that the system can not only identify a single sensitive word, but also identify sensitive information composed of multiple words. Since it is based on unsupervised learning, this method does not rely on a large amount of labeled data, which reduces the implementation cost, and is also suitable for occasions where it is difficult to obtain a large amount of labeled data. The present invention is a method and system that can effectively, comprehensively and automatically label sensitive information.

[0245] Figure 2 1 is a block diagram of a device for automatically labeling sensitive information based on unsupervised learning according to an exemplary embodiment. The device is used in a method for automatically labeling sensitive information based on unsupervised learning. Figure 2 The device includes a processing unit 210, a generating unit 220, a constructing unit 230, an analyzing unit 240, a determining unit 250 and an integrating unit 260. Among them:

[0246] The processing unit 210 is used to obtain input data; perform word segmentation preprocessing on the input data to obtain preprocessed data; the word segmentation preprocessing includes word level preprocessing and word level preprocessing; the preprocessed data includes word level information, word level information, inter-word relationships and inter-word relationships;

[0247] A generating unit 220, configured to generate word embeddings on the preprocessed data using an autoencoder model; the autoencoder model is configured to generate high-dimensional word representations based on word-level information and vocabulary-level information;

[0248] A construction unit 230 is used to construct a graph model according to the preprocessed data; the nodes of the graph model represent the word-level information or the vocabulary-level information from the preprocessed data; the edges of the graph model represent the relationship between words or the relationship between vocabulary;

[0249] An analysis unit 240, configured to perform a graph-based statistical analysis on the graph model to obtain candidate sensitive information;

[0250] The determination unit 250 is used to screen the candidate sensitive information based on the adaptive tagging rule generation mechanism to obtain the sensitive information; based on the graph model, determine the sensitive information node according to the sensitive information; the adaptive tagging rule generation mechanism optimizes the tagging rules through iteration; the adaptive tagging rules are used to match the sensitive information at the single word level and the vocabulary level;

[0251] The integration unit 260 is used to integrate word embedding, analysis results and sensitive information nodes to obtain a hybrid model; make sensitive information labeling decisions based on the hybrid model; the hybrid model is a multi-layer perceptron.

[0252] Among them, the encoder of the autoencoder is used to convert word-level information and vocabulary-level information into latent space vectors;

[0253] The hybrid fusion layer of the autoencoder is used to fuse the latent space vector into a fused latent space vector; the hybrid fusion layer uses the attention mechanism to merge the single word level information and the vocabulary level information;

[0254] The decoder of the autoencoder is used to generate word embeddings based on the fused latent space vectors.

[0255] Among them, the encoder adopts an adaptive learning rate adjustment mechanism; the adaptive learning rate adjustment mechanism dynamically adjusts the learning rate according to the complexity of the input data.

[0256] Among them, the decoder includes a decomposition conversion layer, an information reconstruction layer and a multimodal generation network;

[0257] The decomposition transformation layer is used to expand the fused latent space vector from the hybrid fusion layer;

[0258] The information reconstruction layer is used for nonlinear transformation of the convolutional transposition layer or the fully connected layer and outputs the reconstructed single-word level information and vocabulary level information;

[0259] The multimodal generative network is used to generate multimodal word embeddings based on the latent space vectors.

[0260] Optionally, the construction unit 230 is further configured to:

[0261] Traversing the preprocessed data, creating nodes according to the single word level information and the word level information in the preprocessed data; determining the single word level information and the word level information corresponding to the node as node attributes;

[0262] Determine the relationship between nodes based on the nodes and their attributes, build edges based on the relationship between nodes and calculate edge weights; edge weights are determined based on the physical distance, cosine similarity, and co-occurrence probability of the nodes;

[0263] Construct a graph model based on nodes, edges, and edge weights.

[0264] Optionally, the analysis unit 240 is further configured to:

[0265] Calculate the number of edges directly connected to the nodes based on the nodes of the graph model; obtain the degree of the node based on the number of edges; store the degree of the node in a hash table;

[0266] According to the nodes of the graph model, the neighbor nodes of the nodes are obtained; according to the nodes and the neighbor nodes, the actual number of edges and the maximum number of edges are calculated; according to the actual number of edges and the maximum number of edges, the clustering coefficient of the nodes is calculated; the clustering coefficient is stored in the hash table;

[0267] Traverse the hash table that stores the degree of the node and the clustering coefficient of the node, mark the nodes of the graph model according to the preset threshold, and obtain the candidate sensitive information nodes; according to the graph model and the candidate sensitive information nodes, obtain the candidate sensitive information corresponding to the candidate sensitive information nodes.

[0268] Optionally, the determining unit 250 is further configured to:

[0269] In the initialization phase, a preliminary rule set is created; the preliminary rule set includes at least one regular expression and at least one keyword list; initialization is performed according to the preliminary rule set; a sensitive information hash table is created;

[0270] In the rule optimization stage, the preliminary rule set is evaluated by F1 score to obtain the F1 score result; based on the F1 score result, the preliminary rule set is optimized to obtain the optimized rule set;

[0271] In the sensitive information determination stage, candidate sensitive information is screened based on the optimized rule set to obtain screened sensitive information; the screened sensitive information is cross-validated and reviewed by manual experts to obtain sensitive information; the sensitive information is stored in a sensitive information hash table.

[0272] The present invention proposes a method for automatic labeling of sensitive information based on unsupervised learning. By fusing word-level and vocabulary-level information, the generated high-dimensional word representation captures the semantic and structural features of the text more comprehensively. Constructing a graph model and performing graph-based statistical analysis can take into account the relationship between words or vocabulary, so that the system can not only identify a single sensitive word, but also identify sensitive information composed of multiple words. Since it is based on unsupervised learning, this method does not rely on a large amount of labeled data, which reduces the implementation cost, and is also suitable for occasions where it is difficult to obtain a large amount of labeled data. The present invention is a method and system that can effectively, comprehensively and automatically label sensitive information.

[0273] Figure 3 is a schematic diagram of the structure of a sensitive information automatic labeling device provided by an embodiment of the present invention, such as Figure 3 As shown, the sensitive information automatic labeling device may include the above Figure 2 The apparatus for automatically labeling sensitive information based on unsupervised learning is shown. Optionally, the apparatus for automatically labeling sensitive information 310 may include a first processor 2001 .

[0274] Optionally, the sensitive information automatic labeling device 310 may also include a memory 2002 and a transceiver 2003 .

[0275] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0276] Combine the following Figure 3 The components of the sensitive information automatic labeling device 310 are specifically introduced:

[0277] The first processor 2001 is the control center of the sensitive information automatic labeling device 310, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0278] Optionally, the first processor 2001 can perform various functions of the sensitive information automatic labeling device 310 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0279] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 3 CPU0 and CPU1 are shown in FIG.

[0280] In a specific implementation, as an embodiment, the sensitive information automatic labeling device 310 may also include multiple processors, such as Figure 3 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0281] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0282] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be accessed through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0283] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0284] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 3 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0285] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0286] It should be noted that Figure 3 The structure of the sensitive information automatic labeling device 310 shown in the figure does not constitute a limitation on the router. The actual knowledge structure identification device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0287] In addition, the technical effects of the sensitive information automatic labeling device 310 can refer to the technical effects of the sensitive information automatic labeling method based on unsupervised learning described in the above method embodiment, and will not be repeated here.

[0288] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0289] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0290] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0291] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0292] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0293] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0294] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0295] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0296] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0297] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0298] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0299] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0300] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for automatic labeling of sensitive information based on unsupervised learning, characterized in that: The method comprises: Acquire input data; perform word segmentation preprocessing on the input data to obtain preprocessed data; the word segmentation preprocessing includes word level preprocessing and word level preprocessing; the preprocessed data includes word level information, word level information, inter-word relationships, and inter-word relationships; The steps of the lexical level preprocessing include preliminary word segmentation, part-of-speech tagging, word filtering and polysemous word disambiguation; in the preliminary word segmentation, word segmentation is performed based on an unsupervised learning algorithm; Generate word embeddings on the preprocessed data using an autoencoder model; the autoencoder model is used to generate high-dimensional word representations based on word-level information and vocabulary-level information; The encoder of the autoencoder model is used to convert the word-level information and the vocabulary-level information into a latent space vector; The hybrid fusion layer of the autoencoder model is used to fuse the latent space vector into a fused latent space vector; the hybrid fusion layer uses an attention mechanism to merge the single word level information and the vocabulary level information; The decoder of the autoencoder model is used to generate word embedding according to the fused latent space vector; A graph model is constructed according to the preprocessed data; the nodes of the graph model represent word-level information or vocabulary-level information from the preprocessed data; the edges of the graph model represent the relationship between words or the relationship between words; Performing graph-based statistical analysis on the graph model to obtain candidate sensitive information; The step of performing a graph-based statistical analysis on the graph model to obtain candidate sensitive information includes: Calculating according to the nodes of the graph model to obtain the number of edges directly connected to the nodes; obtaining the degree of the node according to the number of edges; and storing the degree of the node in a hash table; According to the nodes of the graph model, the neighbor nodes of the nodes are obtained; according to the nodes and the neighbor nodes, the actual number of edges and the maximum number of edges are calculated; according to the actual number of edges and the maximum number of edges, the clustering coefficient of the nodes is calculated; and the clustering coefficient is stored in a hash table; Traversing a hash table storing the degree of the node and the clustering coefficient of the node, marking the nodes of the graph model according to a preset threshold, and obtaining candidate sensitive information nodes; obtaining candidate sensitive information corresponding to the candidate sensitive information nodes according to the graph model and the candidate sensitive information nodes; The step of calculating according to the node and the neighboring nodes to obtain the actual number of edges and the maximum number of edges includes: For each node in the graph, find all neighbor nodes; calculate the number of edges that actually exist between all neighbor nodes; calculate the maximum number of possible edges between all neighbor nodes; the clustering coefficient is equal to the number of actual edges divided by the maximum number of possible edges; Based on the adaptive annotation rule generation mechanism, the candidate sensitive information is screened to obtain sensitive information; based on the graph model, sensitive information nodes are determined according to the sensitive information; the adaptive annotation rules are used to match sensitive information at the single word level and the vocabulary level; The method of screening the candidate sensitive information based on the adaptive annotation rule generation mechanism to obtain the sensitive information includes: In the initialization phase, a preliminary rule set is created; the preliminary rule set includes at least one regular expression and at least one keyword list; initialization is performed according to the preliminary rule set; a sensitive information hash table is created; In the rule optimization stage, the preliminary rule set is evaluated by F1 score to obtain an F1 score result; the preliminary rule set is optimized according to the F1 score result to obtain an optimized rule set; In the sensitive information determination stage, the candidate sensitive information is screened based on the optimized rule set to obtain screened sensitive information; the screened sensitive information is cross-validated and manually reviewed by experts to obtain sensitive information; the sensitive information is stored in the sensitive information hash table; Integrate the word embedding, analysis results and sensitive information nodes to obtain a hybrid model; make sensitive information labeling decisions based on the hybrid model; the hybrid model is a multi-layer perceptron; Wherein, the analysis results include the degree and clustering coefficient of the node; The step of integrating the word embedding, the analysis results and the sensitive information nodes to obtain a hybrid model includes: Use multi-layer perceptron as the basic framework of hybrid model; The input of the multi-layer perceptron includes word embedding generated by the autoencoder model, sensitive information nodes determined based on analysis results and an adaptive annotation rule generation mechanism; The output of the multilayer perceptron is the probability of whether the information represented by the node is sensitive.

2. The method for automatic labeling of sensitive information based on unsupervised learning according to claim 1 is characterized in that: The encoder adopts an adaptive learning rate adjustment mechanism; the adaptive learning rate adjustment mechanism dynamically adjusts the learning rate according to the complexity of the input data.

3. The method for automatic labeling of sensitive information based on unsupervised learning according to claim 1 is characterized in that: The decoder includes a decomposition conversion layer, an information reconstruction layer and a multimodal generation network; The decomposition transformation layer is used to expand the fused latent space vector from the hybrid fusion layer; The information reconstruction layer is used for nonlinear conversion of the convolutional transposition layer or the fully connected layer and outputs reconstructed single word level information and vocabulary level information; The multimodal generative network is used to generate multimodal word embeddings according to the latent space vector.

4. The method for automatic labeling of sensitive information based on unsupervised learning according to claim 1, characterized in that: The step of constructing a graph model according to the preprocessed data includes: Traversing the preprocessed data, creating nodes according to the single-word level information and the word-level information in the preprocessed data; determining the single-word level information and the word-level information corresponding to the node as node attributes; Determine the relationship between nodes according to the nodes and the node attributes, construct edges according to the relationship between nodes and calculate edge weights; the edge weights are determined according to the physical distance, cosine similarity and co-occurrence probability of the nodes; A graph model is constructed according to the nodes, the edges, and the edge weights.

5. A device for automatically labeling sensitive information based on unsupervised learning, wherein the device is used to implement the method for automatically labeling sensitive information based on unsupervised learning as claimed in any one of claims 1 to 4, characterized in that: The device comprises: A processing unit is used to obtain input data; perform word segmentation preprocessing on the input data to obtain preprocessed data; the word segmentation preprocessing includes word level preprocessing and word level preprocessing; the preprocessed data includes word level information, word level information, inter-word relations and inter-word relations; The steps of the lexical level preprocessing include preliminary word segmentation, part-of-speech tagging, word filtering and polysemous word disambiguation; in the preliminary word segmentation, word segmentation is performed based on an unsupervised learning algorithm; A generating unit, configured to generate word embeddings on the preprocessed data using an autoencoder model; the autoencoder model is configured to generate high-dimensional word representations based on word-level information and vocabulary-level information; The encoder of the autoencoder model is used to convert the word-level information and the vocabulary-level information into a latent space vector; The hybrid fusion layer of the autoencoder model is used to fuse the latent space vector into a fused latent space vector; the hybrid fusion layer uses an attention mechanism to merge the single word level information and the vocabulary level information; The decoder of the autoencoder model is used to generate word embedding according to the fused latent space vector; A construction unit, configured to construct a graph model according to the preprocessed data; the nodes of the graph model represent word-level information or vocabulary-level information from the preprocessed data; the edges of the graph model represent the relationship between words or the relationship between words; An analysis unit, configured to perform a graph-based statistical analysis on the graph model to obtain candidate sensitive information; The step of performing a graph-based statistical analysis on the graph model to obtain candidate sensitive information includes: Calculating according to the nodes of the graph model to obtain the number of edges directly connected to the nodes; obtaining the degree of the node according to the number of edges; and storing the degree of the node in a hash table; According to the nodes of the graph model, the neighbor nodes of the nodes are obtained; according to the nodes and the neighbor nodes, the actual number of edges and the maximum number of edges are calculated; according to the actual number of edges and the maximum number of edges, the clustering coefficient of the nodes is calculated; and the clustering coefficient is stored in a hash table; Traversing a hash table storing the degree of the node and the clustering coefficient of the node, marking the nodes of the graph model according to a preset threshold, and obtaining candidate sensitive information nodes; obtaining candidate sensitive information corresponding to the candidate sensitive information nodes according to the graph model and the candidate sensitive information nodes; The step of calculating according to the node and the neighboring nodes to obtain the actual number of edges and the maximum number of edges includes: For each node in the graph, find all neighbor nodes; calculate the number of edges that actually exist between all neighbor nodes; calculate the maximum number of possible edges between all neighbor nodes; the clustering coefficient is equal to the number of actual edges divided by the maximum number of possible edges; A determination unit is used to screen the candidate sensitive information based on an adaptive annotation rule generation mechanism to obtain sensitive information; based on the graph model, determine sensitive information nodes according to the sensitive information; the adaptive annotation rule generation mechanism optimizes the annotation rules through iteration; the adaptive annotation rules are used to match sensitive information at the single word level and the vocabulary level; The method of screening the candidate sensitive information based on the adaptive annotation rule generation mechanism to obtain the sensitive information includes: In the initialization phase, a preliminary rule set is created; the preliminary rule set includes at least one regular expression and at least one keyword list; initialization is performed according to the preliminary rule set; a sensitive information hash table is created; In the rule optimization stage, the preliminary rule set is evaluated by F1 score to obtain an F1 score result; the preliminary rule set is optimized according to the F1 score result to obtain an optimized rule set; In the sensitive information determination stage, the candidate sensitive information is screened based on the optimized rule set to obtain screened sensitive information; the screened sensitive information is cross-validated and manually reviewed by experts to obtain sensitive information; the sensitive information is stored in the sensitive information hash table; An integration unit, configured to integrate the word embedding, the analysis result and the sensitive information node to obtain a hybrid model; and make a sensitive information labeling decision based on the hybrid model; the hybrid model is a multi-layer perceptron; Wherein, the analysis results include the degree and clustering coefficient of the node; The step of integrating the word embedding, the analysis results and the sensitive information nodes to obtain a hybrid model includes: Use multi-layer perceptron as the basic framework of hybrid model; The input of the multi-layer perceptron includes word embedding generated by the autoencoder model, sensitive information nodes determined based on analysis results and an adaptive annotation rule generation mechanism; The output of the multilayer perceptron is the probability of whether the information represented by the node is sensitive.

6. A sensitive information automatic labeling device, characterized in that: The sensitive information automatic labeling device includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 4.

Images