A sensitive data identification and desensitization method and system based on federal global prototype

Abstract

The application discloses a sensitive data identification and desensitization method and system based on a federal global prototype, which comprises the following steps: a terminal deploys a local model, for each entity category, the average feature of the text features of all sample data in the category extracted by a BERT model is calculated as a local category prototype, and the local model parameters are uploaded to a server together; the server aggregates the local model parameters to update global model parameters, aggregates the local category prototypes according to the entity categories to update global category prototypes, and issues the terminal; the terminal enhances the training of a CRF layer by using the updated global model parameters and global category prototypes, and calculates a label sequence; a sensitive data identification model is obtained through iterative training; and sensitive data identification and desensitization are performed. In the application, a global category prototype library is constructed, local features of multiple terminals are integrated, the privacy protection advantage of federal learning and the modeling capability of deep learning are combined, and in the meanwhile of protecting data privacy, sensitive data is accurately identified.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data desensitization, and particularly relates to a sensitive data identification and desensitization method and system based on a federal global prototype. BACKGROUND

[0002] With the rapid development of information technology and the modernization transformation of the power system, the traditional power system is gradually changing to a new power system. In recent years, unmanned inspection terminals have become an indispensable inspection tool for grassroots teams. However, in the actual work process of the unmanned inspection terminal, the frequency of data transmission also increases the risk of sensitive data leakage. Once sensitive data such as inspection reports, equipment status information, and environmental parameters are leaked, it will bring serious power grid safety risks and economic losses. Data desensitization technology plays an important role in data privacy protection. By processing sensitive data, the sensitivity of the data is reduced, thereby effectively reducing the risk of sensitive information leakage. In the unmanned inspection terminal, how to accurately identify sensitive information has become an important prerequisite for data desensitization due to the irregularity of text data such as personal information and location information. In recent years, the rise of deep learning technology has provided a new way for sensitive data identification and desensitization. However, unmanned inspection terminals are usually widely distributed, and traditional deep learning methods rely on centralized training, which requires a large amount of data to be aggregated to the center server. This not only increases the cost of data transmission and storage, but also brings many risks due to the privacy of sensitive data, thereby threatening the safe operation and privacy security of the entire power system.

[0003] A "Bert-Crf-based assembly quality knowledge entity recognition method" disclosed on the Chinese patent document, with publication number CN118468874A and publication date 2024.08.09, includes the following steps: (1) establishing a complex product assembly quality knowledge system; (2) establishing a Bert-Crf-based assembly quality knowledge entity recognition model; (3) using a test set to evaluate the performance of the model, and reasonably adjusting the parameters of the model to improve the performance of the model, and realizing complex product assembly quality entity recognition. This technology identifies complex product assembly quality entities based on the Bert-Crf model, realizes automatic extraction and classification of complex product assembly quality entities in natural language text, and improves the accuracy of complex product assembly quality entity recognition. However, the Bert-Crf model in this technology is applied to the assembly quality knowledge entity recognition of product assembly enterprises, and the data is basically stored in the safety location set by the enterprise, so it is suitable to apply deep learning which relies on centralized training and does not have the risk of sensitive data leakage. However, for widely distributed unmanned inspection terminals, using deep learning methods for sensitive data identification and desensitization requires a large amount of data to be aggregated to the central server, which not only increases the cost of data transmission and storage, but also brings many risks due to the privacy of sensitive data, thereby threatening the safe operation and privacy security of the entire power system. SUMMARY

[0004] The present application is to overcome the problem that in the prior art, when using deep learning methods for sensitive data identification and desensitization of unmanned inspection terminals, a large amount of data needs to be aggregated to the central server, which not only increases the cost of data transmission and storage, but also brings many risks due to the privacy of sensitive data, thereby threatening the safe operation and privacy security of the entire power system. A sensitive data identification and desensitization method and system based on federated global prototypes are provided.

[0005] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0006] A sensitive data identification and desensitization method based on federated global prototypes, comprising:

[0007] Deploying a BERT-CRF model on the terminal, for each entity category, calculating the average feature of the text features extracted by the BERT model for all sample data in this category as the local category prototype, and uploading the local model parameters to the server together with the local category prototype;

[0008] The server receives a plurality of terminal data, aggregates the local model parameters to update the global model parameters, aggregates the local category prototypes according to the entity categories to update the global category prototypes, and issues the terminal;

[0009] The terminal uses the updated global model parameters and global category prototypes to perform enhanced training on the CRF layer and calculate the label sequence;

[0010] Iterative training is used to obtain a sensitive data recognition model; sensitive data recognition and desensitization are performed.

[0011] This invention, based on an improved BERT-CRF model and combined with federated learning technology, jointly trains a global model across multiple terminals without sharing local data, effectively protecting the privacy of local data on each terminal. It also introduces contrastive learning to construct a global category prototype library, enhancing the sensitive data recognition model's deep understanding of data semantics and its ability to integrate global features. This allows for accurate identification of sensitive information in unmanned inspection terminal data, providing a more robust safeguard for the data security of unmanned inspection terminals. Based on asymmetric encryption theory, the identified sensitive data is desensitized using the RSA algorithm, ensuring that even if data is intercepted during transmission or storage by unmanned inspection terminals, attackers cannot easily decrypt it, effectively preventing the leakage of sensitive data.

[0012] Preferably, the updating of the global category prototype includes:

[0013] For each entity category, the global average feature of the entity category is calculated and added as the corresponding global category prototype to the global category prototype library;

[0014] The global average feature of the entity category is the sum of the local category prototypes of the terminals that upload the category divided by the number of terminals that upload the category.

[0015] Preferably, the enhanced training of the CRF layer includes:

[0016] The emission probability is expanded to a linear combination of the text features under each entity category and the similarity of the global category prototype to enhance the emission probability;

[0017] The label transfer probability is adjusted according to the product of the similarity between the global category prototypes corresponding to adjacent labels in the label sequence and the second adjustment parameter.

[0018] Preferably, the enhanced emission probability for each entity category is the sum of the first product and the second product;

[0019] The first product is the product of the weight matrix and the text feature; the second product is the product of the first adjustment parameter and the global category prototype similarity; the global category prototype similarity is the similarity between the text feature and the global category prototype.

[0020] Preferably, the updating of global model parameters includes:

[0021] A weighted average aggregation is performed on all local model parameters uploaded to the server, and a weight of the local model parameter is assigned according to the amount of data on the terminal corresponding to the local model parameter.

[0022] Preferably, the text features extracted by the BERT model include: the sample data is obtained by preprocessing the data of the unmanned inspection terminal;

[0023] The text sequence in the sample data is input into the BERT model, and the output features of each layer are extracted through several encoding layers in the model; through the gated attention mechanism, the output features of each layer are weighted and aggregated to obtain text features.

[0024] Preferably, the iterative training to obtain the sensitive data recognition model includes:

[0025] Iterative training is performed using a linear combination of contrast loss and cross entropy loss as the total loss, and the contrast loss weight is set to balance the contributions of the two losses. When the training termination condition is met, the trained sensitive data recognition model is output.

[0026] Preferably, the sample data is obtained by preprocessing data from the unmanned inspection terminal, including:

[0027] Perform text segmentation on data from unmanned inspection terminals. English data is segmented into words or short sentences and then further segmented using the WordPiece algorithm. Chinese data is segmented by characters.

[0028] The segmented text is labeled with part-of-speech sequences. Each type of entity has its own corresponding independent B and I labels. Entities with a length of 1 are marked with S labels. Irrelevant characters are marked with O.

[0029] Preferably, the sensitive data identification and desensitization comprises:

[0030] The RSA algorithm is used to generate a key pair, and the identified sensitive data is encrypted with the public key. The ciphertext is then sent from the unmanned inspection terminal to the receiver. After receiving the ciphertext, the receiver decrypts and restores it using the private key.

[0031] A sensitive data identification and desensitization system based on a federated global prototype, including:

[0032] The data preprocessing module performs text segmentation and prediction annotation on the data from the unmanned inspection terminal;

[0033] The sensitive data identification module uses preprocessed data to build and train a sensitive data identification model that combines the BERT-CRF model with federated learning to identify sensitive data from unmanned inspection terminals.

[0034] The data desensitization module desensitizes the sensitive data of the identified unmanned inspection terminals.

[0035] The present invention has the following beneficial effects: based on the improved BERT-CRF model and combined with federated learning technology, the global model is trained jointly by multiple terminals without sharing local data, effectively protecting the privacy of local data of each terminal; comparative learning is introduced to construct a global category prototype library, and the model's deep understanding of data semantics and integration capabilities of global features are enhanced, thereby realizing accurate identification of sensitive information in unmanned inspection terminal data, and providing a more powerful guarantee for the data security of unmanned inspection terminals; by constructing a global category prototype library and integrating the local features of multiple terminals to form a representative global prototype, combined with the privacy protection advantages of federated learning and the powerful modeling capabilities of deep learning, while protecting data privacy, accurate identification and efficient desensitization of sensitive data are achieved, significantly improving the security and reliability of unmanned inspection data, and providing a strong guarantee for the safe operation of the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flowchart of a sensitive data identification and desensitization method based on a federated global prototype in the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0038] Among existing sensitive data identification technologies, the BERT model excels in handling polysemous words and extracting features, providing strong technical support for accurate sensitive data identification. However, a single BERT model struggles to directly capture the dependencies between labels in sequence labeling tasks. Conditional Random Fields (CRFs), as a discriminative model, excel at sequence labeling. By modeling the label dependencies of input sequences, they can optimize labeling results. Combining BERT with CRFs can fully leverage the advantages of BERT and the sequence labeling capabilities of CRFs to achieve efficient and accurate sensitive data identification.

[0039] However, unmanned inspection terminals are typically widely distributed, and deep learning methods like BERT-CRF rely on centralized training, requiring large amounts of data to be aggregated to a central server. This not only increases data transmission and storage costs, but also poses numerous risks due to privacy issues with sensitive data. Directly transmitting data to a central server for centralized processing may result in data interception, tampering, or leakage during transmission, exposing sensitive information to unauthorized third parties, thereby threatening the safe operation and privacy security of the entire power system. Federated learning can provide a new solution to this problem. By training the model on the local terminal and uploading the model parameters rather than the original data to the server for aggregation, federated learning protects data privacy and improves model performance. However, there are still shortcomings in directly applying federated learning methods to the identification of sensitive data in unmanned inspections. These are mainly reflected in the lack of a deep understanding of data semantics and the lack of global feature integration capabilities, which leads to limited recognition accuracy and generalization.

[0040] In order to solve the above problems, the following Figure 1 The sensitive data identification and desensitization method shown in the figure is based on a federated global prototype, including:

[0041] The terminal deploys the BERT-CRF model. For each entity category, the terminal calculates the average feature of the text features extracted by the BERT model for all sample data in that category as the local category prototype and uploads it to the server together with the local model parameters.

[0042] The server receives data from several terminals, aggregates local model parameters to update global model parameters, aggregates local category prototypes by entity category, updates global category prototypes, and sends them to the terminals;

[0043] The terminal uses the updated global model parameters and global category prototypes to perform enhanced training on the CRF layer and calculate the label sequence;

[0044] Iterative training is used to obtain a sensitive data recognition model; sensitive data recognition and desensitization are performed.

[0045] It should be noted that the present invention is based on an improved BERT-CRF model, combined with federated learning technology, and while not sharing local data, it jointly trains a global model with multiple terminals to effectively protect the privacy of local data on each terminal. At the same time, it introduces comparative learning to build a global category prototype library, enhancing the sensitive data recognition model's deep understanding of data semantics and its ability to integrate global features, thereby achieving accurate identification of sensitive information in unmanned inspection terminal data and providing a more powerful guarantee for the data security of unmanned inspection terminals. Based on asymmetric encryption theory, the identified sensitive data is desensitized using the RSA algorithm to ensure that even if the data is intercepted during the transmission or storage of data by the unmanned inspection terminal, the attacker cannot easily decrypt it, thereby effectively preventing the leakage of sensitive data.

[0046] It is worth noting that the present invention solves the problem of how to combine federated learning with the BERT-CRF model to overcome their respective shortcomings in training and constructing a sensitive data recognition model for unmanned inspection terminals. By sharing global category prototypes, the recognition model's deep understanding of data semantics and its ability to integrate global features are enhanced, thereby more comprehensively maintaining the data security of unmanned inspection terminals and adapting to local data in different terminal scenarios; by constructing a global category prototype library, the local features of multiple terminals are integrated to form a representative global category prototype. Combining the privacy protection advantages of federated learning and the powerful modeling capabilities of deep learning, while protecting data privacy, it achieves accurate identification and efficient desensitization of sensitive data, significantly improving the security and reliability of unmanned inspection data, and providing strong guarantees for the safe operation of the power system.

[0047] As a specific embodiment, the sample data is obtained by preprocessing the data of the unmanned inspection terminal, including:

[0048] Perform text segmentation on data from unmanned inspection terminals. English data is segmented into words or short sentences and then further segmented using the WordPiece algorithm. Chinese data is segmented by characters.

[0049] The segmented text is labeled with part-of-speech sequences. Each type of entity has its own corresponding independent B and I labels. Entities with a length of 1 are marked with S labels. Irrelevant characters are marked with O.

[0050] It should be noted that data preprocessing aims to provide a foundation for building sensitive data recognition models. Data preprocessing uses real data generated and used during the actual operation of unmanned inspection terminals, such as inspection reports, equipment status information, and environmental parameters. This data preprocessing ensures that the subsequent sensitive data recognition model can be trained on a comprehensive and diverse dataset, thereby improving model recognition accuracy.

[0051] Specifically, text segmentation involves segmenting the data from unmanned inspection terminals to adapt the BERT model. For Chinese datasets, the data is segmented by word. For English datasets, the original data is first divided into sentences. This step aims to perform a preliminary segmentation of continuous text. Second, each sentence is further segmented into phrases or words based on delimiters such as spaces and punctuation. This step helps refine the text structure and facilitates the model's understanding of the constituent units of language. Finally, the WordPiece algorithm is used for final segmentation, breaking down phrases or words into smaller word granularity.

[0052] It also includes corpus annotation: before model training, the segmented text is tagged with part-of-speech sequences to construct a sample data set. The "BIOS" tagging method is used to improve text recognition. The "BIOS" sequence tagging method has four tags: B indicates that the character is the starting position of the entity, I indicates that the character is the internal position of the entity, O indicates that the character is a non-entity part, and S indicates that the character is an entity composed of a single character. When tagging, it is necessary to clarify the definition and scope of the entity and consider the semantic information of the specific field of the unmanned inspection terminal to ensure the consistency and accuracy of the tagging. The specific tagging rules are as follows: 1) Each type of entity has its own corresponding independent B and I labels; 2) Entities with a length of 1 are marked with S tags; 3) Irrelevant characters are marked with O; This step aims to provide the required data foundation for the subsequent establishment of sensitive data recognition models to ensure that the model can effectively identify sensitive information in the text.

[0053] As a specific embodiment, after completing the data preprocessing, it is necessary to train the sensitive data recognition model. First, the multi-layer feature extraction capability of the BERT model is utilized, combined with dynamic weight aggregation technology, to fully integrate the output features of each layer, thereby more accurately capturing the contextual information in the text and obtaining text features. On this basis, the sequence labeling capability of the CRF layer is used to further accurately identify sensitive information in the text, ensuring accurate labeling of key information. At the same time, using a federated learning framework, the terminal (i.e., unmanned inspection terminal) independently trains the model locally and uploads the local model parameters and local category prototypes to the server for global aggregation. The server then distributes the updated global model parameters and global category prototypes to each terminal, achieving continuous optimization and synchronization of the model, effectively protecting data privacy, and improving the security and efficiency of the unmanned inspection terminal during data processing.

[0054] Specifically, the text features extracted by the BERT model include: sample data is obtained by preprocessing the data from the unmanned inspection terminal; the text sequence in the sample data is input into the BERT model, and the output features of each layer are extracted through several encoding layers in the model; through the gated attention mechanism, the output features of each layer are weighted and aggregated to obtain text features.

[0055] In the feature extraction stage of the local BERT model, the input is the preprocessed sample data (annotated dataset), which contains a text sequence and its corresponding label sequence. The text sequence is sent to the 12-layer Transformer structure of the BERT model to obtain the output features of each layer. Then, the weight of each layer is dynamically calculated through the gated attention mechanism, and the final text features are obtained by weighted aggregation. The features in the present invention are all represented in the form of vectors. The gated attention mechanism combines global context information to guide the calculation of weights, and then uses the convolutional network to extract the local information of each layer feature, combines it with the corresponding weights for weighted calculation, and finally outputs the text features for subsequent processing.

[0056] It should be noted that before establishing and training the sensitive data recognition model, the model needs to be initialized first. The terminal device and server are initialized and configured separately. The terminal first deploys the local model, including the BERT model and the randomly initialized local conditional random field (CRF) layer. These parameters correspond to the local model parameters. At the same time, the terminal also needs to configure the relevant parameters of federated learning, including aggregation frequency, learning rate, and contrast loss weight. The server initializes the global model, and its initial global model parameters W global Empty, and create an empty global category prototype library P global , used to store the summary information of subsequent global category prototypes.,The terminal is an unmanned inspection terminal, and the server can be a cloud or other,device for central control and processing.

[0057] As a specific embodiment, the operation of performing federation aggregation at the server and sending the update information to the terminal includes updating the global category prototype and updating the global model parameters.

[0058] Updates to the global category prototype include:

[0059] For each entity category c, calculate the global average feature of entity category c as the corresponding global category prototype P c global Add to global category prototype library P global The global average feature of entity category c is the local category prototype P of the terminal that uploaded this category. c local The sum is divided by the number of terminals that upload this category.

[0060] The global model parameter W global Updates include:

[0061] For all local model parameters W uploaded to the server local Perform weighted average aggregation and assign weights to local model parameters based on the amount of data on the terminal corresponding to the local model parameters.

[0062] In the global model delivery phase, the server will update the global model parameters W global , and the global category prototype library P global After receiving this information, the terminal replaces the local model parameters with the global model parameters and loads the global category prototype library for subsequent sensitive data identification tasks.

[0063] As a specific embodiment, after the corresponding deployment is completed at the terminal and server, during the training process of the sensitive data recognition model, the loss value will be recalculated after each parameter update to evaluate the performance of the model under the current parameters. Through continuous iterative training, until the predetermined training rounds are reached or the loss value converges to the preset threshold, it is ensured that the model can accurately identify the sensitive data of the unmanned inspection terminal and has good generalization ability. In order to enhance the model's ability to distinguish the semantic boundaries of different categories, improve the quality of feature representation and the generalization ability of the model, and ultimately improve the accuracy and robustness of sensitive data recognition, and overcome the problems of combining the BERT-CRF model with federated learning, the present invention optimizes the training of the model by introducing a global category prototype library and contrast loss.

[0064] Specifically, using the global category prototype to enhance the training of the CRF layer includes:

[0065] The emission probability is expanded to a linear combination of the text features under each entity category c and the similarity of the global category prototype to enhance the emission probability; the enhanced emission probability under each entity category c is the sum of the first product and the second product;

[0066] The first product is the product of the weight matrix and the text feature; the second product is the product of the first adjustment parameter and the global category prototype similarity; the global category prototype similarity is the similarity between the text feature and the global category prototype; in this way, the information of the global category prototype is incorporated into the calculation of the emission probability.

[0067] The label transition probability is adjusted according to the product of the similarity between the global class prototypes corresponding to adjacent labels in the label sequence and the second adjustment parameter. The dynamic label transition constraint is adjusted, the similarity information between the global class prototypes is introduced, and the rationality of the label transition is enhanced. Specifically, the label transition probability is adjusted according to the similarity between the global class prototypes corresponding to the previous label and the current label, and the influence of the label transition similarity is controlled through the adjustment parameter. The similarity between the global class prototypes is used to adjust the label transition probability, thereby improving the accuracy and rationality of decoding. After completing the CRF enhanced training, the Viterbi algorithm is used to calculate the optimal label sequence according to the enhanced emission probability and the adjusted label transition probability.

[0068] Further, the sensitive data recognition model obtained by iterative training includes:

[0069] The linear combination of the contrast loss and the cross-entropy loss is used as the total loss for iterative training, and the contrast loss weight is set to balance the contributions of the two losses; when the training termination condition is met, the trained sensitive data recognition model is output.

[0070] It is worth noting that, in addition to using global class prototypes to solve the problems existing in the combination of BERT-CRF model and federated learning, the present application further improves the effect of model training by introducing contrast learning technology. During model training, positive and negative sample pairs are constructed. For each entity category c, a positive and negative sample pair is constructed, and the contrast loss is calculated through these sample pairs. The contrast loss is combined with the cross-entropy loss to obtain the total loss, which is used to guide the training and optimization of the model.

[0071] As a specific embodiment, after the training of the sensitive data recognition model is completed, it is redeployed to the terminal and the server, and waits for subsequent sensitive data recognition. When performing sensitive data recognition, the pre-processing process is performed on the data of the unattended inspection terminal to be recognized, and then the pre-processed result is input into the sensitive data recognition model to output the final recognized sensitive data.

[0072] Further, the sensitive data recognition and desensitization include:

[0073] The RSA algorithm is used to generate a key pair, the public key is used to encrypt the recognized sensitive data, and the ciphertext is sent from the unattended inspection terminal to the receiver; the receiver decrypts and restores the ciphertext through the private key after receiving the ciphertext.

[0074] It's important to note that text data is desensitized using the RSA encryption algorithm. As an asymmetric encryption algorithm, RSA uses a pair of public and private keys to encrypt and decrypt data, effectively ensuring the security of the desensitized data. In unmanned inspection scenarios, RSA encryption ensures that sensitive data is protected during transmission or storage. Even if the data is intercepted, attackers cannot easily decrypt it, effectively preventing sensitive data leakage. Furthermore, if necessary, the key can be used to restore the original data, ensuring data privacy and security while also meeting data usage requirements.

[0075] Specifically, generate a key pair: select a key length and use the RSA algorithm to generate a key pair (pk, sk). This key pair includes a public key (pk) and a private key (sk). The public key is used to encrypt the identified sensitive data, while the private key is used to decrypt the data. This step provides a secure foundation for encrypted communication and ensures key security.

[0076] Encrypted Data: The encryption process protects data confidentiality by encrypting plaintext with a public key to generate ciphertext. When operating, the unmanned inspection terminal can send the encrypted ciphertext to recipients such as servers and users, eliminating the need to worry about data theft or leakage during transmission, ensuring secure communication.

[0077] Decrypted data: During the decryption process, the recipient uses their private key to decrypt the received ciphertext and restore the original plaintext data. This allows the recipient to read the encrypted data sent by the sender, ensuring the accuracy and reliability of communication.

[0078] As a specific embodiment, the application of unmanned inspection terminals in the field of power transmission and distribution inspection is becoming increasingly popular. In the actual working process, data transmission is inevitably required. Once sensitive data such as inspection reports and equipment status information are leaked, it will bring serious power grid security risks and economic losses. By performing text segmentation, corpus annotation and other operations on the original unmanned inspection terminal data, data preprocessing is achieved, and the preprocessed text sequence is input into the improved BERT-CRF model for federal collaborative training. The model integrates multi-level semantic features through a gated attention mechanism, and dynamically optimizes entity recognition capabilities based on a global category prototype library. Finally, the best annotation sequence is decoded through the Viterbi algorithm to accurately locate the data to be desensitized. The RSA encryption algorithm is introduced in the data desensitization process to encrypt the identified sensitive data. At the same time, combined with key rotation and multi-factor authentication mechanisms, it is ensured that only authorized users holding private keys can decrypt the data, effectively protecting the confidentiality of sensitive data. Compared with the traditional single-terminal training mode, the present invention realizes multi-terminal data collaborative modeling through a federated learning framework, which significantly improves the model generalization ability while avoiding the leakage of original data.

[0079] It should be noted that the method of the present invention mainly includes three parts: data preprocessing, sensitive data identification model establishment, and sensitive data desensitization. In data preprocessing, the massive text data collected by the unmanned inspection terminals is carefully segmented through word and symbol segmentation and the WordPiece algorithm. The segmented text is then tagged with part-of-speech sequences to construct a sample dataset, providing a foundation for subsequent model training.

[0080] During the development of the sensitive data identification model, the preprocessed text sequence is fed into the BERT layer, where multi-level semantic features are integrated through a dynamic weight allocation mechanism. Based on the global category prototypes sent by the server, a gated attention mechanism is used to calculate the importance weights of the outputs of different encoding layers. The output features of each layer are combined with the global category prototypes as input features, and then the weight parameters of the gated attention network are combined to calculate the weights of the output features of each layer through an activation function. After feature transformation, these weighted features are aggregated into text features. The CRF layer incorporates the global category prototype library for enhancement during the decoding process. The emission probability of each entity category is combined with its semantic similarity in the global category prototype library. Label transfer probabilities are also optimized based on the correlation of global category prototypes in adjacent label sequences. Each terminal calculates the average feature of the local entity category as the local category prototype and uploads it to the server. The federated server aggregates the generated global category prototype and adds it to the global category prototype library, achieving cross-terminal semantic feature alignment. Model training utilizes a joint optimization method of cross-entropy loss and contrastive loss. Parameters are updated iteratively through backpropagation. The final sensitive data identification model is output when the loss converges or a predetermined number of rounds are reached.

[0081] During the sensitive data desensitization process, the identified data is desensitized using the RSA encryption algorithm. A key length is selected based on security requirements, and a corresponding public and private key pair is generated. The public key is used to encrypt the sensitive data, generating difficult-to-crack ciphertext and effectively preventing data leaks. When accessing or using this encrypted data, legitimate users can decrypt it using the private key to restore the original plaintext data. This process ensures data security while meeting data usage requirements, providing secure support for data management in unmanned inspection terminals.

[0082] Specifically, the entire execution process mainly includes the following steps:

[0083] Step 1: Preprocess the data from the unmanned inspection terminals. English data is segmented into words or short sentences. The final segmentation is performed using the WordPiece algorithm, breaking down the phrases or words into smaller word granularities. Chinese data is segmented by character. Proceed to Step 2.

[0084] Step 2: Perform part-of-speech sequence tagging on the segmented text. Each entity type has its own corresponding independent B and I labels. Entities with a length of 1 are marked with S labels. Irrelevant characters are marked with O. Construct a sample dataset according to the above rules. Proceed to Step 3.

[0085] Step 3: Federated Initialization. The terminal first deploys a local model consisting of BERT and CRF. It also configures relevant federated learning parameters, such as the aggregation frequency, learning rate, and contrastive loss weight. The aggregation frequency determines the frequency of parameter synchronization between the terminal and the server, while the learning rate controls the speed of model parameter updates. The server initializes an empty global model and a global category prototype library to store global category prototype information. Proceed to Step 4.

[0086] Step 4: BERT feature extraction. The preprocessed text sequence is input into the BERT model. The BERT model performs deep semantic analysis on the text through its internal encoding layers, extracting feature vectors for each encoding layer. These feature vectors effectively represent the semantic information of the text, providing a foundation for subsequent sensitive data identification. Furthermore, a gated attention mechanism is used to perform weighted aggregation on the extracted feature vectors, further optimizing the feature representation to obtain text features. Proceed to Step 5.

[0087] Step 5: On each terminal, calculate the average text features of all sample data in each entity category to obtain the local category prototype. Then, upload the local model parameters and local category prototype to the server to prepare for subsequent federated aggregation. Proceed to Step 6.

[0088] Step 6: Federated Aggregation: The server receives the local model parameters and local category prototypes uploaded by each terminal. It updates the global model parameters by taking a weighted average of the local model parameters. It also calculates the global category prototype by taking an average of the local category prototypes and adds the updated global category prototype to the global category prototype library. The updated global model parameters and global category prototypes are then distributed to each terminal so that they can update their local models. Proceed to Step 7.

[0089] Step 7: CRF Enhanced Training. On the terminal, perform enhanced training on the CRF layer using the updated global model parameters and global category prototypes. Taking the global category prototypes into account in the emission probability calculation and label transition probability adjustment process further optimizes the model's ability to identify sensitive data. Finally, the optimal label sequence is decoded using the Viterbi algorithm, achieving accurate identification of sensitive data. Proceed to Step 8.

[0090] Step 8: Iterate the training process and define a contrastive loss function. This method uses contrastive learning to further optimize the model's feature representation capabilities. Combined with the cross-entropy loss function, the total loss is calculated, and the model parameters are updated through backpropagation. Repeat Steps 4 through 8 until the iteration termination criteria are met. Finally, the trained sensitive data recognition model is output. Proceed to Step 9.

[0091] Step 9: On the unmanned inspection terminal, use the trained sensitive data recognition model to perform real-time detection on the collected data from the unmanned inspection terminal, thereby accurately identifying sensitive data in the unmanned inspection terminal data. Proceed to Step 10.

[0092] Step 10: Select a key length and generate a key pair using the RSA key generation function. The public key is used to encrypt the identified data to be desensitized, while the private key is used for decryption. The public key is used to encrypt the identified sensitive data, generating the corresponding ciphertext. This ensures the security of sensitive data during transmission and storage, preventing data leakage. When the encrypted data is needed, the ciphertext is decrypted using the private key to restore the original plaintext data.

[0093] In addition to a sensitive data identification method based on a federated global prototype, the present invention also provides a sensitive data identification and desensitization system based on a federated global prototype, including:

[0094] The data preprocessing module performs text segmentation and prediction annotation on the data from the unmanned inspection terminal;

[0095] The sensitive data identification module uses preprocessed data to build and train a sensitive data identification model that combines the BERT-CRF model with federated learning to identify sensitive data from unmanned inspection terminals.

[0096] The data desensitization module desensitizes the sensitive data of the identified unmanned inspection terminals.

[0097] It should be noted that, first, by constructing a multi-terminal collaborative training mechanism under the federated learning framework, differentiated word segmentation and sequence labeling of cross-scenario texts are completed in the data preprocessing module. Secondly, the BERT-CRF model is used in the sensitive data recognition module. Based on the gated attention mechanism, multi-level semantic features are dynamically integrated, and the global category prototype library is combined to achieve cross-terminal entity semantic representation alignment. By introducing contrastive learning, the feature aggregation of similar sensitive data and the discrimination of heterogeneous data are enhanced. During the model training process, each terminal uploads the local category prototype to the server for federated aggregation to form a unified global category prototype to optimize recognition robustness. Finally, in the data desensitization module, based on the asymmetric encryption theory, the RSA encryption algorithm is introduced to encrypt sensitive data, thereby realizing desensitization operations and effectively reducing the risk of sensitive information leakage from unmanned inspection terminals.

[0098] Specifically, the core task of the data preprocessing module is to convert raw unmanned inspection terminal data into a form suitable for subsequent model processing. This module selects real data generated and used during the actual operation of unmanned inspection terminals, obtains raw text data, segments it, and uses the WordPiece algorithm to break phrases or words into smaller word granularity. Furthermore, this module performs part-of-speech sequence tagging on the segmented text to construct a sample dataset, providing the data foundation for the subsequent sensitive data identification model.

[0099] Sensitive Data Identification Module: This module first uses the BERT model to extract text features. It then dynamically assigns weights to the output features of each Transformer encoding layer, combines this with a gated attention mechanism to calculate the weights of each layer's output features, and then transforms these features through a convolutional network. These weighted features are then fused to generate text features containing multi-level semantics. During training, each terminal calculates the average features of text features based on local entity categories and uploads them to the server as local category prototypes. By globally aggregating local model parameters and local category prototypes, a unified representation space is formed, enabling the model to leverage distributed data to enhance the generalized modeling of entity semantic prototypes.

[0100] The CRF layer innovatively introduces global category prototype similarity calculation during the decoding process, expanding the emission probability (or emission score) to a linear combination of the similarity between text features and global category prototypes. It also dynamically adjusts the label transfer probability (or label transfer score) based on the similarity between the global category prototypes of adjacent labels in the label sequence, thereby achieving precise positioning of entity boundaries during the decoding phase. At the same time, a dual optimization strategy of cross-entropy loss and contrastive loss is adopted during the model training phase. Through contrastive learning, similar entity features are closely aggregated in the global category prototype space, while heterogeneous features are kept away from each other. The federated learning mechanism is then combined to continuously optimize the global model parameters and the global category prototype library while protecting data privacy. Ultimately, the optimal annotation sequence is obtained through Viterbi algorithm decoding, which significantly improves the recognition performance of sensitive entities while ensuring that the data of each terminal does not go out of domain.

[0101] Data Desensitization Module: This module uses the RSA encryption algorithm to desensitize sensitive data. A key pair is generated after selecting a key length. The public key encrypts the plaintext to generate ciphertext. During decryption, the private key is used to decrypt the received ciphertext and restore the original plaintext data. The RSA encryption algorithm not only ensures the confidentiality of sensitive data but also implements a security mechanism that only users holding the private key can decrypt the data. This effectively prevents the leakage and misuse of sensitive data, significantly improving the security and privacy of data in unmanned inspection terminals. Furthermore, the desensitized data retains its original value, meeting the practical application scenarios of unmanned inspection terminals.

[0102] The above examples are further elaboration and illustration of the present application, so as to be understood, and are not any limitation of the present application, any modification, equivalent replacement and improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A sensitive data identification and desensitization method based on a federated global prototype, characterized in that: include: The terminal deploys the BERT-CRF model. For each entity category, the terminal calculates the average feature of the text features extracted by the BERT model for all sample data in that category as the local category prototype and uploads it to the server together with the local model parameters. The server receives data from several terminals, aggregates local model parameters to update global model parameters, aggregates local category prototypes by entity category, updates global category prototypes, and sends them to the terminals; The terminal uses the updated global model parameters and global category prototypes to perform enhanced training on the CRF layer and calculate the label sequence; Enhanced training of the CRF layer includes: The emission probability is expanded to a linear combination of the text features under each entity category and the similarity of the global category prototype to enhance the emission probability; The label transfer probability is adjusted according to the product of the similarity between the global category prototypes corresponding to adjacent labels in the label sequence and the second adjustment parameter; the global category prototype similarity is the similarity between the text feature and the global category prototype; Iterative training obtains a sensitive data recognition model that combines the BERT-CRF model with federated learning; sensitive data recognition and desensitization are performed.

2. The method for identifying and desensitizing sensitive data based on a federated global prototype according to claim 1, characterized in that: The updating of the global category prototype includes: For each entity category, the global average feature of the entity category is calculated and added as the corresponding global category prototype to the global category prototype library; The global average feature of the entity category is the sum of the local category prototypes of the terminals that upload the category divided by the number of terminals that upload the category.

3. The sensitive data identification and desensitization method based on the federated global prototype according to claim 1 is characterized in that: The enhanced emission probability for each entity category is the sum of the first product and the second product; The first product is the product of the weight matrix and the text feature; the second product is the product of the first adjustment parameter and the global category prototype similarity.

4. A sensitive data identification and desensitization method based on a federated global prototype according to claim 1 or 2, characterized in that: The updating of global model parameters includes: A weighted average aggregation is performed on all local model parameters uploaded to the server, and a weight of the local model parameter is assigned according to the amount of data on the terminal corresponding to the local model parameter.

5. The method for identifying and desensitizing sensitive data based on a federated global prototype according to claim 1, characterized in that: The text features extracted by the BERT model include: the sample data is obtained by preprocessing the data from the unmanned inspection terminal; The text sequence in the sample data is input into the BERT model, and the output features of each layer are extracted through several encoding layers in the model; through the gated attention mechanism, the output features of each layer are weighted and aggregated to obtain text features.

6. The method for identifying and desensitizing sensitive data based on a federated global prototype according to claim 1, characterized in that: The sensitive data recognition model obtained by iterative training includes: Iterative training is performed using a linear combination of contrast loss and cross entropy loss as the total loss, and the contrast loss weight is set to balance the contributions of the two losses. When the training termination condition is met, the trained sensitive data recognition model is output.

7. A method for identifying and desensitizing sensitive data based on a federated global prototype according to claim 1, 2, 3, 5, or 6, characterized in that: The sample data is obtained by preprocessing the data of the unmanned inspection terminal, including: Perform text segmentation on data from unmanned inspection terminals. English data is segmented into words or short sentences and then further segmented using the WordPiece algorithm. Chinese data is segmented by characters. The segmented text is labeled with part-of-speech sequences. Each type of entity has its own corresponding independent B and I labels. Entities with a length of 1 are marked with S labels. Irrelevant characters are marked with O.

8. The method for identifying and desensitizing sensitive data based on a federated global prototype according to claim 1, 2, 3, 5, or 6, characterized in that: The sensitive data identification and desensitization includes: The RSA algorithm is used to generate a key pair, and the identified sensitive data is encrypted with the public key. The ciphertext is then sent from the unmanned inspection terminal to the receiver. After receiving the ciphertext, the receiver decrypts and restores it using the private key.

9. A sensitive data identification and desensitization system based on a federated global prototype, applicable to the sensitive data identification and desensitization method according to any one of claims 1 to 8, characterized in that: include: The data preprocessing module performs text segmentation and prediction annotation on the data from the unmanned inspection terminal; The sensitive data identification module uses preprocessed data to build and train a sensitive data identification model that combines the BERT-CRF model with federated learning to identify sensitive data from unmanned inspection terminals. The data desensitization module desensitizes the sensitive data of the identified unmanned inspection terminals.

Images