Data desensitization method and device based on BERT model

Abstract

The invention relates to a data desensitization method and device based on a BERT model. The method comprises the following steps: preprocessing business data by utilizing a BERT model; identifying key data in the preprocessed business data by using the trained BERT model; identifying sensitive information in the key data by using the trained BERT model; according to the data type, the data sensitivity and the context information of the sensitive information, a corresponding desensitization strategy is selected from the desensitization strategy library to conduct desensitization processing on the sensitive information, through preprocessing, key data identification and desensitization processing on the service data, complex data content can be efficiently processed and understood, in addition, in the desensitization process, the data processing efficiency is improved, and the data processing efficiency is improved. According to the method, the desensitization strategy is dynamically selected and adjusted based on the data type, the data sensitivity and the context information, so that the data desensitization process can be adaptively adjusted according to the specific data use environment and the sensitivity level, and the desensitization technology can keep effectiveness and applicability in different application situations.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data desensitization method and device based on a BERT model. Background Art

[0002] In the field of social governance and public services, data desensitization plays a vital role in processing public opinion texts. Data desensitization refers to removing or replacing sensitive information while maintaining the practicality of the data to prevent the leakage of personal privacy or other sensitive information during the use of the data. This is particularly important when processing public opinion texts, as these texts usually contain a large amount of personal information and sensitive data.

[0003] However, with the high concentration and frequent use of data, data security and privacy protection are facing unprecedented challenges. In order to deal with these data security risks, data desensitization technology has emerged and become a key solution. Data desensitization aims to blur, replace, encrypt and process sensitive data through specific algorithms and technical means, so that sensitive information can be effectively hidden or replaced while retaining the original characteristics and value of the data, thereby ensuring that the data can meet the business needs of social governance and public services and maximize the privacy security of the data subject in the subsequent analysis, sharing and application process. However, the current desensitization technology still needs to be improved in terms of accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a data desensitization method and device based on the BERT model that can accurately perform data desensitization in response to the above technical problems.

[0005] A data desensitization method based on a BERT model, comprising the following steps: Collect business data of the digital city and pre-process the business data using the BERT model; the business data at least includes public feedback information, policy document information, and social event information; Train the BERT model based on the training corpus, and use the trained BERT model to identify key data in the preprocessed business data; Use the trained BERT model to identify sensitive information in key data; According to the data type, data sensitivity and contextual information of the sensitive information, a corresponding desensitization strategy is selected from the desensitization strategy library, and the sensitive information is desensitized based on the desensitization strategy; the desensitization strategy library includes the following desensitization strategies: data masking, pseudonymization, encryption, generalization and perturbation.

[0006] In one embodiment, in the step of collecting business data of a digital city and preprocessing the business data using a BERT model, the BERT model includes multiple encoder layers, and each encoder layer performs the following steps: Use the BERT model to generate corresponding vector representations for the words in the business data based on the word embedding and position embedding corresponding to the words in the business data; Use the BERT model to perform a linear transformation on the vector representation based on the self-attention layer to obtain the corresponding query, key, and value, and perform a weighted update on the vector representation based on the query, key, and value; The vector representation updated by the self-attention layer is nonlinearly transformed using a feedforward network; The vector representation after nonlinear transformation of the feedforward network is subjected to layer normalization and residual connection processing.

[0007] In one embodiment, the step of collecting business data of a digital city and preprocessing the business data using a BERT model further includes the following steps: The BERT model is used to segment the business data based on the dictionary or context information corresponding to the digital city and by selecting a word-level segmentation method or a subword-level segmentation method according to the category of the vocabulary in the business data.

[0008] In one embodiment, the step of collecting business data of a digital city and preprocessing the business data using a BERT model further includes the following steps: If the length of the business data exceeds the input limit of the BERT model, the business data is processed by truncation or padding methods so that the length of the processed business data meets the input limit of the BERT model.

[0009] In one embodiment, the step of training a BERT model based on a training corpus and using the trained BERT model to identify key data in the preprocessed business data includes the steps of: Randomly select some words in the training corpus for masking, train the BERT model to predict the masked words according to the unmasked context corresponding to the words, and obtain the first loss function; Randomly select a pair of sentences in the training corpus, train the BERT model to determine whether the second sentence in the pair is a direct sequel to the first sentence, and obtain the second loss function; Obtain the sum of the first loss function and the second loss function of the BERT model during the training process to determine the training degree of the BERT model.

[0010] In one embodiment, the step of training a BERT model based on a training corpus and using the trained BERT model to identify key data in the preprocessed business data includes the steps of: Use the trained BERT model to extract features from the preprocessed business data to obtain the first-class feature representation; The trained BERT model is used to perform feature classification or recognition tasks to identify key data from the first type of feature representation; the key data includes at least event type, specific entity, and sentiment level.

[0011] In one embodiment, the step of using the trained BERT model to identify sensitive information in key data includes the steps of: Use the trained BERT model to extract features from key data and obtain the second type of feature representation; The trained BERT model is used to perform feature classification or recognition tasks to identify sensitive information from the second type of feature representation; sensitive information includes sensitive information content, data type, data sensitivity, and contextual information.

[0012] In one embodiment, according to the data type, data sensitivity and context information of the sensitive information, a corresponding desensitization strategy is selected from a desensitization strategy library, and the step of desensitizing the sensitive information based on the desensitization strategy further includes the steps of: Mark sensitive data fields in sensitive information; According to the data type, data sensitivity and context information of sensitive information, a corresponding desensitization strategy is selected from the desensitization strategy library, and the sensitive data fields of the digital city are desensitized based on the desensitization strategy; Record the log of digital city desensitization process.

[0013] In one embodiment, according to the data type, data sensitivity and context information of the sensitive information, a corresponding desensitization strategy is selected from the desensitization strategy library, and after the step of desensitizing the sensitive information based on the desensitization strategy, the step further includes: Real-time monitoring of the desensitization effect of desensitizing sensitive information based on desensitization strategies; After desensitizing sensitive information based on the desensitization strategy, collect user feedback and abnormal situations, and analyze the effectiveness of the desensitization strategy based on the digital city user feedback and abnormal situations; Adjust the desensitization strategy based on the desensitization effect and effectiveness of the digital city.

[0014] A data desensitization device based on a BERT model, the digital city device comprising: The data collection and preprocessing module is used to collect the business data of the digital city and preprocess the business data using the BERT model; the business data at least includes public feedback information, policy document information and social event information; Model training and data identification module, which is used to train the BERT model based on the training corpus and use the trained BERT model to identify key data in the preprocessed business data; Sensitive information identification module, which is used to identify sensitive information in key data using the trained BERT model; The desensitization processing module is used to select the corresponding desensitization strategy from the desensitization strategy library according to the data type, data sensitivity and context information of the sensitive information, and desensitize the sensitive information based on the desensitization strategy; the desensitization strategy library includes the following desensitization strategies: data masking, pseudonymization, encryption, generalization and perturbation.

[0015] One of the above technical solutions has the following advantages and beneficial effects: The data desensitization method provided in each embodiment of the present application includes the following steps: collecting business data of the digital city and preprocessing the business data using the BERT model; training the BERT model based on the training corpus, and using the trained BERT model to identify key data in the preprocessed business data; using the trained BERT model to identify sensitive information in the key data; selecting a corresponding desensitization strategy from a desensitization strategy library according to the data type, data sensitivity and context information of the sensitive information, and desensitizing the sensitive information based on the desensitization strategy. The present application provides a comprehensive solution for desensitizing sensitive information by preprocessing business data, identifying key data and desensitizing the business data, which can efficiently process and understand complex data content. In addition, in the desensitization process, the desensitization strategy is dynamically selected and adjusted based on the data type, data sensitivity and context information, so that the data desensitization process can be adaptively adjusted according to the specific environment and sensitivity level of data use, thereby ensuring that the desensitization technology can maintain its effectiveness and applicability in different application scenarios, thereby greatly improving the security and practicality of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the process of data desensitization method in the embodiment of the present application.

[0017] Figure 2 Schematic diagram of the process of the pretreatment step in the embodiment of the present application.

[0018] Figure 3 Schematic diagram of the model training process in the embodiment of the present application.

[0019] Figure 4Schematic diagram of the data identification process in the embodiment of the present application.

[0020] Figure 5 Schematic diagram of the process of adjusting the strategy in the embodiment of the present application.

[0021] Figure 6 Schematic diagram of the internal structure of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0023] In order to solve the problem that the existing desensitization technology is not accurate enough, in one embodiment, Figure 1 As shown, a data desensitization method based on the BERT model is provided, including the following steps: Step S110, collect the business data of the digital city, and use the BERT model to pre-process the business data. Digital city refers to the use of digital information technology to collect, integrate, store, analyze and visualize various information such as the city's geography, resources, environment, economy, society, etc., to achieve a digital, intelligent and networked urban development model for urban planning, construction, management and services. The business data of the digital city is the data obtained by the computer system when processing various businesses related to the city. For example, digital cities play a huge role in the fields of social governance and public services. In this field, business data at least includes public feedback information, policy document information and social event information. Public feedback information refers to the opinions, suggestions, evaluations, complaints, demands and other information expressed by the public through the digital city system for various affairs, products, services, policies, social events and other contents; policy document information refers to the document information composed of a series of rules, guidelines, action plans and other contents formulated by organizations or institutions to achieve specific goals. Social event information refers to data and descriptions related to events that occur in various fields of society and have a certain impact on social order, public life, etc. The data involved in this application have been fully authorized for acquisition, and the collection, use and processing of relevant information must comply with the relevant laws, regulations and standards of the relevant countries and regions. In the process of collecting data in the digital city, sensitive information will inevitably be collected, including personal sensitive information, corporate sensitive information, etc.

[0024] Before desensitizing business data, the business data is preprocessed, that is, the collected data is preliminarily standardized using the BERT model. The purpose of preprocessing is to identify and correct semantic inconsistencies and clean irrelevant information, thereby improving the accuracy and consistency of the data. Among them, the BERT model is a pre-trained language model based on the Transformer architecture.

[0025] In a specific example, Figure 2 As shown in the figure, in the steps of collecting the business data of the digital city and preprocessing the business data using the BERT model, the BERT model contains multiple encoder layers (the encoder layer is the core component of the BERT model). The BERT model contains multiple layers of Transformer encoders, which collect the output features of each layer of the model and merge them using strategies such as splicing, element-by-element addition, or weighted summation to integrate semantic information at different levels, thereby enhancing the model's ability to represent input data. This process is optimized and integrated into the model through learnable weight parameters to enhance the model's ability to extract information at different levels. Each encoder layer performs the following steps: Step S210, using the BERT model based on the word embedding and position embedding corresponding to the vocabulary in the business data, to generate corresponding vector representations for the vocabulary in the business data.

[0026] Among them, word embedding is a technique in natural language processing that maps words into low-dimensional vector space representations. In natural language processing, word position embedding is a technical means introduced to capture the position information of words in the text.

[0027] Specifically, the vector representation of the vocabulary is E=Word Embedding(xi)+Position Embedding(xi), where Word Embedding(xi) is the word embedding of vocabulary xi, and Position Embedding(xi) is the position embedding of vocabulary xi.

[0028] Step S220, using the BERT model to perform a linear transformation on the vector representation based on the self-attention layer to obtain the corresponding query, key and value, and perform a weighted update on the vector representation according to the query, key and value.

[0029] The self-attention layer aims to enable each word in the data to adaptively pay attention to other words in the data, so as to learn the long-term dependencies and semantic information within the data. It calculates the relevance weights between words, performs a weighted sum of all elements, and obtains a new representation of each word, so that the model can better understand the importance and mutual relationship of each word in the data. The self-attention layer is expressed as: Attention(Q,K,V)=softmax( )V; Among them, Q represents the query matrix, K represents the key matrix, and V represents the value matrix. is the feature dimension, is the scaling factor, is the matrix obtained by multiplying the transpose of K and Q, is the attention score, which is obtained by the softmax function and then multiplied by the value matrix V.

[0030] Attention(Score) = Attention weight Attention(Weight) = softmax(Attention(Score)) Final output Output = Attention(Weight) V Step S230, the vector representation updated by the self-attention layer is nonlinearly transformed using a feedforward network.

[0031] The feedforward network consists of an input layer, several hidden layers, and an output layer. There is no feedback connection between neurons. Information starts from the input layer, passes through each hidden layer in turn, and finally reaches the output layer. It is a one-way information transmission process. In an example, the feedforward network FFN(x)=max(0,xW1+b1)W2+b2, where W1 is the weight matrix, the dimension of W1 is the input d_hidden×d_in, where d_in is the dimension of the input vector x, and d_hidden is the number of neurons in the hidden layer. b1 is the bias vector, and its dimension is the same as the number of neurons in the hidden layer. Max(0,x*w1+b1) is the RELU activation function, which is used to set negative values ​​to 0 and keep positive values ​​unchanged to avoid the problem of gradient disappearance during the calculation process. W2 and b2 are the weight matrix and bias vector of x after the activation function is obtained to obtain x2 and then linearly transformed. The dimension of w2 is d_out×d_hidden, and the dimension of b2 is d_out, where d_out is the dimension of the output layer, and d_hidden is the number of neurons in the hidden layer. This is a two-layer feed-forward network that further transforms the output of the self-attention layer.

[0032] Step S240, performing layer normalization and residual connection processing on the vector representation that has been nonlinearly transformed by the feedforward network.

[0033] Layer normalization is a normalization method that aims to normalize the input of a feedforward network. Residual connection is a skip connection method that introduces a shortcut in the neural network so that the output of a layer can be directly added to the output of the subsequent layer. Layer normalization and residual connection help avoid the vanishing gradient problem, speed up training, and improve model performance.

[0034] If the word segmentation is inaccurate, sensitive data may not be completely desensitized, thereby increasing the risk of data leakage and causing the quality of the data to degrade after desensitization, affecting the availability and subsequent analysis of the data. In one example, the steps of collecting business data of a digital city and preprocessing the business data using a BERT model also include the steps of: using the BERT model to segment the business data based on the dictionary or context information corresponding to the digital city and selecting a word-level segmentation method or a subword-level segmentation method according to the category of the vocabulary in the business data. The dictionary corresponding to the digital city contains specific terms and knowledge that conform to the technical field of the digital city. Context information refers to relevant information around a specific element or theme in a text, an event, an exchange, or a specific context. The word-level segmentation method is a technology for segmenting text in natural language processing, including string matching-based methods, statistics-based methods, machine learning-based methods, deep learning-based methods, etc. The subword-level segmentation method is a more detailed text segmentation technology in natural language processing, including rule-based methods, etc. Specifically, the word-level segmentation method is used to process common words, and the subword-level segmentation method is used to process new words or proper nouns.

[0035] Due to the particularity of each field, in order to perform more targeted word segmentation on the data in each field, in one example, the word segmentation strategy is dynamically selected according to the task and field of the business data, and the word segmentation strategy is combined with the BERT model based on the dictionary or context information corresponding to the digital city, and the word-level word segmentation method or the subword-level word segmentation method is selected according to the category of the vocabulary in the business data to segment the business data. The word segmentation strategy refers to the word segmentation techniques for different types of content, such as word-level word segmentation strategies, word-level word segmentation strategies, or word segmentation strategies combined with word-word mixtures. According to different types of content, a word segmentation strategy suitable for task requirements is selected. In one example, in the process of word segmentation of business data, specific terms and knowledge in the field of digital city technology can be marked.

[0036] For some ambiguous data, after using different methods such as rule-based, statistical, and deep learning to segment words, manual review and comparison are performed to improve the accuracy of word segmentation. New words and words that are easily misclassified are added to the current dictionary, and the sensitive vocabulary library and word segmentation rules are updated regularly. After the sensitive vocabulary library is updated, the existing word segmentation model is trained and tuned to ensure its accuracy and reliability. This application method introduces a dictionary in the field of digital cities to improve the accuracy of word segmentation of business data in digital cities.

[0037] There are specific restrictions on the amount of input data that the BERT model can process at one time. This is to prevent the amount of data input at one time from failing to meet the restrictions of the BERT model, which would affect the data processing process. In one example, the steps of collecting business data of a digital city and preprocessing the business data using the BERT model also include the following steps: if the length of the business data exceeds the input limit of the BERT model, the business data is processed using a truncation or padding method so that the length of the processed business data meets the input limit of the BERT model. The truncation method refers to intercepting the data according to certain rules or thresholds, removing the excess part, and retaining data of a specified length. The padding method refers to padding the data to a specified length by adding specific values ​​or elements when the data length is insufficient.

[0038] Before using the BERT model to preprocess the business data, the business data is cleaned and standardized, including removing noise characters, unifying upper and lower cases, and processing abbreviations, to ensure the quality and consistency of the input business data. Preprocess the business data after text cleaning and standardization.

[0039] Step S120, training the BERT model based on the training corpus, and using the trained BERT model to identify key data in the preprocessed business data.

[0040] The BERT model is trained to enhance its ability to understand complex data content. The training corpus is a large collection of text data used to train models in natural language processing. In an example, Figure 3 As shown, the steps of training the BERT model based on the training corpus and using the trained BERT model to identify key data in the preprocessed business data include the following steps: Step S310, randomly select some words in the training corpus for masking, train the BERT model to predict the masked words according to the unmasked context corresponding to the words, and obtain the first loss function. This step trains the BERT model based on the Masked Language Model (MLM). In this task, the masked language model randomly selects some words in the input business data for masking, and then tries to predict these masked words based on the unmasked context. The first loss function is the negative value of the sum of the logarithms of the probability of predicting the masked words based on the masked language model. First loss function: LMLM=-∑i∈M logP(xi|x\i;Θ) In this task, some words xi are randomly masked (M represents the index set of masked words), and the model tries to predict these masked words. x\i represents other words except masked words. Θ represents the model parameters, and P(xi|x\i;Θ) represents the probability of the model predicting the masked word xi given the context x\i.

[0041] Step S320, randomly select a pair of sentences in the training corpus, train the BERT model to determine whether the second sentence in the pair is the direct sequel of the first sentence, and obtain the second loss function. In this step, the BERT model is trained based on the next sentence prediction model. The next sentence prediction model receives a pair of sentences as input and predicts whether the second sentence is the direct sequel of the first sentence. The second loss function is the negative logarithm of the probability of predicting whether the sentence pair is continuous based on the next sentence prediction model. Second loss function: LNSP=-logP(y|x1,x2;Θ) In this task, the model receives two sentences x1 and x2 as input and predicts whether x2 is the next sentence of x1 (y is a binary label, y=1 means "yes", y=0 means "no"). Θ also represents the model parameters, P(y|x1,x2;Θ) represents the probability that the model predicts that x2 is the next sentence of x1.

[0042] Step S330, obtaining the sum of the first loss function and the second loss function of the BERT model during the training process to determine the training degree of the BERT model. During the training process of the BERT model, the total loss function is the sum of the first loss function and the second loss function: L=LMLM+LNSP.

[0043] By pre-training the BERT model on a rich training corpus, the model can learn to understand the complex structure of language and improve its performance in subsequent specific tasks (such as key data identification). This training method enables the BERT model to effectively process and understand complex data content.

[0044] After the BERT model is trained, the trained BERT model is used to identify key data in the preprocessed business data, for example, key data such as events, entities, sentiment tendencies, etc.

[0045] In one example, if Figure 4 As shown, the steps of training the BERT model based on the training corpus and using the trained BERT model to identify key data in the preprocessed business data include the following steps: Step S410, use the trained BERT model to extract features from the preprocessed business data to obtain the first type of feature representation. In Input Representation = BERT preprocessed (text), BERT preprocessed represents the representation of the preprocessed business data after passing through the BERT model. Features = BERT encode layers (Input Representation) After being processed by multiple encoder layers of the BERT model, the input representation of the business data is converted into a higher-level feature representation. This step is the key for the BERT model to capture and understand the context of the text. When performing feature extraction, not only the information of the current word should be considered, but also its contextual information should be considered to better capture the association and semantic information between words. As a pre-trained language representation method in natural language processing, BERT model embedding generates deep context-aware word vectors through a deep bidirectional Transformer model. According to the complexity and characteristics of the task, an adaptive feature selection method is used to select the most representative and discriminative features for recognition and classification.

[0046] Step S420, use the trained BERT model to perform feature classification or recognition tasks, and identify key data from the first type of feature representation. For each key data to be identified (such as events, entities, sentiment tendencies, etc.), the BERT model usually includes a specific output layer for performing classification or recognition tasks based on features. Elementi=soft max(Wi•Features+bi), where Elementi represents the i-th key data to be identified (such as event type, specific entity, sentiment level, etc.). Wi and bi are the weights and biases of the output layer. The softmax function is used to convert the output into a probability distribution. Among them, the key data includes at least event type, specific entity and sentiment level.

[0047] The above process allows the model to identify and classify key data in the input business data. For example, if the goal is to identify named entities in business data (such as names of people, places, organizations, etc.), the BERT model can predict the entity category of each word or phrase in the business data in this way. Similarly, for sentiment analysis, the model can determine the sentiment tendency of the entire sentence or document.

[0048] Step S130, using the trained BERT model to identify sensitive information in key data.

[0049] Categorize key data and divide the data into sensitive data and non-sensitive data. For example, sensitive data includes personal identity information, event core data, health data, etc. It is necessary to identify sensitive information from key data in order to desensitize the sensitive information. In one example, the step of using the trained BERT model to identify sensitive information in key data includes the steps of: extracting features from key data using the trained BERT model to obtain a second type of feature representation; performing feature classification or recognition tasks using the trained BERT model to identify sensitive information from the second type of feature representation. Among them, sensitive information includes sensitive information content, data type, data sensitivity, and contextual information.

[0050] Use the trained BERT model to identify sensitive information in the data, including both explicit and implicit sensitive data.

[0051] Sensitive information identification model: Sensitive Info = BERT trained (text), where BERT trained represents the trained BERT model that can identify and process text data. text is the key input data.

[0052] Feature extraction: Features = BER Tencoder layers (Sensitive Info). After being processed by multiple encoder layers of the BERT model, the model extracts key features from key data, which help identify sensitive information.

[0053] Sensitive data classification: Sensitive Typei=soft max(Wi•Features+bi), where Sensitive Typei represents the i-th category of sensitive information (such as personal identity information, financial information, etc.). Wi and bi are the weights and biases of the classification layer. The softmax function converts the output into a probability distribution, which is used to determine whether each data unit belongs to a specific sensitive type.

[0054] In this way, the BERT model can effectively identify various sensitive information in key data. This includes not only direct sensitive information (such as name, address, phone number, etc.), but also implicit sensitive data that may indirectly leak personal information. This step is a key link in data security desensitization processing to ensure that user privacy can be protected in subsequent use and analysis.

[0055] Step S140, according to the data type, data sensitivity and context information of the sensitive information, a corresponding desensitization strategy is selected from the desensitization strategy library, and the sensitive information is desensitized based on the desensitization strategy. The desensitization strategy library stores various desensitization strategies, including the following desensitization strategies: data masking, pseudonymization, encryption, generalization and perturbation.

[0056] In a specific process, the BERT model is used to adaptively select or adjust the desensitization strategy based on historical data and current context information. The BERT model can continuously optimize the dynamic strategy selection mechanism of the rule engine by analyzing the past desensitization effects and performance. According to the data type, data sensitivity and context information, the access logic selector automatically selects the most appropriate desensitization strategy: the trained BERT model is used to extract and classify sensitive information from the business data, and the corresponding desensitization strategy is matched from the desensitization strategy library for the different types of sensitive data extracted. If the extracted sensitive data is personal identity information (such as name, address, telephone number, etc.) or financial information (such as credit card number, bank account information, etc.), then the data masking and encryption strategy are associated; if it is user behavior data, then the pseudonymization and encryption method are associated; if it is password or data transmission related information, then the masking and encryption strategy are associated; if it is statistical data (such as demographics, economic data, etc.), geographic location and other related information, then the generalization and perturbation strategy is associated.

[0057] This application introduces a dynamic adjustment mechanism, which uses the adaptive learning ability of the BERT model to dynamically adjust the desensitization strategy according to the data type, data sensitivity and contextual information of the data. For example, the trained BERT model is used as a classifier to evaluate the sensitivity of the input data, and the input data is marked as high or low sensitivity. Based on the sensitivity assessment results, conditional logic is used to decide what desensitization measures to take: when facing highly sensitive data, the BERT model can automatically enhance the desensitization strategy, and when facing low-sensitivity data, a lighter desensitization strategy can be selected, thereby ensuring data security while retaining more data practicality. For the situation where multiple desensitization strategies exist at the same time, the workflow engine is connected to orchestrate the desensitization tasks, manage the execution order and conditions of different desensitization measures, and integrate the error handling mechanism to ensure that a desensitization step can be rolled back or retried when it fails.

[0058] In a specific example, according to the data type, data sensitivity and context information of sensitive information, a corresponding desensitization policy is selected from the desensitization policy library, and the step of desensitizing the sensitive information based on the desensitization policy also includes the steps of: marking sensitive data fields in sensitive information; according to the data type, data sensitivity and context information of sensitive information, a corresponding desensitization policy is selected from the desensitization policy library, and desensitizing the sensitive data fields of the digital city based on the desensitization policy; recording the digital city desensitization process log. By recording the log well, subsequent audits and adjustments are facilitated. The log includes which data is desensitized, which policies are used, and the results of the desensitization.

[0059] In order to continuously improve the desensitization effect, the desensitization effect of the current desensitization strategy is supervised. In an example, Figure 5 As shown, according to the data type, data sensitivity and context information of the sensitive information, a corresponding desensitization strategy is selected from the desensitization strategy library, and after the step of desensitizing the sensitive information based on the desensitization strategy, the step further includes: Step S510: monitor in real time the desensitization effect of desensitizing sensitive information based on the desensitization strategy to ensure that the desensitized data can still meet business requirements.

[0060] Step S520: After desensitizing sensitive information based on the desensitization strategy, user feedback and abnormal situations are collected, and the effectiveness of the desensitization strategy is analyzed based on the digital city user feedback and abnormal situations. User feedback can help identify which strategies perform better in actual applications and which strategies need to be improved. Abnormal situations are situations where the data is desensitized incorrectly or not desensitized during the data desensitization process.

[0061] Step S530: based on the digital city desensitization effect and digital city effectiveness, the desensitization strategy is adjusted to improve the desensitization effect and system performance.

[0062] Through continuous training and optimization, the BERT model can continuously adapt to new data environments and sensitivity requirements. For example, over time, the BERT model can maintain its high accuracy and desensitization effect on sensitive data by continuously learning new data samples.

[0063] This application data desensitization method incorporates professional knowledge in the field of social governance into the data element extraction process, ensuring that the output of the BERT model is closely related to the actual application scenario and enhancing the practicality of data interpretation. The identified data elements are classified and labeled, and the data structure is optimized. Through structural optimization, the efficiency and consistency of data processing are improved.

[0064] The data desensitization method of this application also includes the following steps: the system will automatically retrieve and integrate the latest social governance data regularly to maintain the timeliness and relevance of the training corpus; in addition to updating the data, the system will automatically iterate the BERT model regularly to adapt to new data patterns. The system will continuously monitor the quality of the processed data to ensure its accuracy, consistency and completeness; introduce automated tools to promptly identify and handle any data anomalies or deviations. Based on new data sets and feedback, the system automatically adjusts the desensitization processing strategy to maintain its security and effectiveness; implements a sensitivity rating system to automatically evaluate the sensitivity of new data sets and adjust the desensitization intensity and strategy accordingly. The system regularly collects user feedback and performance data as a basis for continuous improvement; based on the collected feedback, automatically adjusts the data processing strategy to improve the overall performance of the system. Regular system maintenance is performed to ensure that all components and modules are running in the best state; as new technologies and methods develop, the system will integrate these innovations in a timely manner to ensure continued leadership. The complete data processing system is deployed on the corresponding hardware and software platforms; the system is applied to the field of social governance to improve the efficiency and accuracy of policy making and public opinion analysis.

[0065] The data desensitization method provided in each embodiment of the present application includes the following steps: collecting business data of the digital city and preprocessing the business data using the BERT model; training the BERT model based on the training corpus, and using the trained BERT model to identify key data in the preprocessed business data; using the trained BERT model to identify sensitive information in the key data; selecting a corresponding desensitization strategy from a desensitization strategy library according to the data type, data sensitivity and context information of the sensitive information, and desensitizing the sensitive information based on the desensitization strategy. The present application provides a comprehensive solution for desensitizing sensitive information by preprocessing business data, identifying key data and desensitizing the business data, which can efficiently process and understand complex data content. In addition, in the desensitization process, the desensitization strategy is dynamically selected and adjusted based on the data type, data sensitivity and context information, so that the data desensitization process can be adaptively adjusted according to the specific environment and sensitivity level of data use, thereby ensuring that the desensitization technology can maintain its effectiveness and applicability in different application scenarios, thereby greatly improving the security and practicality of data processing.

[0066] This application provides a comprehensive solution through data element identification and security desensitization based on the BERT model, which can efficiently process and understand complex data content. Compared with the existing technology, this application not only focuses on improving the efficiency of data processing, but also pays more attention to protecting the security and privacy of data, especially in processing data containing implicit sensitive information, such as personal habits or behavior patterns, providing more comprehensive protection measures.

[0067] This application can intelligently identify sensitive information in data and perform effective desensitization. Compared with the prior art, the method of this application can not only process obvious personal identification information, but also process more hidden sensitive content, thereby providing a safer data processing environment.

[0068] A significant advantage of this application is its dynamic adjustment mechanism, which allows the data desensitization process to be adaptively adjusted according to the specific environment and sensitivity level of data use. This flexibility ensures that the desensitization technology can maintain its effectiveness and applicability in different application scenarios, thereby greatly improving the security and practicality of data processing.

[0069] Compared with the existing technology, this application has a wider scope of application and better effect in data desensitization. Whether facing changing data types or different processing requirements, the dynamic adjustment mechanism of this application can provide accurate desensitization strategies to ensure the security of data during use.

[0070] It should be understood that although Figures 1 to 5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figures 1 to 5 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0071] In one embodiment, a data desensitization device based on a BERT model is provided, and the digital city device includes: The data collection and preprocessing module is used to collect the business data of the digital city and preprocess the business data using the BERT model; the business data at least includes public feedback information, policy document information and social event information; Model training and data identification module, which is used to train the BERT model based on the training corpus and use the trained BERT model to identify key data in the preprocessed business data; Sensitive information identification module, which is used to identify sensitive information in key data using the trained BERT model; The desensitization processing module is used to select the corresponding desensitization strategy from the desensitization strategy library according to the data type, data sensitivity and context information of the sensitive information, and desensitize the sensitive information based on the desensitization strategy; the desensitization strategy library includes the following desensitization strategies: data masking, pseudonymization, encryption, generalization and perturbation.

[0072] For the specific limitations of the data desensitization device based on the BERT model, please refer to the limitations of the data desensitization method based on the BERT model above, which will not be repeated here. Each module in the above-mentioned data desensitization device based on the BERT model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0073] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store business data of the digital city. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data desensitization method based on the BERT model is implemented.

[0074] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0075] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented: Collect business data of the digital city and pre-process the business data using the BERT model; the business data at least includes public feedback information, policy document information, and social event information; Train the BERT model based on the training corpus, and use the trained BERT model to identify key data in the preprocessed business data; Use the trained BERT model to identify sensitive information in key data; According to the data type, data sensitivity and contextual information of the sensitive information, a corresponding desensitization strategy is selected from the desensitization strategy library, and the sensitive information is desensitized based on the desensitization strategy; the desensitization strategy library includes the following desensitization strategies: data masking, pseudonymization, encryption, generalization and perturbation.

[0076] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Collect business data of the digital city and pre-process the business data using the BERT model; the business data at least includes public feedback information, policy document information, and social event information; Train the BERT model based on the training corpus, and use the trained BERT model to identify key data in the preprocessed business data; Use the trained BERT model to identify sensitive information in key data; According to the data type, data sensitivity and contextual information of the sensitive information, a corresponding desensitization strategy is selected from the desensitization strategy library, and the sensitive information is desensitized based on the desensitization strategy; the desensitization strategy library includes the following desensitization strategies: data masking, pseudonymization, encryption, generalization and perturbation.

[0077] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0078] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A data desensitization method based on the BERT model, characterized in that: The following steps are involved: Collecting business data of the digital city and preprocessing the business data using the BERT model; the business data at least includes public feedback information, policy document information and social event information; Training the BERT model based on a training corpus, and using the trained BERT model to identify key data in the preprocessed business data; Using the trained BERT model to identify sensitive information in the key data; According to the data type, data sensitivity and context information of the sensitive information, a corresponding desensitization strategy is selected from a desensitization strategy library, and the sensitive information is desensitized based on the desensitization strategy; the desensitization strategy library includes the following desensitization strategies: data masking, pseudonymization, encryption, generalization and perturbation.

2. The data desensitization method based on the BERT model according to claim 1, characterized in that: In the step of collecting the business data of the digital city and preprocessing the business data using the BERT model, the BERT model includes multiple encoder layers, and each encoder layer performs the following steps: Generate corresponding vector representations for the words in the business data using the BERT model based on word embeddings and position embeddings corresponding to the words in the business data; Using the BERT model to perform a linear transformation on the vector representation based on a self-attention layer to obtain a corresponding query, key, and value, and performing a weighted update on the vector representation according to the query, the key, and the value; The vector representation weighted and updated by the self-attention layer is nonlinearly transformed using a feedforward network; The vector representation that has been nonlinearly transformed by the feedforward network is subjected to layer normalization and residual connection processing.

3. The data desensitization method based on the BERT model according to claim 2, characterized in that: The step of collecting the business data of the digital city and preprocessing the business data using the BERT model also includes the following steps: The BERT model is used to segment the business data based on the dictionary or context information corresponding to the digital city and by selecting a word-level segmentation method or a subword-level segmentation method according to the category of the vocabulary in the business data.

4. The data desensitization method based on the BERT model according to claim 2, characterized in that: The step of collecting the business data of the digital city and preprocessing the business data using the BERT model also includes the following steps: If the length of the business data exceeds the input limit of the BERT model, the business data is processed by truncation or padding method so that the length of the processed business data meets the input limit of the BERT model.

5. The data desensitization method based on the BERT model according to claim 1, characterized in that: The step of training the BERT model based on a training corpus and using the trained BERT model to identify key data in the preprocessed business data includes the following steps: Randomly selecting some words in the training corpus for masking, training the BERT model to predict the masked part of the words according to the unmasked context corresponding to the part of the words, and obtaining a first loss function; Randomly selecting a pair of sentences in the training corpus, training the BERT model to determine whether the second sentence in the pair of sentences is a direct sequel to the first sentence, and obtaining a second loss function; Obtain the sum of the first loss function and the second loss function of the BERT model during the training process to determine the training degree of the BERT model.

6. The data desensitization method based on the BERT model according to claim 5, characterized in that: The step of training the BERT model based on a training corpus and using the trained BERT model to identify key data in the preprocessed business data includes the following steps: Using the trained BERT model to perform feature extraction on the preprocessed business data to obtain a first type of feature representation; The trained BERT model is used to perform feature classification or recognition tasks to identify key data from the first type of feature representation; the key data includes at least event type, specific entity and sentiment level.

7. The data desensitization method based on the BERT model according to claim 1, characterized in that: The step of using the trained BERT model to identify sensitive information in the key data includes the following steps: Using the trained BERT model to extract features from the key data to obtain a second type of feature representation; The trained BERT model is used to perform feature classification or recognition tasks to identify sensitive information from the second type of feature representation; the sensitive information includes sensitive information content, data type, data sensitivity and contextual information.

8. The data desensitization method based on the BERT model according to claim 1, characterized in that: The step of selecting a corresponding desensitization strategy from a desensitization strategy library according to the data type, data sensitivity and context information of the sensitive information, and performing desensitization processing on the sensitive information based on the desensitization strategy, further includes the steps of: marking sensitive data fields in the sensitive information; According to the data type, data sensitivity and context information of the sensitive information, a corresponding desensitization strategy is selected from a desensitization strategy library, and the sensitive data field is desensitized based on the desensitization strategy; Record the desensitization process log.

9. The data desensitization method based on the BERT model according to claim 1, characterized in that: After selecting a corresponding desensitization strategy from a desensitization strategy library according to the data type, data sensitivity and context information of the sensitive information, and performing desensitization processing on the sensitive information based on the desensitization strategy, the method further includes the following steps: Real-time monitoring of the desensitization effect of desensitizing the sensitive information based on the desensitization strategy; After desensitizing the sensitive information based on the desensitization strategy, collecting user feedback and abnormal situations, and analyzing the effectiveness of the desensitization strategy based on the user feedback and abnormal situations; Based on the desensitization effect and the effectiveness, the desensitization strategy is adjusted.

10. A data desensitization device based on the BERT model, characterized in that: The device comprises: The data collection and preprocessing module is used to collect the business data of the digital city and preprocess the business data using the BERT model; the business data at least includes public feedback information, policy document information and social event information; A model training and data identification module, used to train the BERT model based on a training corpus, and use the trained BERT model to identify key data in the preprocessed business data; A sensitive information identification module, used to identify sensitive information in the key data using the trained BERT model; A desensitization processing module is used to select a corresponding desensitization strategy from a desensitization strategy library according to the data type, data sensitivity and context information of the sensitive information, and perform desensitization processing on the sensitive information based on the desensitization strategy; the desensitization strategy library includes the following desensitization strategies: data masking, pseudonymization, encryption, generalization and perturbation.

Images