A method and apparatus for desensitizing data

By performing semantic encoding and projection decomposition on text data and injecting noise only into the privacy perturbation component, the problem of low semantic fidelity in existing technologies is solved, and high-fidelity text desensitization is achieved.

CN123116340APending Publication Date: 2026-11-17AGRICULTURAL BANK OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611559097.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-09
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies, in the process of desensitizing text data, directly add Gaussian noise to the word embedding vector, causing synchronous perturbation of sensitive and non-sensitive semantic information, thus reducing the semantic fidelity of the desensitized text.

Method used

By acquiring the text to be processed and performing semantic encoding, sensitive entities are identified. The semantic decoupling projection matrix is ​​used to decompose the word vector into semantic fidelity components and privacy perturbation components. Differential privacy noise is injected only into the privacy perturbation components, and desensitization results are generated by combining the structured information extraction model.

Benefits of technology

While achieving the desensitization of sensitive information, the semantic fidelity of the desensitized semantic vector sequence is improved, ensuring the integrity of the contextual semantics.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123116340A_ABST
    Figure CN123116340A_ABST
Patent Text Reader

Abstract

This application discloses a data desensitization method and apparatus, comprising: performing semantic encoding and sensitive entity identification on the text to be processed, determining the target word vector corresponding to the sensitive entity annotation information, and using a semantic decoupling projection matrix to project and decompose the target word vector into semantically faithful component vectors and privacy perturbation component vectors, so that the information related to the context semantics and the information related to the sensitive attributes are respectively carried in different components; further performing differential privacy noise injection processing on the privacy perturbation component vector, and combining it with the semantically faithful component vector for vector desensitization processing, thereby avoiding the addition of noise to the target word vector as a whole, which would cause the information related to the context semantics to be simultaneously perturbed. While achieving sensitive information desensitization, it improves the semantic fidelity of the desensitized semantic vector sequence, and can generate desensitized structured information records based on the desensitized semantic vector sequence to support subsequent information processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and apparatus for de-identifying data. Background Technology

[0002] With the development of artificial intelligence and big data technologies, text data is widely used in natural language processing, data analysis, and information retrieval. However, text often contains sensitive information such as names, ID numbers, phone numbers, and bank account numbers. Therefore, before sharing and analyzing text data, it is necessary to de-identify sensitive information and maintain the original contextual semantics of the text as much as possible, so that the processed data can still be used for subsequent analysis and information extraction.

[0003] Existing technology converts text into word embedding vectors, adds Gaussian noise to the word embedding vectors, records the gradient directions of the target model and the attack model, and then uses the recorded gradient directions to guide the diffusion model to perform inverse diffusion sampling to generate desensitized text, thereby achieving text desensitization based on semantic representation.

[0004] Because existing technologies directly add Gaussian noise to word embedding vectors, sensitive and non-sensitive semantic information are simultaneously disturbed, thereby reducing the semantic fidelity of desensitized text. Summary of the Invention

[0005] In view of the shortcomings of the prior art, this application provides a data desensitization method and apparatus to solve the problem of low semantic fidelity of desensitized text.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] The first aspect of this application provides a method for de-identifying data, including:

[0008] The text to be processed is obtained, the text to be processed is semantically encoded to obtain an initial semantic vector sequence, and the text to be processed is sensitive entity identification to obtain sensitive entity annotation information in the text to be processed.

[0009] The target word vector corresponding to the sensitive entity annotation information is determined from the initial semantic vector sequence, and the target word vector is projected and decomposed according to the semantic decoupling projection matrix to obtain the semantic fidelity component vector and the privacy perturbation component vector; wherein, the semantic fidelity component vector carries information related to the semantic context of the text to be processed, and the privacy perturbation component vector carries information related to the sensitive attributes of the text to be processed.

[0010] Differential privacy noise injection is performed on the privacy perturbation component vector to obtain the perturbed privacy perturbation component vector;

[0011] The semantic fidelity component vector and the perturbed privacy perturbation component vector are subjected to vector desensitization processing to obtain a desensitized semantic vector sequence;

[0012] The desensitized semantic vector sequence is input into the structured information extraction model, and the structured information extraction model performs reasoning processing on the desensitized semantic vector sequence to generate desensitized structured information records.

[0013] The desensitized semantic vector sequence and the desensitized structured information record are associated and encapsulated to obtain the desensitized result corresponding to the text to be processed.

[0014] Optionally, in the above data desensitization method, the training method for the semantic decoupling projection matrix includes:

[0015] Acquire training text and perform semantic encoding on the training text to obtain an initial training semantic vector sequence;

[0016] The training text is subjected to sensitive entity recognition using a sensitive entity recognition model to obtain the training sensitive entity annotation information of the training text, and the word vectors corresponding to the training sensitive entity annotation information are determined from the initial training semantic vector sequence.

[0017] Construct a dual-branch encoder network; wherein the dual-branch encoder network includes at least a shared bottom-level feature extraction layer, a semantic fidelity branch, and a privacy perturbation branch;

[0018] The word vector is input into the dual-branch encoder network, and the word vector is subjected to feature extraction through the shared bottom-level feature extraction layer to obtain the intermediate representation vector;

[0019] The intermediate representation vector is processed by the semantic fidelity branch and the privacy perturbation branch respectively to obtain the semantic fidelity component prediction value and the privacy perturbation component prediction value;

[0020] Based on the training sensitive entity annotation information, the semantic fidelity component prediction value, and the privacy perturbation component prediction value, a joint loss function is calculated;

[0021] Determine whether the joint loss function converges;

[0022] If the joint loss function converges, the first weight matrix corresponding to the semantic fidelity branch and the second weight matrix corresponding to the privacy perturbation branch are obtained from the dual-branch encoder network, and the semantic decoupling projection matrix is ​​generated based on the first weight matrix and the second weight matrix.

[0023] If the joint loss function does not converge, the parameters of the dual-branch encoder network are adjusted, and the process of inputting the word vector into the dual-branch encoder network and extracting features from the word vector through the shared bottom-level feature extraction layer is returned to obtain the intermediate representation vector.

[0024] Optionally, in the above data desensitization method, the step of performing differential privacy noise injection processing on the privacy perturbation component vector to obtain the perturbated privacy perturbation component vector includes:

[0025] Calculate the norm of the privacy perturbation component vector;

[0026] When the norm is greater than the preset pruning threshold, the privacy perturbation component vector is scaled to a norm equal to the pruning threshold to obtain the pruned vector;

[0027] Obtain the privacy budget parameter corresponding to the sensitive entity category to which the target word vector belongs;

[0028] A random noise vector is generated based on the privacy budget parameter and the preset noise multiplier;

[0029] The random noise vector is superimposed on the clipped vector to obtain the perturbed privacy perturbation component vector.

[0030] Optionally, in the above data desensitization method, before calculating the norm of the privacy perturbation component vector, the method further includes:

[0031] Extract all target privacy perturbation component vectors marked as sensitive entity words from the initial semantic vector sequence, and construct a privacy perturbation component sample set based on each extracted target privacy perturbation component vector;

[0032] Calculate the norm of each target privacy perturbation component vector in the privacy perturbation component sample set to obtain a norm sequence;

[0033] The norm sequence is sorted in ascending order, and the norm value located at the preset percentile position after sorting is selected as the clipping threshold.

[0034] Optionally, in the above data desensitization method, the step of performing vector desensitization processing on the semantic fidelity component vector and the perturbed privacy perturbation component vector to obtain a desensitized semantic vector sequence includes:

[0035] The vector sum of the semantic fidelity component vector and the perturbed privacy perturbation component vector is calculated to obtain the reconstructed target word vector;

[0036] Obtain the original target word vector corresponding to the reconstructed target word vector in the initial semantic vector sequence;

[0037] The original target word vectors in the initial semantic vector sequence are replaced with the reconstructed target word vectors to obtain the desensitized semantic vector sequence.

[0038] Optionally, in the above data desensitization method, after replacing the original target word vector in the initial semantic vector sequence with the reconstructed target word vector to obtain the desensitized semantic vector sequence, the method further includes:

[0039] Obtain the context window vector set of each preset number of non-sensitive word vectors before and after the original target word vector in the text to be processed;

[0040] Calculate the first cosine similarity between the reconstructed target word vector and the mean vector of the context window vector set, and calculate the second cosine similarity between the original target word vector and the mean vector of the context window vector set;

[0041] When the difference between the first cosine similarity and the second cosine similarity is greater than the preset semantic offset tolerance threshold, the reconstructed target word vector is subjected to offset correction operation until the difference is not greater than the semantic offset tolerance threshold or the maximum number of iterations is reached.

[0042] Optionally, in the above-described data desensitization method, the training method for the structured information extraction model includes:

[0043] Obtaining non-sensitive public text;

[0044] The non-sensitive public text is input into a pre-trained language representation model to obtain a training semantic vector sequence;

[0045] The trained semantic vector sequence is input into the initialized structured information extraction model for multi-task fine-tuning training to obtain the trained desensitized structured information record;

[0046] Based on the aforementioned desensitized structured information records from the training, the joint task loss function is determined;

[0047] Determine whether the joint task loss function has converged;

[0048] If the joint task loss function converges, the initialized structured information extraction model will be determined as the trained structured information extraction model.

[0049] If the joint task loss function fails to converge, the parameters of the structured information extraction model are adjusted, and the process returns to inputting the training semantic vector sequence into the initialized structured information extraction model for multi-task fine-tuning training to obtain the training desensitized structured information record.

[0050] Optionally, in the above data desensitization method, the step of associating and encapsulating the desensitized semantic vector sequence and the desensitized structured information record to obtain the desensitization result corresponding to the text to be processed includes:

[0051] Construct a data encapsulation structure; wherein the data encapsulation structure includes vector data fields and structured data fields;

[0052] The desensitized semantic vector sequence is arranged according to a preset word order, and the arranged desensitized semantic vector sequence is written into the vector data field;

[0053] The desensitized structured information record is serialized to obtain structured serialized data, and the structured serialized data is written into the structured data field;

[0054] Obtain the text identifier information corresponding to the text to be processed and the privacy parameter information corresponding to the differential privacy noise injection processing;

[0055] The vector data field, the structured data field, the text identification information, and the privacy parameter information are associated and encapsulated to obtain the de-identification result corresponding to the text to be processed.

[0056] A second aspect of this application provides a data desensitization apparatus, comprising:

[0057] The text processing unit is used to acquire the text to be processed, perform semantic encoding on the text to be processed to obtain an initial semantic vector sequence, and perform sensitive entity recognition on the text to be processed to obtain sensitive entity annotation information in the text to be processed.

[0058] The projection decomposition unit is used to determine the target word vector corresponding to the sensitive entity annotation information from the initial semantic vector sequence, and to perform projection decomposition on the target word vector according to the semantic decoupling projection matrix to obtain semantic fidelity component vector and privacy perturbation component vector; wherein, the semantic fidelity component vector carries information related to the semantic context of the text to be processed, and the privacy perturbation component vector carries information related to the sensitive attributes of the text to be processed.

[0059] A noise injection unit is used to perform differential privacy noise injection processing on the privacy perturbation component vector to obtain the perturbed privacy perturbation component vector;

[0060] The vector desensitization unit is used to perform vector desensitization processing on the semantic fidelity component vector and the perturbed privacy perturbation component vector to obtain a desensitized semantic vector sequence.

[0061] The information extraction unit is used to input the desensitized semantic vector sequence into the structured information extraction model, and to perform reasoning processing on the desensitized semantic vector sequence through the structured information extraction model to generate desensitized structured information records.

[0062] The association and encapsulation unit is used to associate and encapsulate the desensitized semantic vector sequence and the desensitized structured information record to obtain the desensitization result corresponding to the text to be processed.

[0063] Optionally, the above-mentioned data desensitization device further includes:

[0064] The text acquisition unit is used to acquire training text and perform semantic encoding on the training text to obtain an initial training semantic vector sequence;

[0065] The entity recognition unit is used to perform sensitive entity recognition on the training text using a sensitive entity recognition model, obtain the training sensitive entity annotation information of the training text, and determine the word vector corresponding to the training sensitive entity annotation information from the initial training semantic vector sequence.

[0066] A network construction unit is used to construct a dual-branch encoder network; wherein the dual-branch encoder network includes at least a shared bottom-level feature extraction layer, a semantic fidelity branch, and a privacy perturbation branch;

[0067] The feature extraction unit is used to input the word vector into the dual-branch encoder network, and extract features from the word vector through the shared bottom feature extraction layer to obtain an intermediate representation vector;

[0068] The branch processing unit is used to process the intermediate representation vector through the semantic fidelity branch and the privacy perturbation branch respectively to obtain the semantic fidelity component prediction value and the privacy perturbation component prediction value.

[0069] The loss calculation unit is used to calculate the joint loss function based on the training sensitive entity annotation information, the semantic fidelity component prediction value, and the privacy perturbation component prediction value.

[0070] A convergence determination unit is used to determine whether the joint loss function has converged;

[0071] The matrix generation unit is used to obtain the first weight matrix corresponding to the semantic fidelity branch and the second weight matrix corresponding to the privacy perturbation branch from the dual-branch encoder network if the joint loss function converges, and generate the semantic decoupling projection matrix based on the first weight matrix and the second weight matrix.

[0072] The parameter adjustment unit is used to adjust the parameters of the dual-branch encoder network if the joint loss function fails to converge, and then return to the process of inputting the word vector into the dual-branch encoder network and extracting features from the word vector through the shared bottom-level feature extraction layer to obtain an intermediate representation vector.

[0073] Optionally, in the above-described data desensitization device, the noise injection unit includes:

[0074] Norm calculation unit, used to calculate the norm of the privacy perturbation component vector;

[0075] The vector clipping unit is used to scale the privacy perturbation component vector to a norm equal to the clipping threshold when the norm is greater than a preset clipping threshold, so as to obtain a clipped vector.

[0076] The parameter acquisition unit is used to acquire the privacy budget parameter corresponding to the sensitive entity category to which the target word vector belongs;

[0077] A noise generation unit is used to generate a random noise vector based on the privacy budget parameter and a preset noise multiplier;

[0078] The noise superposition unit is used to superimpose the random noise vector onto the clipped vector to obtain the perturbed privacy perturbation component vector.

[0079] Optionally, the above-mentioned data desensitization device further includes:

[0080] The sample construction unit is used to extract all target privacy perturbation component vectors marked as sensitive entity words from the initial semantic vector sequence, and to construct a privacy perturbation component sample set based on the extracted target privacy perturbation component vectors.

[0081] The norm statistics unit is used to calculate the norm of each target privacy perturbation component vector in the privacy perturbation component sample set, and obtain the norm sequence.

[0082] The threshold determination unit is used to sort the norm sequence in ascending order and select the norm value located at a preset percentile position after sorting as the clipping threshold.

[0083] Optionally, in the above-described data desensitization device, the vector desensitization unit includes:

[0084] The vector reconstruction unit is used to calculate the vector sum of the semantic fidelity component vector and the perturbed privacy perturbation component vector to obtain the reconstructed target word vector;

[0085] A vector acquisition unit is used to acquire the original target word vector corresponding to the reconstructed target word vector in the initial semantic vector sequence;

[0086] The vector replacement unit is used to replace the original target word vector in the initial semantic vector sequence with the reconstructed target word vector to obtain a desensitized semantic vector sequence.

[0087] Optionally, the above-mentioned data desensitization device further includes:

[0088] The window acquisition unit is used to acquire a set of context window vectors for each preset number of non-sensitive word vectors before and after the original target word vector in the text to be processed.

[0089] The similarity calculation unit is used to calculate the first cosine similarity between the reconstructed target word vector and the mean vector of the context window vector set, and to calculate the second cosine similarity between the original target word vector and the mean vector of the context window vector set.

[0090] The offset correction unit is used to perform offset correction operation on the reconstructed target word vector when the difference between the first cosine similarity and the second cosine similarity is greater than a preset semantic offset tolerance threshold, until the difference is not greater than the semantic offset tolerance threshold or the maximum number of iterations is reached.

[0091] Optionally, the above-mentioned data desensitization device further includes:

[0092] The acquisition unit is used to acquire non-sensitive public text;

[0093] The generation unit is used to input the non-sensitive public text into a pre-trained language representation model to obtain a training semantic vector sequence;

[0094] The fine-tuning training unit is used to input the training semantic vector sequence into the initialized structured information extraction model for multi-task fine-tuning training, so as to obtain the training desensitized structured information record;

[0095] The loss determination unit is used to determine the joint task loss function based on the training desensitized structured information record;

[0096] The judgment unit is used to determine whether the joint task loss function has converged;

[0097] The model determination unit is used to determine the initialized structured information extraction model as the trained structured information extraction model if the joint task loss function converges.

[0098] The adjustment unit is used to adjust the parameters of the structured information extraction model if the joint task loss function fails to converge, and then return to the process of inputting the training semantic vector sequence into the initialized structured information extraction model for multi-task fine-tuning training to obtain the training desensitized structured information record.

[0099] Optionally, in the above-described data desensitization device, the associated encapsulation unit includes:

[0100] A structural building unit is used to construct a data encapsulation structure; wherein the data encapsulation structure includes vector data fields and structured data fields;

[0101] The vector writing unit is used to arrange the desensitized semantic vector sequence according to a preset word order, and write the arranged desensitized semantic vector sequence into the vector data field;

[0102] The data serialization unit is used to serialize the desensitized structured information record to obtain structured serialized data, and write the structured serialized data into the structured data field;

[0103] The information acquisition unit is used to acquire the text identifier information corresponding to the text to be processed and the privacy parameter information corresponding to the differential privacy noise injection processing;

[0104] The data encapsulation unit is used to associate and encapsulate the vector data field, the structured data field, the text identification information, and the privacy parameter information to obtain the de-identification result corresponding to the text to be processed.

[0105] This application provides a data desensitization method. First, it acquires the text to be processed, performs semantic encoding on the text to obtain an initial semantic vector sequence, and then identifies sensitive entities in the text to obtain sensitive entity annotation information. Next, it determines the target word vectors corresponding to the sensitive entity annotation information from the initial semantic vector sequence, and performs projection decomposition on the target word vectors according to the semantic decoupling projection matrix to obtain semantic fidelity component vectors and privacy perturbation component vectors. The semantic fidelity component vectors carry information related to the semantic context of the text to be processed, while the privacy perturbation component vectors carry information related to the semantic context of the text to be processed. The process begins by decomposing the sensitive attributes of the text to be processed. Next, differential privacy noise injection is applied to the privacy perturbation component vector to obtain a perturbed privacy perturbation component vector. Then, vector desensitization processing is performed on both the semantic fidelity component vector and the perturbed privacy perturbation component vector to obtain a desensitized semantic vector sequence. This desensitized semantic vector sequence is then input into a structured information extraction model. The model performs inference processing on the desensitized semantic vector sequence to generate a desensitized structured information record. Finally, the desensitized semantic vector sequence and the desensitized structured information record are associated and encapsulated to obtain the desensitized result corresponding to the text to be processed. Thus, by projectively decomposing the target word vector and applying differential privacy noise injection only to the privacy perturbation component vector, the perturbation to the semantic fidelity component vector is reduced, thereby improving the semantic fidelity of the desensitized semantic vector sequence while achieving sensitive information desensitization. Attached Figure Description

[0106] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0107] Figure 1 A flowchart illustrating a data desensitization method provided in an embodiment of this application;

[0108] Figure 2 A flowchart illustrating a method for generating privacy perturbation component vectors after perturbation, provided for another embodiment of this application;

[0109] Figure 3 A flowchart illustrating a method for determining a clipping threshold, provided in another embodiment of this application;

[0110] Figure 4 A flowchart illustrating a method for generating a desensitized semantic vector sequence, provided in another embodiment of this application;

[0111] Figure 5This is a schematic diagram of a data desensitization device provided in another embodiment of this application. Detailed Implementation

[0112] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0113] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0114] This application provides a data desensitization method, such as... Figure 1 As shown, the specific steps include:

[0115] S101. Obtain the text to be processed, perform semantic encoding on the text to be processed to obtain an initial semantic vector sequence, and perform sensitive entity recognition on the text to be processed to obtain sensitive entity annotation information in the text to be processed.

[0116] Understandably, the text to be processed refers to the raw text data that needs to be protected for privacy, which may come from database storage, file uploads, API calls, or real-time streaming input.

[0117] Optionally, text data stored in a database can be retrieved in batches via a query interface. For text data uploaded from files, the file format can be parsed and the text content extracted. For text data from API calls, text fields can be extracted from the request body. For real-time streaming text data, it can be processed in batches according to a preset time window or data volume threshold. Text from different sources is uniformly converted to a standard text format before entering subsequent processing, thus enabling the same de-identification process to cover scenarios such as offline batch processing and online real-time processing.

[0118] Semantic encoding refers to the process of converting words or characters in text into continuous numerical vectors that can be processed by a computer. Specifically, pre-trained language representation models can be used to perform semantic encoding on the text to be processed. These pre-trained language representation models can employ BERT, RoBERTa, ERNIE, or other Transformer architecture models trained on large-scale corpora. Through semantic encoding, each word in the text to be processed can be converted into a high-dimensional dense numerical representation, forming an initial sequence of semantic vectors. Each vector in the initial sequence of semantic vectors carries the semantic information of the corresponding word in its context.

[0119] Sensitive entity recognition refers to the technology of automatically detecting and labeling word segments belonging to preset sensitive entity categories from text using natural language processing models. Preset sensitive entity categories can include personal privacy or legally protected information categories such as names, ID numbers, phone numbers, home addresses, bank account numbers, and email addresses. Sensitive entity recognition models can employ deep learning-based sequence labeling models, such as BiLSTM-CRF, BERT-CRF, or other named entity recognition models. Sensitive entity labeling information is used to mark the location of sensitive entity words and their categories. A corresponding privacy budget parameter can be preset for each sensitive entity category. The privacy budget parameter quantifies the strength of differential privacy protection; a smaller value indicates a relatively larger amount of added noise and a higher degree of privacy protection. Different privacy budget configurations can be used for entities with different levels of sensitivity; for example, highly sensitive entities such as ID numbers use smaller privacy budget values, while general personal information uses larger privacy budget values.

[0120] In some embodiments, the text can be preprocessed before semantic encoding. Preprocessing may include removing irrelevant whitespace characters, standardizing encoding formats, standardizing punctuation marks, and word segmentation. Word segmentation divides the text into a sequence of tokens. A token is the smallest semantic unit obtained after word segmentation, typically a word or subword. Word segmentation can employ dictionary-based, statistical, or deep learning-based methods.

[0121] Optionally, in another embodiment of this application, to adapt to text data of different language types, the corresponding pre-trained language representation model and sensitive entity recognition model can be selected according to the language type of the text to be processed. For example, for Chinese text, a model pre-trained on Chinese corpora can be used. For English text, a model pre-trained on English corpora can be used. For multilingual mixed text, a multilingual pre-trained model can be used.

[0122] S102. Determine the target word vector corresponding to the sensitive entity annotation information from the initial semantic vector sequence, and perform projection decomposition on the target word vector according to the semantic decoupling projection matrix to obtain the semantic fidelity component vector and the privacy perturbation component vector.

[0123] Among them, the semantic fidelity component vector carries information related to the semantic context of the text to be processed, and the privacy perturbation component vector carries information related to the sensitive attributes of the text to be processed.

[0124] It is understandable that the target lexical vector refers to the vector in the initial semantic vector sequence that is marked as a sensitive entity lexical. Semantic decoupling refers to the process of decomposing a lexical vector into multiple independent sub-vectors carrying information of different properties, so that each sub-vector corresponds to different factors such as semantic content or privacy attributes. Through semantic decoupling, the semantic information of sensitive entities can be separated from their privacy attribute information, so that in subsequent privacy protection operations, perturbations are applied only to the privacy attribute information, avoiding collateral damage to the semantic information.

[0125] Specifically, based on the sensitive entity word positions recorded in the sensitive entity annotation information, the target word vector at the corresponding position can be determined from the initial semantic vector sequence. For each target word vector marked as a sensitive entity word, projection decomposition can be performed using a pre-trained semantic decoupling projection matrix.

[0126] In one implementation, a pre-trained semantic decoupling projection matrix is ​​used. For target word vectors Perform a linear transformation to obtain the semantically faithful component vector. and privacy perturbation component vector ,in and To decouple the projection matrix based on semantics The constructed orthogonal projection operator satisfies and Semantic decoupling projection matrix The network is obtained by training a dual-branch encoder network on a non-sensitive public corpus with the joint training objective of maximizing the correlation between semantic fidelity components and contextual semantic labels and minimizing the correlation between privacy perturbation components and sensitive attribute labels.

[0127] Therefore, optionally, in this embodiment of the application, a method for training a semantic decoupling projection matrix is ​​provided, specifically including the following processes A1-A9:

[0128] A1: Obtain the training text and perform semantic encoding on the training text to obtain the initial semantic vector sequence for training.

[0129] Specifically, the training text can be provided by a training corpus consisting of non-sensitive publicly available text. The training corpus should cover text data in similar domains and styles to the text to be processed to ensure that the semantic decoupling projection matrix obtained through training has good generalization ability. The semantic encoding processing of the training text can adopt the same pre-trained language representation model as in step S101 to ensure the consistency of the vector space.

[0130] A2: Use a sensitive entity recognition model to perform sensitive entity recognition on the training text, obtain the training sensitive entity annotation information of the training text, and determine the word vectors corresponding to the training sensitive entity annotation information from the initial semantic vector sequence of training.

[0131] Specifically, a sensitive entity recognition model can be used to label the categories of sensitive entity lexical units in the training text, and dependency parsing tools can be used to generate contextual semantic labels corresponding to the training text. The training sensitive entity annotation information can be used to provide sensitive entity category labels, and the contextual semantic labels can be used to represent the contextual semantics corresponding to the lexical units.

[0132] A3: Construct a dual-branch encoder network.

[0133] The dual-branch encoder network includes at least a shared bottom-level feature extraction layer, a semantic fidelity branch, and a privacy perturbation branch.

[0134] Understandably, the shared underlying feature extraction layer is used to extract features from the input word vectors. The semantic fidelity branch is used to obtain the predicted values ​​of the semantic fidelity components associated with the context semantics. The privacy perturbation branch is used to obtain the predicted values ​​of the privacy perturbation components associated with sensitive attributes.

[0135] A4: Input the word vector into the dual-branch encoder network, and extract features from the word vector by sharing the bottom feature extraction layer to obtain the intermediate representation vector.

[0136] Specifically, the word vectors in the initial semantic vector sequence are used as input to the dual-branch encoder network, and mapped to the intermediate representation space through a shared bottom-level feature extraction layer to obtain the intermediate representation vector. The intermediate representation is a vector. It serves as the common input for both the semantic fidelity branch and the privacy perturbation branch.

[0137] A5: The intermediate representation vector is processed by the semantic fidelity branch and the privacy perturbation branch respectively to obtain the predicted values ​​of the semantic fidelity component and the privacy perturbation component.

[0138] Specifically, the semantic fidelity branch may include a first fully connected layer, which converts the intermediate representation vector... Inputting the first fully connected layer yields the semantic fidelity component predictions. The privacy perturbation branch may include a second fully connected layer, which will pass the intermediate representation vector to the next layer. Inputting the second fully connected layer yields the predicted values ​​of the privacy perturbation components. .

[0139] A6: Calculate the joint loss function based on the training sensitive entity annotation information, semantic fidelity component prediction value, and privacy perturbation component prediction value.

[0140] Specifically, the formula for calculating the joint loss function is as follows: .

[0141] in, The semantic fidelity loss term is calculated as follows: Cross-entropy loss with contextual semantic labels, The loss item for privacy protection is calculated as follows: Negative cross-entropy loss with sensitive entity category labels, To reconstruct the loss term, the calculation method is based on the original word vector. With reconstructed vector The mean square error between them , , These are the preset weight hyperparameters.

[0142] A7: Determine whether the joint loss function converges.

[0143] Specifically, the stochastic gradient descent optimization algorithm can be used to train the two-branch encoder network, and the joint loss function is continuously calculated during training. Whether the joint loss function has converged is used as the criterion for determining whether training is complete. When the joint loss function converges, process A8 is executed; when the joint loss function has not converged, process A9 is executed.

[0144] A8: Obtain the first weight matrix corresponding to the semantic fidelity branch and the second weight matrix corresponding to the privacy perturbation branch from the dual-branch encoder network, and generate the semantic decoupling projection matrix based on the first weight matrix and the second weight matrix.

[0145] Specifically, when the joint loss function converges, training is complete, and at this point, the first weight matrix can be... Second weight matrix Concatenating to form a semantic decoupling projection matrix Based on the semantically decoupled projection matrix, the orthogonal projection operator can be computed. and The superscript Indicates matrix transpose, superscript This represents finding the inverse of a matrix.

[0146] Understandably, orthogonal projection operators include semantic-fidelity projection operators and privacy-perturbation projection operators, which are used to map target lexical vectors to semantic-fidelity subspaces and privacy-perturbation subspaces, respectively. The computation of orthogonal projection operators involves matrix transpose and matrix inversion operations to ensure that the projection results satisfy the orthogonality constraint, that is, the semantic-fidelity component vectors and the privacy-perturbation component vectors are mutually orthogonal.

[0147] A9: Adjust the parameters of the dual-branch encoder network.

[0148] Specifically, when the joint loss function fails to converge, the parameters of the dual-branch encoder network can be adjusted. Once the parameters are adjusted, the process returns to step A4 until the joint loss function converges.

[0149] It should be noted that parameter tuning can employ stochastic gradient descent or other optimization algorithms. The gradient of the loss function with respect to each parameter is calculated via backpropagation, and the parameter values ​​are updated according to the learning rate. During optimization, techniques such as learning rate decay and momentum acceleration can be used to improve convergence speed and stability.

[0150] The semantic decoupling projection matrix obtained through the above training process can effectively decompose the target word vectors orthogonally during the inference stage, achieving a refined separation of semantic information and privacy attributes.

[0151] S103. Perform differential privacy noise injection processing on the privacy perturbation component vector to obtain the perturbed privacy perturbation component vector.

[0152] As is understandable, differential privacy refers to a privacy-preserving mathematical framework that adds controlled random noise to data or models, making it impossible for attackers to infer the existence of any individual in the dataset from the output. By applying differential privacy noise to the privacy-perturbing component vectors rather than the entire lexical vector, the contextual semantic information carried by the semantically faithful components can be preserved to the greatest extent while ensuring privacy.

[0153] Optionally, in another embodiment of this application, one specific implementation of step S103 is as follows: Figure 2 As shown, the specific steps include:

[0154] S201. Calculate the norm of the privacy perturbation component vector.

[0155] Specifically, the norm can be adopted Norm or other suitable vector norm metrics. Norm calculation is used to evaluate privacy perturbation component vectors. The sensitivity level provides a basis for subsequent cutting operations.

[0156] Optionally, in another embodiment of this application, in order to determine the pruning threshold corresponding to the norm of the privacy perturbation component vector in step S201, before performing step S201, as follows: Figure 3 As shown, the specific steps also include:

[0157] S301. Extract all target privacy perturbation component vectors marked as sensitive entity words from the initial semantic vector sequence, and construct a privacy perturbation component sample set based on the extracted target privacy perturbation component vectors.

[0158] Specifically, based on the sensitive entity annotation information generated in step S101, the target word vectors of all words marked as sensitive entities in the initial semantic vector sequence can be determined, and the target privacy perturbation component vectors corresponding to each target word vector can be obtained according to the projection decomposition method in step S102. The obtained target privacy perturbation component vectors are then collected to form a privacy perturbation component sample set. .in This represents the number of samples.

[0159] S302. Calculate the norm of each target privacy perturbation component vector in the privacy perturbation component sample set to obtain the norm sequence.

[0160] Specifically, the privacy perturbation component sample set can be calculated separately. Each target privacy perturbation component vector of norm And according to the privacy perturbation component vector of each target Corresponding The correspondence between norms forms a norm sequence. .

[0161] S303. Sort the norm sequence in ascending order, and select the norm value that is located at the preset percentile position after sorting as the clipping threshold.

[0162] In one implementation, the preset percentile parameter α can take values ​​in the range of 0.90 ≤ α ≤ 0.99. If the privacy perturbation component sample set contains N target privacy perturbation component vectors, the corresponding sorting position in the norm sequence can be determined according to the preset percentile parameter α. and sort the position The norm value is determined as the clipping threshold C.

[0163] Therefore, the target privacy perturbation component vector can be made to have a proportion in the sample set of privacy perturbation components that is not less than α. If the norm is less than or equal to the pruning threshold C, step S202 mainly scales the target privacy perturbation component vector with an abnormally large norm, thereby preserving most of the original norm distribution characteristics of the target privacy perturbation component vector while establishing an upper bound on the norm.

[0164] S202. When the norm is greater than the preset pruning threshold, the privacy perturbation component vector is scaled to a norm equal to the pruning threshold to obtain the pruned vector.

[0165] Specifically, the cropping operation is used to limit the maximum impact of a single sample on the noise scale.

[0166] It should be noted that if the privacy perturbation component vector is The cropping threshold is Then the clipped vector v p 'Can be represented as:

[0167]

[0168] Among them, | |2 represents the privacy perturbation component vector. of Norm.

[0169] From the above calculation relationship, it can be seen that when the privacy perturbation component vector... When the norm is not greater than the pruning threshold C, it means no pruning is needed, the privacy perturbation component vector remains unchanged, and the original privacy perturbation component vector is directly used as the pruned vector. When the privacy perturbation component vector... When the norm is greater than the pruning threshold C, the privacy perturbation component vector is scaled proportionally to make the pruned vector... The norm is equal to the clipping threshold C.

[0170] S203. Obtain the privacy budget parameter corresponding to the sensitive entity category to which the target word vector belongs.

[0171] Specifically, the sensitive entity category corresponding to the target lexical vector can be determined based on the sensitive entity annotation information generated in step S101, and then the pre-set privacy budget parameter can be obtained based on the sensitive entity category. Therefore, different sensitive entity categories can correspond to different privacy budget parameters, enabling differential privacy noise injection processing to determine the corresponding privacy protection strength based on the sensitive entity category.

[0172] S204. Generate a random noise vector based on the privacy budget parameter and the preset noise multiplier.

[0173] Specifically, noise multiplier Privacy Budget Parameters The relationship is determined in the following ways:

[0174] For the preset differential privacy failure probability parameter, the noise multiplier satisfies the conditions of the differential privacy Gaussian mechanism, specifically:

[0175]

[0176] in For privacy budget parameters corresponding to sensitive entity categories, This is a preset differential privacy relaxation term, with a value less than the reciprocal of the number of samples in the training corpus.

[0177] Next, based on the privacy budget parameters corresponding to the sensitive entity category to which the target word vector belongs. and preset noise multiplier Generate a value that conforms to the mean. Standard deviation is Gaussian distributed random noise vector .

[0178] It should be noted that random noise vectors Each dimension component Independently from the mean variance is Gaussian distribution Obtained from sampling, among which The cropping threshold, Indexed by vector dimension.

[0179] S205. Superimpose the random noise vector onto the clipped vector to obtain the perturbed privacy perturbation component vector.

[0180] Specifically, the privacy perturbation component vector after perturbation can be represented as:

[0181] random noise vector Superimposed on the clipped vector Above, we obtain the privacy-perturbed component vector after differential privacy protection. .

[0182] Among them, semantic fidelity component vector The original values ​​remain unchanged during the noise injection process. Thus, random noise only acts on the privacy perturbation component vector, which can reduce the impact of noise on information that is semantically related to the context of the text to be processed.

[0183] It should be noted that during the differential privacy noise injection process described above, the semantic fidelity component vectors retain their original values ​​and do not participate in the noise superposition operation. This design ensures that the privacy protection of sensitive attribute information does not affect the integrity of the contextual semantic information.

[0184] Furthermore, for any two adjacent initial semantic vector sequences that differ only in a single sensitive entity word, the perturbed privacy perturbation component vector obtained after decomposition in step S102, pruning and noise injection in step S103 is... satisfy - Differential privacy, that is, for any measurable subset of outputs ,have ,in D' represents adjacent datasets. This mathematical guarantee ensures that attackers cannot infer the existence of sensitive information about a specific individual in the original dataset by observing the anonymized vector sequence.

[0185] S104. Perform vector desensitization processing on the semantic fidelity component vector and the perturbation privacy perturbation component vector to obtain a desensitized semantic vector sequence.

[0186] Specifically, vector desensitization process re-integrates the privacy-protected perturbation components with the original semantically faithful components to generate a vector representation that combines privacy security and semantic usability.

[0187] Optionally, in another embodiment of this application, one specific implementation of step S104 is as follows: Figure 4 As shown, the specific steps include:

[0188] S401. Calculate the vector sum of the semantic fidelity component vector and the privacy perturbation component vector after perturbation to obtain the reconstructed target word vector.

[0189] Specifically, the vector summation operation merges the semantically faithful component vector that remains unchanged with the privacy-perturbed component vector that has undergone differential privacy noise perturbation, forming a reconstructed target word vector that retains semantic information while protecting privacy.

[0190] Therefore, the formula for reconstructing the target word vector is: .

[0191] Wherein, the semantic fidelity component vector is Privacy perturbation component vector after perturbation .

[0192] S402. Obtain the original target word vector corresponding to the reconstructed target word vector in the initial semantic vector sequence.

[0193] S403. Replace the original target word vector in the initial semantic vector sequence with the reconstructed target word vector to obtain the desensitized semantic vector sequence.

[0194] Specifically, the original target word vectors at corresponding positions in the initial semantic vector sequence generated by S101 are replaced with the reconstructed target word vector v', thereby generating a desensitized semantic vector sequence. Through this replacement operation, word vectors marked as sensitive entities in the initial semantic vector sequence are replaced with privacy-preserving reconstructed vectors, while other non-sensitive word vectors remain unchanged, thus generating a complete desensitized semantic vector sequence. This desensitized semantic vector sequence is used to replace the original text data input to downstream natural language processing tasks, protecting personal privacy while maintaining semantic usability.

[0195] Optionally, in another embodiment of this application, in order to reduce the semantic shift generated after replacing the original target word vector with the reconstructed target word vector in step S403, after obtaining the desensitized semantic vector sequence, the reconstructed target word vector can also be subjected to semantic consistency check and shift correction based on the semantic correlation between the reconstructed target word vector and the original target word vector and the context window vector set, respectively. Therefore, after executing step S403, the following processes B1-B3 are further included:

[0196] B1: Obtain the context window vector set of the preset number of non-sensitive word vectors before and after the original target word vector in the text to be processed.

[0197] Specifically, for each reconstructed target word vector v' in the replaced desensitized semantic vector sequence, obtain its corresponding original target word vector in the initial semantic vector sequence. and the original target word vector The sentences before and after it contain A set of context window vectors for non-sensitive word meta-vectors.

[0198] Among them, the preset quantity The window radius is a positive integer and can be set according to text length and semantic coherence requirements; for example, it can be set to 3, 5, or 10. The context window vector set includes the original target word vectors before and after the preset number of elements in the sentence. The non-sensitive word vector is used to represent the local semantic environment in which the word is located.

[0199] B2: Calculate the first cosine similarity between the reconstructed target word vector and the mean vector of the context window vector set, and calculate the second cosine similarity between the original target word vector and the mean vector of the context window vector set.

[0200] Specifically, first, the mean vector of all vectors in the context window vector set is calculated. Then, the cosine similarity between the reconstructed target word vector and the original target word vector and this mean vector is calculated respectively. Cosine similarity measures the degree of closeness between two vectors in a direction, and its value ranges from -1 to 1. The larger the value, the more consistent the directions.

[0201] B3: When the difference between the first cosine similarity and the second cosine similarity is greater than the preset semantic offset tolerance threshold, perform offset correction operation on the reconstructed target word vector until the difference is no greater than the semantic offset tolerance threshold or the maximum number of iterations is reached.

[0202] It should be noted that when Greater than the preset semantic offset tolerance threshold At this time, an offset correction operation needs to be performed on the reconstructed target word vector v'. Here, the first cosine similarity is s', and the second cosine similarity is... .

[0203] The specific correction method is as follows ,in The mean vector of the context window vector set. To correct the step size factor, Repeat the offset correction operation until... (The difference is not greater than the semantic offset tolerance threshold) meets or reaches the maximum number of iterations limit, and the correction is completed with... Replace the original target word vector v' with the reconstructed word vector.

[0204] Specifically, the offset correction operation adjusts the reconstructed target word vector along the direction of the mean vector of the context window vector set. The correction method can employ an iterative approximation approach, where each iteration moves the vector a certain distance towards the mean vector according to a correction step size coefficient. The correction step size coefficient is a positive number less than 1, used to control the adjustment magnitude of each iteration and avoid over-correction that could cause the vector to deviate from a reasonable range.

[0205] By employing semantic consistency checks and offset correction, the cumulative effect of vector space offsets caused by noise injection can be effectively suppressed, ensuring the semantic coherence and reasonable positional distribution of the desensitized vector sequence within the overall context. This correction method does not rely on the precise values ​​of the original sensitive vectors but rather performs calibration based on contextual information within the desensitized vector space, thus meeting privacy protection requirements.

[0206] S105. Input the desensitized semantic vector sequence into the structured information extraction model, and use the structured information extraction model to perform reasoning processing on the desensitized semantic vector sequence to generate desensitized structured information records.

[0207] Understandably, structured information extraction models are used to extract structured information from desensitized semantic vector sequences obtained through vector desensitization processing. Since the structured information extraction process directly uses the desensitized semantic vector sequence as input, there is no need to recover sensitive entity information from the text to be processed; subsequent information extraction can be performed based on the desensitized semantic representation.

[0208] Specifically, the desensitized semantic vector sequence generated in step S104 can be input into a pre-trained structured information extraction model. The structured information extraction model can then perform reasoning processing on the desensitized semantic vector sequence to extract at least one of entity, relation, event argument, and attribute information, and generate a desensitized structured information record based on a preset structured data template.

[0209] Among them, the de-identified structured information record is used to support non-privacy-sensitive analysis and retrieval of the original text data, and is output together with the de-identified semantic vector sequence to reduce the data integration overhead of downstream systems.

[0210] It should be noted that the structured information extraction model can be a model fine-tuned through multi-task training on non-sensitive public text. The structured information extraction model may include a shared bidirectional long short-term memory network layer, a conditional random field sequence labeling layer, a dual affine attention layer, and a span-based event argument extraction layer. Specifically, the conditional random field sequence labeling layer is used for entity recognition, the dual affine attention layer for relation classification, and the span-based event argument extraction layer for event argument extraction.

[0211] Optionally, in this embodiment of the application, a training method for a structured information extraction model is provided, specifically including the following processes C1-C7:

[0212] C1: Obtain non-sensitive public text.

[0213] Specifically, the training data can be provided as a training set consisting of non-sensitive public text and corresponding structured labeled data. The structured labeled data includes at least two of the following: entity mention boundary labels, entity type labels, entity relationship type labels, event trigger word labels, and event argument role labels. Entity mention boundary labels are used to identify the start and end positions of entities in the text. Entity type labels are used to distinguish different entity categories such as person names, place names, and organization names. Entity relationship type labels are used to describe the semantic relationships between entities, such as membership, location, and causality. Event trigger word labels are used to identify keywords that trigger specific events. Event argument role labels are used to identify the roles of each participating entity in the event, such as agent, patient, time, and location.

[0214] C2: Input non-sensitive public text into the pre-trained language representation model to obtain a sequence of training semantic vectors.

[0215] Specifically, the same pre-trained language representation model as used in step S101 for semantic encoding can be used to perform semantic encoding on non-sensitive public text, converting each word in the non-sensitive public text into a corresponding semantic vector, and forming a training semantic vector sequence according to the order of word arrangement in the non-sensitive public text.

[0216] C3: Input the training semantic vector sequence into the initialized structured information extraction model for multi-task fine-tuning training to obtain the training desensitized structured information record.

[0217] Specifically, the training semantic vector sequence can be used as the input feature of the structured information extraction model. The context features in the training semantic vector sequence can be extracted by a shared bidirectional long short-term memory network layer, and then processed by a conditional random field sequence labeling layer, a dual affine attention layer, and a span-based event argument extraction layer.

[0218] Among them, the Conditional Random Field Sequence Labeling Layer performs entity recognition based on the training semantic vector sequence, the Dual Affine Attention Layer performs relation classification based on the recognized entities, the Span-based Event Argument Extraction Layer performs event argument extraction, and forms a training desensitized structured information record based on the processing results.

[0219] C4: Determine the joint task loss function based on the desensitized structured information records from training.

[0220] Specifically, in the multi-task fine-tuning training process, the joint task loss function can be expressed as:

[0221]

[0222] in, To train the negative log-likelihood loss for named entity recognition in desensitized structured information records, To train the cross-entropy loss for classifying relationships in desensitized structured information records, To train a binary classification cross-entropy loss function for extracting event arguments from desensitized structured information records, , , These are the preset task weight coefficients.

[0223] C5: Determine whether the joint task loss function has converged.

[0224] Specifically, during the multi-task fine-tuning training of the structured information extraction model, the joint task loss function can be continuously calculated, and the convergence of the structured information extraction model can be determined based on the changes in the joint task loss function. When the joint task loss function converges, process C6 is executed. When the joint task loss function does not converge, process C7 is executed.

[0225] C6: The initialized structured information extraction model is determined to be the trained structured information extraction model.

[0226] Specifically, when the joint task loss function reaches convergence, the adjustment of the parameters of the structured information extraction model can be stopped, and the model parameters after training can be fixed so that the trained structured information extraction model can be used for inference processing in step S105.

[0227] C7: Adjust the parameters of the structured information extraction model.

[0228] Specifically, when the joint task loss function has not reached convergence, the parameters of the structured information extraction model can be adjusted according to the joint task loss function, and the multi-task fine-tuning training can be re-executed based on the adjusted model parameters, that is, the execution process C3 is returned until the joint task loss function reaches convergence.

[0229] Through the above training process, the structured information extraction model is able to learn the structured information extraction relationships corresponding to entities, relations, and event arguments based on the training semantic vector sequence. After training is completed, the model parameters are fixed, and the trained structured information extraction model is used in step S105 to perform inference processing with the desensitized semantic vector sequence as input to generate desensitized structured information records.

[0230] S106. Associate and encapsulate the desensitized semantic vector sequence and the desensitized structured information record to obtain the desensitized result corresponding to the text to be processed.

[0231] Understandably, the de-identified semantic vector sequence is used to form a privacy-preserving semantic representation, while the de-identified structured information record is used to form the structured information extracted from the de-identified semantic vector sequence. By associating and encapsulating the two, the de-identified semantic representation and the corresponding structured information can form a related output result.

[0232] Optionally, in another embodiment of this application, a specific implementation of step S106 includes the following processes D1-D5:

[0233] D1: Construct a data encapsulation structure.

[0234] The data encapsulation structure includes vector data fields and structured data fields.

[0235] D2: Arrange the desensitized semantic vector sequence according to the preset word order, and write the arranged desensitized semantic vector sequence into the vector data field.

[0236] Specifically, the word vectors in the desensitized semantic vector sequence can be arranged according to the order of the word elements in the text to be processed, and the arrangement result can be organized into a two-dimensional floating-point array.

[0237] In this array, the first dimension of the two-dimensional floating-point array is equal to the length of the word sequence, and the second dimension is equal to the vector dimension output by the pre-trained language representation model. After permutation, the two-dimensional floating-point array is written into the vector data field of the data encapsulation structure, so that the vector data field stores the de-identified semantic representation of the text to be processed.

[0238] D3: Serialize the de-identified structured information records to obtain structured serialized data, and write the structured serialized data into the structured data field.

[0239] Specifically, JSON or Protocol Buffers formats can be used to serialize and encode de-identified structured information records.

[0240] Structured serialized data can include entity list objects, relationship list objects, and event list objects.

[0241] Each entity entry in the entity list object can include a unique entity identifier, entity type, the start and end indexes of the entity mention in the lexical sequence, and entity attribute key-value pairs.

[0242] Each relation entry in the relation list object can include a head entity identifier, a tail entity identifier, and a relation type.

[0243] Each event entry in the event list object may include the event trigger word position index, the event type, and a set of key-value pairs consisting of the argument role name and the corresponding entity identifier.

[0244] Through the above serialization process, the desensitized structured information records generated by the structured information extraction model can be converted into structured serialized data that can be encapsulated together with the desensitized semantic vector sequence.

[0245] D4: Obtain the text identifier information corresponding to the text to be processed and the privacy parameter information corresponding to the differential privacy noise injection processing.

[0246] The text identification information is used to establish the correspondence between the de-identification result and the text to be processed. The privacy parameter information is used to record the parameter information corresponding to the differential privacy noise injection processing in step S103.

[0247] Specifically, the original text identifier and processing timestamp corresponding to the text to be processed can be obtained, and the privacy budget parameters corresponding to the differential privacy noise injection processing can be obtained. The above information can be used as a component of the data in subsequent association and encapsulation.

[0248] D5: Associate and encapsulate the vector data fields, structured data fields, text identification information, and privacy parameter information to obtain the de-identification result corresponding to the text to be processed.

[0249] Specifically, the desensitized semantic vector sequence stored in the vector data field, the structured serialized data stored in the structured data field, and the original text identifier, processing timestamp, and privacy budget parameters can be packaged together into a serialized data file for output.

[0250] Therefore, the anonymized result includes both anonymized semantic vector sequences protected by differential privacy and anonymized structured information records generated based on these sequences. The anonymized semantic vector sequences can replace the original text data input to downstream natural language processing tasks, while the anonymized structured information records can be used to support non-privacy-sensitive analysis and retrieval of the original text data.

[0251] This application provides a data desensitization method. First, it acquires the text to be processed, performs semantic encoding on the text to obtain an initial semantic vector sequence, and then identifies sensitive entities in the text to obtain sensitive entity annotation information. Next, it determines the target word vectors corresponding to the sensitive entity annotation information from the initial semantic vector sequence, and performs projection decomposition on the target word vectors according to the semantic decoupling projection matrix to obtain semantic fidelity component vectors and privacy perturbation component vectors. The semantic fidelity component vectors carry information related to the semantic context of the text to be processed, while the privacy perturbation component vectors carry information related to the semantic context of the text to be processed. The process begins by decomposing the sensitive attributes of the text to be processed. Next, differential privacy noise injection is applied to the privacy perturbation component vector to obtain a perturbed privacy perturbation component vector. Then, vector desensitization processing is performed on both the semantic fidelity component vector and the perturbed privacy perturbation component vector to obtain a desensitized semantic vector sequence. This desensitized semantic vector sequence is then input into a structured information extraction model. The model performs inference processing on the desensitized semantic vector sequence to generate a desensitized structured information record. Finally, the desensitized semantic vector sequence and the desensitized structured information record are associated and encapsulated to obtain the desensitized result corresponding to the text to be processed. Thus, by projectively decomposing the target word vector and applying differential privacy noise injection only to the privacy perturbation component vector, the perturbation to the semantic fidelity component vector is reduced, thereby improving the semantic fidelity of the desensitized semantic vector sequence while achieving sensitive information desensitization.

[0252] Another embodiment of this application provides a data desensitization device, such as... Figure 5 As shown, it specifically includes the following units:

[0253] The text processing unit 501 is used to acquire the text to be processed, perform semantic encoding on the text to be processed to obtain an initial semantic vector sequence, and perform sensitive entity recognition on the text to be processed to obtain sensitive entity annotation information in the text to be processed.

[0254] The projection decomposition unit 502 is used to determine the target word vectors corresponding to the sensitive entity annotation information from the initial semantic vector sequence, and to perform projection decomposition on the target word vectors according to the semantic decoupling projection matrix to obtain semantic fidelity component vectors and privacy perturbation component vectors. Among them, the semantic fidelity component vectors carry information related to the semantic context of the text to be processed, and the privacy perturbation component vectors carry information related to the sensitive attributes of the text to be processed.

[0255] The noise injection unit 503 is used to perform differential privacy noise injection processing on the privacy perturbation component vector to obtain the perturbed privacy perturbation component vector.

[0256] Vector desensitization unit 504 is used to perform vector desensitization processing on semantic fidelity component vector and perturbation privacy perturbation component vector to obtain desensitized semantic vector sequence.

[0257] The information extraction unit 505 is used to input the desensitized semantic vector sequence into the structured information extraction model, and to perform reasoning processing on the desensitized semantic vector sequence through the structured information extraction model to generate desensitized structured information records.

[0258] The association encapsulation unit 506 is used to associate and encapsulate the desensitized semantic vector sequence and the desensitized structured information record to obtain the desensitized result corresponding to the text to be processed.

[0259] It should be noted that the specific working process of the above modules in the embodiments of this application can be referred to steps S101 to S106 in the above method embodiments, and will not be repeated here.

[0260] Optionally, another embodiment of this application provides a data desensitization device that further includes:

[0261] The text acquisition unit is used to acquire training text and perform semantic encoding on the training text to obtain an initial training semantic vector sequence.

[0262] The entity recognition unit is used to perform sensitive entity recognition on the training text using a sensitive entity recognition model, obtain the training sensitive entity annotation information of the training text, and determine the word vectors corresponding to the training sensitive entity annotation information from the initial training semantic vector sequence.

[0263] Network building blocks are used to construct dual-branch encoder networks. A dual-branch encoder network includes at least a shared bottom-level feature extraction layer, a semantic fidelity branch, and a privacy perturbation branch.

[0264] The feature extraction unit is used to input the word vector into the dual-branch encoder network, and extract features from the word vector by sharing the bottom feature extraction layer to obtain the intermediate representation vector.

[0265] The branch processing unit is used to process the intermediate representation vector through the semantic fidelity branch and the privacy perturbation branch respectively to obtain the semantic fidelity component prediction value and the privacy perturbation component prediction value.

[0266] The loss calculation unit is used to calculate the joint loss function based on the training sensitive entity annotation information, the semantic fidelity component prediction value, and the privacy perturbation component prediction value.

[0267] The convergence judgment unit is used to determine whether the joint loss function has converged.

[0268] The matrix generation unit is used to obtain the first weight matrix corresponding to the semantic fidelity branch and the second weight matrix corresponding to the privacy perturbation branch from the dual-branch encoder network if the joint loss function converges, and to generate the semantic decoupling projection matrix based on the first weight matrix and the second weight matrix.

[0269] The parameter adjustment unit is used to adjust the parameters of the dual-branch encoder network if the joint loss function fails to converge, and then return to the execution to input the word vector into the dual-branch encoder network. The word vector is then used to extract features from the word vector through the shared bottom feature extraction layer to obtain the intermediate representation vector.

[0270] Optionally, in another embodiment of this application, a data desensitization device includes a noise injection unit 503, comprising:

[0271] The norm calculation unit is used to calculate the norm of the privacy perturbation component vector.

[0272] The vector clipping unit is used to scale the privacy perturbation component vector to a norm equal to the clipping threshold when the norm is greater than a preset clipping threshold, thus obtaining the clipped vector.

[0273] The parameter acquisition unit is used to obtain the privacy budget parameters corresponding to the sensitive entity category to which the target word vector belongs.

[0274] The noise generation unit is used to generate random noise vectors based on privacy budget parameters and preset noise multipliers.

[0275] The noise superposition unit is used to superimpose the random noise vector onto the clipped vector to obtain the perturbed privacy perturbation component vector.

[0276] Optionally, another embodiment of this application provides a data desensitization device that further includes:

[0277] The sample construction unit is used to extract all target privacy perturbation component vectors marked as sensitive entity words from the initial semantic vector sequence, and to construct a privacy perturbation component sample set based on the extracted target privacy perturbation component vectors.

[0278] The norm statistics unit is used to calculate the norm of each target privacy perturbation component vector in the privacy perturbation component sample set, resulting in a norm sequence.

[0279] The threshold determination unit is used to sort the norm sequence in ascending order and select the norm value located at the preset percentile position after sorting as the clipping threshold.

[0280] Optionally, in another embodiment of this application, a data desensitization device includes a vector desensitization unit 504, comprising:

[0281] The vector reconstruction unit is used to calculate the vector sum of the semantically faithful component vector and the perturbed privacy perturbation component vector to obtain the reconstructed target word vector.

[0282] The vector acquisition unit is used to acquire the original target word vector corresponding to the reconstructed target word vector in the initial semantic vector sequence.

[0283] The vector replacement unit is used to replace the original target word vector in the initial semantic vector sequence with the reconstructed target word vector to obtain the desensitized semantic vector sequence.

[0284] Optionally, another embodiment of this application provides a data desensitization device that further includes:

[0285] The window acquisition unit is used to acquire the context window vector set of a preset number of non-sensitive word vectors before and after the original target word vector in the text to be processed.

[0286] The similarity calculation unit is used to calculate the first cosine similarity between the reconstructed target word vector and the mean vector of the context window vector set, and to calculate the second cosine similarity between the original target word vector and the mean vector of the context window vector set.

[0287] The offset correction unit is used to perform offset correction operation on the reconstructed target word vector when the difference between the first cosine similarity and the second cosine similarity is greater than the preset semantic offset tolerance threshold, until the difference is no greater than the semantic offset tolerance threshold or the maximum number of iterations is reached.

[0288] Optionally, another embodiment of this application provides a data desensitization device that further includes:

[0289] The acquisition unit is used to acquire non-sensitive public text.

[0290] The generation unit is used to input non-sensitive public text into a pre-trained language representation model to obtain a sequence of training semantic vectors.

[0291] The fine-tuning training unit is used to input the training semantic vector sequence into the initialized structured information extraction model for multi-task fine-tuning training, thereby obtaining the training desensitized structured information record.

[0292] The loss determination unit is used to determine the joint task loss function based on the training desensitized structured information records.

[0293] The judgment unit is used to determine whether the joint task loss function has converged.

[0294] The model determination unit is used to determine the initialized structured information extraction model as the trained structured information extraction model if the joint task loss function converges.

[0295] The adjustment unit is used to adjust the parameters of the structured information extraction model if the joint task loss function fails to converge, and then return to the initialization of the structured information extraction model to perform multi-task fine-tuning training by inputting the training semantic vector sequence into the model to obtain the training desensitized structured information record.

[0296] Optionally, in another embodiment of this application, a data desensitization device includes an associated encapsulation unit 506, comprising:

[0297] The structural building block is used to construct the data encapsulation structure. This data encapsulation structure includes vector data fields and structured data fields.

[0298] The vector writing unit is used to arrange the desensitized semantic vector sequence according to a preset word order and write the arranged desensitized semantic vector sequence into the vector data field.

[0299] The data serialization unit is used to serialize the de-identified structured information records to obtain structured serialized data, and then write the structured serialized data into the structured data field.

[0300] The information acquisition unit is used to acquire the text identifier information corresponding to the text to be processed and the privacy parameter information corresponding to the differential privacy noise injection processing.

[0301] The data encapsulation unit is used to associate and encapsulate vector data fields, structured data fields, text identification information, and privacy parameter information to obtain the de-identification result corresponding to the text to be processed.

[0302] It should be noted that the specific working process of each module provided in the above embodiments of this application can be referred to the corresponding steps in the above method embodiments, and will not be repeated here.

[0303] It should also be noted that the data desensitization device provided in this application has the technical effects of any of the above embodiments, and will not be described in detail here.

[0304] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0305] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for desensitizing data, characterized in that, include: The text to be processed is obtained, the text to be processed is semantically encoded to obtain an initial semantic vector sequence, and the text to be processed is sensitive entity identification to obtain sensitive entity annotation information in the text to be processed. The target word vector corresponding to the sensitive entity annotation information is determined from the initial semantic vector sequence, and the target word vector is projected and decomposed according to the semantic decoupling projection matrix to obtain the semantic fidelity component vector and the privacy perturbation component vector; wherein, the semantic fidelity component vector carries information related to the semantic context of the text to be processed, and the privacy perturbation component vector carries information related to the sensitive attributes of the text to be processed. Differential privacy noise injection is performed on the privacy perturbation component vector to obtain the perturbed privacy perturbation component vector; The semantic fidelity component vector and the perturbed privacy perturbation component vector are subjected to vector desensitization processing to obtain a desensitized semantic vector sequence; The desensitized semantic vector sequence is input into the structured information extraction model, and the structured information extraction model performs reasoning processing on the desensitized semantic vector sequence to generate desensitized structured information records. The desensitized semantic vector sequence and the desensitized structured information record are associated and encapsulated to obtain the desensitized result corresponding to the text to be processed.

2. The method according to claim 1, characterized in that, The training method for the semantic decoupling projection matrix includes: Acquire training text and perform semantic encoding on the training text to obtain an initial training semantic vector sequence; The training text is subjected to sensitive entity recognition using a sensitive entity recognition model to obtain the training sensitive entity annotation information of the training text, and the word vectors corresponding to the training sensitive entity annotation information are determined from the initial training semantic vector sequence. Construct a dual-branch encoder network; wherein the dual-branch encoder network includes at least a shared bottom-level feature extraction layer, a semantic fidelity branch, and a privacy perturbation branch; The word vector is input into the dual-branch encoder network, and the word vector is subjected to feature extraction through the shared bottom-level feature extraction layer to obtain the intermediate representation vector; The intermediate representation vector is processed by the semantic fidelity branch and the privacy perturbation branch respectively to obtain the semantic fidelity component prediction value and the privacy perturbation component prediction value; Based on the training sensitive entity annotation information, the semantic fidelity component prediction value, and the privacy perturbation component prediction value, a joint loss function is calculated; Determine whether the joint loss function converges; If the joint loss function converges, the first weight matrix corresponding to the semantic fidelity branch and the second weight matrix corresponding to the privacy perturbation branch are obtained from the dual-branch encoder network, and the semantic decoupling projection matrix is ​​generated based on the first weight matrix and the second weight matrix. If the joint loss function does not converge, the parameters of the dual-branch encoder network are adjusted, and the process of inputting the word vector into the dual-branch encoder network and extracting features from the word vector through the shared bottom-level feature extraction layer is returned to obtain the intermediate representation vector.

3. The method according to claim 1, characterized in that, The step of performing differential privacy noise injection processing on the privacy perturbation component vector to obtain the perturbed privacy perturbation component vector includes: Calculate the norm of the privacy perturbation component vector; When the norm is greater than the preset pruning threshold, the privacy perturbation component vector is scaled to a norm equal to the pruning threshold to obtain the pruned vector; Obtain the privacy budget parameter corresponding to the sensitive entity category to which the target word vector belongs; A random noise vector is generated based on the privacy budget parameter and the preset noise multiplier; The random noise vector is superimposed on the clipped vector to obtain the perturbed privacy perturbation component vector.

4. The method according to claim 3, characterized in that, Before calculating the norm of the privacy perturbation component vector, the method further includes: Extract all target privacy perturbation component vectors marked as sensitive entity words from the initial semantic vector sequence, and construct a privacy perturbation component sample set based on each extracted target privacy perturbation component vector; Calculate the norm of each target privacy perturbation component vector in the privacy perturbation component sample set to obtain a norm sequence; The norm sequence is sorted in ascending order, and the norm value located at the preset percentile position after sorting is selected as the clipping threshold.

5. The method according to claim 1, characterized in that, The step of performing vector desensitization processing on the semantic fidelity component vector and the perturbed privacy perturbation component vector to obtain a desensitized semantic vector sequence includes: The vector sum of the semantic fidelity component vector and the perturbed privacy perturbation component vector is calculated to obtain the reconstructed target word vector; Obtain the original target word vector corresponding to the reconstructed target word vector in the initial semantic vector sequence; The original target word vectors in the initial semantic vector sequence are replaced with the reconstructed target word vectors to obtain the desensitized semantic vector sequence.

6. The method according to claim 5, characterized in that, After replacing the original target word vectors in the initial semantic vector sequence with the reconstructed target word vectors to obtain the desensitized semantic vector sequence, the method further includes: Obtain the context window vector set of each preset number of non-sensitive word vectors before and after the original target word vector in the text to be processed; Calculate the first cosine similarity between the reconstructed target word vector and the mean vector of the context window vector set, and calculate the second cosine similarity between the original target word vector and the mean vector of the context window vector set; When the difference between the first cosine similarity and the second cosine similarity is greater than the preset semantic offset tolerance threshold, the reconstructed target word vector is subjected to offset correction operation until the difference is not greater than the semantic offset tolerance threshold or the maximum number of iterations is reached.

7. The method according to claim 1, characterized in that, The training method for the structured information extraction model includes: Obtaining non-sensitive public text; The non-sensitive public text is input into a pre-trained language representation model to obtain a training semantic vector sequence; The trained semantic vector sequence is input into the initialized structured information extraction model for multi-task fine-tuning training to obtain the trained desensitized structured information record; Based on the aforementioned desensitized structured information records from the training, the joint task loss function is determined; Determine whether the joint task loss function has converged; If the joint task loss function converges, the initialized structured information extraction model will be determined as the trained structured information extraction model. If the joint task loss function fails to converge, the parameters of the structured information extraction model are adjusted, and the process returns to inputting the training semantic vector sequence into the initialized structured information extraction model for multi-task fine-tuning training to obtain the training desensitized structured information record.

8. The method according to claim 1, characterized in that, The step of associating and encapsulating the desensitized semantic vector sequence and the desensitized structured information record to obtain the desensitized result corresponding to the text to be processed includes: Construct a data encapsulation structure; wherein the data encapsulation structure includes vector data fields and structured data fields; The desensitized semantic vector sequence is arranged according to a preset word order, and the arranged desensitized semantic vector sequence is written into the vector data field; The desensitized structured information record is serialized to obtain structured serialized data, and the structured serialized data is written into the structured data field; Obtain the text identifier information corresponding to the text to be processed and the privacy parameter information corresponding to the differential privacy noise injection processing; The vector data field, the structured data field, the text identification information, and the privacy parameter information are associated and encapsulated to obtain the de-identification result corresponding to the text to be processed.

9. A data desensitization device, characterized in that, include: The text processing unit is used to acquire the text to be processed, perform semantic encoding on the text to be processed to obtain an initial semantic vector sequence, and perform sensitive entity recognition on the text to be processed to obtain sensitive entity annotation information in the text to be processed. The projection decomposition unit is used to determine the target word vector corresponding to the sensitive entity annotation information from the initial semantic vector sequence, and to perform projection decomposition on the target word vector according to the semantic decoupling projection matrix to obtain semantic fidelity component vector and privacy perturbation component vector; wherein, the semantic fidelity component vector carries information related to the semantic context of the text to be processed, and the privacy perturbation component vector carries information related to the sensitive attributes of the text to be processed. A noise injection unit is used to perform differential privacy noise injection processing on the privacy perturbation component vector to obtain the perturbed privacy perturbation component vector; The vector desensitization unit is used to perform vector desensitization processing on the semantic fidelity component vector and the perturbed privacy perturbation component vector to obtain a desensitized semantic vector sequence. The information extraction unit is used to input the desensitized semantic vector sequence into the structured information extraction model, and to perform reasoning processing on the desensitized semantic vector sequence through the structured information extraction model to generate desensitized structured information records. The association and encapsulation unit is used to associate and encapsulate the desensitized semantic vector sequence and the desensitized structured information record to obtain the desensitization result corresponding to the text to be processed.

10. The apparatus according to claim 9, characterized in that, Also includes: The text acquisition unit is used to acquire training text and perform semantic encoding on the training text to obtain an initial training semantic vector sequence; The entity recognition unit is used to perform sensitive entity recognition on the training text using a sensitive entity recognition model, obtain the training sensitive entity annotation information of the training text, and determine the word vector corresponding to the training sensitive entity annotation information from the initial training semantic vector sequence. A network construction unit is used to construct a dual-branch encoder network; wherein the dual-branch encoder network includes at least a shared bottom-level feature extraction layer, a semantic fidelity branch, and a privacy perturbation branch; The feature extraction unit is used to input the word vector into the dual-branch encoder network, and extract features from the word vector through the shared bottom feature extraction layer to obtain an intermediate representation vector; The branch processing unit is used to process the intermediate representation vector through the semantic fidelity branch and the privacy perturbation branch respectively to obtain the semantic fidelity component prediction value and the privacy perturbation component prediction value. The loss calculation unit is used to calculate the joint loss function based on the training sensitive entity annotation information, the semantic fidelity component prediction value, and the privacy perturbation component prediction value. A convergence determination unit is used to determine whether the joint loss function has converged; The matrix generation unit is used to obtain the first weight matrix corresponding to the semantic fidelity branch and the second weight matrix corresponding to the privacy perturbation branch from the dual-branch encoder network if the joint loss function converges, and generate the semantic decoupling projection matrix based on the first weight matrix and the second weight matrix. The parameter adjustment unit is used to adjust the parameters of the dual-branch encoder network if the joint loss function fails to converge, and then return to the process of inputting the word vector into the dual-branch encoder network and extracting features from the word vector through the shared bottom-level feature extraction layer to obtain an intermediate representation vector.