Data desensitization and feature extraction method based on user privacy protection

Abstract

The invention provides a data desensitization and feature extraction method based on user privacy protection. The method comprises a sensitive data automatic grading step, a desensitization processing step, a feature extraction step and a privacy risk verification step. Through mechanisms such as dynamic hierarchical desensitization, a differential desensitization algorithm, hardware security module key management, a multi-level feature extraction framework, differential privacy secondary protection, a privacy-utility evaluation model and the like, the problems of feature loss, contradiction between security and performance, fragile key management and the like in a traditional desensitization technology can be solved; therefore, the balance between privacy protection and data availability is realized.

Description

Technical Field

[0001] This invention relates to the fields of network security and data processing technology, specifically to a data desensitization and feature extraction method based on user privacy protection, applicable to hierarchical desensitization and feature retention extraction of user privacy data in big data platforms or artificial intelligence model training scenarios. Background Technology

[0002] In the digital age, lifestyle service platforms (such as food delivery, e-commerce, and social networks) have accumulated massive amounts of user data, including phone numbers, ID card numbers, addresses, and behavioral tags. To optimize recommendation algorithms and improve service quality, these platforms need to conduct in-depth analysis of this data. However, directly using plaintext data poses serious privacy risks, such as user identity re-identification and misuse of sensitive information.

[0003] Existing data anonymization technologies have the following main drawbacks: Feature loss problem: Traditional desensitization methods (such as full-field masking or hash replacement) can protect privacy, but they will completely lose the business features in the data. For example, after replacing all mobile phone numbers with MD5 hash values, the features such as "location" and "carrier" contained therein cannot be used by the model, causing the data to become "dead data".

[0004] Security and performance are at odds: simple string replacement is easily restored by rainbow table attacks; high-strength encryption (such as AES full encryption) is secure, but it will significantly slow down the system response speed in high-concurrency scenarios.

[0005] Vulnerable key management: Most systems hardcode encryption keys in code or configuration files, so once the server is compromised, encrypted data is equivalent to plaintext.

[0006] Unique privacy protection: Traditional K-anonymity technology cannot resist background knowledge attacks and does not consider the differentiated protection needs of data sensitivity levels.

[0007] Therefore, there is an urgent need for a desensitization method that can differentiate data based on its sensitivity level, retain statistical features while desensitizing, and possess strong key management capabilities. Summary of the Invention

[0008] This invention provides a data desensitization and feature extraction method based on user privacy protection. This method solves the problems of feature loss, security and performance contradiction, and fragile key management in traditional desensitization technology by means of dynamic hierarchical desensitization, differentiated desensitization algorithm, hardware security module key management, multi-level feature extraction framework, differential privacy secondary protection, and privacy-utility evaluation model, so as to achieve a balance between privacy protection and data availability.

[0009] The present invention achieves the above objectives through the following technical solutions: A data anonymization and feature extraction method based on user privacy protection includes: Automated sensitive data classification steps: Use regular expressions and natural language processing techniques to scan database fields and classify data into three levels: high sensitivity, medium sensitivity, and low sensitivity. Desensitization steps: Based on the sensitivity level of the data, a differentiated desensitization algorithm is used to desensitize the data. Highly sensitive data is masked and preserved or encrypted using a hybrid method, medium sensitive data is truncated or obfuscated, and low sensitive data is formatted. Feature extraction steps: By constructing a domain knowledge graph, business semantics in the anonymized data are dynamically identified to extract feature information related to business analysis or model training; among them, periodic patterns of user behavior are extracted through an LSTM network; GeoHash encoding is used to convert address information into gridded coordinates to construct a user activity heatmap; and sentiment and key entity information in user reviews are analyzed based on the BERT model. Privacy risk verification steps: By simulating attacker behavior, link attack tests are conducted using external public datasets and de-identified data. The K-anonymity of the de-identified data is calculated and evaluated, and a differential privacy mechanism is introduced to provide secondary protection for the K-anonymity results. When a privacy risk is detected that does not meet the preset requirements, a generalization operation is automatically triggered until the K-anonymity requirements are met.

[0010] According to the present invention, a data desensitization and feature extraction method based on user privacy protection is provided. For mobile phone numbers in highly sensitive data, a mask preservation method is adopted, which is achieved by a string replacement function: the first 3 and last 4 characters of the mobile phone number are retained, and the 4th to 7th characters are replaced with mask characters. For address information in sensitive data, a spatial truncation method is used. The address resolution library is used to identify the administrative division level of the address, and the address field after the street or road is truncated, so that the address is generalized to the level of administrative division + street / road. For payment accounts in highly sensitive data, a hybrid encryption method is adopted, specifically using the AES-256-GCM algorithm. While encrypting the plaintext of the payment account, an authentication tag is generated, and the authentication tag is verified during decryption. Furthermore, a hardware security module is used to generate, store, and manage the lifecycle of encryption keys.

[0011] According to the data desensitization and feature extraction method based on user privacy protection provided by the present invention, a three-layer key architecture is constructed within the hardware security module: Root Key: Stored internally in the security chip of the hardware security module and cannot be exported to external systems; the root key destruction process is automatically triggered when unauthorized disassembly, tampering, or brute-force attacks are detected. Key Encryption Key KEK: Generated by encryption from the root key and stored in the secure storage area of ​​the hardware security module; used to encrypt data encryption key DEK. Data Encryption Key (DEK): A temporary key randomly generated by the hardware security module, used only for encrypting plaintext payment account data in a single transaction or batch.

[0012] According to the data desensitization and feature extraction method based on user privacy protection provided by the present invention, when the application server needs to encrypt the plaintext of the payment account, the following steps are performed: The application server sends a payment account encryption request to the hardware security module; The hardware security module responds to the request, randomly generates an initialization vector (IV) and a data encryption key (DEK), and uses the IV and DEK to perform AES-256-GCM encryption on the plaintext payment account. Simultaneously, it generates an authentication tag, which is a hash derivative of the ciphertext and is used to uniquely identify the ciphertext content and integrity status. The hardware security module uses KEK to encrypt DEK to obtain the ciphertext DEK. The hardware security module returns an encrypted result packet containing the payment account ciphertext, authentication tag, and ciphertext DEK, and returns the payment account ciphertext and ciphertext DEK to the application server, which then associates and stores them in the database. The hardware security module is configured with a timed key rotation strategy, which triggers a key rotation every 30 days via a scheduled task. For newly written payment account data, the corresponding DEK is directly encrypted using the newly generated KEK. For stored historical payment account data, when the data is accessed, the hardware security module automatically calls the new KEK to re-encrypt the original ciphertext DEK until all historical data DEKs are protected by the latest KEK.

[0013] According to the data anonymization and feature extraction method based on user privacy protection provided by the present invention, in the key security management based on hardware security modules, the root key is generated through quantum key distribution technology, specifically including: Generate a one-time random root key using a QKD device; Set the quantum key update threshold to trigger the QKD device to regenerate the root key and synchronize it to the key encryption key KEK through the physical tamper-proof channel of the hardware security module; During key rotation, the hardware security module performs a quantum state destruction operation on the old root key to prevent quantum computing attacks from restoring historical keys.

[0014] According to the present invention, a data desensitization and feature extraction method based on user privacy protection is provided. In the feature extraction step, the target dataset after desensitization is obtained. The target dataset includes the original fields after masking, encryption or noise addition processing. Based on a pre-trained feature mapping model, a non-linear transformation is performed on the desensitized target dataset to generate intermediate feature vectors. The intermediate feature vectors are reduced in dimensionality and made sparse to extract a subset of key features, which are then used as input features for business analysis indicators or model training. Specifically, for the user behavior logs of the target dataset: a time-series feature extraction model is built using an LSTM network to mine periodic patterns in the time-series data of the user behavior logs; GeoHash encoding is used to convert the address information in the user behavior logs into gridded coordinates, and a user activity heatmap is constructed based on the coordinate density; and a pre-trained BERT model is used to parse the text comments in the user behavior logs to extract sentiment and key entity information.

[0015] According to the data desensitization and feature extraction method based on user privacy protection provided by the present invention, the following processing strategy is adopted for ID card number data in the feature extraction step, including: Place of origin feature extraction: Extract the first 6 digits of the ID card number as the address code, match the address code with a pre-constructed address code-administrative division mapping dictionary, and obtain the corresponding administrative division information as the place of origin feature; Age feature extraction: Extract the birth year digits from the 7th to the 10th digits of the ID number, obtain the current system year, and obtain the age feature through the calculation logic of current year minus birth year; Gender feature extraction: Extract the 17th digit of the ID number and determine the gender feature based on the parity of the digit: if the digit is odd, the person is identified as male, and if the digit is even, the person is identified as female. Data storage method: The original ID number field is masked and then stored; at the same time, the extracted place of origin, age and gender features are stored in a separate feature table in the form of structured fields, and the feature table is stored in association with the masked ID number data.

[0016] According to the present invention, a data anonymization and feature extraction method based on user privacy protection includes the following steps when calculating and evaluating the K-anonymity of the anonymized data: Anonymity calculation: Traverse the anonymized target dataset, identify all quasi-identifier combinations, count the number of records in each equivalence class, and determine the minimum K-anonymity value K of the current dataset. min ; Security assessment: Based on the minimum K-anonymity value K minand the preset safety threshold K threshold Determine whether the current anonymized data meets the security release standards; if K min <K threshold If so, it is determined to be a high-risk state; Information loss quantification: Calculate the information loss degree of the current de-identified data. The information loss degree is determined based on the generalization level height or the coefficient of variation. Iterative optimization: When a high-risk state is identified, the adaptive re-anonymization engine is activated. If the information loss is less than the preset upper limit, the micro-aggregation algorithm is used to further split or merge the high-risk equivalence classes in order to increase the size of the equivalence classes. If the information loss exceeds the preset upper limit, differential privacy noise is introduced into the high-risk equivalence class, and the K-anonymity value is recalculated. Output: Repeat the anonymity calculation step until the iterative optimization step, until K is reached. min ≥ K threshold Furthermore, the information loss converges, and the optimized K-anonymous dataset is output.

[0017] According to the data desensitization and feature extraction method based on user privacy protection provided by the present invention, in the privacy risk verification step, the system simulates the role of an attacker, obtains an external public dataset, links the external public dataset with the desensitized target dataset for a link attack, and attempts to locate a specified natural person by matching quasi-identifier combinations; According to the core definition of the K-anonymity model, it is required that the quasi-identifier combination of any record in the target dataset after anonymization is completely consistent with the quasi-identifier combinations of k-1 other records; the size of all equivalence classes in the target dataset after anonymization is counted. If the size of any equivalence class is less than the preset k value, it is determined that the anonymized data has the risk of re-identification. When the privacy risk detection fails, a generalization operation is automatically triggered to correct the anonymized target dataset. The generalization operation includes generalizing specific attribute values ​​to range values ​​or category values ​​until the anonymized dataset meets the K-anonymity requirement.

[0018] According to the data anonymization and feature extraction method based on user privacy protection provided by the present invention, a differential privacy mechanism is introduced in the privacy risk verification step to provide secondary protection for the K-anonymity results, specifically including: Based on the sensitivity level labels of data features, the initial privacy budget for each numerical field is determined. Among them, the sensitivity level and The values ​​are negatively correlated; In the equivalence classes generated by K-anonymity, based on the initial privacy budget And the global sensitivity of the feature function, generate Laplace distribution noise and superimpose it on the feature value; Calculate the data utility metrics after noise injection, including mean squared error (MSE) or mean absolute error (MAE). Determine whether the utility indicators meet the preset business analysis or model training accuracy thresholds; If the conditions are not met, the privacy budget is adjusted in reverse using a PID control algorithm based on the deviation between the utility index and the threshold. And re-execute the noise-adding operation; If the conditions are met, the target dataset with differential privacy protection will be output.

[0019] Therefore, compared with existing technologies, the data desensitization and feature extraction method based on user privacy protection proposed in this invention has the following beneficial effects: 1. This invention constructs a multi-level privacy defense system through a dual protection mechanism of K-anonymity and differential privacy: K-anonymity ensures that any combination of quasi-identifiers in the dataset cannot be uniquely identified, and differential privacy further blocks the re-identification attack path through noise injection, effectively resisting common privacy threats such as link attacks and inference attacks, and meeting the requirements of the Data Security Law and Level 3 Information Security Protection for user privacy protection. 2. This invention proposes an integrated de-identification and feature extraction framework. Feature preservation logic is embedded during the data de-identification stage. Combined with domain knowledge graphs and deep learning models, it can accurately extract multi-dimensional business features such as temporal, spatial, and semantic features. Furthermore, the de-identified data can be directly used for model training, with model accuracy loss controlled at an extremely low level, ensuring a balance between privacy protection and data value mining, and supporting the efficient operation of core business scenarios such as recommendation systems and risk control models.

[0020] 3. This invention introduces an HSM hardware security module to achieve full lifecycle management of keys. Through a key hierarchical architecture and dynamic rotation mechanism, it completely eliminates the risk of hard-coded keys and compresses encryption response time to meet the real-time processing requirements of high-concurrency scenarios, thereby significantly reducing the performance loss caused by encryption and improving the overall throughput of the big data platform.

[0021] 4. This invention achieves automated closed-loop control in all aspects, from sensitive data classification and differentiated desensitization to feature extraction and privacy risk verification, which can reduce manual intervention and lower operational risks. The system can dynamically sense changes in data distribution and adjustments in business needs, automatically optimize desensitization strategies and feature extraction rules, and form an adaptive capability of perception-decision-execution, which greatly improves the flexibility and scalability of privacy protection solutions.

[0022] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0023] Figure 1 This is a flowchart of an embodiment of a data desensitization and feature extraction method based on user privacy protection according to the present invention.

[0024] Figure 2 This is a schematic diagram illustrating the steps of an embodiment of a data desensitization and feature extraction method based on user privacy protection according to the present invention.

[0025] Figure 3 This is a mapping diagram of the strategy for sensitive data classification and differentiated desensitization in an embodiment of a data desensitization and feature extraction method based on user privacy protection according to the present invention.

[0026] Figure 4 This is a flowchart illustrating the key management and encryption process based on a hardware security module in an embodiment of a data desensitization and feature extraction method for user privacy protection according to the present invention.

[0027] Figure 5 This is a logical diagram illustrating the verification of desensitized data and simulated attacks in an embodiment of a data desensitization and feature extraction method based on user privacy protection according to the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] See Figure 1 and Figure 2 This embodiment provides a data anonymization and feature extraction method based on user privacy protection, including: Sensitive data automated classification step - S1: Use regular expressions and natural language processing technology to scan database fields and classify data into three levels: high sensitivity, medium sensitivity and low sensitivity. Among them, high sensitivity data is a strong identifier used to directly locate natural persons, medium sensitivity data is a quasi-identifier used to narrow down the location scope, and low sensitivity data is an attribute used to describe user habits but does not directly expose identity. Desensitization Processing Step - S2: Based on the sensitivity level of the data, a differentiated desensitization algorithm is used to desensitize the data. Among them, high-sensitivity data is masked and preserved or mixed encryption is used, medium-sensitivity data is spatially truncated or blurred, and low-sensitivity data is formatted. Feature Extraction Step - S3: By constructing a domain knowledge graph, business semantics in the anonymized data are dynamically identified to extract feature information about business analysis or model training, and the feature information is stored in a structured manner; among them, the periodic patterns of user behavior are extracted through an LSTM network; GeoHash encoding is used to convert address information into gridded coordinates to construct a user activity heatmap; and the sentiment and key entity information in user comments are analyzed based on the BERT model. Privacy Risk Verification Step - S4: By simulating attacker behavior, link attack tests are conducted using external public datasets and anonymized internal data. The K-anonymity of the anonymized data is calculated and evaluated, and a differential privacy mechanism is introduced to provide secondary protection for the K-anonymity results. When a privacy risk is detected that does not meet the preset requirements, a generalization operation is automatically triggered until the K-anonymity requirements are met.

[0031] like Figure 3 As shown, for mobile phone numbers in highly sensitive data, a masking method is used, which is achieved through a string replacement function: the first 3 digits of the mobile phone number (operator / number segment information) and the last 4 digits (customer service verification information) are retained, and the 4 characters from the 4th to the 7th digits are replaced with "*". The algorithm expression is: Replace(mobile phone number string, start position=4, replacement length=4, replacement character=""); For address information in sensitive data, a spatial truncation method is used. An address resolution library (such as GeocodingAPI) is used to identify the administrative division level of the address (such as "district / county-street / road"). The detailed address fields such as the specific house number and building name after the street or road are truncated, and the address is generalized to the level of administrative district + street / road. For payment account information in highly sensitive data, a hybrid encryption method is adopted, specifically the AES-256-GCM algorithm. A temporary key is randomly generated by the hardware security module. This algorithm supports Authenticated Encryption (AEAD) mode, which generates an authentication tag while encrypting the plaintext of the payment account. During decryption, the authentication tag is verified to ensure that the ciphertext has not been tampered with, thus achieving integrated processing of encryption and integrity verification. Furthermore, the hardware security module (HSM) is used to generate, store, and manage the lifecycle of encryption keys.

[0032] In this embodiment, as Figure 4 As shown, a three-layer key architecture is constructed within the hardware security module: Root Key: Permanently stored inside the security chip of the hardware security module and cannot be exported to external systems; HSM has a physical anti-tamper self-destruct function, which automatically triggers the root key destruction process when unauthorized disassembly, tampering or brute-force cracking is detected; Key Encryption Key KEK: Generated by encryption from the root key and stored in the secure storage area of ​​the hardware security module; used to encrypt data encryption key DEK. Data Encryption Key (DEK): A temporary key randomly generated by the hardware security module, used only for encrypting plaintext payment account data in a single transaction or batch.

[0033] In this embodiment, when the application server needs to encrypt the plaintext of the payment account, it performs the following steps: The application server sends a payment account encryption request to the hardware security module; The hardware security module responds to the request by randomly generating an initialization vector (IV) and a data encryption key (DEK). It then uses the IV and DEK to perform AES-256-GCM encryption on the plaintext payment account (simultaneously generating an authentication tag to ensure the integrity of the ciphertext). The authentication tag is a hash-derived value of the ciphertext, used to uniquely identify the ciphertext content and its integrity status. The hardware security module uses the KEK to encrypt the DEK, obtaining the ciphertext DEK. The hardware security module returns an encrypted result packet, which includes the payment account ciphertext, authentication tag, and ciphertext DEK. The hardware security module returns the payment account ciphertext and the "ciphertext DEK" to the application server, which then associates and stores the two in the database. The hardware security module is configured with a timed key rotation strategy, which triggers key rotation every 30 days through a timed task (such as a Cron Job). For newly written payment account data, the corresponding DEK is directly encrypted using the newly generated KEK. For stored historical payment account data, when the data is accessed, the hardware security module automatically calls the new KEK to perform a re-wrap operation on the original ciphertext DEK until all historical data DEKs are protected by the latest KEK.

[0034] When the application server needs to use the ciphertext, it sends a decryption request to the HSM and submits the stored payment account ciphertext and authentication tag. The HSM uses the DEK and IV required for decryption to re-perform AES-256-GCM decryption on the ciphertext and synchronously regenerates the authentication tag. The newly generated authentication tag is compared byte-by-byte with the authentication tag submitted in the decryption request. If the comparison matches, it is determined that the ciphertext has not been tampered with, and the HSM outputs the decrypted payment account plaintext. If the comparison does not match, the security exception handling process is triggered (such as blocking the decryption operation, recording the security log and alarming). Furthermore, the GCM mode of the AES-256-GCM algorithm simultaneously supports encryption (CTR mode) and integrity verification (GMAC mode), generating ciphertext and authentication tags in a single algorithm execution, avoiding the performance loss of the traditional step-by-step operation of "encryption first and then hashing". Furthermore, the generation of authentication tags depends on three elements: plaintext content, DEK, and IV. Any tampering with the ciphertext or key will result in a mismatch in the authentication tags, ensuring that attackers cannot bypass the integrity check by partially tampering with the data. Furthermore, as a trusted execution environment, HSM isolates keys and encryption operations throughout the process, preventing interference from man-in-the-middle or side-channel attacks during the generation or comparison of authentication tags.

[0035] In key security management based on hardware security modules, the root key is generated using quantum key distribution (QKD) technology, specifically including: A one-time random root key is generated using a QKD device (such as a BB84 protocol optical quantum system), and the quantum no-cloning theorem is used to ensure the absolute security of the key. Set a quantum key update threshold (e.g., every 100 encryption operations or every 24 hours) to trigger the QKD device to regenerate the root key and synchronize it to the key encryption key KEK through the physical tamper-proof channel of the hardware security module. During key rotation, the hardware security module performs quantum state destruction operations on the old root key (such as laser interference with the storage medium) to prevent quantum computing attacks from restoring historical keys.

[0036] In the feature extraction step, inverse feature derivation is performed on the de-identified data: Strong identifier feature extraction: For mobile phone numbers with mask preservation processing (such as 138****5678), derived features such as operator (China Mobile), number segment (138), and geographical region (East China) are decoupled through the number segment location database and the last digit verification algorithm; Quasi-identifier feature generalization: For spatially truncated addresses (such as those near Danling Street in Haidian District, Beijing), combine geocoding services and POI interest point clustering to generate multi-level administrative division labels (Haidian District → Beijing → North China) and functional area attributes (commercial area / science park). Semanticization of behavioral features: For low-sensitivity label data (such as night owls), NLP word vector mapping is used to convert them into quantifiable temporal distribution features (active period: 23:00-4:00).

[0037] In the feature extraction step, a domain knowledge graph is constructed to dynamically identify the business semantics hidden in the anonymized data. For example, for the masked mobile phone number 138****5678, the "number segment-operator" association rule in the graph is used to automatically parse the first 3 digits 138 to correspond to China Mobile and the last 4 digits 5678 as the customer service verification code. For the spatially truncated address near Danling Street, Haidian District, Beijing, the administrative division level (province-city-district-street) is reversed through geocoding services to extract the regional economic attributes (such as Haidian District being a high-tech industry cluster).

[0038] Specifically, the nodes of the domain knowledge graph include original anonymized fields (such as masked ID number 110108********1234), intermediate layer features (place of origin: Haidian District, Beijing), and business target variables (user credit rating). When calculating edge weights, the correlation between features and business targets is quantified based on mutual information theory. For example, the mutual information value between place of origin features and e-commerce consumption preferences is calculated, and the weights of features in model training are dynamically adjusted. Finally, the graph structure is continuously optimized through an online learning framework, and when business requirements change (such as adding risk control rules), the recalculation of feature mapping relationships is automatically triggered.

[0039] In the feature extraction step, the target dataset after desensitization is obtained. The target dataset includes the original fields that have been masked, encrypted, or noise-added. Based on a pre-trained feature mapping model, a nonlinear transformation is performed on the desensitized target dataset to generate intermediate feature vectors. The feature mapping model is trained using a neural network architecture that includes an autoencoder structure, which is used to recover the statistical distribution characteristics of the data in the latent space. The intermediate feature vectors are reduced in dimensionality and sparsified to extract a subset of key features, which are then used as input features for business analysis indicators or model training. Specifically, for the user behavior logs of the target dataset: a time-series feature extraction model is built using an LSTM network to mine periodic patterns in the time-series data of the user behavior logs; GeoHash encoding is used to convert the address information in the user behavior logs into gridded coordinates, and a user activity heatmap is constructed based on the coordinate density; and a pre-trained BERT model is used to parse the text comments in the user behavior logs to extract sentiment and key entity information.

[0040] Specifically, a time-series feature extraction model is built based on a Long Short-Term Memory (LSTM) network to perform periodic pattern mining on time-series data in user behavior logs, including: Discretize the timestamps of user behavior to generate time-series feature vectors; The temporal feature vector is input into the LSTM network, and the periodic patterns of behavior are captured through the hidden layer units; Output user behavior periodic pattern tags, including but not limited to "high-frequency behavior at fixed times" (such as ordering takeout every Wednesday at 8 pm), "low-frequency behavior periodically" (such as making a payment on the 15th of each month), or "random behavior".

[0041] Specifically, the GeoHash encoding algorithm is used to convert the address information in user behavior logs into gridded coordinates, and a user activity heatmap is constructed based on the coordinate density, including: Standardize the original address data to eliminate format differences, such as "Beijing Haidian District" versus "Haidian District, Beijing"; GeoHash encoding maps standardized addresses to grid coordinates of a preset precision, such as a 6-level GeoHash corresponding to an area of ​​approximately 1.2km × 1.2km. Count the frequency of user behavior within each grid coordinate and generate a spatial density distribution matrix; Gaussian filtering is applied to the spatial density distribution matrix to output a smoothed user activity heatmap, which is used to characterize user spatial behavior preferences, such as "resident area" and "high-frequency activity range".

[0042] Specifically, it uses a pre-trained BERT language model to parse text comments in user behavior logs, extracting sentiment and key entity information; this includes: Perform word segmentation and part-of-speech tagging on user comments, and filter out irrelevant symbols and stop words; The preprocessed text is input into the BERT model, and a global semantic vector is output using [CLS] tags. The global semantic vector is classified into sentiment categories such as positive, neutral, and negative based on a fully connected layer, and sentiment tendency labels are output. Key entities such as "logistics speed" and "product quality" are extracted from the comments using Named Entity Recognition (NER) to generate structured semantic feature pairs, such as <logistics speed, positive>.

[0043] Finally, the decoupling results from the temporal dimension, spatial dimension, and semantic dimension are correlated and fused to generate a multimodal feature representation of user behavior, including: Using user ID as an index, align time-series pattern labels, spatial heatmap data, and semantic feature pairs; Construct joint feature vectors through feature concatenation or attention mechanisms; The joint feature vector is normalized to output a standardized feature set suitable for downstream tasks (such as recommendation systems and risk control models).

[0044] As can be seen, this embodiment achieves explicit expression of implicit features in composite data by decoupling the three dimensions of time, space and semantics, which greatly improves feature coverage, reduces the cost of manual intervention in feature engineering, reduces the amount of data required for model training, and ensures the relevance of features to business objectives.

[0045] In the feature extraction step, the following processing strategies are adopted for ID card number data, including: Place of origin feature extraction: Extract the first 6 digits of the ID card number as the address code, match the address code with the pre-constructed address code-administrative division mapping dictionary, and obtain the corresponding administrative division information (such as "Haidian District, Beijing") as the place of origin feature; Age feature extraction: Extract the birth year digits from the 7th to the 10th digits of the ID number (e.g., "1990"), obtain the current system year, and obtain the age feature (e.g., "34") through the calculation logic of the current year minus the birth year. Gender feature extraction: Extract the 17th digit of the ID number (e.g., "3") and determine the gender feature based on the parity of the digit: if the digit is odd, the person is male; if the digit is even, the person is female. Data storage method: The original ID card number field is masked and stored (the first 6 digits of the address code and the last 4 digits of the sequence code are retained, and the middle digits are replaced with mask characters, such as "110108****1234"); at the same time, the extracted place of origin, age, and gender features are stored in a separate feature table in the form of structured fields. The feature table is stored in association with the masked ID card number data for subsequent model training or user profile analysis, which can solve the problem that the masked data cannot be used for model training. The following steps are included when calculating and evaluating the K-anonymity of desensitized data: Anonymity calculation: Traverse the anonymized target dataset, identify all quasi-identifier combinations, count the number of records in each equivalence class, and determine the minimum K-anonymity value K for the current dataset. min ; Security assessment: Based on the minimum K-anonymity value K min and the preset safety threshold K threshold Determine whether the current anonymized data meets the security release standards; if K min <K threshold If so, it is determined to be a high-risk state; Information loss quantification: Calculate the information loss (IL) of the current de-identified data. The information loss is determined based on the generalization level height or the coefficient of variation. Iterative optimization: When a high-risk state is identified, the adaptive re-anonymization engine is activated. If the information loss is less than the preset upper limit, the micro-aggregation algorithm is used to further split or merge the high-risk equivalence classes in order to increase the size of the equivalence classes; If the information loss exceeds the preset upper limit, differential privacy noise is introduced into the high-risk equivalence class, and the K-anonymity value is recalculated. Output: Repeat the anonymity calculation step until the iterative optimization step, until K is reached. min ≥ K threshold Furthermore, the information loss converges, and the optimized K-anonymous dataset is output.

[0046] In the privacy risk verification process, such as Figure 5 As shown, the system simulates the role of an attacker, obtains external public datasets (including but not limited to voter lists and public social network user behavior data), performs a linkage attack on the external public datasets and the anonymized target datasets, and attempts to locate a specific natural person by matching quasi-identifier combinations (such as age, postal code, and gender). According to the core definition of the K-anonymity model, it is required that the quasi-identifier combination of any record in the target dataset after anonymization is completely consistent with the quasi-identifier combinations of at least k-1 other records; the size of all equivalence classes in the anonymized dataset is counted, and if the size of any equivalence class is less than the preset k value (e.g., k=5), it is determined that the anonymized data has the risk of re-identification. When the privacy risk detection fails (e.g., the success rate of link attacks exceeds the preset threshold of 0.1%), a generalization operation is automatically triggered to correct the anonymized data. The generalization operation includes generalizing specific attribute values ​​to range values ​​or category values ​​(e.g., generalizing the age "34" to the age range "30-40", or generalizing a specific postal code to a postal code prefix range), until the anonymized dataset meets the K-anonymity requirement (i.e., the size of all equivalence classes is not less than the value of k).

[0047] In the privacy risk verification step, a differential privacy mechanism is introduced to provide secondary protection for the K-anonymity results, specifically including: Based on the sensitivity level labels of data features, the initial privacy budget for each numerical field is determined. Among them, the sensitivity level and The values ​​are negatively correlated; In the equivalence classes generated by K-anonymity, based on the initial privacy budget And the global sensitivity of the feature function, generate Laplace distribution noise and superimpose it on the feature value; Calculate the data utility metrics after noise injection, including mean squared error (MSE) or mean absolute error (MAE). Determine whether the utility indicators meet the preset business analysis or model training accuracy thresholds; If the conditions are not met, the privacy budget is adjusted in reverse using a PID control algorithm based on the deviation between the utility index and the threshold. And re-execute the noise-adding operation; If the conditions are met, the target dataset with differential privacy protection will be output.

[0048] For example, adding Laplace noise to the K-anonymization results dynamically adjusts the privacy budget ε: Highly sensitive data (such as ID card numbers): Assign ε=0.1 and inject high-intensity noise; Low-sensitivity data (such as user tags): Assign ε=1.0 to preserve data distribution characteristics.

[0049] The noise generation formula is expressed as: Noise ( x )= x +Lap(Δ f / ) Where Δf is the global sensitivity of the feature function.

[0050] In this embodiment, a privacy-utility evaluation model is also provided, which calculates the data utility loss after noise injection using the mean squared error (MSE), expressed as the following formula:

[0051] If the MSE exceeds the preset threshold, the ε value is adjusted in the opposite direction and noise is added again until the accuracy requirements are met.

[0052] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0053] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A data anonymization and feature extraction method based on user privacy protection, characterized in that, include: Automated sensitive data classification steps: Use regular expressions and natural language processing techniques to scan database fields and classify data into three levels: high sensitivity, medium sensitivity, and low sensitivity. Desensitization steps: Based on the sensitivity level of the data, a differentiated desensitization algorithm is used to desensitize the data. Highly sensitive data is masked and preserved or encrypted using a hybrid method, medium sensitive data is truncated or obfuscated, and low sensitive data is formatted. Feature extraction steps: By constructing a domain knowledge graph, business semantics in the anonymized data are dynamically identified to extract feature information related to business analysis or model training; among them, periodic patterns of user behavior are extracted through an LSTM network; GeoHash encoding is used to convert address information into gridded coordinates to construct a user activity heatmap; and sentiment and key entity information in user reviews are analyzed based on the BERT model. Privacy risk verification steps: By simulating attacker behavior, link attack tests are conducted using external public datasets and de-identified data to calculate and evaluate the K-anonymity of the de-identified data, and a differential privacy mechanism is introduced to provide secondary protection for the K-anonymity results.

2. The method according to claim 1, characterized in that: For mobile phone numbers in highly sensitive data, a masking method is used, which is achieved through a string replacement function: the first 3 and last 4 characters of the mobile phone number are retained, and the 4th to 7th characters are replaced with mask characters; For address information in sensitive data, a spatial truncation method is used. The address resolution library is used to identify the administrative division level of the address, and the address field after the street or road is truncated, so that the address is generalized to the level of administrative division + street / road. For payment accounts in highly sensitive data, a hybrid encryption method is adopted, specifically using the AES-256-GCM algorithm. While encrypting the plaintext of the payment account, an authentication tag is generated, and the authentication tag is verified during decryption. Furthermore, a hardware security module is used to generate, store, and manage the lifecycle of encryption keys.

3. The method according to claim 2, characterized in that, A three-tier key architecture is built within the hardware security module: Root Key: Stored internally in the security chip of the hardware security module and cannot be exported to external systems; the root key destruction process is automatically triggered when unauthorized disassembly, tampering, or brute-force attacks are detected. Key Encryption Key KEK: Generated by encryption from the root key and stored in the secure storage area of ​​the hardware security module; used to encrypt data encryption key DEK. Data Encryption Key (DEK): A temporary key randomly generated by the hardware security module, used only for encrypting plaintext payment account data in a single transaction or batch.

4. The method according to claim 3, characterized in that, When the application server needs to encrypt the plaintext of the payment account, it performs the following steps: The application server sends a payment account encryption request to the hardware security module; The hardware security module responds to the request, randomly generates an initialization vector (IV) and a data encryption key (DEK), and uses the IV and DEK to perform AES-256-GCM encryption on the plaintext payment account. Simultaneously, it generates an authentication tag, which is a hash derivative of the ciphertext and is used to uniquely identify the ciphertext content and integrity status. The hardware security module uses KEK to encrypt DEK to obtain the ciphertext DEK. The hardware security module returns an encrypted result packet containing the payment account ciphertext, authentication tag, and ciphertext DEK, and returns the payment account ciphertext and ciphertext DEK to the application server, which then associates and stores them in the database. The hardware security module is configured with a timed key rotation strategy, which triggers a key rotation every 30 days via a scheduled task. For newly written payment account data, the corresponding DEK is directly encrypted using the newly generated KEK. For stored historical payment account data, when the data is accessed, the hardware security module automatically calls the new KEK to re-encrypt the original ciphertext DEK until all historical data DEKs are protected by the latest KEK.

5. The method according to claim 3, characterized in that, In key security management based on hardware security modules, the root key is generated using quantum key distribution technology, specifically including: Generate a one-time random root key using a QKD device; Set the quantum key update threshold to trigger the QKD device to regenerate the root key and synchronize it to the key encryption key KEK through the physical tamper-proof channel of the hardware security module; During key rotation, the hardware security module performs a quantum state destruction operation on the old root key to prevent quantum computing attacks from restoring historical keys.

6. The method according to claim 1, characterized in that: In the feature extraction step, the target dataset after desensitization is obtained. The target dataset includes the original fields that have been masked, encrypted, or noise-added. Based on a pre-trained feature mapping model, a non-linear transformation is performed on the desensitized target dataset to generate intermediate feature vectors. The intermediate feature vectors are reduced in dimensionality and made sparse to extract a subset of key features, which are then used as input features for business analysis indicators or model training. Specifically, for the user behavior logs of the target dataset: a time-series feature extraction model is built using an LSTM network to mine periodic patterns in the time-series data of the user behavior logs; GeoHash encoding is used to convert the address information in the user behavior logs into gridded coordinates, and a user activity heatmap is constructed based on the coordinate density; and a pre-trained BERT model is used to parse the text comments in the user behavior logs to extract sentiment and key entity information.

7. The method according to claim 1, characterized in that, In the feature extraction step, the following processing strategies are adopted for ID card number data, including: Place of origin feature extraction: Extract the first 6 digits of the ID card number as the address code, match the address code with a pre-constructed address code-administrative division mapping dictionary, and obtain the corresponding administrative division information as the place of origin feature; Age feature extraction: Extract the birth year digits from the 7th to the 10th digits of the ID number, obtain the current system year, and obtain the age feature through the calculation logic of current year minus birth year; Gender feature extraction: Extract the 17th digit of the ID number and determine the gender feature based on the parity of the digit: if the digit is odd, the person is identified as male, and if the digit is even, the person is identified as female. Data storage method: The original ID number field is masked and then stored; at the same time, the extracted place of origin, age and gender features are stored in a separate feature table in the form of structured fields, and the feature table is stored in association with the masked ID number data.

8. The method according to claim 1, characterized in that, The following steps are included when calculating and evaluating the K-anonymity of desensitized data: Anonymity calculation: Traverse the anonymized target dataset, identify all quasi-identifier combinations, count the number of records in each equivalence class, and determine the minimum K-anonymity value K of the current dataset. min ; Security assessment: Based on the minimum K-anonymity value K min and the preset safety threshold K threshold Determine whether the current anonymized data meets the security release standards; if K min <K threshold If so, it is determined to be a high-risk state; Information loss quantification: Calculate the information loss degree of the current de-identified data. The information loss degree is determined based on the generalization level height or the coefficient of variation. Iterative optimization: When a high-risk state is identified, the adaptive re-anonymization engine is activated. If the information loss is less than the preset upper limit, the micro-aggregation algorithm is used to further split or merge the high-risk equivalence classes in order to increase the size of the equivalence classes. If the information loss exceeds the preset upper limit, differential privacy noise is introduced into the high-risk equivalence class, and the K-anonymity value is recalculated. Output: Repeat the anonymity calculation step until the iterative optimization step, until K is reached. min ≥ K threshold Furthermore, the information loss converges, and the optimized K-anonymous dataset is output.

9. The method according to claim 8, characterized in that: In the privacy risk verification step, the system simulates the role of an attacker, obtains external public datasets, links the external public datasets with the de-identified target datasets, and attempts to locate a specific natural person by matching quasi-identifier combinations. According to the core definition of the K-anonymity model, it is required that the quasi-identifier combination of any record in the target dataset after anonymization is completely consistent with the quasi-identifier combinations of at least k-1 other records. The size of all equivalence classes in the target dataset after anonymization is counted. If the size of any equivalence class is less than the preset k value, it is determined that the anonymized data has a risk of re-identification. When the privacy risk detection fails, a generalization operation is automatically triggered to correct the anonymized target dataset. The generalization operation includes generalizing specific attribute values ​​to range values ​​or category values ​​until the anonymized dataset meets the K-anonymity requirement.

10. The method according to any one of claims 1 to 9, characterized in that, In the privacy risk verification step, a differential privacy mechanism is introduced to provide secondary protection for the K-anonymity results, specifically including: Based on the sensitivity level labels of data features, the initial privacy budget for each numerical field is determined. Among them, the sensitivity level and The values ​​are negatively correlated; In the equivalence classes generated by K-anonymity, based on the initial privacy budget And the global sensitivity of the feature function, generate Laplace distribution noise and superimpose it on the feature value; Calculate the data utility metrics after noise injection, including mean squared error (MSE) or mean absolute error (MAE). Determine whether the utility indicators meet the preset business analysis or model training accuracy thresholds; If the conditions are not met, the privacy budget is adjusted in reverse using a PID control algorithm based on the deviation between the utility index and the threshold. And re-execute the noise-adding operation; If the conditions are met, the target dataset with differential privacy protection will be output.

Images