Data processing method and apparatus, storage medium, and computer program product

Abstract

This application provides a data processing method and apparatus, storage medium, and computer program product. The method includes: receiving a query request sent by a client; identifying sensitive data in the data to be anonymized corresponding to the query request through semantic analysis, rule matching, and association semantic mining; marking the data to be anonymized with sensitive tags; acquiring real-time network environment information; and determining a risk score for accessing the data to be anonymized based on the sensitive tags and the real-time network environment information; selecting a corresponding anonymization algorithm for the data to be anonymized based on the risk score; performing data anonymization on the data to be anonymized according to the anonymization algorithm; and sending the anonymization result to the client.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method and apparatus, storage medium, and computer program product. Background Technology

[0002] Sensitive data in current databases (such as user privacy and trade secrets) mainly faces three types of risks: first, internal threats, including unauthorized access by operations and maintenance personnel and unauthorized export by developers, which constitute intentional or unintentional data misuse; second, external attacks, such as malicious theft methods like Structured Query Language (SQL) injection and vulnerability exploitation; and third, the risk of secondary diffusion in data sharing scenarios. The widely adopted static data masking technology has four key drawbacks: first, once data is masked, it is permanently distorted, making it impossible to recreate the real-world scenario during production problem investigation and testing; second, the crude global masking rules destroy the business relevance between data, severely affecting the validity of test data; third, it lacks fine-grained control mechanisms, making it impossible to implement differentiated masking based on different access roles and scenarios; and finally, traditional masking often destroys the original data format, causing parsing anomalies in downstream systems.

[0003] However, current data anonymization solutions are mainly based on static rules, which puts a lot of pressure on storage. Static rules lack fine-grained control, or they involve complex semantic mining of all data, resulting in wasted resources. They generally suffer from incomplete identification of sensitive data and low flexibility and effectiveness of data anonymization. Summary of the Invention

[0004] This application provides a data processing method and apparatus, a storage medium, and a computer program product.

[0005] This application provides a data processing method, including: The system receives query requests sent by clients and identifies sensitive data in the data to be anonymized corresponding to the query request through semantic analysis, rule matching, and association semantic mining, and marks the data to be anonymized with sensitive tags. Obtain real-time network environment information, and determine the risk score for accessing the data to be de-identified based on the sensitive tags and the real-time network environment information; Based on the risk score, a corresponding de-identification algorithm is selected for the data to be de-identified; The data to be de-identified is de-identified according to the de-identification algorithm, and the de-identification result is sent to the client.

[0006] This application provides a data processing apparatus, including: a processor, a memory, and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute one or more computer programs stored in the memory to implement the data processing method.

[0007] This application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a data processing method.

[0008] This application provides a computer program product, including a computer program that, when executed by a processor, implements a data processing method.

[0009] This application provides a data processing method, apparatus, storage medium, and computer program product. The method includes: receiving a query request sent by a client; identifying sensitive data in the data to be anonymized corresponding to the query request through semantic analysis, rule matching, and association semantic mining; marking the data to be anonymized with sensitive tags; acquiring real-time network environment information; and determining a risk score for accessing the data to be anonymized based on the sensitive tags and the real-time network environment information; selecting a corresponding anonymization algorithm for the data to be anonymized based on the risk score; performing data anonymization on the data to be anonymized according to the anonymization algorithm; and sending the anonymization result to the client. The technical solution provided in this application, through a multi-layered identification mechanism of semantic analysis, rule matching, and association semantic mining, combined with dynamic risk assessment of the real-time network environment, achieves accurate identification and graded anonymization of sensitive data. This reduces the risk of sensitive information leakage while ensuring data availability, and completes real-time anonymization processing before the data is returned to the client, improving the flexibility and effectiveness of data anonymization. Attached Figure Description

[0010] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application; Figure 2 This application provides an exemplary data processing timing diagram. Figure 3 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 1 ; Figure 4 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 2 . Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0012] The technical solutions of this application and how they solve the aforementioned technical problems will be described in detail below through embodiments and in conjunction with the accompanying drawings. The embodiments below can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0013] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0014] This application provides a data processing method implemented by a data processing device. The data processing device can be any electronic device, and this application is not limited thereto. The main problems solved by the technical solution of this application include the following: Existing sensitive information identification technologies driven by Natural Language Processing (NLP) cannot be combined with the relationships between database tables, resulting in the omission of sensitive information identification that is scattered across tables; The dynamic data masking strategy is not integrated with the underlying database execution process (such as query parsing and execution plan generation), resulting in significant performance loss in high-concurrency scenarios. The autonomous optimization mechanism is not adapted to the native characteristics of the database (such as partitioned tables and row-level security policies), making it difficult to cope with the dynamic changes in complex database environments.

[0015] The data processing method of this application, through deep integration with PostgreSQL database executor hooks, forms a closed-loop processing flow of "identification-decision-execution-optimization," effectively solving the technical bottlenecks of traditional solutions in terms of identification capability, dynamic adaptability, and data utility. The technical solution provided by this application is described in detail below.

[0016] Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the data processing method mainly includes the following steps: S101. Receive the query request sent by the client, and identify sensitive data for the data to be de-identified corresponding to the query request through semantic analysis, rule matching and association semantic mining, and mark the data to be de-identified with sensitive tags.

[0017] In the embodiments of this application, the data processing device can receive a query request sent by the client, and through semantic analysis, rule matching and association semantic mining, identify sensitive data in the data to be de-identified corresponding to the query request, and mark the data to be de-identified with sensitive tags.

[0018] It should be noted that, in the embodiments of this application, the query request is used to request access to certain data in the database. The database may be PostgreSQL. The data to be anonymized corresponding to the query request, that is, the data information retrieved based on the query request, is stored under the field name. This embodiment of the application does not limit this.

[0019] In the embodiments of this application, considering the significant limitations of traditional metadata parsing methods in dealing with modern database environments, the core problem lies in the over-reliance on static rules and human experience, leading to inadequacy in dynamic and complex business scenarios. Specifically, this manifests in three aspects: First, in terms of technical implementation, matching methods based on regular expressions and predefined rules lack semantic understanding capabilities. For example, they cannot recognize the semantic equivalence between "medical record number" and "medical record identifier (ID)," and their accuracy in recognizing unstructured names (such as the mixed Chinese and English field "Patient ID_card") is insufficient. Second, in terms of business adaptability, static type-sensitivity mapping tables struggle to handle cross-industry scenarios. The same field name might belong to highly sensitive PHI data in a medical scenario, while in the retail industry it might only be ordinary user information. Traditional methods require manual reconfiguration of rules for each business scenario, and maintenance costs increase exponentially with business line expansion. More importantly, these methods completely fail to capture inter-table relationship characteristics, such as sensitive information stored in a scattered manner through foreign keys (e.g., ID card fragments stored separately in the user base table and transaction table), resulting in an overall sensitive field recall rate generally below 60%. These shortcomings are particularly prominent today with the rise of microservice architecture and agile development. Every database structure adjustment requires manual re-auditing, severely hindering the efficiency and responsiveness of enterprise data governance. Therefore, the proposed sensitive data identification method employs an innovative multi-model fusion architecture, achieving high-precision automatic identification of sensitive information through a collaborative processing flow of semantic analysis, rule matching, and association semantic mining.

[0020] In the embodiments of this application, the data processing device identifies sensitive information in the data to be anonymized corresponding to the query request through semantic analysis, rule matching, and association semantic mining, and labels the data to be anonymized with sensitive tags, including: extracting and analyzing the semantic features of the data to be anonymized based on a pre-trained language model, and generating a first tag for the data to be anonymized containing sensitive data types and confidence levels; matching the data to be anonymized with rules in a regular expression rule base, and generating a second tag for the data to be anonymized containing sensitive data types and confidence levels; the regular expression rule base is a set of rules that describe the format features of sensitive data using regular expressions; performing semantic similarity calculation on the data to be anonymized and the data associated in the inter-table association graph to obtain a cross-table association semantic vector; the inter-table association graph is constructed based on foreign key relationships extracted from database system tables and is used to identify interrelated sensitive data stored in a distributed manner across tables; and fusing the first tag, the second tag, and the cross-table association semantic vector to generate a sensitive tag containing sensitive data types and confidence levels.

[0021] It should be noted that, in the embodiments of this application, the pre-trained language model can be a pre-trained Bidirectional Encoder Representations from Transformers (BERT) multilingual model. This model can be used to construct a deep semantic understanding layer for Named Entity Recognition (NER). For the data to be anonymized, the model's 12-layer Transformer encoder captures semantic features in field names and text content. The bottom layer (layers 1-3) captures local word order patterns and basic semantics, identifies key words (such as ID card, patient_id) in the stored field names, and learns character-level features (such as consecutive digits possibly representing an ID number). The middle layer (layers 4-9) establishes syntactic associations, understands the composition structure of field names (such as the modification relationship of "user_phone_number"), and captures cross-language correspondences (such as "telephone" ≈ "telephone"). The top layer (layers 10-12) forms semantic representations, establishes domain concept associations (in the medical field, "medical record number" ≈ "MRN"), and identifies sensitive data context patterns (address fields are often accompanied by postal codes). The model employs a standard Transformer architecture, where each encoder layer contains two core components: a multi-head attention mechanism and a feedforward neural network. This allows the model to capture complex dependencies in the input sequence. The final model outputs the first label of the data to be anonymized, which may include the field name, sensitive data type, and confidence level.

[0022] It should be noted that, in the embodiments of this application, the regular expression rule base is a set of rules that describe the format characteristics of sensitive data using regular expressions. The rules in the regular expression rule base can be dynamically changed. The rule regularity adopts a hierarchical design architecture. For the data to be anonymized corresponding to the query request, the regular expression rule base is used for scanning. Based on the matching results, a second label is finally generated, similar to the first label, including the field name, sensitive data type, and confidence level. For example, firstly, more than 200 pre-compiled regular expression rules (such as the patterns ^

[17] [\dXx] for ID cards and ^[1-9]\d{15,18}$ for bank cards) are stored in PostgreSQL. During the application phase, all pre-compiled regular expression rules are loaded from PostgreSQL into the cache (Redis) and simultaneously loaded into the application memory. When a rule is updated in PostgreSQL, Redis is updated simultaneously, and a notification is published via Redis Pub to update the rules in memory for each application instance. Upon receiving the notification, the rules are reloaded from Redis. An atomic update strategy is used to ensure that dynamic addition and deletion of rules at runtime does not affect service availability.

[0023] It should be noted that, in the embodiments of this application, semantic mining can extract foreign key relationships based on PostgreSQL system tables (such as the constraint information table pg_constraint, attribute information table pg_attribute, etc.) to construct an inter-table association graph; the BERT model is used to calculate the semantic similarity of the associated data (such as the semantic equivalence of "medical case number" and "medical record ID") to generate cross-table association semantic vectors. For example, when "user_id" in the user table and "patient_user_id" in the medical record table are associated through a foreign key, the "ID card fragment" and "medical insurance card number fragment" associated with "user_id" in both tables are automatically identified as combined sensitive information.

[0024] It should be noted that, in the embodiments of this application, the first label, the second label, and the cross-table association semantic vector can be fused by an adaptive weighted fusion algorithm. The weights of each part can be dynamically adjusted according to the F1 value of the corresponding algorithm on the validation set. For data associated through foreign keys, its weight in the fusion calculation can be increased (the weight coefficient increases by 0.2). Finally, the output is a sensitive label with confidence (range 0-1) and sensitive data type. The confidence can be calculated using the following formula (1): (1) Where i=1 The confidence score for the first label (output of the pre-trained language model), when i=2 The confidence score for the second tag (output from the regular expression rule base); This represents the relevance weight of the data to be de-identified, determined by the cross-table association semantic vector, with 0.2 being the increase in the weight coefficient. This represents the final confidence level after fusion.

[0025] It should be noted that, in the embodiments of this application, the performance of the most recent N semantic analysis, rule matching, and association semantic mining processes can be recorded in real time simultaneously, and the weights are recalculated every K processing cycles (e.g., K=100). (2) (3) Where i=1, This represents the statistical measure of recognition performance in the most recent N semantic analyses. This represents the number of times sensitive data was correctly identified in the most recent N semantic analyses. This represents the number of times sensitive data was incorrectly identified in the most recent N semantic analysis operations. This indicates the number of times sensitive data was missed in the most recent N semantic analyses. This represents the weights before the semantic analysis update. This represents the weights after semantic analysis updates. Similarly, i=2 corresponds to the performance of the relevant parameters in rule matching, and i=3 corresponds to the performance of the relevant parameters in association semantic mining. As a smoothing factor, it can be set to 0.9 by default to prevent drastic fluctuations in weights.

[0026] It should be noted that, in the embodiments of this application, the weighted algorithm for generating sensitive labels and the mechanism for updating weights can be dynamically designed or adjusted according to actual needs and application scenarios, and are not limited in the embodiments of this application.

[0027] S102. Obtain real-time network environment information, and determine the risk score for accessing the data to be de-identified based on the sensitive labels and real-time network environment information.

[0028] In the embodiments of this application, the data processing device can acquire real-time network environment information and determine the risk score of accessing the data to be de-identified based on the sensitive tags and the real-time network environment information.

[0029] It should be noted that, in the embodiments of this application, a risk score is determined based on sensitive tags and real-time network environment information, which is used to dynamically determine the de-identification algorithm and intelligently select the optimal de-identification algorithm during subsequent query execution.

[0030] In the embodiments of this application, a risk score for accessing data to be de-identified is determined based on sensitive labels and real-time network environment information, including: quantifying the sensitivity of the data to be de-identified based on sensitive labels to obtain a data sensitivity score for the data to be de-identified; assessing the degree of environmental anomaly based on real-time network environment information to obtain an environmental anomaly score; and fusing the data sensitivity score and the environmental anomaly score to obtain a risk score.

[0031] It should be noted that, in the embodiments of this application, the risk assessment adopts a parallel pipeline processing architecture, comprising two independent assessment parts: data sensitivity assessment based on sensitive tags and environmental anomaly assessment based on real-time network environment information. The environmental anomaly assessment evaluates the potential threats to accessing the data to be de-identified in the current scenario. The two assessment scores are weighted and fused to generate a risk score for accessing the data to be de-identified, with a score range of [0,1]. The scoring formula is as follows: (4) in, Indicates risk score, Indicates data-sensitive scoring. For the corresponding weights, Indicates an environmental anomaly score. The corresponding weights are used. The specific weight values ​​can be set according to actual needs and application scenarios. For example, the weights are 0.5 and 0.5 respectively. This application embodiment does not limit the weights.

[0032] In the embodiments of this application, the assessment of the degree of environmental anomaly based on real-time network environment information to obtain an environmental anomaly score includes: acquiring sample network environment information at different time points and extracting feature values ​​of the sample network environment information at each time point under multiple feature dimensions to obtain a set of environmental feature values ​​for each time point; determining the time weight of each time point based on the time difference between each time point and the current time; assessing the correlation between different feature dimensions and determining the dimension weight of each feature dimension; constructing an isolated forest by selecting environmental feature values ​​at some time points based on the time weight of each time point and the dimension weight of each feature dimension; extracting feature values ​​of real-time network environment information under multiple feature dimensions, and determining the environmental anomaly score based on the expected path length of the extracted feature values ​​in the isolated forest, the number of segmentations on each feature dimension, and the corresponding dimension weight.

[0033] It should be noted that, in the embodiments of this application, in terms of feature dimension, real-time network environment information is mainly collected through distributed probes, including but not limited to timestamps, Global Positioning System (GPS) coordinates, Internet Protocol (IP) addresses, terminal types, and other raw data. After parsing and preliminary cleaning, this network environment data generates raw records containing multi-dimensional environmental features. The feature engineering layer is responsible for converting the raw data into a spatiotemporal feature tensor E: (5) In E, each row represents a set of environmental feature values ​​at a specific point in time. , representing the m feature values ​​across m feature dimensions at time point 1, and so on. , representing the m feature values ​​under m feature dimensions at time point t, where t is the number of time points and m is the feature dimension. In the embodiments of this application, the main process is to use a sliding window (e.g., the window size of the sliding window is 50) to statistically analyze the sequence data such as access time and operation interval through the time feature pipeline, extract frequency domain features (FFT transform) and time domain features (moving average, standard deviation), and normalize them to the [0,1] interval; the spatial feature pipeline uses the Geohash algorithm to convert GPS coordinates into grid codes, and resolves IP addresses to administrative regions through the MaxMind database, and then maps them to hierarchical spatial codes; the device feature pipeline performs one-hot encoding on categorical variables such as terminal type, operating system, and security patch status, and discretizes continuous variables (such as CPU utilization) by binning, and finally, splices them together to form a t×m-dimensional spatiotemporal feature tensor E.

[0034] It should be noted that, in the embodiments of this application, Isolation Forest is well-suited for environmental anomaly detection due to its linear time complexity and the fact that it does not require labeled data. However, traditional algorithms suffer from randomness bias and dimensionality sensitivity. This application proposes the following improvements: Dynamic subsampling strategy: Traditional isolated forests randomly select subsamples to construct isolation trees (iTrees), leading to unstable detection results. This application employs time-weighted sampling, where recent data receives a higher sampling probability, adapting to dynamic changes in environmental characteristics. The time weight of each time point is determined based on the time difference between each time point and the current time. (6) in, The time weight represents the time point t. Indicates the current moment. Representing time point t, The decay coefficient (default 0.1) makes the model focus more on recent behavioral patterns. Simultaneously, a cross-correlation metric is introduced during the sampling phase, prioritizing feature dimensions with low correlation for segmentation, thus improving isolation efficiency. Anomaly scoring is improved; the original algorithm uses expected path length to calculate anomaly scores. This application introduces feature importance weights into the denominator to correct for differences in contributions from different dimensions: (7) in, This represents the environmental anomaly score, where x represents the feature value extracted from multiple feature dimensions of the real-time network environment information. Indicates the expected path length. Represents the standardization factor. This represents the dimensional weight of the i-th feature dimension. This represents the number of times x is segmented in the i-th feature dimension.

[0035] S103. Based on the risk score, select the corresponding de-identification algorithm for the data to be de-identified.

[0036] In the embodiments of this application, after determining the risk score of accessing the data to be de-identified, the data processing device can select the corresponding de-identification algorithm for the data to be de-identified based on the risk score.

[0037] It should be noted that, in the embodiments of this application, differential privacy and format-preserving encryption technologies can be used to automatically match appropriate de-identification algorithms according to different business scenarios.

[0038] In the embodiments of this application, based on risk scores, a corresponding de-identification algorithm is selected for the data to be de-identified, including: if the risk score is less than or equal to a first threshold, a format-preserving encryption algorithm is selected as the de-identification algorithm; if the risk score is greater than the first threshold and less than or equal to a second threshold, a dynamic masking algorithm is selected as the de-identification algorithm; if the risk score is greater than the second threshold and less than or equal to a third threshold, a differential privacy algorithm is selected as the de-identification algorithm; and if the risk score is greater than the third threshold, a full-field generalization algorithm is selected as the de-identification algorithm.

[0039] In the embodiments of this application, if the risk score is greater than the second threshold and less than or equal to the third threshold, the noise parameters of the differential privacy algorithm are dynamically calculated based on the risk score.

[0040] For example, in the embodiments of this application, firstly, it is determined whether the risk score is ≤0.4 (first threshold). If it is, it is determined to be a low-risk scenario, and the Format-Preserving Encryption (FPE) algorithm is selected to perform encryption while maintaining the integrity of the data structure and verification rules (e.g., a bank card number can still be verified by Luhn after encryption). Otherwise, it enters the medium-risk handling stage. For requests with a risk score in the range of 0.4 (first threshold) - 0.6 (second threshold), a dynamic masking algorithm is enabled to precisely control the degree of information exposure according to the score (e.g., when the score is 0.45, the first 4 digits + the last 2 digits are displayed; when the score is 0.55, only the first and last 1 digits are displayed). The data correlation requirements are checked simultaneously. If there is a cross-table correlation, an encryption token is automatically injected to maintain the integrity of the business logic. When the risk score is greater than 0.6 (second threshold) - 0.9 (third threshold), it enters the high-risk response stage. Advanced protection is triggered, and differential privacy algorithm is applied first. The noise parameter is dynamically calculated according to the score. Among them, the score in the range of 0.6 (-0.7) adopts a different approach. The Laplace noise score is in the range of 0.7-0.9. In strict mode, for POINT type coordinate data, a geographic range generalization is applied, for example, generalizing specific coordinates "(XXX.XXX,XX.XXX)" to the range "XX City XX District", while preserving the calculation validity of geographic information functions (such as ST_Distance); for extremely high scores greater than 0.9 (third threshold), a full-field generalization algorithm is initiated, converting the original value into an irreversible range interval (such as converting the specific value into "..."). (Level identifier).

[0041] S104. Perform data desensitization on the data to be desensitized according to the desensitization algorithm, and send the desensitization result to the client.

[0042] In the embodiments of this application, after a suitable desensitization algorithm is determined, the data to be desensitized can be desensitized according to the desensitization algorithm, and the desensitization result can be sent to the client.

[0043] In the embodiments of this application, data anonymization is performed on the data to be anonymized according to the anonymization algorithm, including: determining the access scope of the database corresponding to the query request based on the database row-level security policy, and executing the query request; intercepting the data to be anonymized obtained by executing the query request through the database executor hook mechanism, and anonymizing the data to be anonymized according to the anonymization algorithm.

[0044] It is understood that, in the embodiments of this application, in order to solve the problem of the de-identification strategy being disconnected from the database execution process, this application can embed the de-identification logic into the key nodes of database query execution through the Executor Hook mechanism of the PostgreSQL database, as follows: Real-time data masking execution: After PostgreSQL generates the execution plan, before the data to be masked is returned to the client, data is intercepted through hook functions, and the aforementioned masking algorithm is called for real-time processing, avoiding complete data exposure in memory. Compared to traditional application-layer masking, this reduces the amount of data transferred between the application and the database, and can reduce query latency in high-concurrency scenarios.

[0045] In conjunction with Row-Level Security (RLS): The masking algorithm is linked with PostgreSQL's RLS. For example, when RLS determines that "user A can only access data in department B", the executor hook will automatically apply the masking algorithm to highly sensitive data to be masked (such as mobile phone numbers) in department B's data, achieving a two-layer protection of "access control + masking".

[0046] Based on the above steps S101-S104, see, for example, [link to example]. Figure 2 The technical solution of this application mainly involves the following: 1. The client sends a query request; 2. PostgreSQL extracts the data to be de-identified corresponding to the query request and sends it to the sensitive data identification engine; 3. The sensitive data identification engine marks the data to be de-identified with sensitive tags and notifies PostgreSQL of the sensitive tags; among them, the sensitive data identification engine identifies sensitive data in the data to be de-identified corresponding to the query request through semantic analysis, rule matching and association semantic mining; 4. PostgreSQL requests a desensitization policy from the policy engine; 5. The strategy engine requests the utility optimization engine to obtain the de-identification algorithm; 6. Utility optimization prompts the policy engine to indicate the de-identification algorithm (including algorithm parameters); 7. The policy engine instructs the desensitization plan to the PostgreSQL executor hook; 8. The PostgreSQL executor hook performs data de-identification and returns the de-identification results to the client.

[0047] In the embodiments of this application, based on the above, a "database metadata linkage learning" mechanism is also proposed. Based on incremental learning of PostgreSQL logs, the query logs in pg_log are parsed to extract newly emerging sensitive field patterns (such as "medical insurance number_v2") and automatically update the training samples of the BERT model. The masking effect of partitioned tables is statistically analyzed (such as the masking coverage of each partition) and the partition-level masking strategy is dynamically adjusted. Rules are adaptively generated. Based on the tablespace distribution of PostgreSQL (such as tables distributed across nodes), distributed masking rules are automatically generated to ensure the consistency of masking across different nodes. "Session-level masking" is automatically applied to temporary tables (pg_temp), and the rules automatically expire after the session ends.

[0048] It also ensures the adaptability and agility of the entire process. By utilizing efficient online learning algorithms or lightweight fine-tuning techniques, it can actively learn the desensitization effect of historical data (statistical feature bias, query result accuracy) and user feedback on misjudgment results without the need to retrain the entire model. It can also dynamically update the built-in sensitive information identification mechanism (model, pattern matching rule base, etc.).

[0049] This application can form a closed-loop "perception-analysis-generation-deployment-feedback-learning" process that can keenly capture and respond to changes in the data environment, continuously optimize the accuracy and efficiency of de-identification rules, and proactively adapt to emerging new sensitive data patterns, thereby significantly reducing long-term operation and maintenance costs, enhancing proactive data security defense capabilities, and ensuring that de-identification strategies are always in sync with business needs and security posture.

[0050] Compared with existing technologies, the technical solution provided in this application uses a fusion architecture (including semantic analysis, rule matching, and association semantic mining) to automatically scan database content and classify sensitive data in structured / unstructured data without the need for pre-defined rules. Secondly, it dynamically adjusts the de-identification algorithm by combining real-time network environment information and sensitive tags with risk scoring, and intelligently selects the optimal de-identification algorithm during query execution. Thirdly, it adopts technologies such as differential privacy and format-preserving encryption to automatically match appropriate de-identification algorithms according to different business scenarios. In addition, it can continuously optimize de-identification rules and adapt to new sensitive data patterns through an incremental learning mechanism.

[0051] This application also provides a data processing apparatus. Figure 3 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 1 .like Figure 3 As shown, in an embodiment of this application, the data processing device 1 includes: The receiving module 11 is used to receive query requests sent by the client; Processing module 12 is used to identify sensitive data in the data to be anonymized corresponding to the query request through semantic analysis, rule matching, and association semantic mining; to mark the data to be anonymized with sensitive tags; to obtain real-time network environment information; and to determine the risk score of accessing the data to be anonymized based on the sensitive tags and the real-time network environment information; to select a corresponding anonymization algorithm for the data to be anonymized based on the risk score; and to anonymize the data to be anonymized according to the anonymization algorithm. The sending module 13 is used to send the de-identification result to the client.

[0052] In one embodiment of this application, the processing module 12 is configured to extract and analyze the semantic features of the data to be anonymized based on a pre-trained language model, and generate a first label for the data to be anonymized containing sensitive data types and confidence levels; match the data to be anonymized with rules in a regular expression rule base, and generate a second label for the data to be anonymized containing sensitive data types and confidence levels; the regular expression rule base is a set of rules that describe the format features of sensitive data using regular expressions; perform semantic similarity calculation on the data to be anonymized and the data associated in the inter-table association graph, and generate a cross-table association semantic vector; the inter-table association graph is constructed based on foreign key relationships extracted from database system tables, and is used to identify interrelated sensitive data stored across tables; and fuse the first label, the second label, and the cross-table association semantic vector to generate the sensitive label containing sensitive data types and confidence levels.

[0053] In one embodiment of this application, the processing module 12 is used to quantify the sensitivity of the data to be de-identified based on the sensitive label to obtain a data sensitivity score for the data to be de-identified; to assess the degree of environmental anomaly based on the real-time network environment information to obtain an environmental anomaly score; and to fuse the data sensitivity score and the environmental anomaly score to obtain the risk score.

[0054] In one embodiment of this application, the processing module 12 is used to acquire sample network environment information at different time points, and extract feature values ​​of the sample network environment information at each time point under multiple feature dimensions to obtain a set of environmental feature values ​​for each time point; determine the time weight of each time point based on the time difference between each time point and the current time; evaluate the cross-correlation between different feature dimensions and determine the dimension weight of each feature dimension; construct an isolated forest by selecting environmental feature values ​​at some time points based on the time weight of each time point and the dimension weight of each feature dimension; extract the feature values ​​of the real-time network environment information under multiple feature dimensions, and determine the environmental anomaly score based on the expected path length of the extracted feature values ​​in the isolated forest, the number of segmentations on each feature dimension, and the corresponding dimension weight.

[0055] In one embodiment of this application, the processing module 12 is configured to: select a format-preserving encryption algorithm as the de-identification algorithm if the risk score is less than or equal to a first threshold; select a dynamic masking algorithm as the de-identification algorithm if the risk score is greater than the first threshold and less than or equal to a second threshold; select a differential privacy algorithm as the de-identification algorithm if the risk score is greater than the second threshold and less than or equal to a third threshold; and select a full-field generalization algorithm as the de-identification algorithm if the risk score is greater than the third threshold.

[0056] In one embodiment of this application, the processing module 12 is used to dynamically calculate the noise parameters of the differential privacy algorithm based on the risk score if the risk score is greater than the second threshold and less than or equal to the third threshold.

[0057] In one embodiment of this application, the processing module 12 is used to determine the access scope of the database corresponding to the query request based on the database row-level security policy, and execute the query request; through the executor hook mechanism of the database, intercept the data to be de-identified obtained by executing the query request, and perform data de-identification on the data to be de-identified according to the de-identification algorithm.

[0058] Based on the same inventive concept Figure 4 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 2 .like Figure 4 As shown in the embodiments of this application, the data processing device 1 includes: a processor 14, a memory 15, and a communication bus 16; The communication bus 16 is used to realize the communication connection between the processor 14 and the memory 15; The processor 14 is used to execute one or more computer programs stored in the memory 15 to implement the data processing method.

[0059] This application also provides a computer program product, including a computer program that, when executed by a processor, implements a data processing method.

[0060] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a data processing method. The computer-readable storage medium can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or it can be a device including one or any combination of the above-mentioned memories, such as a mobile phone, computer, tablet device, personal digital assistant, etc.

[0061] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0062] This application is described with reference to schematic and / or block diagrams of implementations of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the schematic and / or block diagrams can be implemented by computer program instructions, and combinations of blocks in the schematic and / or block diagrams can be implemented. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the schematic and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0063] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the implementation flow diagram. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0065] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, include: The system receives query requests sent by clients and identifies sensitive data in the data to be anonymized corresponding to the query request through semantic analysis, rule matching, and association semantic mining, and marks the data to be anonymized with sensitive tags. Obtain real-time network environment information, and determine the risk score for accessing the data to be de-identified based on the sensitive tags and the real-time network environment information; Based on the risk score, a corresponding de-identification algorithm is selected for the data to be de-identified; The data to be de-identified is de-identified according to the de-identification algorithm, and the de-identification result is sent to the client.

2. The method according to claim 1, characterized in that, The process involves semantic analysis, rule matching, and association semantic mining to identify sensitive information in the data to be anonymized corresponding to the query request, and to label the data with sensitive tags, including: Based on a pre-trained language model, the semantic features of the data to be de-identified are extracted and analyzed to generate a first label for the data to be de-identified, which includes the sensitive data type and confidence level. The data to be anonymized is matched with rules in the regular expression rule base to generate a second label for the data to be anonymized, which includes the sensitive data type and confidence level; the regular expression rule base is a set of rules that use regular expressions to describe the format characteristics of sensitive data. Semantic similarity is calculated between the data to be desensitized and the data associated in the inter-table association graph to generate cross-table association semantic vectors; the inter-table association graph is constructed based on foreign key relationships extracted from tables in the database system and is used to identify sensitive data that are interrelated and stored in a scattered manner across tables; By fusing the first tag, the second tag, and the cross-table association semantic vector, a sensitive tag containing sensitive data types and confidence levels is generated.

3. The method according to claim 1, characterized in that, The step of determining a risk score for accessing the data to be de-identified based on the sensitive tags and the real-time network environment information includes: The sensitivity of the data to be de-identified is quantified based on the sensitive labels to obtain a data sensitivity score for the data to be de-identified; The degree of environmental anomaly is assessed based on the real-time network environment information to obtain an environmental anomaly score. The risk score is obtained by combining the data sensitivity score and the environmental anomaly score.

4. The method according to claim 3, characterized in that, The process of assessing the degree of environmental anomaly based on the real-time network environment information to obtain an environmental anomaly score includes: Obtain sample network environment information at different time points, and extract the feature values ​​of the sample network environment information at each time point under multiple feature dimensions to obtain a set of environmental feature values ​​for each time point; The time weight of each time point is determined based on the time difference between each time point and the current time. Evaluate the cross-correlation between different feature dimensions and determine the dimensional weight of each feature dimension; Based on the time weight of each time point and the dimensional weight of each feature dimension, an isolated forest is constructed by selecting environmental feature values ​​at some time points. The feature values ​​of the real-time network environment information under multiple feature dimensions are extracted. Based on the expected path length of the extracted feature values ​​in the isolated forest, the number of segmentations in each feature dimension and the corresponding dimension weights, the environment anomaly score is determined.

5. The method according to claim 1, characterized in that, The step of selecting a corresponding de-identification algorithm for the data to be de-identified based on the risk score includes: If the risk score is less than or equal to the first threshold, the format-preserving encryption algorithm is selected as the de-identification algorithm. If the risk score is greater than the first threshold and less than or equal to the second threshold, the dynamic masking algorithm is selected as the desensitization algorithm. If the risk score is greater than the second threshold and less than or equal to the third threshold, the differential privacy algorithm is selected as the de-identification algorithm. If the risk score is greater than the third threshold, the full-field generalization algorithm is selected as the de-identification algorithm.

6. The method according to claim 5, characterized in that, The method further includes: If the risk score is greater than the second threshold and less than or equal to the third threshold, the noise parameters of the differential privacy algorithm are dynamically calculated based on the risk score.

7. The method according to claim 1, characterized in that, The step of performing data desensitization on the data to be desensitized according to the desensitization algorithm includes: Based on the database's row-level security policy, the access scope of the database corresponding to the query request is determined, and the query request is executed. The database uses an executor hook mechanism to intercept the data to be de-identified obtained from the query request and performs data de-identification on the data according to the de-identification algorithm.

8. A data processing apparatus, characterized in that, include: Processor, memory, and communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is configured to execute one or more computer programs stored in the memory to implement the method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images