Differential privacy fused compliance test data generation method and device and readable storage medium thereof

Abstract

The invention provides a compliance test data generation method and device fusing differential privacy and a readable storage medium. The compliance test data generation device comprises a data acquisition module, a data sensitivity analysis module, a differential privacy parameter adjustment module, a data generation module and a test data verification module. The data acquisition module acquires and unifies a data format from multi-source heterogeneous data; the data sensitivity analysis module divides data into high, medium and low sensitivity levels by adopting a K-means clustering algorithm and integrating data types, sources and purpose quantitative weights; a differential privacy parameter adjustment module dynamically adjusts a privacy budget epsilon and a noise scale parameter lambda based on the sensitivity level and the test purpose; the data generation module selects a Laplace or exponential mechanism to generate test data according to the data distribution characteristics; and the verification module feeds back optimization parameters through statistical test. According to the method, the problem of test data deviation caused by insufficient privacy protection and static differential privacy parameters of a traditional anonymization method is solved.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and specifically relates to a compliance test data generation method and device integrating differential privacy and a readable storage medium thereof, and is particularly suitable for compliance test data generation in industries with high privacy requirements such as finance and medical care. Background Art

[0002] In the context of digital transformation, the contradiction between compliance testing's reliance on real data and the need for privacy protection has become increasingly prominent. Traditional data anonymization methods (such as simply deleting or replacing sensitive fields) are difficult to resist combined attacks, leading to the risk of privacy leakage; and when differential privacy technology is directly applied, the balance between the amount of noise added and data availability is difficult to dynamically adjust, and there is a lack of a parameter adjustment mechanism that can be flexibly adapted according to test scenarios (such as functional testing and performance testing). Existing technologies cannot achieve accurate and dynamic privacy protection while ensuring data availability, resulting in distorted test results or insufficient privacy protection.

[0003] Therefore, there is an urgent need for a compliance test data generation method, device and readable storage medium that integrates differential privacy to solve the problems existing in the prior art. Summary of the invention

[0004] The embodiments of the present invention provide a compliance test data generation method and device and a readable storage medium thereof that integrate differential privacy, which address the problems existing in current technologies, such as the inability to balance data privacy protection and test data availability, and the lack of a mechanism for dynamically adjusting privacy parameters according to data sensitivity and test purposes.

[0005] The core technology of this invention is to achieve precise adaptation of privacy protection strength and test data availability through data sensitivity grading (based on K-Means clustering algorithm) and dynamic differential privacy parameter adjustment (privacy budget ε, noise scale λ), combined with data distribution optimization and verification feedback mechanism.

[0006] In a first aspect, the present invention provides a method for generating compliance test data integrating differential privacy, the method comprising the following steps: S1. Collect original test data from multiple data sources, and perform format conversion and consistency processing on the data. The multiple data sources include relational databases, non-relational databases and file systems; S2. Use clustering algorithms to classify the sensitivity of data, calculate comprehensive sensitivity scores based on the quantitative weights of data type, source, and purpose, and divide the data into three sensitivity levels: high, medium, and low; S3. Dynamically adjust differential privacy parameters based on sensitivity level and test purpose. The parameters include privacy budget ε and noise scale parameter λ, where high-sensitivity data corresponds to a smaller ε value and low-sensitivity data corresponds to a larger ε value; S4. Based on the adjusted ε and λ, a Laplace mechanism or an exponential mechanism is used to generate test data that meets the differential privacy requirements, and noise parameters are optimized based on data distribution characteristics; S5. Compare the statistical feature deviation between the original data and the generated data through a statistical test method. If the deviation exceeds a preset threshold, feedback is given to step S3 to readjust the parameters and generate data.

[0007] Furthermore, in step S2, the clustering algorithm is a K-means clustering algorithm, the initial cluster centers are determined based on domain knowledge or random initialization, and the number of cluster centers is fixed at 3, corresponding to high, medium, and low sensitivity levels, respectively.

[0008] Furthermore, the data format conversion in step S1 includes: Perform type mapping, field naming conversion and date format unification for relational and non-relational database data; Parse text files in the file system in CSV, JSON, or XML format, and record metadata information for binary files.

[0009] Furthermore, in step S3, parameter adjustment rules are selected according to the test purpose: In functional testing, reduce the ε value to improve data accuracy; In performance testing, the ε value is increased to reduce the impact of noise on data availability.

[0010] Furthermore, in step S4, a differential privacy algorithm is selected for different data distributions: The Laplace mechanism is used for normally distributed data, and the noise scale parameter λ is linearly related to the standard deviation σ; An exponential mechanism is used for uniformly distributed data, and the noise scale parameter is dynamically adjusted based on the data value range.

[0011] Furthermore, the statistical test method in step S5 includes: For normally distributed data, t test or F test was used; KS test or Mann-Whitney U test was used for non-normally distributed data.

[0012] Furthermore, the data consistency processing in step S1 includes: Set the value range for numerical data to clean invalid data; Use hash algorithm to remove duplicate data; Verify field logic relationships and user identification consistency across data sources.

[0013] In a second aspect, the present invention provides a compliance test data generation device integrating differential privacy, comprising: The data collection module is used to collect raw test data from multiple data sources, and perform format conversion and consistency processing on the data. The multiple data sources include relational databases, non-relational databases and file systems; The data sensitivity analysis module uses a clustering algorithm to classify the sensitivity of data, calculates a comprehensive sensitivity score based on the quantitative weights of data type, source, and purpose, and divides the data into three sensitivity levels: high, medium, and low; The differential privacy parameter adjustment module dynamically adjusts the privacy budget ε and the noise scale parameter λ based on the sensitivity level and the test purpose. High-sensitivity data corresponds to a smaller ε value, and low-sensitivity data corresponds to a larger ε value. The data generation module uses the Laplace mechanism or exponential mechanism to generate test data that meets the differential privacy requirements based on the adjusted ε and λ, and optimizes the noise parameters based on the data distribution characteristics; The test data verification module compares the statistical feature deviation between the original data and the generated data through statistical test methods. If the deviation exceeds the preset threshold, it is fed back to the differential privacy parameter adjustment module to regenerate the data.

[0014] In a third aspect, the present invention provides an electronic device including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the above-mentioned compliance test data generation method integrating differential privacy.

[0015] In a fourth aspect, the present invention provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process. The process includes the compliance test data generation method based on the above-mentioned differential privacy integration.

[0016] The main contributions and innovations of the present invention are as follows: 1. The precise balance between privacy protection and data availability By dynamically adjusting differential privacy parameters (privacy budget ε, noise scale λ), according to the data sensitivity level (high / medium / low) and test purpose (functional test / performance test), it can flexibly adapt to different scenario requirements. For example, functional testing of highly sensitive data uses ε=0.1, and performance testing of low-sensitivity data uses ε=1. While ensuring privacy (the smaller the ε value, the stronger the protection), the test data deviation is controlled to ≤10%, which is significantly better than the traditional k-anonymity method (deviation>30%), and the usability is improved by 66.7%.

[0017] 2. Data sensitivity adaptive classification The K-Means clustering algorithm is used to combine the quantitative weights of data types (such as high sensitivity of ID card number and low sensitivity of age), data sources (such as low risk of financial institution database and high risk of third-party platform) and uses (such as low sensitivity of internal audit and high sensitivity of fund transaction) to achieve dynamic sensitivity classification. For example, financial transaction amount > 100,000 yuan is automatically classified as high sensitivity, 10,000-100,000 yuan is medium sensitivity, and < 10,000 yuan is low sensitivity, solving the rigidity problem of traditional static classification.

[0018] 3. Distribution-optimized differential privacy noise addition Select the Laplace mechanism or exponential mechanism for different data distributions (normal / uniform / Poisson distribution) and optimize the noise parameters. For example, in normal distribution data, the noise scale parameter λ is linearly related to the standard deviation σ (λ=kσ), ensuring that the data distribution characteristics (such as mean, variance) after adding noise are highly consistent with the original data, avoiding statistical distortion caused by mismatch of noise distribution in traditional methods.

[0019] 4. Efficient large-scale data processing capabilities Through parallel computing frameworks (such as Apache Spark) and distributed storage optimization, it supports real-time processing of ultra-large-scale data. For example, after the data set is sharded and noise is added in parallel, the processing efficiency is increased by more than 5 times compared with the traditional single-threaded solution, meeting the needs of generating massive data in the fields of finance, the Internet of Things, etc.

[0020] 5. Cross-industry scalability and verification assurance The modular design supports the adaptation of multiple industries such as finance, medical care, and the Internet of Things. For example, when generating electronic medical record test data in the medical field, the sensitivity level is dynamically increased and the ε value is reduced (from 0.5 to 0.2) by real-time monitoring of changes in data usage (such as from research to clinical diagnosis) to ensure patient privacy protection; at the same time, data deviations are verified through hypothesis testing (t-test / KS test) and visual analysis (boxplot / histogram) to ensure the reliability of test results.

[0021] 6. Reproducibility and stability of technical solutions Provides a configurable management interface, users can customize parameters such as sensitivity weights, ε / λ initial values ​​according to industry needs without modifying the core code. For example, financial users can preset the sensitivity threshold of transaction amounts, while medical users can adjust the privacy weight of medical record fields, achieving "one-time development, multiple scenarios reuse".

[0022] The details of one or more embodiments of the invention are set forth in the following drawings and description so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of a method for generating compliance test data integrating differential privacy according to an embodiment of the present invention; Figure 2 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0025] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0026] Existing technologies cannot balance data privacy protection and test data availability, and lack a mechanism to dynamically adjust privacy parameters based on data sensitivity and test purposes.

[0027] Based on this, the present invention solves the problems existing in the prior art based on multi-dimensional data sensitivity analysis and dynamic differential privacy parameter adjustment.

[0028] Embodiment 1 The present invention aims at a compliance test data generation method integrating differential privacy, specifically, referring to Figure 1 , the method comprising: S1. Collect original test data from multiple data sources, and perform format conversion and consistency processing on the data. The multiple data sources include relational databases, non-relational databases and file systems; In this embodiment, the original test data is collected from multiple data sources such as databases and file systems. For example, in the compliance test of the financial system, user account transaction data, personal information data, etc. are collected. After collection, the data is initially cleaned and preprocessed to remove invalid data and duplicate data, unify the data format, and improve the efficiency and accuracy of subsequent processing.

[0029] S2. Use clustering algorithms to classify the sensitivity of data, calculate comprehensive sensitivity scores based on the quantitative weights of data type, source, and purpose, and divide the data into three sensitivity levels: high, medium, and low; In this embodiment, the sensitivity of data is analyzed by comprehensively considering factors such as data type, source and purpose. Data such as user ID number and password are considered highly sensitive data, while age, gender and the like are considered low-sensitivity data. The K-Means clustering algorithm is used to classify the data, and three cluster centers are set to represent high, medium and low sensitivity. Financial data with transaction amounts exceeding 100,000 yuan are classified as high-sensitivity data, 10,000-100,000 yuan as medium-sensitivity data, and less than 10,000 yuan as low-sensitivity data.

[0030] S3. Dynamically adjust differential privacy parameters based on sensitivity level and test purpose. The parameters include privacy budget ε and noise scale parameter λ, where high-sensitivity data corresponds to a smaller ε value and low-sensitivity data corresponds to a larger ε value; In this embodiment, the differential privacy parameters are dynamically adjusted according to the data sensitivity analysis results and the test purpose. For highly sensitive data, a smaller privacy budget (such as ε=0.1) is set to provide stronger privacy protection; for low-sensitivity data, a larger privacy budget (such as ε=1) is set to reduce the impact on data availability. At the same time, the parameters are adjusted according to the test purpose. Performance testing has lower requirements for data accuracy, and the privacy budget can be appropriately increased; functional testing requires more accurate data, so the privacy budget is reduced. In addition to the privacy budget ε, the scale parameter λ of the noise distribution is also adjusted accordingly.

[0031] Specifically, the differential privacy parameter dynamic adjustment algorithm (formula description): define the function adjust_parameters(sensitivity_level,test_purpose), where sensitivity_level is the data sensitivity level (high, medium, low) determined by the data sensitivity analysis module, and test_purpose is the test purpose (performance test, functional test, etc.). By establishing a mapping table, different (sensitivity_level, test_purpose) combinations are mapped to corresponding (ε, λ) and other parameter values. The mapping table is as follows: | sensitivity_level | test_purpose | ε | λ | | --- | --- | --- | --- | | High | Functional Test | 0.1 | 0.01 | | High | Performance Test | 0.3 | 0.05 | | Medium | Functional Test | 0.5 | 0.1 | | Medium | Performance Test | 0.8 | 0.2 | | Low | Functional Test | 0.8 | 0.3 | | Low | Performance Test | 1 | 0.5 | S4. Based on the adjusted ε and λ, a Laplace mechanism or an exponential mechanism is used to generate test data that meets the differential privacy requirements, and noise parameters are optimized based on data distribution characteristics; In this embodiment, according to the adjusted differential privacy parameters, a differential privacy algorithm such as the Laplace mechanism or the exponential mechanism is used to process the original data to generate test data that meets the differential privacy requirements. 2 ), when applying the Laplace mechanism to add noise, the noise scale parameters are adjusted according to the normal distribution parameters, so that the result after adding noise is more consistent with the distribution characteristics of the original data and improves data availability.

[0032] S5. Compare the statistical feature deviation between the original data and the generated data through a statistical test method. If the deviation exceeds a preset threshold, feedback is given to step S3 to readjust the parameters and generate data.

[0033] In this embodiment, the generated test data is verified to ensure that the differential privacy requirements and test requirements are met. The statistical characteristics such as the mean and variance of the original data and the generated data are compared through hypothesis testing methods such as t-tests to evaluate data availability. If the data availability deviation exceeds the set threshold (such as 10%), the differential privacy parameters are readjusted and data is generated.

[0034] Embodiment 2 Based on the same concept, the first embodiment is applied to the generation of compliance test data for a financial system, and the specific steps are as follows: 1. Data collection and preprocessing Data sources: MySQL user account table (including user_id, name, gender, balance, transaction_amount) and MongoDB transaction log documents.

[0035] Format conversion: MySQL field user_id (INT) → unified to user_id (STRING).

[0036] MongoDB nested document transactions is expanded into independent fields txn_date and txn_amount.

[0037] Cleaning rules: Filter abnormal records with negative balance (retain special accounts).

[0038] De-duplicate based on user_id+txn_date+txn_amount hash.

[0039] 2. Data sensitivity analysis Multi-factor weighted calculation: Data types: user_id (0.8), name (0.8), transaction_amount (0.6).

[0040] Data source: internal core database (0.3).

[0041] Data usage: Anti-money laundering rules testing (0.6).

[0042] Comprehensive score: transaction_amount score = 0.6×0.8 (type) + 0.3×0.3 (source) + 0.6×0.6 (purpose) = 0.75.

[0043] K-Means Clustering: Initial center: set according to historical data (low: 500 yuan, medium: 5000 yuan, high: 500,000 yuan).

[0044] Dynamic adjustment: The cluster center is recalculated every week. If the proportion of high-sensitivity transactions exceeds 30%, the high center is increased to 800,000 yuan.

[0045] 3. Differential Privacy Parameter Adjustment Mapping table call: High sensitivity function test →ε=0.1,λ=0.01.

[0046] Real-time feedback: Verification found that the statistical characteristics deviated by 12%, triggering adjustments: ε=0.1×1.2=0.12, λ=0.01×0.8=0.008.

[0047] 4. Test Data Generation Algorithm selection: transaction_amount follows a normal distribution N(5000,2000²) → Laplace mechanism.

[0048] Noise addition: The noise scale λ=0.008×2000=16.

[0049] Generate data: noisy_amount=transaction_amount+Laplace(0,16).

[0050] Parallel Processing: By using SparkRDD partitioning and parallel generation, the processing time for 100,000 data records was reduced from 2 hours to 25 minutes.

[0051] 5. Data Validation Statistical tests: t-test: the mean of original data is 4980 yuan vs the mean of generated data is 5012 yuan (p=0.12>0.05, no significant difference).

[0052] Analysis of variance: F value 1.03 (p=0.45>0.05).

[0053] Visual verification: The QQ (Quantile-Quantile Plot) graph shows that the data points are basically distributed along the reference line, and the histogram shape is consistent with the original data.

[0054] 6. Test the application Anti-money laundering rules testing: The accuracy rate of identifying suspicious transactions using raw data is 98.7%.

[0055] The test accuracy of generated data is 97.2% (deviation 1.5%<10%).

[0056] Embodiment 3 Based on the same concept, the first embodiment is applied to the generation of electronic medical record test data in a medical system, and the specific steps are as follows: 1. Data sensitivity analysis Key fields: patient_id (0.8), diagnosis (0.7), age (0.2).

[0057] Clustering results: High sensitivity: cancer diagnosis record (ε=0.05).

[0058] Medium sensitivity: chronic disease diagnosis record (ε=0.5).

[0059] Low sensitivity: Routine physical examination records (ε=1.0).

[0060] 2. Data generation optimization Distribution adaptation: age is a skewed distribution → Laplace + exponential mixing mechanism.

[0061] Laplace noise: λ1=0.1×σ(σ=12)=1.2.

[0062] Exponential noise: θ = 0.2 × (max_age - min_age) = 0.2 × 80 = 16.

[0063] 3. Verify the results Availability Bias: Diagnostic classification accuracy: original data 92% → generated data 89% (deviation 3.3%).

[0064] Privacy protection verification: Re-identification attack success rate: 0.3% (traditional method 5.2%).

[0065] Embodiment 4 Based on the same concept, the first embodiment is applied to another financial transaction data compliance test, and the specific steps are as follows: Step 1: Data collection and preprocessing 1. Data source configuration: Relational database: Collect user transaction data tables from the MySQL database (fields include user_id, transaction_amount, timestamp), with data types of INT, DECIMAL (10,2), and DATETIME.

[0066] Non-relational database: Collect user profile documents (nested fields such as user_profile.name and user_profile.id_card) from MongoDB and convert them into relational structures through flattening.

[0067] File system: Parse transaction log files in CSV format (with commas as delimiters), and process field values ​​containing commas (such as the description field wrapped in double quotes).

[0068] 2. Data consistency processing: Invalid data cleaning: Filter records where transaction_amount is negative (the amount of financial transactions cannot be negative).

[0069] Cross-data source verification: Associate MySQL's user_id with MongoDB's user_profile.user_id to verify the logical consistency of the ID number and name.

[0070] Step 2: Data Sensitivity Analysis 1. Sensitivity weight calculation: Data types: ID number (weight 0.8), transaction amount (weight 0.6), transaction time (weight 0.2).

[0071] Data sources: internal database (weight 0.3), third-party payment platform (weight 0.7).

[0072] Data usage: Anti-money laundering rule testing (weight 0.6), performance stress testing (weight 0.4).

[0073] Overall sensitivity score: S=0.8×0.7×0.6+0.6×0.3×0.4=0.336+0.072=0.408 2. K-Means Clustering Classification: Initial cluster center: high sensitivity (500,000 yuan), medium sensitivity (50,000 yuan), and low sensitivity (5,000 yuan) are set based on historical data.

[0074] Dynamic adjustment: re-cluster daily incremental data and exclude outliers (such as single transaction amount > 1 million yuan).

[0075] Step 3: Dynamically adjust differential privacy parameters 1. The parameter mapping rules are as follows:

[0076] 2. Adjust trigger conditions in real time: When the transaction purpose changes from "Performance Testing" to "Anti-Money Laundering Testing", the sensitivity level increases from medium to high and ε decreases from 0.8 to 0.1.

[0077] Step 4: Test data generation and optimization 1. Algorithm selection: The transaction amount follows a normal distribution (μ=50,000 yuan, σ=20,000 yuan), adopts the Laplace mechanism, and the noise scale parameter λ=0.1σ=2000 yuan.

[0078] 10,000 test data were generated, and the mean value after adding noise was 49,800 yuan (deviation 0.4%).

[0079] 2. Parallel computing optimization: Apache Spark was used to partition the data into shards (1,000 records per shard) and add noise in a distributed manner, reducing the time required from 120 seconds for a single thread to 25 seconds.

[0080] Step 5: Test data verification and feedback 1. Statistical test: t-test: the original data mean is 50,000 yuan vs. the generated data mean is 49,800 yuan (p=0.12>0.05, no significant difference).

[0081] KS test: The consistency test of the two sets of data distribution passed (D=0.03<critical value 0.05).

[0082] 2. Visual verification: Comparison of box plots: The interquartile range of the generated data (IQR=32,000 yuan) is highly consistent with the original data (IQR=31,000 yuan).

[0083] Deviation feedback: If the deviation is >10%, parameter adjustment is triggered (such as increasing ε from 0.1 to 0.2).

[0084] Embodiment 5 Based on the same concept, the first embodiment is applied to the medical electronic medical record privacy protection test Step 1: Multi-source data integration Collect patient medical records from the HIS system (Oracle database), parse the examination reports in XML format, and unify the date format into YYYY-MM-DD.

[0085] Step 2: Dynamic sensitivity grading Disease type: cancer diagnosis (weight 0.9) > chronic disease (weight 0.5) > routine physical examination (weight 0.3).

[0086] Dynamic Adjustment: When a medical record changes from “research use” to “clinical diagnosis”, the sensitivity level increases from medium to high.

[0087] Step 3: Parameter optimization and generation Application of the exponential mechanism: The diagnosis result field is categorical data (uniform distribution), and the noise parameter θ = 0.05 × (ba) = 0.05 × 10 = 0.5 (assuming that the disease code range is 1-10).

[0088] Generated data distribution: cancer accounted for 5% in the original data, and 4.8% in the generated data (deviation 4%).

[0089] Step 4: Verify cross-industry scalability Configuration management: Set medical field weights (such as diagnosis_type:0.9) through the JSON configuration file without modifying the core code.

[0090] Key technical implementation details Data sensitivity analysis module: Cluster center update strategy: recalculate the cluster center based on new data every week and exclude outliers outside 3σ.

[0091] Sensitivity warning: When the sensitivity score of a certain type of data changes by more than 20%, the manual review process is triggered.

[0092] Differential privacy parameter adjustment module: Machine learning assisted: Random forest model trained based on historical data to predict the optimal ε value (R²=0.89).

[0093] Data generation efficiency optimization: In-memory computing optimization: Use the Apache Arrow format to accelerate Spark data processing and reduce memory usage by 40%.

[0094] Based on Example 4 and Example 5, the experimental data comparison results are as follows:

[0095] Therefore, the present invention is significantly superior to traditional methods in terms of privacy protection strength (ε=0.1 vs. k=3), data availability (deviation ≤10% vs.>30%) and processing efficiency (increased by 4.8 times).

[0096] Embodiment 6 Based on the same concept, the present invention also proposes a compliance test data generation device integrating differential privacy, comprising: The data collection module is used to collect raw test data from multiple data sources, and perform format conversion and consistency processing on the data. The multiple data sources include relational databases, non-relational databases and file systems; The data sensitivity analysis module uses a clustering algorithm to classify the sensitivity of data, calculates a comprehensive sensitivity score based on the quantitative weights of data type, source, and purpose, and divides the data into three sensitivity levels: high, medium, and low; The differential privacy parameter adjustment module dynamically adjusts the privacy budget ε and the noise scale parameter λ based on the sensitivity level and the test purpose. High-sensitivity data corresponds to a smaller ε value, and low-sensitivity data corresponds to a larger ε value. The data generation module uses the Laplace mechanism or exponential mechanism to generate test data that meets the differential privacy requirements based on the adjusted ε and λ, and optimizes the noise parameters based on the data distribution characteristics; The test data verification module compares the statistical feature deviation between the original data and the generated data through statistical test methods. If the deviation exceeds the preset threshold, it is fed back to the differential privacy parameter adjustment module to regenerate the data.

[0097] Embodiment 7 This embodiment also provides an electronic device, referring to Figure 2 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0098] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0099] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0100] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0101] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the compliance test data generation methods integrating differential privacy in the above embodiments.

[0102] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0103] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0104] Input / output devices 408 are used to input or output information.

[0105] Embodiment 8 This embodiment also provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute the process. The process includes the compliance test data generation method integrating differential privacy according to the first embodiment.

[0106] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0107] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0108] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 1 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0109] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A compliance test data generation method integrating differential privacy, characterized in that: The following steps are involved: S1. Collecting raw test data from multiple data sources, and performing format conversion and consistency processing on the data, wherein the multiple data sources include relational databases, non-relational databases and file systems; S2. Use clustering algorithms to classify the sensitivity of data, calculate comprehensive sensitivity scores based on the quantitative weights of data type, source, and purpose, and divide the data into three sensitivity levels: high, medium, and low; S3. Dynamically adjust differential privacy parameters based on the sensitivity level and the test purpose, the parameters including the privacy budget ε and the noise scale parameter λ, wherein high-sensitivity data corresponds to a smaller ε value and low-sensitivity data corresponds to a larger ε value; S4. Based on the adjusted ε and λ, a Laplace mechanism or an exponential mechanism is used to generate test data that meets the differential privacy requirements, and noise parameters are optimized based on data distribution characteristics; S5. Compare the statistical feature deviation between the original data and the generated data through a statistical test method. If the deviation exceeds a preset threshold, feedback is given to step S3 to readjust the parameters and generate data.

2. The method for generating compliance test data integrating differential privacy as claimed in claim 1, characterized in that: In step S2, the clustering algorithm is the K-means clustering algorithm, the initial cluster centers are determined based on domain knowledge or random initialization, and the number of cluster centers is fixed at 3, corresponding to high, medium, and low sensitivity levels, respectively.

3. The method for generating compliance test data integrating differential privacy as claimed in claim 1, characterized in that: The data format conversion in step S1 includes: Perform type mapping, field naming conversion and date format unification for relational and non-relational database data; Parse text files in the file system in CSV, JSON, or XML format, and record metadata information for binary files.

4. The method for generating compliance test data integrating differential privacy as claimed in claim 1, characterized in that: In step S3, select parameter adjustment rules according to the test purpose: In functional testing, reduce the ε value to improve data accuracy; In performance testing, the ε value is increased to reduce the impact of noise on data availability.

5. The method for generating compliance test data integrating differential privacy as claimed in claim 1, characterized in that: In step S4, a differential privacy algorithm is selected for different data distributions: The Laplace mechanism is used for normally distributed data, and the noise scale parameter λ is linearly related to the standard deviation σ; An exponential mechanism is used for uniformly distributed data, and the noise scale parameter is dynamically adjusted based on the data value range.

6. The method for generating compliance test data integrating differential privacy as claimed in claim 1, characterized in that: The statistical test method in step S5 includes: For normally distributed data, t test or F test was used; KS test or Mann-Whitney U test was used for non-normally distributed data.

7. The method for generating compliance test data integrating differential privacy according to any one of claims 1 to 6, characterized in that: The data consistency processing in step S1 includes: Set the value range for numerical data to clean invalid data; Use hash algorithm to remove duplicate data; Verify field logic relationships and user identification consistency across data sources.

8. A compliance test data generation device integrating differential privacy, characterized in that: include: A data collection module is used to collect original test data from multiple data sources, and perform format conversion and consistency processing on the data. The multiple data sources include relational databases, non-relational databases and file systems; The data sensitivity analysis module uses a clustering algorithm to classify the sensitivity of data, calculates a comprehensive sensitivity score based on the quantitative weights of data type, source, and purpose, and divides the data into three sensitivity levels: high, medium, and low; A differential privacy parameter adjustment module dynamically adjusts the privacy budget ε and the noise scale parameter λ based on the sensitivity level and the test purpose, wherein high-sensitivity data corresponds to a smaller ε value and low-sensitivity data corresponds to a larger ε value; The data generation module uses the Laplace mechanism or exponential mechanism to generate test data that meets the differential privacy requirements based on the adjusted ε and λ, and optimizes the noise parameters based on the data distribution characteristics; The test data verification module compares the statistical feature deviation between the original data and the generated data through statistical test methods. If the deviation exceeds the preset threshold, it is fed back to the differential privacy parameter adjustment module to regenerate the data.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the compliance test data generation method integrating differential privacy as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, and the process includes the compliance test data generation method integrating differential privacy according to any one of claims 1 to 7.

Images