Dynamic desensitization test data automatic generation method and system based on large language model

Abstract

This application discloses a method and system for automatically generating dynamic de-identification test data based on a large language model. The method includes: collecting and parsing database metadata; extracting semantic feature vectors of fields; identifying and classifying sensitive types using a multi-path fusion strategy with sample verification; dynamically constructing structured LLM prompt word templates and injecting a dedicated generation strategy; calling the large language model to simultaneously generate four types of test data—normal values, boundary values, outliers, and null values—in a single call; performing format verification, constraint verification, and automatic correction on the generated data; and storing the data by field dimension and establishing a mapping index. This invention achieves accurate understanding of field business semantics through multi-level semantic feature extraction and multi-path fusion recognition. Combined with LLM multi-morphological generation and a three-level quality assurance mechanism, it solves the problems of insufficient semantic understanding, incomplete data morphological coverage, and disconnect between strategy and data in existing technologies, significantly improving the effectiveness and coverage of dynamic de-identification SDK testing.

Description

Technical Field

[0001] This application relates to the field of database security testing technology, specifically to a method, system, computer-readable storage medium, and electronic device for automatically generating dynamic de-identified test data based on a large language model. Background Technology

[0002] A dynamic data masking SDK is a security component that performs real-time data masking based on preset strategies before database query results are returned. During SDK testing, the quality and diversity of test data directly affect the effectiveness and coverage of the tests.

[0003] Currently, test data is mainly prepared using the following methods:

[0004] (1) Template-based static data generation

[0005] Testers predefine data templates (e.g., phone number format: 138****0000) and generate test data in batches using scripts. This method produces test data with a limited format, making it difficult to cover complex scenarios such as boundary values ​​and outliers, thus hindering the detection of defects in the anonymization algorithm under extreme conditions.

[0006] (2) Production data sampling and desensitization

[0007] Real data is sampled from the production environment and anonymized before being used for testing. This approach has the following problems: First, sampling production data involves privacy compliance risks, and the sampling process requires a strict approval process; second, the anonymized data may lose the statistical characteristics and business relevance of the original data; third, for highly sensitive fields (such as ID card numbers and bank card numbers), complete real data may not even exist in the production environment.

[0008] (3) Rule-based synthetic data generation

[0009] CN119513212A (Method for Generating Test Data, October 2024) discloses a technical solution: obtaining business data associated with each business field based on a data anonymization strategy adapted to the field type of each business field. The core of this solution is to generate test data based on field type (such as string, integer, date, etc.) and primary / foreign key relationships, but it has the following shortcomings:

[0010] A. Unable to understand the business semantics of the fields (e.g., although "id_card" and "phone" are both string types, their data formats and generation logic are completely different).

[0011] B. The generated test data lacks semantic authenticity (e.g., the generated names may be meaningless combinations of characters).

[0012] C. It is difficult to generate test data with various forms such as boundary values ​​and outliers.

[0013] (4) Sensitive data generation based on large language model (LLM)

[0014] CN121390041A (A Method for Synthesizing Diverse Sensitive Text Data for De-identification Effectiveness Evaluation, November 2025) discloses a technical solution: using a large language model, and designing diverse strategies based on entity type and sentence structure to generate diverse sensitive text data for de-identification effectiveness evaluation. The main drawback of this solution is:

[0015] A. Its goal is to generate test data for evaluating the effectiveness of de-identification tools, rather than for generating test data for automated testing of dynamic de-identification SDKs;

[0016] B. The generated data is mainly natural language text and lacks specific support for structured database fields (such as mobile phone numbers, ID card numbers, bank card numbers and other data types with strict format constraints);

[0017] C. The generation process does not fully utilize the semantic features of the fields (such as field names, field comments, and pattern features in the data samples) as generation constraints;

[0018] D. The issue of linking generated data with the requirements of de-identification strategies remains unresolved.

[0019] Comprehensive analysis reveals the following core shortcomings in the generation of dynamic de-identification SDK test data:

[0020] Insufficient semantic understanding: Unable to distinguish different business semantics of fields of the same type, the generated test data lacks business authenticity, and the existing solution relies on field type (such as VARCHAR, INT) rather than field semantics (such as name, mobile phone number, ID card number).

[0021] Incomplete data format coverage: It mainly generates normal values, and requires manual supplementation of various data formats such as boundary values, outliers, and null values. It lacks input domain analysis based on the characteristics of desensitization algorithms and automatic boundary value generation mechanisms.

[0022] Strategy and data are disconnected: The generated test data lacks an inherent connection with the de-identification strategy, making it impossible to verify the strategy's boundary conditions. The data generation process is independent of the strategy configuration process, with no linkage mechanism between the two.

[0023] Weak support for format constraints: For fields with strict format constraints (such as check digits in ID card numbers or the LUHN algorithm for bank card numbers), the generated data may be non-compliant. There is a lack of parsing and validation capabilities for field format constraints.

[0024] Poor scalability: Adding new sensitive types requires manually writing generation rules, resulting in high maintenance costs. The generation rules are implemented in a hard-coded manner, lacking dynamic extension capabilities based on semantic templates.

[0025] The aforementioned deficiencies result in insufficient quality and diversity of test data in dynamic de-identification SDK testing, making it difficult to effectively verify the correctness of SDK de-identification under various data formats and boundary conditions, thus affecting the effectiveness and coverage of the test. Summary of the Invention

[0026] To overcome the aforementioned deficiencies in the existing technology, this application proposes a novel method and system for automatically generating test data for automated testing of dynamic de-identified SDKs (Software Development Kits) based on a large language model.

[0027] More precisely, this invention aims to address the problems of insufficient semantic understanding, incomplete data form coverage, disconnect between strategy and data, weak support for format constraints, and poor scalability in the generation of dynamic desensitization SDK test data in the prior art. It provides a method and system for automatically generating dynamic desensitization test data based on a large language model, realizing full-process automation from field semantic understanding to sensitive type identification, LLM prompt word construction, multi-form test data generation, and format constraint verification.

[0028] To achieve the above objectives, the present invention employs the following technical strategies:

[0029] 1. Multi-level field semantic feature extraction mechanism: Combining field name parsing, annotation semantic analysis, sample pattern mining and constraint feature extraction, a field semantic feature vector is constructed to achieve an accurate understanding of the field's business semantics.

[0030] 2. Multi-path fusion sensitive type identification mechanism: It adopts three parallel identification methods: rule matching, semantic analysis and sample verification. The sensitive type is output through weighted voting fusion to improve the identification accuracy.

[0031] 3. Dynamic construction mechanism for structured LLM prompts: Based on the semantic features of fields and sensitive types, dynamically construct structured prompts that include task description, field information, generation requirements, output format, and generation constraints, and inject exclusive generation strategies for sensitive types.

[0032] 4. Automatic generation mechanism for multi-form test data: In a single LLM call, test data of four forms—normal value, boundary value, abnormal value, and null value—are generated simultaneously, covering all data forms required for testing.

[0033] 5. Data quality assurance mechanism: A three-level quality assurance mechanism of format verification, constraint verification, and automatic correction is adopted to ensure that the generated test data meets the field constraint requirements.

[0034] Specifically, this application provides the following technical solutions:

[0035] The first aspect of this application provides a method for automatically generating dynamic de-identified test data based on a large language model, such as... Figure 1 As shown, it includes the following steps:

[0036] S1. Database Metadata Acquisition and Parsing: Acquire metadata information from the target database, standardize the metadata, and construct a set of field metadata objects;

[0037] S2. Field semantic feature extraction: Based on the field metadata object set, perform multi-level semantic feature extraction on each field to construct a field semantic feature vector that includes business semantic type, format mode, length range, numerical range, null value constraint and uniqueness constraint;

[0038] S3. Sensitive Type Identification and Classification: Based on the semantic feature vector of the field, a multi-path fusion strategy of rule matching, semantic analysis and sample verification is used to identify sensitive types, and the final sensitive type is output through weighted voting fusion; the sensitivity level is classified according to the sensitive type and business context.

[0039] S4. Dynamic construction of structured LLM prompts: Based on the sensitivity type, sensitivity level and field semantic feature vector, dynamically construct structured LLM prompt templates and inject a dedicated generation strategy corresponding to the sensitivity type;

[0040] S5. Generation of multi-form test data based on LLM: The structured LLM prompt word template is sent to the large language model to request the generation of test data; the large language model generates multi-form test data simultaneously in a single call according to the constraints and requirements in the prompt words, and organizes and outputs the data according to the form classification.

[0041] S6. Data Format and Constraint Validation: Automatically validate the test data generated by the large language model to ensure that the generated test data meets the field constraint requirements, and calculate a quality score for each data item.

[0042] S7. Test Dataset Storage and Management: Store validated test data by field dimension, establish a mapping index between fields and test datasets, and provide incremental supplementation and version management functions.

[0043] Furthermore, in the method of this application, the metadata information mentioned in step S1 includes: field name, field type, field annotation, and field-level sample data.

[0044] Furthermore, in the method of this application, the multi-level semantic feature extraction in step S2 includes: field name semantic parsing, field annotation semantic analysis, sample data pattern mining, and field constraint feature extraction.

[0045] Furthermore, in the method of this application, the semantic parsing of field names adopts a pre-trained NLP classification model, and performs semantic parsing based on the character-level features, word embedding vectors and naming patterns of the field name string to extract the semantic information of the field name; the semantic analysis of field annotations adopts named entity recognition technology to extract sensitive entity types from the annotation text; the sample data pattern mining adopts regular expression matching to identify the format features of the data; the field constraint feature extraction includes extracting length limits, numerical ranges, null value allowances, unique constraints and foreign key dependencies.

[0046] Furthermore, in the method of this application, the sample verification multi-path fusion strategy in step S3 includes:

[0047] (1) Rule matching: Fast matching is performed based on field name keywords and regular expression patterns, and the candidate set of sensitive types and matching weights are output;

[0048] (2) Semantic analysis: Based on the NLP classification model, the field names and annotations are semantically classified, and the sensitive types and confidence scores are output;

[0049] (3) Sample validation: Validate the format of the sample data to confirm whether it conforms to the format specifications of sensitive types, and output the compliance rate;

[0050] (4) Fusion decision: The above three types of identification results are weighted and fused to output the final sensitive type.

[0051] Furthermore, in the method of this application, the LLM prompt template mentioned in step S4 includes a task description, field information, generation requirements, output format, and generation constraints.

[0052] Furthermore, in the method of this application, the multi-form test data in step S5 includes test data in four forms: normal value, boundary value, outlier value, and null value; wherein: normal value is valid data that meets the format requirements, boundary value is data that is at the constraint boundary, outlier value is illegal data that does not meet the format requirements, and null value is NULL value or empty string.

[0053] Furthermore, in the method of this application, the automatic verification in step S6 includes format verification, constraint verification, and data correction; wherein: the format verification includes regular expression matching verification, verification bit algorithm verification, length verification, and type verification; the data correction includes minor deviation correction, format error regeneration, and correction record;

[0054] The quality score is calculated based on a weighted average of three dimensions: format compliance, business authenticity, and form coverage. Data with a quality score below a preset threshold is marked as requiring manual review.

[0055] Furthermore, in the method of this application, the storage structure of the test dataset storage in step S7 includes a field metadata table, a generation configuration table, and a test data table;

[0056] The mapping index supports multi-dimensional queries by field ID, sensitivity type, data type, and quality score;

[0057] The incremental supplementation identifies new data format requirements based on historical generation configurations, reuses already generated high-quality data and only supplements the missing parts, while maintaining data versions to support rollback.

[0058] A second aspect of this application provides an automatic generation system for dynamic de-identification test data based on a large language model. The system, when running, implements the steps of the aforementioned automatic generation method for dynamic de-identification test data based on a large language model, such as... Figure 2 As shown, the system includes:

[0059] The metadata acquisition module is used to collect metadata information from the target database, standardize the metadata, and construct a set of field metadata objects.

[0060] The semantic feature extraction module is used to perform multi-level semantic feature extraction on each field based on the field metadata object set, and construct a field semantic feature vector that includes business semantic type, format mode, length range, numerical range, null value constraint and uniqueness constraint.

[0061] The sensitivity identification module is used to identify sensitive types based on the semantic feature vector of the field, using a multi-path fusion strategy of rule matching, semantic analysis and sample verification, and output the final sensitivity type through weighted voting fusion; and to classify the sensitivity level according to the sensitivity type and business context.

[0062] The prompt word construction module is used to dynamically construct a structured LLM prompt word template based on the sensitivity type, sensitivity level and field semantic feature vector, and inject a dedicated generation strategy corresponding to the sensitivity type;

[0063] The LLM calling module is used to send the structured LLM prompt word template to the large language model to request the generation of test data; the large language model generates test data of multiple forms in a single call according to the constraints and requirements in the prompt words, and organizes the output according to the form classification.

[0064] The data validation module is used to automatically validate the test data generated by the large language model to ensure that the generated test data meets the field constraint requirements and to calculate a quality score for each data item.

[0065] The storage management module is used to store validated test data by field dimension, establish a mapping index between fields and test datasets, and provide incremental supplementation and version management functions.

[0066] A third aspect of this application provides an electronic device, including: a memory and a processor;

[0067] Memory: Used to store computer programs;

[0068] Processor: Used to execute the computer program to implement the steps of the aforementioned method for automatically generating dynamic de-identified test data based on a large language model.

[0069] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the aforementioned method for automatically generating dynamic de-identified test data based on a large language model.

[0070] In summary, compared with the prior art, the present invention has the following technical advantages:

[0071] (1) Strong semantic comprehension ability

[0072] Existing technologies mainly rely on field types (such as VARCHAR and INT) for data generation, which cannot distinguish different business semantics of fields of the same type (for example, even though they are both VARCHAR types, the generation logic of mobile phone numbers and ID card numbers is completely different).

[0073] This invention employs a multi-layered semantic feature extraction method combining field name semantic parsing, field annotation semantic analysis, and sample data pattern mining. This method can accurately identify the business semantic type of fields (such as mobile phone numbers, ID card numbers, names, and email addresses), and adopts specific data generation strategies for different sensitive types. The generated test data has a high degree of business authenticity.

[0074] (2) Comprehensive coverage of data formats

[0075] Existing technologies mainly generate normal value data, while boundary values, outliers, null values ​​and other types of data need to be manually supplemented, resulting in insufficient test coverage.

[0076] This invention explicitly requires the generation of test data in four types of formats—normal values, boundary values, outliers, and null values—in the LLM prompts, and designs specific generation strategies for each format (such as boundary value coverage of minimum length, maximum length, minimum value, maximum value, etc.), to achieve fully automatic generation of multi-format data and significantly improve test coverage.

[0077] (3) Strategy and data linkage

[0078] In existing technologies, the generation of test data and the configuration of the de-identification strategy are independent of each other, which may result in the generated test data failing to effectively verify the boundary conditions of the de-identification strategy.

[0079] Although this invention primarily focuses on test data generation, it achieves linkage between test data and desensitization strategy requirements through a technical path of sensitive type identification → desensitization strategy matching → data generation constraint injection. For example, when a mobile phone number field is identified, the system automatically matches the test requirements of the mobile phone number desensitization strategy (such as truncation, masking, etc.) and generates test data that can verify the correctness of the strategy under various boundary conditions.

[0080] (4) Format constraint validation is complete

[0081] Existing technologies may generate test data that does not conform to the format constraints of the fields (such as incorrect ID number verification digits or incorrect number of digits in the mobile phone number), making the test data unusable.

[0082] This invention adds a data quality assurance mechanism after LLM generation, including format validation, constraint verification, and automatic correction. Through multiple checks such as regular expression matching, checksum algorithms, and length verification, it ensures that the generated test data meets field constraint requirements. For data with slight deviations, the system automatically corrects them to guarantee data quality.

[0083] (5) Good scalability

[0084] When adding new sensitive types, existing technologies require manually writing generation rules, which results in high maintenance costs and poor scalability.

[0085] This invention adopts an extensible architecture of LLM prompt word templates + sensitive type-specific strategies. When adding a new sensitive type, only the corresponding prompt word template and generation strategy configuration need to be added, without modifying the core code, which has good extensibility.

[0086] (6) Comparison and summary with existing technologies

[0087]

[0088] Other features and advantages of this application will be set forth in detail in the following description, or will become apparent through the implementation of the relevant technical solutions of this application. The objectives and other advantages of this application can be achieved through the technical features and means explicitly pointed out in the description, claims, and drawings, and will be obtained through the implementation of these technical contents. Attached Figure Description

[0089] To more clearly illustrate the technical solution of this application, the accompanying drawings involved in the description of this invention will be briefly introduced below. It should be noted that the drawings only show some embodiments of the invention. For those skilled in the art, other related drawings can be derived from these drawings without creative effort.

[0090] Figure 1 This is a flowchart illustrating the overall implementation of the method for automatically generating dynamic desensitization test data in this application.

[0091] Figure 2 This is a structural diagram of the automatic generation system for dynamic desensitization test data in this application.

[0092] Figure 3 This is a flowchart of the semantic feature extraction process for fields in the method of this application.

[0093] Figure 4 This is a flowchart illustrating the LLM prompt word template construction and data generation process in the method described in this application.

[0094] Figure 5 This is a flowchart of the data format verification and correction process in the method of this application.

[0095] Figure 6 This is a schematic diagram of the sensitive type multi-path fusion identification design architecture in the method of this application.

[0096] Figure 7 This is the overall design architecture diagram of the system in this application.

[0097] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0098] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0099] In this document, the term "comprising" and any variations thereof (such as "including," "including," etc.) are open-ended expressions and should be understood as "including but not limited to," meaning that the listed content is not exhaustive and may include other content not explicitly mentioned. The term "based on" should be understood as "at least partially based on," meaning that the basis or condition referred to may not be the only factor and may involve other relevant factors. The term "one embodiment" should be understood as "at least one embodiment," meaning that the described embodiment is not the only possible implementation, and other similar embodiments may exist.

[0100] In this application, the terms "a" and "a plurality of" are used to modify related elements or features, and their expression is illustrative rather than restrictive. Unless otherwise expressly stated in the context, "a" should be understood as "at least one," and "a plurality of" should be understood as "at least two." Those skilled in the art should reasonably interpret these terms based on the semantic and logical relationships of the context to ensure that they cover the possibility of "one or more."

[0101] Example: An Automatic Method for Dynamic De-identification Test Data Generation Based on a Large Language Model

[0102] The core of this solution is to automatically generate test datasets covering multiple forms (normal values, boundary values, outliers, and null values) by using a large language model as the generation engine, field semantic features as generation constraints, and desensitization strategy requirements as the generation guide.

[0103] Overall technical approach:

[0104] Database metadata (field names, field types, field comments, sample data)

[0105] ↓

[0106] Semantic feature extraction of fields (NLP classification model)

[0107] ↓

[0108] Sensitive Type Identification and Classification (Rule Matching + Semantic Analysis)

[0109] ↓

[0110] LLM prompt word template construction (injecting semantic features + generating requirements)

[0111] ↓

[0112] LLM Multimorphic Test Data Generation

[0113] ↓

[0114] Data format and constraint validation (regular expressions + validation algorithms)

[0115] ↓

[0116] Test dataset storage and management (indexed by field)

[0117] I. An Automatic Generation Method for Dynamic De-identification Test Data Based on a Large Language Model

[0118] Includes the following steps:

[0119] Step 1: Database Metadata Collection and Parsing

[0120] Collect metadata information from the target database, including but not limited to:

[0121] Field names (e.g., phone, id_card, user_name);

[0122] Field types (such as VARCHAR(11), CHAR(18), INT);

[0123] Field comments (e.g., "user's mobile phone number", "ID card number");

[0124] Field-level sample data (a small amount of real data is sampled for pattern recognition).

[0125] The collected metadata is standardized to construct a set of field metadata objects F = {f1, f2, ...,fn}, where each field object contains attributes such as field name, type, comments, and sample.

[0126] Step 2: Extraction of semantic features of fields

[0127] Figure 3 The diagram shows the semantic feature extraction process for each field. For each field, the following semantic feature extraction operations are performed:

[0128] 2.1 Semantic parsing of field names

[0129] A pre-trained NLP classification model is used to perform semantic parsing on field names and output the business semantic category of the field. The classification model is based on the following features:

[0130] Character-level characteristics of field name strings (such as whether they contain keywords like phone, mobile, tel);

[0131] Word embedding of field names;

[0132] The naming pattern of field names (such as camelCase or underscore).

[0133] 2.2 Semantic Analysis of Field Annotations

[0134] When field names are semantically ambiguous, supplementary analysis is performed using field annotations. Named Entity Recognition (NER) technology is used to extract sensitive entity types (such as "phone number", "ID card number", "bank card number") from the annotation text.

[0135] 2.3 Pattern Mining from Sample Data

[0136] Perform regular expression pattern mining on the field sample data to identify the data format characteristics:

[0137] Mobile phone number: matches the pattern ^1[3-9]\d{9}$;

[0138] ID card number: Match the pattern ^[1-9]\d{5}(18|19|20)\d{2}(0[1-9]|1[0-2])(0[1-9]|

[12] \d|3

[01] )\d{3}[\dXx]$;

[0139] Email address: Matches the pattern ^[\w\.-]+@[\w\.-]+\.\w+$;

[0140] Date: Matches patterns such as \d{4}-\d{2}-\d{2} or \d{4} / \d{2} / \d{2}.

[0141] 2.4 Field Constraint Feature Extraction

[0142] Extracting data constraint features from fields:

[0143] Length limits (minimum length, maximum length);

[0144] Numerical range (minimum value, maximum value);

[0145] Whether null values ​​are allowed;

[0146] Is it a unique constraint?

[0147] Foreign key dependency.

[0148] Based on the above analysis, the field semantic feature vector Sv = (semantic_type, format_pattern, length_range, value_range, is_nullable, is_unique) is constructed.

[0149] Step 3: Sensitive Type Identification and Classification

[0150] Based on the semantic feature vector extracted in step 2, the fields are classified into sensitive types and sensitivity levels:

[0151] 3.1 Sensitive Type Identification

[0152] Figure 6 The diagram shows a schematic of a multi-path fusion identification design architecture for sensitive types.

[0153] A multi-path fusion recognition strategy is adopted:

[0154] (1) Rule matching layer: Based on field name keywords and regular expression patterns, fast matching is performed to output a candidate set of sensitive types;

[0155] (2) Semantic analysis layer: Based on the NLP classification model, the field names and annotations are semantically classified, and the confidence scores of sensitive types are output;

[0156] (3) Sample validation layer: Perform format validation on the sample data to confirm whether it conforms to the format specifications of sensitive types;

[0157] (4) Fusion decision layer: The three identification results are weighted and fused to output the final sensitive type (such as TYPE_MOBILE, TYPE_ID_CARD, TYPE_BANK_CARD, TYPE_NAME, TYPE_ADDRESS, TYPE_EMAIL, TYPE_NON_SENSITIVE, etc.).

[0158] 3.2 Sensitivity Level Classification

[0159] Based on the sensitivity type and business context, fields are divided into three sensitivity levels:

[0160] Sensitive (L3): ID card number, bank card number, password, biometrics, etc.;

[0161] Controlled (L2): Phone number, email address, detailed address, name, etc.;

[0162] Internal (L1): Gender, age group, city, etc.

[0163] Step 4: LLM prompt template construction

[0164] Figure 4 The diagram shows the process of constructing LLM prompt word templates and generating data.

[0165] Based on sensitivity type, sensitivity level, semantic features, and generation requirements, LLM prompt word templates are dynamically constructed.

[0166] 4.1 Prompt Word Template Structure Design

[0167] The prompt template uses a structured design and includes the following components:

[0168]

Task Description

[0169] You are a test data generation assistant. Please generate diverse test data for the dynamic anonymization SDK test according to the following requirements.

[0170]

Field Information

[0171] - Sensitive type: {sensitive_type}

[0172] - Sensitivity level: {sensitive_level}

[0173] - Format requirement: {format_pattern}

[0174] - Length limit: {length_range}

[0175] - Value range: {value_range}

[0176] [Generate Requirements]

[0177] Please generate {count} records for each of the following data formats:

[0178] 1. Normal value: Valid data that conforms to the format requirements.

[0179] 2. Boundary values: data at length boundaries and numerical boundaries.

[0180] 3. Outliers: Illegal data such as incorrect format, incorrect data type, or excessive length.

[0181] 4. Null value: NULL value or empty string.

[0182] Output Format

[0183] Please output the data in JSON array format, with each record containing the following fields: value, morphology, and description.

[0184] Generate constraints

[0185] - The data should be business-authentic and consistent with real business scenarios.

[0186] - Boundary values ​​should cover minimum length, maximum length, minimum value, and maximum value.

[0187] - Outliers should include common error formats.

[0188] 4.2 Dynamic Parameter Injection for Prompt Words

[0189] Based on the field features extracted in steps 2 and 3, dynamically populate the parameters in the prompt word template:

[0190] When the field is a mobile phone number, the format must be filled in with the regular expression pattern ^1[3-9]\d{9}$, and the length limit must be filled in with {11,11};

[0191] When the field is an ID card number, the format requires the ID card regular expression pattern to be filled in, and at the same time, the logic for calculating the check digit is injected to prompt the user.

[0192] When the field is a numeric type, the numeric range is extracted from the field constraints, and the boundary value generation prompts automatically include minimum value -1, minimum value, maximum value, maximum value +1, etc.

[0193] 4.3 Generation Strategy Specific for Sensitive Types

[0194] For different sensitivity types, inject specific generation strategies into the prompt words:

[0195]

[0196] Step 5: Generation of Multimorphic Test Data Based on LLM

[0197] 5.1 LLM Calls and Data Generation

[0198] The prompt word template constructed in step 4 is sent to a large language model (such as GPT-4, Claude, or Wenxin Yiyan) to request the generation of test data. The LLM returns a structured test dataset based on the constraints and requirements in the prompt words.

[0199] 5.2 Multi-format data classification and organization

[0200] The data returned by LLM is organized according to its morphology, generating the following four types of data:

[0201]

[0202] 5.3 Control of Data Generation Volume

[0203] Supports configuring the number of data generated for each field and each data format; the default configuration is:

[0204] Normal values: 5;

[0205] Boundary values: 2 for each boundary point (e.g., for length boundaries: 2 for minimum length and 2 for maximum length);

[0206] Outliers: 5 (covering common error types);

[0207] Null values: 2 (1 NULL and 1 empty string).

[0208] Step 6: Data Format and Constraint Validation

[0209] Figure 5The diagram shows the data format verification and correction process.

[0210] Automatically validate the data generated by LLM to ensure that the data meets the field constraint requirements.

[0211] 6.1 Format Validation

[0212] For each generated data entry, validate it according to the format requirements of the fields:

[0213] (1) Regular expression matching validation: Validates whether the data matches the preset regular expression pattern;

[0214] (2) Check bit algorithm verification: For data containing check bits, such as ID card numbers, verify whether the check bits are correct;

[0215] (3) Length verification: Verify whether the data length is within the allowed range;

[0216] (4) Type validation: Verify whether the data type matches the field definition.

[0217] 6.2 Data Correction Mechanism

[0218] For data that fails validation, perform automatic correction:

[0219] (1) Minor deviation correction: If a mobile phone number is missing a digit, it will be automatically filled in according to the context;

[0220] (2) Regeneration of format errors: For data that cannot be automatically corrected, mark it as a generation failure and call LLM again to generate alternative data;

[0221] (3) Correction record: Record each correction operation for data quality assessment and prompt word optimization.

[0222] 6.3 Data Quality Score

[0223] A quality score is calculated for each generated data point, with the following scoring dimensions:

[0224] Format compliance (0-1 point): Whether the data fully conforms to the format requirements;

[0225] Business Authenticity (0-1 point): Does the data have reasonable business semantics?

[0226] Morphological coverage (0-1 point): Whether it accurately belongs to the target data morphology.

[0227] The overall score is a weighted average of three dimensions, and data with a quality score below 0.8 are marked as "requires manual review".

[0228] Step 7: Test Dataset Storage and Management

[0229] 7.1 Data Storage Structure

[0230] The generated test data will be stored in the following structure:

[0231]

[0232] 7.2 Data Indexing and Querying

[0233] Create a field-test dataset mapping index to support the following query methods:

[0234] Query by field ID: Retrieves all test data for a specific field;

[0235] Query by sensitivity type: Retrieve test data for a specific type of sensitive field;

[0236] Query by data format: Retrieve test data in a specified format;

[0237] Search by quality score: Get high-quality or low-quality data.

[0238] 7.3 Incremental Supplementation and Updates

[0239] Incremental generation is supported when field constraints change or test requirements are updated:

[0240] Identify new data format requirements based on historical generated configurations;

[0241] Reuse existing high-quality data, only supplementing the missing parts;

[0242] Maintain data versions and support rolling back to historical versions.

[0243] II. Automatic Generation System for Dynamic De-identification Test Data Based on Large Language Model

[0244] Figure 7 The diagram shows the overall design architecture of this system, which includes the following modules:

[0245]

[0246] System data flow:

[0247] Database → Metadata Acquisition Module → Semantic Feature Extraction Module → Sensitive Feature Identification Module → Prompt Word Construction Module → LLM Call Module → Data Validation Module → Storage Management Module → Test Dataset Output.

[0248] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementations of systems, methods, and computer program products according to various embodiments of this application, including architecture, functionality, and operation. In these figures, each block may represent a module, program segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should be noted that each block in the block diagrams and / or flowcharts, and combinations thereof, can be implemented using either a dedicated hardware-based system or a combination of dedicated hardware and computer instructions to achieve the specified function or operation.

[0249] like Figure 8 As shown in the illustration, an embodiment of this application also discloses an electronic device, including: a processor 310, a communication interface 320, a memory 330 for storing a processor-executable computer program, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the steps of the above-described method for automatically generating dynamic de-identified test data based on a large language model.

[0250] It is understood that, in addition to memory and a processor, this electronic device may also include input devices (such as a keyboard), output devices (such as a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor through I / O interfaces (i.e., input / output interfaces).

[0251] The operations described in this application can be implemented by writing computer program code using one or more programming languages ​​or a combination thereof. The programming languages ​​include, but are not limited to, the following types:

[0252] Object-oriented programming languages, such as Java, Smalltalk, C++, etc.

[0253] Conventional procedural programming languages, such as "C" or similar programming languages.

[0254] The execution methods of program code include, but are not limited to:

[0255] It runs entirely on the user's computer;

[0256] Part of it executes on the user's computer, and part of it executes on a remote computer;

[0257] Execute as a standalone software package;

[0258] It is executed entirely on a remote computer or server.

[0259] In scenarios involving remote computers, the remote computer can connect to the user's computer via any type of network, including but not limited to local area networks (LANs) or wide area networks (WANs). Furthermore, the remote computer can also connect to external computers through an internet service provider, for example, by utilizing the internet for connection.

[0260] Furthermore, this application also discloses a computer-readable storage medium, which, when the instructions in the computer-readable storage medium are executed by the processor of an electronic device, enables the electronic device to perform the various steps of the automatic generation method for dynamic de-identification test data based on a large language model disclosed in this application.

[0261] In the context of this application, a computer-readable storage medium refers to a tangible medium capable of storing computer program code and related data. Such computer-readable storage media can be used to store the program code and related data described in this application to support program execution and persistent data storage.

[0262] Specifically, according to embodiments of this application, the processes described in the flowcharts can be implemented as computer software programs. For example, embodiments of this application relate to a computer program product comprising a computer program carried on a non-transitory computer-readable medium. This computer program contains program code for executing the automatic generation method for dynamic de-identification test data based on a large language model disclosed in this application. When this computer program is executed by a processing system, it can achieve the functions defined in the embodiments of this application.

[0263] While the foregoing discussion contains several specific implementation details, these details should not be construed as limiting the scope of this application. The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features. Furthermore, this application should also cover other technical solutions formed by any combination of the above-described technical features or their equivalents without departing from the foregoing disclosed concept.

[0264] Those skilled in the art should also understand that modifications can be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features, without departing from the spirit and scope of the technical solutions of the embodiments of this application. These modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the core spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for automatically generating dynamic de-identified test data based on a large language model, characterized in that, Includes the following steps: S1. Database Metadata Acquisition and Parsing: Acquire metadata information from the target database, standardize the metadata, and construct a set of field metadata objects; S2. Field semantic feature extraction: Based on the field metadata object set, perform multi-level semantic feature extraction on each field to construct a field semantic feature vector that includes business semantic type, format mode, length range, numerical range, null value constraint and uniqueness constraint; S3. Sensitive Type Identification and Classification: Based on the semantic feature vector of the field, a multi-path fusion strategy of rule matching, semantic analysis and sample verification is adopted to identify sensitive types, and the final sensitive type is output through weighted voting fusion. Sensitivity levels are classified according to the sensitivity type and business context; S4. Dynamic construction of structured LLM prompts: Based on the sensitivity type, sensitivity level and field semantic feature vector, dynamically construct structured LLM prompt templates and inject a dedicated generation strategy corresponding to the sensitivity type; S5. Generation of multi-morphological test data based on LLM: The structured LLM prompt word template is sent to the large language model to request the generation of test data; The large language model generates multi-form test data in a single call based on the constraints and requirements in the prompt words, and organizes the output according to the form classification. S6. Data Format and Constraint Validation: Automatically validate the test data generated by the large language model to ensure that the generated test data meets the field constraint requirements, and calculate a quality score for each data item. S7. Test Dataset Storage and Management: Store validated test data by field dimension, establish a mapping index between fields and test datasets, and provide incremental supplementation and version management functions.

2. The method according to claim 1, characterized in that, The metadata information mentioned in step S1 includes: field name, field type, field comment, and field-level sample data.

3. The method according to claim 1, characterized in that, The multi-level semantic feature extraction in step S2 includes: semantic parsing of field names, semantic analysis of field annotations, pattern mining of sample data, and extraction of field constraint features.

4. The method according to claim 3, characterized in that, The semantic parsing of the field name adopts a pre-trained NLP classification model, which performs semantic parsing based on the character-level features, word embedding vectors and naming patterns of the field name string to extract the semantic information of the field name; The semantic analysis of the field annotations uses named entity recognition technology to extract sensitive entity types from the annotation text; the sample data pattern mining uses regular expression matching to identify the format features of the data. The field constraint feature extraction includes extracting length limits, numerical ranges, null value allowances, unique constraints, and foreign key dependencies.

5. The method according to claim 1, characterized in that, The sample verification multi-path fusion strategy described in step S3 includes: (1) Rule matching: Fast matching is performed based on field name keywords and regular expression patterns, and the candidate set of sensitive types and matching weights are output; (2) Semantic analysis: Based on the NLP classification model, the field names and annotations are semantically classified, and the sensitive types and confidence scores are output; (3) Sample validation: Validate the format of the sample data to confirm whether it conforms to the format specifications of sensitive types, and output the compliance rate; (4) Fusion decision: The above three types of identification results are weighted and fused to output the final sensitive type.

6. The method according to claim 1, characterized in that, The LLM prompt template mentioned in step S4 includes task description, field information, generation requirements, output format, and generation constraints.

7. The method according to claim 1, characterized in that, The multi-form test data mentioned in step S5 includes test data in four forms: normal values, boundary values, outliers, and null values. Among them, normal values ​​are valid data that meets the format requirements, boundary values ​​are data that is at the constraint boundary, outliers are illegal data that does not meet the format requirements, and null values ​​are NULL values ​​or empty strings.

8. The method according to claim 1, characterized in that, The automatic verification in step S6 includes format verification, constraint verification, and data correction; wherein: the format verification includes regular expression matching verification, verification bit algorithm verification, length verification, and type verification; the data correction includes minor deviation correction, format error regeneration, and record correction; The quality score is calculated based on a weighted average of three dimensions: format compliance, business authenticity, and form coverage. Data with a quality score below a preset threshold is marked as requiring manual review.

9. The method according to claim 1, characterized in that, The storage structure of the test dataset mentioned in step S7 includes a field metadata table, a generation configuration table, and a test data table; The mapping index supports multi-dimensional queries by field ID, sensitivity type, data type, and quality score; The incremental supplementation identifies new data format requirements based on historical generation configurations, reuses already generated high-quality data and only supplements the missing parts, while maintaining data versions to support rollback.

10. A dynamic de-identification test data automatic generation system based on a large language model, characterized in that, The system runtime implements the steps of the automatic generation method for dynamic de-identification test data based on a large language model as described in any one of claims 1-9, including: The metadata acquisition module is used to collect metadata information from the target database, standardize the metadata, and construct a set of field metadata objects. The semantic feature extraction module is used to perform multi-level semantic feature extraction on each field based on the field metadata object set, and construct a field semantic feature vector that includes business semantic type, format mode, length range, numerical range, null value constraint and uniqueness constraint. The sensitivity identification module is used to identify sensitive types based on the semantic feature vector of the field, using a multi-path fusion strategy of rule matching, semantic analysis and sample verification, and output the final sensitivity type through weighted voting fusion; and to classify the sensitivity level according to the sensitivity type and business context. The prompt word construction module is used to dynamically construct a structured LLM prompt word template based on the sensitivity type, sensitivity level and field semantic feature vector, and inject a dedicated generation strategy corresponding to the sensitivity type; The LLM calling module is used to send the structured LLM prompt word template to the large language model to request the generation of test data; the large language model generates test data of multiple forms in a single call according to the constraints and requirements in the prompt words, and organizes the output according to the form classification. The data validation module is used to automatically validate the test data generated by the large language model to ensure that the generated test data meets the field constraint requirements and to calculate a quality score for each data item. The storage management module is used to store validated test data by field dimension, establish a mapping index between fields and test datasets, and provide incremental supplementation and version management functions.

Images