Simulation data synthesis methods, devices, electronic equipment and storage media

Abstract

This application discloses a simulation data synthesis method, apparatus, electronic device, and storage medium, belonging to the field of computer technology. The method includes: acquiring target structured raw data and first target data information for each target raw field in the target structured raw data; the first target data information includes the privacy properties of the target raw fields; when the privacy properties of the target raw fields are in a first state, generating a first simulation field based on the target raw fields and the first target data information using a first data generation model; when the privacy properties of the target raw fields are in a second state, generating a second simulation field based on the target raw fields and the first target data information using a second data generation model; and generating target simulation data for the target structured raw data based on the first and second simulation fields.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a method, apparatus, electronic device and storage medium for simulating data synthesis. Background Technology

[0002] Public data operation platforms are fundamental infrastructures for leveraging data elements to empower socio-economic development and are a vital force in promoting the digital economy. With the development of big data technology, the application scope of public management data, such as government data, is becoming increasingly broad. However, in the process of extracting value from synthetic simulation data, how to protect privacy and sensitive information within public management data has become an urgent issue to be addressed. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a simulation data synthesis method, apparatus, electronic device, and storage medium to solve the problem of the inability to protect the privacy information of the original data during the synthesis of simulation data.

[0004] Firstly, this application provides a method for synthesizing simulation data, including:

[0005] Obtain the target structured raw data and the first target data information for each target raw field in the target structured raw data; the first target data information includes the privacy properties of the target raw fields;

[0006] For each target original field, under the condition that the privacy property of the target original field is in the first state, the first simulation field is generated based on the target original field and the first target data information through the first data generation model; the first state indicates that the target original field includes privacy information;

[0007] When the privacy property of the target original field is in the second state, the second simulation field is generated based on the target original field and the first target data information through the second data generation model; the second state indicates that the target original field does not include privacy information.

[0008] Based on the first simulation field and the second simulation field, target simulation data and first target data information are generated for the target structured original data.

[0009] According to the simulation data synthesis method of this application, by acquiring the target structured original data and the first target data information of each target original field in the target structured original data, for each target original field, under the condition that the privacy property of the target original field is in a first state, a first simulation field is generated based on the target original field and the first target data information through a first data generation model; the first state indicates that the target original field includes privacy information; under the condition that the privacy property of the target original field is in a second state, a second simulation field is generated based on the target original field and the first target data information through a second data generation model; the second state indicates that the target original field does not include privacy information; based on the first simulation field and the second simulation field, target simulation data for the target structured original data is generated, so as to avoid the leakage of target original fields including privacy information by means of the collaboration of the first data generation model and the second data generation model, the first data generation model processes the target original fields including privacy information, and the second data generation model processes the target original fields not including privacy information, thereby solving the problem that it is impossible to protect the privacy information of the original data during the process of synthesizing simulation data.

[0010] According to one embodiment of this application, when the privacy nature of the target original field is in a first state, a first simulation field is generated based on the target original field and first target data information through a first data generation model, including:

[0011] With the privacy property of the target original field in the first state, the first simulation data synthesis method is matched to the target original field through the target large model;

[0012] The first simulation data synthesis method is executed through the first data generation model to generate the first simulation field.

[0013] According to one embodiment of this application, a first simulation data synthesis method is executed through a first data generation model to generate a first simulation field, including:

[0014] Based on the prompt word templates pre-stored in the first data generation model, the first simulation data synthesis method is executed to generate the first simulation field.

[0015] According to one embodiment of this application, when the privacy nature of the target original field is in a second state, a second simulation field is generated based on the target original field and the first target data information using a second data generation model, including:

[0016] When the privacy property of the target original field is in the second state, the second simulation data synthesis method is matched to the target original field through the target large model;

[0017] The second simulation data synthesis method is executed using the second data generation model to generate the second simulation field.

[0018] According to one embodiment of this application, a second simulation data synthesis method is executed through a second data generation model to generate a second simulation field, including:

[0019] The target original field is input into the high-dimensional feature space of the second data generation model, and the feature distribution and association pattern are obtained through adversarial learning.

[0020] The second simulation field is obtained by sampling based on feature distribution and association pattern.

[0021] According to one embodiment of this application, obtaining target structured raw data and first target data information for each target raw field in the target structured raw data includes:

[0022] Obtain the target structured raw data; the target structured raw data includes several raw fields;

[0023] Perform data parsing operations on the target original field to obtain the original field name and data type; the target original field can be any one of several original fields.

[0024] Based on the original field name and the preset privacy field rule base, determine the privacy nature of the target original field;

[0025] The original field name, original field data type, and privacy properties are determined as the primary target data information for the target original field.

[0026] According to one embodiment of this application, after generating target simulation data for the target structured raw data based on a first simulation field and a second simulation field, the method includes:

[0027] Obtain the evaluation value of the target simulation data under the target evaluation index; the target evaluation index includes at least one of the following: data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and raw data leakage rate.

[0028] Based on the weights and evaluation values ​​corresponding to the target evaluation indicators, obtain the overall quality evaluation score of the target simulation data;

[0029] Based on the overall quality assessment score, it is determined whether the target simulation data is qualified.

[0030] According to one embodiment of this application, after determining whether the target simulation data is qualified based on the overall quality assessment score, the method includes:

[0031] If the target simulation data is unqualified, the original target fields are parsed again to obtain the second target data information; the second target data information includes the updated privacy properties.

[0032] Based on the updated privacy properties, the target data generation model is determined from the first data generation model and the second data generation model, and the target simulation data is regenerated based on the target data generation model.

[0033] Manual intervention is triggered if the number of data parsing operations performed on the target original field is greater than or equal to the target threshold.

[0034] Secondly, this application provides a simulation data synthesis apparatus, comprising:

[0035] The first acquisition module is used to acquire the target structured raw data and the first target data information of each target raw field in the target structured raw data; the first target data information includes the privacy properties of the target raw fields;

[0036] The first generation module is used to generate a first simulation field for each target original field, provided that the privacy property of the target original field is in a first state, by using a first data generation model based on the target original field and the first target data information; the first state indicates that the target original field includes privacy information;

[0037] The second generation module is used to generate a second simulation field based on the target original field and the first target data information through a second data generation model when the privacy property of the target original field is in a second state; the second state indicates that the target original field does not include privacy information.

[0038] The third generation module is used to generate target simulation data for the target structured original data based on the first simulation field and the second simulation field.

[0039] According to the simulation data synthesis apparatus of this application, by acquiring target structured raw data and first target data information of each target raw field in the target structured raw data, for each target raw field, when the privacy nature of the target raw field is in a first state, a first simulation field is generated based on the target raw field and the first target data information through a first data generation model; the first state indicates that the target raw field includes privacy information; when the privacy nature of the target raw field is in a second state, a second simulation field is generated based on the target raw field and the first target data information through a second data generation model; the second state indicates that the target raw field does not include privacy information; based on the first simulation field and the second simulation field, target simulation data for the target structured raw data is generated, so as to avoid the leakage of target raw fields including privacy information by means of the collaboration of the first data generation model and the second data generation model, the first data generation model processes the target raw field including privacy information, and the second data generation model processes the first target data information of the target raw field not including privacy information, thereby solving the problem of not being able to protect the privacy information of the raw data during the process of synthesizing simulation data.

[0040] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the simulation data synthesis method described in the first aspect.

[0041] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0042] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0043] Figure 1 This is one of the flowcharts illustrating the simulation data synthesis method provided in the embodiments of this application;

[0044] Figure 2 This is a second schematic flowchart of the simulation data synthesis method provided in the embodiments of this application;

[0045] Figure 3 This is a schematic diagram of the simulation data synthesis device provided in the embodiments of this application;

[0046] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0048] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0049] The simulation data synthesis method, apparatus, electronic device, and storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0050] The simulation data synthesis method can be applied to the terminal, specifically executed by the hardware or software in the terminal.

[0051] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0052] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0053] The simulation data synthesis method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the simulation data synthesis method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The simulation data synthesis method provided in this application embodiment will be described below using an electronic device as the execution subject.

[0054] like Figure 1As shown, the simulation data synthesis method includes steps 110, 120, 130 and 140.

[0055] Step 110: Obtain the target structured raw data and the first target data information for each target raw field in the target structured raw data; the first target data information includes the privacy properties of the target raw fields.

[0056] In some embodiments, target structured raw data can be obtained, which may include several raw fields. Parsing operations are performed on the target raw fields to obtain first target data information for the target raw fields. The target raw fields may be any one of the several raw fields.

[0057] In practice, the target structured raw data can be data from any industry. For example, the target structured raw data can be public management data in a government setting (which may include census data, employment analysis data, newborn population data, etc.), or it can be data from the medical industry, the financial industry, or any theoretically feasible data in a non-government setting. The target structured raw data can also be data in any format.

[0058] In some embodiments, before parsing the target original fields, the target structured original data can be preprocessed, and then the target original fields in the preprocessed target structured original data can be parsed. Data preprocessing may include mean filtering, Gaussian filtering, or any theoretically feasible data preprocessing method. Data preprocessing may also include steps such as data format validation, null value imputation, meaningless field removal, and field value normalization.

[0059] In some embodiments, data parsing operations can be performed on the target original field to obtain first target data information such as the privacy nature of the target original field. The privacy nature indicates whether the target original field contains privacy information (such as name, ID number, etc.).

[0060] Step 120: For each target original field, if the privacy property of the target original field is in the first state, generate the first simulation field based on the target original field and the first target data information through the first data generation model; the first state indicates that the target original field includes privacy information.

[0061] In some embodiments, the first data generation model can be a large model. The first data generation model can be a model used to obtain simulation data based on raw fields including privacy information.

[0062] In some embodiments, if the target original field includes privacy information, a first simulation field can be generated based on the target original field and the first target data information through a first data generation model.

[0063] In actual implementation, the number of first simulation fields is at least one. The first simulation field can be data after removing, masking, or simulating privacy information.

[0064] In some embodiments, when the privacy state of the target field is in a first state, a simulation data synthesis method specifically designed for processing fields including privacy information can be matched to the target original field, and the simulation data synthesis method can be executed through a first data generation model to generate a first simulation field.

[0065] Step 130: When the privacy property of the target original field is in the second state, generate the second simulation field based on the target original field and the first target data information through the second data generation model; the second state means that the target original field does not include privacy information.

[0066] In some embodiments, the second data generation model can be an adversarial generative small model, and the training hyperparameter configuration of the second data generation model includes model category, number of iterations, batch size, learning strategy, etc. The second data generation model can be a model used to obtain simulation data based on raw fields that do not include privacy information.

[0067] In actual execution, the number of second simulation fields is at least one.

[0068] In some embodiments, when the privacy state of the target field is in the second state, a simulation data synthesis method specifically designed for processing fields that do not include privacy information can be matched to the target original field, and the simulation data synthesis method can be executed through the second data generation model to generate the second simulation field.

[0069] Step 140: Based on the first simulation field and the second simulation field, generate the first target data information of the target simulation data for the target structured original data.

[0070] In some embodiments, target simulation data for the target structured raw data can be constructed based on all first simulation fields and all second simulation fields.

[0071] In some embodiments, after generating the target simulation data, a quality assessment can be performed on the newly generated target simulation data. If the assessment result is satisfactory, the target simulation data is output. If the assessment result is unsatisfactory, the original target fields are parsed again.

[0072] According to the simulation data synthesis method of this application embodiment, by acquiring target structured raw data and first target data information of each target raw field in the target structured raw data, for each target raw field, when the privacy nature of the target raw field is in a first state, a first simulation field is generated based on the target raw field and the first target data information through a first data generation model; the first state indicates that the target raw field includes privacy information; when the privacy nature of the target raw field is in a second state, a second simulation field is generated based on the target raw field and the first target data information through a second data generation model; the second state indicates that the target raw field does not include privacy information; based on the first simulation field and the second simulation field, target simulation data for the target structured raw data is generated, so as to avoid the leakage of target raw fields including privacy information by means of the collaboration of the first data generation model and the second data generation model, the first data generation model processes the target raw field including privacy information, and the second data generation model processes the target raw field first target data information first target data information first target data information that does not include privacy information, thereby solving the problem that the privacy information of the raw data cannot be protected in the process of synthesizing simulation data.

[0073] In some embodiments, when the privacy property of the target original field is in a first state, the target original field is matched with a first simulation data synthesis method through the target large model; the first simulation data synthesis method is executed through the first data generation model to generate the first simulation field.

[0074] In some embodiments, the target large model can be a pre-trained model, which can be used to match simulation data synthesis methods for the target original field based on the privacy properties of the target original field.

[0075] In practice, the first simulation data synthesis method can be used to generate specific privacy simulation data based on original fields including privacy information. All privacy simulation data are newly generated data.

[0076] In actual implementation, the first simulation data synthesis method can be a discrete data synthesis method, an enumeration-type data synthesis method, or any other theoretically feasible data synthesis method.

[0077] In some embodiments, a first simulation data synthesis method can be executed based on a first data generation model to generate a first simulation field based on the target original field and the first target data information.

[0078] In some embodiments, the first target data information may include target parameters, which may include a preset generation quantity. The first simulation data synthesis method may be executed based on the first data generation model to obtain the preset generation quantity of first simulation fields.

[0079] According to the simulation data synthesis method of this application embodiment, by acquiring target structured raw data and first target data information for each target raw field in the target structured raw data, for each target raw field, when the privacy nature of the target raw field is in a first state, a first simulation data synthesis method is matched for the target raw field through a target large model; the first simulation data synthesis method is executed through a first data generation model corresponding to the first simulation data synthesis method to generate a first simulation field; the first state indicates that the target raw field includes privacy information; when the privacy nature of the target raw field is in a second state, a second simulation field is generated based on the target raw field and the first target data information through a second data generation model; the second state indicates that the target raw field does not include privacy information; based on the first simulation field and the second simulation field, target simulation data for the target structured raw data is generated, so as to avoid the leakage of target raw fields including privacy information by means of the collaboration of the first data generation model and the second data generation model, by means of the processing of target raw fields including privacy information by means of the first data generation model and the processing of target raw fields not including privacy information by means of the second data generation model, thereby solving the problem that it is impossible to protect the privacy information of the raw data during the process of synthesizing simulation data.

[0080] In some embodiments, a first simulation data synthesis method is executed based on a prompt word template pre-stored in a first data generation model to generate a first simulation field.

[0081] In some embodiments, the first target data information may include the field name, field description, data type, and target parameters of the target original field. The target parameters may include a preset generation quantity and field constraint rules. The field constraint rules include specific constraints such as additive constraints, multiplicative constraints, inclusion constraints, and causal constraints.

[0082] In actual implementation, the first data generation model has domain knowledge reserves and logical reasoning capabilities, including prompt word templates. The prompt word templates include sample restriction descriptions and sample format descriptions. The sample restriction descriptions are used to limit the value range of the generated target simulation data, and the sample format descriptions are used to provide format references for generating target simulation data.

[0083] In some embodiments, the first data generation model can execute the first simulation data synthesis method according to the preset generation quantity and field constraint rules in the target parameters, and synthesize the first simulation field based on the prompt word template pre-stored in the first data generation model, so as to generate a first simulation field with a quantity equal to the preset generation data and conforming to the field constraint rules.

[0084] According to the simulation data synthesis method of this application embodiment, by acquiring target structured raw data and first target data information for each target raw field in the target structured raw data, for each target raw field, when the privacy nature of the target raw field is in a first state, a first simulation data synthesis method is matched for the target raw field through a target large model; the first simulation data synthesis method is executed through a first data generation model corresponding to the first simulation data synthesis method to generate a first simulation field; the first state indicates that the target raw field includes privacy information; when the privacy nature of the target raw field is in a second state, a second simulation field is generated based on the target raw field and the first target data information through a second data generation model; the second state indicates that the target raw field does not include privacy information; based on the first simulation field and the second simulation field, target simulation data for the target structured raw data is generated, so as to avoid the leakage of target raw fields including privacy information by means of the collaboration of the first data generation model and the second data generation model, by means of the processing of target raw fields including privacy information by means of the processing of target raw fields not including privacy information by means of the second data generation model, thereby solving the problem that it is impossible to protect the privacy information of the raw data during the process of synthesizing simulation data.

[0085] In some embodiments, when the privacy property of the target original field is in the second state, the target original field is matched with a second simulation data synthesis method through the target large model; the second simulation data synthesis method is executed through the second data generation model to generate the second simulation field.

[0086] In practice, the second simulation data synthesis method can be used to generate simulation data based on raw fields that do not include privacy information.

[0087] In practice, the second simulation data synthesis method can be a discrete data synthesis method, an enumeration-based data synthesis method, or any other theoretically feasible data synthesis method.

[0088] In some embodiments, where the first target data information of the target original field indicates that the target original field does not include privacy information, the target large model matches the target original field with a second simulation data synthesis method.

[0089] In actual implementation, the second data generation model is the model used to execute the second simulation data synthesis method.

[0090] According to the simulation data synthesis method of this application embodiment, by acquiring target structured raw data and first target data information for each target raw field in the target structured raw data, for each target raw field, when the privacy nature of the target raw field is in a first state, a first simulation field is generated based on the target raw field and the first target data information through a first data generation model; the first state indicates that the target raw field includes privacy information; when the privacy nature of the target raw field is in a second state, a second simulation data synthesis method is matched for the target raw field through a target large model; the second simulation data synthesis method is executed through a second data generation model corresponding to the second simulation data synthesis method to generate a second simulation field; the second state indicates that the target raw field does not include privacy information; based on the first simulation field and the second simulation field, target simulation data for the target structured raw data is generated, so as to avoid the leakage of target raw fields including privacy information by means of the collaboration of the first data generation model and the second data generation model, by means of the processing of target raw fields including privacy information by means of the first data generation model and the processing of target raw fields not including privacy information by means of the second data generation model, thereby solving the problem that it is impossible to protect the privacy information of the raw data during the process of synthesizing simulation data.

[0091] In some embodiments, the target original field is input into the high-dimensional feature space of the second data generation model, and the feature distribution and association pattern are obtained through adversarial learning; based on the feature distribution and association pattern, the second simulation field is sampled and obtained.

[0092] In some embodiments, the second data generation model can be based on the second simulation data synthesis method, mapping the target original field to a continuous high-dimensional feature space, learning the feature distribution and association pattern of the target original field by adversarial learning against the discriminative model, and then sampling in the high-dimensional feature space and back-mapping it to the field space to achieve the generation of the second simulation field.

[0093] In some embodiments, the second data generation model can execute the second simulation data synthesis method according to the preset generation quantity and field constraint rules in the target parameters to synthesize the second simulation fields, so as to generate a second simulation field with a quantity equal to the preset generation data and conforming to the field constraint rules.

[0094] According to the simulation data synthesis method of this application embodiment, by acquiring target structured raw data and first target data information for each target raw field in the target structured raw data, for each target raw field, when the privacy nature of the target raw field is in a first state, a first simulation field is generated based on the target raw field and the first target data information through a first data generation model; the first state indicates that the target raw field includes privacy information; when the privacy nature of the target raw field is in a second state, a second simulation data synthesis method is matched for the target raw field through a target large model; the second simulation data synthesis method is executed through a second data generation model corresponding to the second simulation data synthesis method to generate a second simulation field; the second state indicates that the target raw field does not include privacy information; based on the first simulation field and the second simulation field, target simulation data for the target structured raw data is generated, so as to avoid the leakage of target raw fields including privacy information by means of the collaboration of the first data generation model and the second data generation model, by means of the processing of target raw fields including privacy information by means of the first data generation model and the processing of target raw fields not including privacy information by means of the second data generation model, thereby solving the problem that it is impossible to protect the privacy information of the raw data during the process of synthesizing simulation data.

[0095] In some embodiments, target structured raw data is obtained; the target structured raw data includes several raw fields; data parsing operations are performed on the target raw fields to obtain the raw field name and raw field data type; the target raw field is any one of the several raw fields; based on the raw field name and a preset privacy field rule base, the privacy nature of the target raw field is determined; the raw field name, raw field data type and privacy nature are determined as the first target data information of the target raw field.

[0096] In actual execution, the original field name can represent the meaning and function of the target original field (such as name, age, gender or ID number, etc.), and the original field data type can represent the value range of the target original field (for example, the original field data type of name can be character type, and the original field data type of gender can be male or female).

[0097] In actual implementation, the preset privacy field rule base includes several original field names and the privacy properties corresponding to each original field name. After performing data parsing operations on the target original field to obtain the original field name, the privacy properties corresponding to the original field name are searched in the privacy field rule base to determine the privacy properties of the target original field.

[0098] In some embodiments, target parameters for a target original field can be obtained, and the target parameters, the original field name, the original field data type, and the privacy properties can be determined as the first target data information of the target original field.

[0099] The target parameters may include the preset number of generated items and field constraint rules. The field constraint rules include specific constraints such as additive constraints, multiplicative constraints, inclusion constraints, and causal constraints.

[0100] According to the simulation data synthesis method of this application embodiment, the following steps are taken: First, target structured raw data is acquired. The target structured raw data includes several raw fields. Data parsing is performed on the target raw fields to obtain the raw field names and raw field data types. The target raw field is any one of the several raw fields. Based on the raw field names, the privacy properties of the target raw fields are determined. The raw field names, raw field data types, and privacy properties are determined as the first target data information of the target raw fields. If the privacy properties of the target raw fields in the first target data information are in a first state (i.e., the target raw fields include privacy information), the target raw fields are matched using a target large model. The first simulation data synthesis method involves: obtaining a first simulation field by executing a first simulation data synthesis method through a first data generation model; when the privacy property of the target original field is in a second state; matching the target original field with a second simulation data synthesis method through a target large model; and generating the second simulation field by executing the second simulation data synthesis method through the second data generation model. Based on the first and second simulation fields, target simulation data is generated to ensure that the first simulation field generated by the first data generation model is entirely newly generated data, thereby avoiding the leakage of target original fields including privacy information and solving the problem of not being able to protect the privacy information of original data during the synthesis of simulation data. Furthermore, the second data generation model is used to learn the characteristics and implicit constraint patterns of other types of fields to generate a second simulation field consistent with the distribution of the target structured original data, ensuring the consistency of the distribution of the second simulation field with the target structured original data.

[0101] In some embodiments, after generating target simulation data for the target structured original data based on the first simulation field and the second simulation field, the evaluation value of the target simulation data under the target evaluation index can be obtained; the target evaluation index includes at least one of data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and original data leakage rate; based on the weight and evaluation value corresponding to the target evaluation index, the overall quality evaluation score of the target simulation data is obtained; based on the overall quality evaluation score, it is determined whether the target simulation data is qualified.

[0102] In some embodiments, after generating target simulation data based on the first simulation field and the second simulation field, the evaluation value of each target evaluation index among the target evaluation indices such as data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and original data leakage rate can be obtained.

[0103] In some embodiments, the target original field and the target simulation data can be compared based on dimensions such as data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and original data leakage rate, so as to obtain the evaluation value of the target simulation data for each target evaluation index among the target evaluation indicators such as data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and original data leakage rate.

[0104] In actual implementation, the weight of each target evaluation indicator among the target evaluation indicators such as data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and raw data leakage rate is a set value.

[0105] In some embodiments, the weights corresponding to each target evaluation index can be adjusted according to the needs of actual applications to meet the quality preference requirements of generating target simulation data in different scenarios.

[0106] In some embodiments, the overall quality assessment score of the target simulation data can be obtained based on the following formula:

[0107] G=G1*W1+G2*W2+G3*W3+G4*W4+G5*W5;

[0108] Wherein, G represents the overall quality assessment score of the target simulation data; G1 represents the assessment value of the target simulation data on the data validity index; W1 represents the weight corresponding to the data validity index; G2 represents the assessment value of the target simulation data on the data structure standardization index; W2 represents the weight corresponding to the data structure standardization index; G3 represents the assessment value of the target simulation data on the numerical distribution consistency index; W3 represents the weight corresponding to the numerical distribution consistency index; G4 represents the assessment value of the target simulation data on the category distribution consistency index; W4 represents the weight corresponding to the category distribution consistency index; G5 represents the assessment value of the target simulation data on the original data leakage rate index; W5 represents the weight corresponding to the original data leakage rate index.

[0109] In some embodiments, the target simulation data can be determined to be qualified if the total comprehensive quality assessment score is greater than or equal to the target quality score. For example, the target simulation data is determined to be qualified if the total comprehensive quality assessment score is greater than or equal to 60. Conversely, the target simulation data is determined to be unqualified if the total comprehensive quality assessment score is less than the target quality score.

[0110] According to the simulation data synthesis method of this application embodiment, by acquiring first target data information for a target original field, and when the privacy nature of the target original field in the first target data information is in a first state, that is, when the target original field includes privacy information, a first simulation data synthesis method is matched for the target original field through a target large model; a first data generation model corresponding to the first simulation data synthesis method is determined as the target data generation model; and target simulation data is generated based on the target original field and the first target data information through the target data generation model, so as to ensure that when the target data generation model is the first data generation model, the target simulation data generated by the target data generation model is all newly generated data, thereby avoiding the leakage of the target original field including privacy information, and solving the problem of not being able to protect the privacy information of the original data during the process of synthesizing simulation data.

[0111] In some embodiments, after determining whether the target simulation data is qualified based on the overall quality assessment score, if the target simulation data is unqualified, the target original field is re-parsed to obtain second target data information; the second target data information includes updated privacy properties; based on the updated privacy properties, a target data generation model is determined from the first data generation model and the second data generation model, and the target simulation data is regenerated based on the target data generation model; if the number of data parsing operations on the target original field is greater than or equal to the target threshold, manual intervention is triggered.

[0112] In some embodiments, if the target simulation data is unqualified, the target original fields are re-parsed, and the simulation data synthesis method and the target data generation model can be re-matched based on the second target data information obtained from the re-parsed operation.

[0113] In some embodiments, after obtaining the second target data information, a target data generation model can be determined from the first data generation model and the second data generation model based on the updated privacy properties in the second target data information. For example, if the updated privacy properties of the target original field are in a first state, the first data generation model is determined as the target data generation model; if the updated privacy properties of the target original field are in a second state, the second data generation model is determined as the target data generation model.

[0114] In some embodiments, when the target data generation model is a first data generation model, a first simulation data synthesis method is executed through the first data generation model to generate a first simulation field.

[0115] In some embodiments, when the target data generation model is a second data generation model, a second simulation data synthesis method is executed through the second data generation model to generate a second simulation field.

[0116] In actual implementation, the target threshold can be a set value.

[0117] In some embodiments, manual intervention is triggered if the number of data parsing operations on the target original field is greater than or equal to a target threshold. For example, if the target threshold is 5, if the number of data parsing operations on the target original field is 5, the data parsing operations on the target original field are stopped, and manual intervention is triggered.

[0118] In some embodiments, manual intervention may include operations such as manually parsing the privacy properties of the target's original fields.

[0119] According to the simulation data synthesis method of this application embodiment, by acquiring first target data information for a target original field, and when the privacy nature of the target original field in the first target data information is in a first state, that is, when the target original field includes privacy information, a first simulation data synthesis method is matched for the target original field through a target large model; a first data generation model corresponding to the first simulation data synthesis method is determined as the target data generation model; and target simulation data is generated based on the target original field and the first target data information through the target data generation model, so as to ensure that when the target data generation model is the first data generation model, the target simulation data generated by the target data generation model is all newly generated data, thereby avoiding the leakage of the target original field including privacy information, and solving the problem of not being able to protect the privacy information of the original data during the process of synthesizing simulation data.

[0120] To better understand the simulation data synthesis method provided in the embodiments of this application, further explanation is provided below. It should be understood that the following discussion is merely exemplary.

[0121] This application provides a method for synthesizing simulation data, the specific steps of which are as follows: Figure 2 As shown:

[0122] Step 210: Obtain the target structured raw data; the target structured raw data includes several raw fields; perform data parsing operations on the target raw fields to obtain the raw field names and raw field data types; the target raw field is any one of the several raw fields; based on the raw field names and the preset privacy field rule base, determine the privacy nature of the target raw field; determine the raw field names, raw field data types, and privacy nature as the first target data information of the target raw field.

[0123] In actual execution, the original field name can represent the meaning and function of the target original field (such as name, age, gender or ID number, etc.), and the original field data type can represent the value range of the target original field (for example, the original field data type of name can be character type, and the original field data type of gender can be male or female).

[0124] In practice, the target structured raw data can be data from any industry, such as government public data (which may include census data, employment analysis data, newborn population data, etc.), medical industry data, or financial industry data—any theoretically feasible data. The target structured raw data can also be data in any format.

[0125] In some embodiments, before parsing the target original fields, the target structured original data can be preprocessed, and then the target original fields in the preprocessed target structured original data can be parsed. Data preprocessing may include mean filtering, Gaussian filtering, or any theoretically feasible data preprocessing method. Data preprocessing may also include steps such as data format validation, null value imputation, meaningless field removal, and field value normalization.

[0126] In some embodiments, data parsing operations can be performed on the target original field to obtain first target data information such as the privacy nature of the target original field. The privacy nature indicates whether the target original field contains privacy information (such as name, ID number, etc.).

[0127] In practice, the pre-defined privacy field rule base can include several original field names and the privacy properties corresponding to each original field name. After performing data parsing operations on the target original field to obtain the original field name, the privacy properties corresponding to the original field name are searched in the privacy field rule base to determine the privacy properties of the target original field.

[0128] In actual implementation, the preset privacy field rule base can also include predefined privacy information detection rules, such as names and ID cards being considered privacy information. If the original field name of the target original field conforms to the privacy information detection rules, the privacy nature of the target original field is determined to be in the first state; if the original field name of the target original field does not conform to the privacy information detection rules, the privacy nature of the target original field is determined to be in the second state.

[0129] In some embodiments, target parameters for a target original field can be obtained, and the target parameters, the original field name, the original field data type, and the privacy properties can be determined as the first target data information of the target original field.

[0130] The target parameters may include the preset number of generated items and field constraint rules. The field constraint rules include specific constraints such as additive constraints, multiplicative constraints, inclusion constraints, and causal constraints.

[0131] Step 220: When the privacy property of the target original field is in the first state, match the first simulation data synthesis method for the target original field through the target large model; the first state means that the target original field includes privacy information.

[0132] In some embodiments, the target large model can be a pre-trained model, which can be used to match simulation data synthesis methods for the target original field based on the privacy properties of the target original field.

[0133] In practice, the first simulation data synthesis method can be used to generate specific privacy simulation data based on original fields including privacy information. All privacy simulation data are newly generated data.

[0134] In actual implementation, the first simulation data synthesis method can be a discrete data synthesis method, an enumeration-type data synthesis method, or any other theoretically feasible data synthesis method.

[0135] Step 230: Using the first data generation model, execute the first simulation data synthesis method to generate the first simulation field.

[0136] In some embodiments, the first target data information may include the field name, field description, data type, and target parameters of the target original field. The target parameters may include a preset generation quantity and field constraint rules. The field constraint rules include specific constraints such as additive constraints, multiplicative constraints, inclusion constraints, and causal constraints.

[0137] In actual implementation, the first data generation model has domain knowledge reserves and logical reasoning capabilities, including prompt word templates. The prompt word templates include sample restriction descriptions and sample format descriptions. The sample restriction descriptions are used to limit the value range of the generated target simulation data, and the sample format descriptions are used to provide format references for generating target simulation data.

[0138] In some embodiments, the first data generation model can execute the first simulation data synthesis method according to the preset generation quantity and field constraint rules in the target parameters, and synthesize the first simulation field based on the prompt word template pre-stored in the first data generation model, so as to generate a first simulation field with a quantity equal to the preset generation data and conforming to the field constraint rules.

[0139] Step 240: When the privacy property of the target original field is in the second state, match the second simulation data synthesis method to the target original field through the target large model; the second state means that the target original field does not include privacy information.

[0140] In practice, the second simulation data synthesis method can be used to generate simulation data based on raw fields that do not include privacy information.

[0141] In practice, the second simulation data synthesis method can be a discrete data synthesis method, an enumeration-based data synthesis method, or any other theoretically feasible data synthesis method.

[0142] In some embodiments, where the first target data information of the target original field indicates that the target original field does not include privacy information, the target large model matches the target original field with a second simulation data synthesis method.

[0143] Step 250: Using the second data generation model, execute the second simulation data synthesis method to generate the second simulation field.

[0144] In actual implementation, the second data generation model is the model used to execute the second simulation data synthesis method.

[0145] In some embodiments, the second data generation model can be an adversarial generative small model, and the training hyperparameter configuration of the second data generation model includes model category, number of iterations, batch size, learning strategy, etc. The second data generation model can be a model used to execute the second simulation data synthesis method.

[0146] In some embodiments, the second data generation model takes the target original field and the first target data information as inputs to the second simulation data synthesis method, executes the second simulation data synthesis method, and generates the second simulation field.

[0147] In some embodiments, the second data generation model can be based on the second simulation data synthesis method to map the target original field to a continuous high-dimensional feature space, learn the feature distribution and association pattern of the target original field by adversarial with the discriminative model, and back-map the sampled data in the high-dimensional feature space to the field space to generate the target simulation data.

[0148] In some embodiments, the second data generation model can execute the second simulation data synthesis method according to the preset generation quantity and field constraint rules in the target parameters to synthesize the second simulation fields, so as to generate a second simulation field with a quantity equal to the preset generation data and conforming to the field constraint rules.

[0149] Step 260: Based on the first simulation field and the second simulation field, generate target simulation data for the target structured original data.

[0150] In some embodiments, target simulation data for the target structured raw data can be constructed based on all first simulation fields and all second simulation fields.

[0151] Step 270: Obtain the evaluation value of the target simulation data under the target evaluation indicators; the target evaluation indicators include at least one of data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and original data leakage rate; based on the weights and evaluation values ​​corresponding to the target evaluation indicators, obtain the overall quality evaluation score of the target simulation data; based on the overall quality evaluation score, determine whether the target simulation data is qualified.

[0152] In some embodiments, after generating target simulation data through the target data generation model, the evaluation value of each target evaluation index among the target evaluation indices such as data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and original data leakage rate can be obtained.

[0153] In some embodiments, the target original field and the target simulation data can be compared based on dimensions such as data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and original data leakage rate, so as to obtain the evaluation value of the target simulation data for each target evaluation index among the target evaluation indicators such as data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and original data leakage rate.

[0154] In actual implementation, the weight of each target evaluation indicator among the target evaluation indicators such as data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and raw data leakage rate is a set value.

[0155] In some embodiments, the weights corresponding to each target evaluation index can be adjusted according to the needs of actual applications to meet the quality preference requirements of generating target simulation data in different scenarios.

[0156] In some embodiments, the overall quality assessment score of the target simulation data can be obtained based on the following formula:

[0157] G=G1*W1+G2*W2+G3*W3+G4*W4+G5*W5;

[0158] Wherein, G represents the overall quality assessment score of the target simulation data; G1 represents the assessment value of the target simulation data on the data validity index; W1 represents the weight corresponding to the data validity index; G2 represents the assessment value of the target simulation data on the data structure standardization index; W2 represents the weight corresponding to the data structure standardization index; G3 represents the assessment value of the target simulation data on the numerical distribution consistency index; W3 represents the weight corresponding to the numerical distribution consistency index; G4 represents the assessment value of the target simulation data on the category distribution consistency index; W4 represents the weight corresponding to the category distribution consistency index; G5 represents the assessment value of the target simulation data on the original data leakage rate index; W5 represents the weight corresponding to the original data leakage rate index.

[0159] In some embodiments, the target simulation data can be determined to be qualified if the total comprehensive quality assessment score is greater than or equal to the target quality score. For example, the target simulation data is determined to be qualified if the total comprehensive quality assessment score is greater than or equal to 60. Conversely, the target simulation data is determined to be unqualified if the total comprehensive quality assessment score is less than the target quality score.

[0160] In some embodiments, after determining whether the target simulation data is qualified based on the overall quality assessment score, if the target simulation data is unqualified, the target original field is re-parsed to obtain second target data information; the second target data information includes updated privacy properties; based on the updated privacy properties, a target data generation model is determined from the first data generation model and the second data generation model, and the target simulation data is regenerated based on the target data generation model; if the number of data parsing operations on the target original field is greater than or equal to the target threshold, manual intervention is triggered.

[0161] In some embodiments, if the target simulation data is unqualified, the target original fields are re-parsed, and the simulation data synthesis method and the target data generation model can be re-matched based on the second target data information obtained from the re-parsed operation.

[0162] In some embodiments, after obtaining the second target data information, a target data generation model can be determined from the first data generation model and the second data generation model based on the updated privacy properties in the second target data information. For example, if the updated privacy properties of the target original field are in a first state, the first data generation model is determined as the target data generation model; if the updated privacy properties of the target original field are in a second state, the second data generation model is determined as the target data generation model.

[0163] In some embodiments, when the target data generation model is a first data generation model, a first simulation data synthesis method is executed through the first data generation model to generate a first simulation field.

[0164] In some embodiments, when the target data generation model is a second data generation model, a second simulation data synthesis method is executed through the second data generation model to generate a second simulation field.

[0165] In actual implementation, the target threshold can be a set value.

[0166] In some embodiments, manual intervention is triggered if the number of data parsing operations on the target original field is greater than or equal to a target threshold. For example, if the target threshold is 5, if the number of data parsing operations on the target original field is 5, the data parsing operations on the target original field are stopped, and manual intervention is triggered.

[0167] In some embodiments, manual intervention may include operations such as manually parsing the privacy properties of the target's original fields.

[0168] This application also provides a simulation data synthesis device.

[0169] like Figure 3 As shown, the simulation data synthesis device 300 includes: a first acquisition module 310, a first generation module 320, a second generation module 330, and a third generation module 340.

[0170] The first acquisition module 310 is used to acquire the target structured raw data and the first target data information of each target raw field in the target structured raw data; the first target data information includes the privacy properties of the target raw fields;

[0171] The first generation module 320 is used to generate a first simulation field for each target original field, provided that the privacy property of the target original field is in a first state, by using a first data generation model based on the target original field and the first target data information; the first state indicates that the target original field includes privacy information;

[0172] The second generation module 330 is used to generate a second simulation field based on the target original field and the first target data information through a second data generation model when the privacy property of the target original field is in a second state; the second state indicates that the target original field does not include privacy information.

[0173] The third generation module 340 is used to generate target simulation data for the target structured original data based on the first simulation field and the second simulation field.

[0174] According to the simulation data synthesis apparatus of this application, by acquiring target structured raw data and first target data information of each target raw field in the target structured raw data, for each target raw field, when the privacy nature of the target raw field is in a first state, a first simulation field is generated based on the target raw field and the first target data information through a first data generation model; the first state indicates that the target raw field includes privacy information; when the privacy nature of the target raw field is in a second state, a second simulation field is generated based on the target raw field and the first target data information through a second data generation model; the second state indicates that the target raw field does not include privacy information; based on the first simulation field and the second simulation field, target simulation data for the target structured raw data is generated, so as to avoid the leakage of target raw fields including privacy information through the collaboration of the first data generation model and the second data generation model, the first data generation model processes the target raw field including privacy information, and the second data generation model processes the target raw field first target data information ...

[0175] In some embodiments, the first generation module 320 includes:

[0176] The first matching unit is used to match the first simulation data synthesis method to the target original field through the target large model when the privacy property of the target original field is in the first state.

[0177] The first generation unit is used to generate the first simulation field by executing the first simulation data synthesis method through the first data generation model.

[0178] In some embodiments, the first generation unit is used to execute a first simulation data synthesis method based on a prompt word template pre-stored in a first data generation model to generate a first simulation field.

[0179] In some embodiments, the second generation module 330 includes:

[0180] The second matching unit is used to match the second simulation data synthesis method for the target original field through the target large model when the privacy property of the target original field is in the second state.

[0181] The second generation unit is used to execute the second simulation data synthesis method through the second data generation model to generate the second simulation field.

[0182] In some embodiments, the second generation unit is used to input the target original field into the high-dimensional feature space of the second data generation model, obtain the feature distribution and association pattern through adversarial learning, and sample to obtain the second simulation field based on the feature distribution and association pattern.

[0183] In some embodiments, the first acquisition module 310 includes:

[0184] The first acquisition unit is used to acquire the target structured raw data; the target structured raw data includes several raw fields;

[0185] The second acquisition unit is used to perform data parsing operations on the target original field to obtain the original field name and the original field data type; the target original field can be any one of several original fields;

[0186] The first determining unit is used to determine the privacy nature of the target original field based on the original field name and a preset privacy field rule base;

[0187] The second determining unit is used to determine the original field name, original field data type, and privacy nature as the first target data information of the target original field.

[0188] In some embodiments, the simulation data synthesis apparatus 300 further includes:

[0189] The second acquisition module is used to acquire the evaluation value of the target simulation data under the target evaluation index; the target evaluation index includes at least one of data validity, data structure standardization, numerical distribution consistency, category distribution consistency, and original data leakage rate.

[0190] The third acquisition module is used to obtain the overall quality assessment score of the target simulation data based on the weights and assessment values ​​corresponding to the target assessment indicators.

[0191] The first determination module is used to determine whether the target simulation data is qualified based on the overall quality assessment score.

[0192] In some embodiments, the simulation data synthesis apparatus 300 further includes:

[0193] The operation module is used to re-parse the original target fields to obtain second target data information when the target simulation data is unqualified; the second target data information includes updated privacy properties.

[0194] The second determining module is used to determine the target data generation model from the first data generation model and the second data generation model based on the updated privacy properties, and to regenerate the target simulation data based on the target data generation model.

[0195] The trigger module is used to trigger manual intervention when the number of data parsing operations on the target original field is greater than or equal to the target threshold.

[0196] The simulation data synthesis device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0197] The simulation data synthesis device in this application embodiment can be a device with an operating system. This operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit it.

[0198] The simulation data synthesis device 300 provided in this embodiment can achieve... Figures 1 to 2 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0199] In some embodiments, such as Figure 4As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, it implements the various processes of the above-described simulation data synthesis method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0200] It should be noted that the computer equipment in this application embodiment includes the mobile electronic equipment and non-mobile electronic equipment described above.

[0201] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described simulation data synthesis method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0202] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0203] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described simulation data synthesis method.

[0204] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0205] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described simulation data synthesis method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0206] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0207] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0208] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0209] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0210] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0211] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method of synthetic data simulation, characterized by, The method comprises: obtaining target structured original data and first target data information of each target original field in the target structured original data; the first target data information comprises a privacy property of the target original field; for each target original field, if the privacy property of the target original field is in a first state, a first simulation data synthesis method is matched for the target original field by a target large model; the first state indicates that the target original field comprises privacy information; based on a prompt word template pre-stored in a first data generation model, the first simulation data synthesis method is executed to generate a first simulation field; if the privacy property of the target original field is in a second state, a second simulation field is generated based on the target original field and the first target data information by a second data generation model; the second state indicates that the target original field does not comprise privacy information; based on the first simulation field and the second simulation field, target simulation data for the target structured original data is generated.

2. The method of claim 1, wherein, The method comprises: if the privacy property of the target original field is in the second state, a second simulation data synthesis method is matched for the target original field by the target large model; the second simulation data synthesis method is executed by the second data generation model to generate the second simulation field.

3. The method of claim 2, wherein, The method comprises: the target original field is input into a high-dimensional feature space of the second data generation model, and a feature distribution and a correlation pattern are obtained through adversarial learning; based on the feature distribution and the correlation pattern, a second simulation field is sampled and obtained.

4. The method of claim 1, wherein, The method comprises: target structured original data is obtained; the target structured original data comprises a plurality of original fields; data analysis is performed on a target original field to obtain an original field name and an original field data type; the target original field is any one of the plurality of original fields; based on the original field name and a preset privacy field rule library, a privacy property of the target original field is determined; the original field name, the original field data type and the privacy property are determined as first target data information of the target original field.

5. The synthetic method of emulating data according to any one of claims 1-4, wherein, After the target simulation data is generated based on the first simulation field and the second simulation field, the method comprises: an evaluation value of the target simulation data under a target evaluation index is obtained; the target evaluation index comprises at least one of data validity, data structure specification, numerical value distribution consistency, category distribution consistency and original data leakage rate. obtaining a comprehensive quality evaluation total score of the target simulation data based on the weight corresponding to the target evaluation index and the evaluation value; determining whether the target simulation data is qualified based on the comprehensive quality evaluation total score.

6. The method of claim 5, wherein, After determining whether the target simulation data is qualified based on the comprehensive quality evaluation total score, the method comprises: In the case that the target simulation data is unqualified, re-performing data parsing operation on the target original field to obtain second target data information; the second target data information comprises updated privacy properties; determining a target data generation model from the first data generation model and the second data generation model based on the updated privacy properties, and regenerating target simulation data based on the target data generation model; In the case that the number of operations of performing data parsing operation on the target original field is greater than or equal to a target threshold, triggering manual intervention.

7. An artificial data synthesizing apparatus characterized by comprising: Comprise: a first obtaining module configured to obtain target structured original data and first target data information of each target original field in the target structured original data; the target structured original data is original data, and the first target data information comprises privacy properties of the target original field; a first matching unit configured to, in the case that the privacy properties of the target original field are in a first state, match a first simulation data synthesis method for the target original field by a target large model; the first state indicates that the target original field comprises privacy information; a first generating unit configured to generate a first simulation field by executing the first simulation data synthesis method based on a prompt word template pre-stored in a first data generation model; a second generating module configured to, in the case that the privacy properties of the target original field are in a second state, generate a second simulation field based on the target original field and the first target data information by a second data generation model; the second state indicates that the target original field does not comprise privacy information; a third generating module configured to generate target simulation data first target data information for the target structured original data based on the first simulation field and the second simulation field.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the simulation data synthesis method of any one of claims 1-6 when executing the program.

Images