Two-stage structured data generation method combining language model and generative model

Abstract

The invention provides a two-stage structured data generation method combining a language model and a generative model, which comprises the following steps: introducing a concept of information entropy to classify all features of original data, then generating low-entropy synthetic data through the generative model in a first stage, and finely adjusting the language model by using the original data in a second stage to generate low-entropy synthetic data; and inputting the low-entropy synthesis data generated in the first stage into the fine-tuned language model, so that the language model can better generate high-entropy synthesis data. Namely, the method combines the advantages of the language model and the generative model, and improves the effect of structured data generation by using the characteristic that the language model can naturally learn and use the context information. The method is suitable for a structured data set of any feature condition, the generated data set can replace an original data set to serve as training data and is applied to other downstream tasks, and downstream model training difficulty caused by the problems of data safety and privacy protection and too small sample number is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of data generation methods, and particularly relates to a two-stage structured data generation method combining a language model with other generation models. Background Art

[0002] The application of structured data in modern information technology is crucial. It not only greatly improves the efficiency and accuracy of data processing, but also greatly facilitates data exchange and integration between different systems. In many important fields, such as finance, healthcare, and industrial production, structured data plays a crucial role. In addition, a large amount of training data is the basis for artificial intelligence-related technologies, which in turn enriches and improves the application scope and efficiency of downstream tasks. However, in actual applications, the situation of existing data is not satisfactory. In some scenarios, the scarcity and extreme distribution imbalance of samples make it impossible to directly use complex high-performance models. In addition, the increasing emphasis on data security and privacy protection further restricts the availability of raw data. To solve this problem, data generation can be adopted. Combining the actual application requirements, the original data set is first expanded, and then subsequent work is carried out.

[0003] In the early days, simple connection functions and Bayesian networks were used for data generation. However, due to the limitations of statistical models in terms of model structure and computational performance, the quality of the generated data was low, and large-scale complex data sets could not be processed. With the gradual development of artificial intelligence technology, deep learning models have shown powerful performance in various tasks, and technologies such as Variational Auto-Encoders (VAE), Generative Adversarial Networks (GAN), and Denoise Diffusion Probabilistic Model (DDPM) have also improved the effect of structured data generation tasks.

[0004] Among them, VAE converts the preprocessed tabular data into distribution parameters in the latent space through an encoder, then samples a latent variable z from this distribution, and then maps z back to the data space through a decoder to generate new tabular data samples, thereby realizing the generation of tabular data. This process combines the reconstruction error and KL divergence as the loss function to ensure that the generated data is both close to the original data distribution and diverse.

[0005] GAN generates tabular data through the cooperation and competition of two neural network modules, the Generator and the Discriminator: the Generator creates data samples from random noise, while the Discriminator tries to distinguish real data from generated data; through continuous iterative optimization, the Generator gradually learns to generate realistic tabular data, ultimately achieving the goal of generating new samples that are indistinguishable from real data.

[0006] Diffusion models generate new tabular data samples by gradually adding noise to blur the data and then gradually removing the noise through an inverse process to restore the data; in this inverse process, the model learns how to gradually restore the data to its original state, thereby achieving the generation of new data similar to real data.

[0007] However, considering that structured data usually contains the relationships between features and a large amount of semantic information, classical deep learning models cannot learn the above information well. In recent years, the emerging large language models (LLMs) have achieved remarkable results in the field of natural language processing and can also play a role in solving the feature associations and semantic information processing in structured data.

[0008] The first to apply the language model to the structured data generation task was the GReaT model proposed by Vadim Borisov et al. First, each feature of each sample was connected by the "is" character into a "subject-predicate-object" text structure to encode the structured data as text data. Subsequently, all the generated "subject-predicate-object" phrases were shuffled, and on this basis, the language model was fine-tuned. Finally, the fine-tuned language model was used for data generation. After that, many works have been improved using different encoding methods and fine-tuning methods, such as TabuLa, TabSAL, etc. However, the structured data generation method based on the language model also has its drawbacks, that is, when dealing with discrete feature data with fewer feature value categories, the generated results are prone to being too concentrated. On the contrary, traditional deep learning models perform better when dealing with such data. Summary of the Invention

[0009] This invention is made to solve the above problems, and its purpose is to provide a data generation method that combines the advantages of two types of models to further improve the data generation effect. This invention adopts the following technical solutions:

[0010] The present invention provides a two-stage structured data generation method combining a language model and a generative model, which has the following technical features and includes the following steps: Step S1, calculate the information entropy of the features contained in each piece of original data, and divide each piece of the original data into high-entropy data or low-entropy data based on the information entropy; Step S2, based on the low-entropy data, generate corresponding low-entropy synthetic data through the generative model; Step S3, based on the original data and the low-entropy synthetic data, generate corresponding high-entropy synthetic data through the language model, so as to obtain the final synthetic data.

[0011] The two-stage structured data generation method combining a language model and a generative model provided by the present invention may also have the following technical features, wherein in Step S1, for a structured original data set D composed of n pieces of the original data and having m features, all of the features are [X 1 , X 2 , …, X m , where the j-th feature of the i-th piece of the original data is denoted as x ij , and the information entropy of the k-th feature in the data set D is:

[0012]

[0013] Based on the magnitude of the total information entropy of all the features of each piece of the original data, divide this piece of the original data into the high-entropy data or the low-entropy data:

[0014]

[0015] In the formula, threshold is a threshold for dividing high-entropy data and low-entropy data, which is set according to the characteristics of the data set.

[0016] The two-stage structured data generation method combining a language model and a generative model provided by the present invention may also have the following technical features, wherein Step S2 includes the following sub-steps: Step S2-1, input all the feature columns of multiple pieces of the low-entropy data into the generative model, and the generative model converts the discrete data therein into one-hot vectors and normalizes the continuous data therein into combined vectors composed of one-hot vectors and scalars; Step S2-2, combine all the one-hot vectors and combined vectors to obtain a large vector, and use the large vector as real data to train the generative model to obtain the trained generative model; Step S2-3, generate low-entropy feature columns with similar statistical characteristics to the input feature columns through the trained generative model; Step S2-4, decode and denormalize the generated low-entropy feature columns so that they are restored to the same format as the features in the original data, thereby obtaining the low-entropy synthetic data.

[0017] The two-stage structured data generation method combining a language model and a generative model provided by the present invention may also have the following technical features. In step S2-1, for each column of the continuous data C i , the number of modes m is estimated using a variational Gaussian mixture model (VGM) i , and the Gaussian mixture model is fitted:

[0018]

[0019] In the formula, μ k is the weight of mode k, η k is the mean of k, and φ k is the standard deviation of mode k. According to the calculated probability density, random sampling is performed from the modes of each value. For the selected mode k, c i,j is represented as a mode vector with the k-th position being 1 and a normalized scalar. The mode vector is represented as:

[0020] β i,j =[0,...,1,...,0]

[0021] The normalized scalar is represented as:

[0022]

[0023] In step S2-2, the large vector is represented as:

[0024]

[0025] In the formula, represents the concatenation operation of vectors. α i,j and β i,j are respectively the normalized scalar and one-hot vector of the continuous data column C i , and d i,j is the one-hot representation of the discrete data column D i .

[0026] The two-stage structured data generation method combining a language model and a generative model provided by the present invention may also have the following technical features. Among them, step S3 includes the following sub-steps: Step S3-1, encoding all feature columns of the original data to obtain an encoded data set; Step S3-2, fine-tuning the language model with the encoded data set; Step S3-3, encoding the low-entropy synthetic data to obtain encoded low-entropy synthetic data; Step S3-4, using the encoded low-entropy synthetic data as the above input to fine-tune the language model, and generating encoded high-entropy synthetic data through the language model, so as to obtain encoded synthetic data; Step S3-5, decoding the encoded synthetic data into the tabular form of the original data, thereby obtaining the final synthetic data.

[0027] The two-stage structured data generation method combining a language model and a generative model provided by the present invention may also have the following technical features. Among them, in step S3-1, all feature columns of the original data are encoded into the form of feature name + feature value, thereby obtaining the encoded data set. In step S3-3, all feature columns of the low-entropy synthetic data are encoded into the form of feature name + feature value, thereby obtaining the encoded low-entropy synthetic data.

[0028] The two-stage structured data generation method combining a language model and a generative model provided by the present invention may also have the following technical features. Among them, in step S3-2, during the inference process of the autoregressive language model q, the output distribution is expressed as:

[0029] z = q(ω 1 ,…,ω k-1 )

[0030] In the formula, ω 1 ,…,ω k-1 represents the token sequence of the input text. The probability that the language model outputs the token ω at each position k according to the sequence of the input text is:

[0031]

[0032] In the formula, T is the sampling temperature that affects the creativity of the language model generation.

[0033] The two-stage structured data generation method combining a language model and a generative model provided by the present invention may also have the following technical features. Among them, the generative model is any one of TVAE, CTGAN, and TabDDPM, and the language model is DistilGPT2.

[0034] Functions and effects of the invention

[0035] According to the two-stage structured data generation method that combines a language model and a generative model provided by the present invention, the concept of information entropy is introduced to classify all features of the original data. Then, in the first stage, a generative model is used to generate low-entropy synthetic data. In the second stage, the original data is used to fine-tune the language model, and the low-entropy synthetic data generated in the first stage is used as the above input to the fine-tuned language model to help the language model understand the potential relationship between low-entropy data and high-entropy data, so that the language model can better generate high-entropy synthetic data. That is to say, this method combines the advantages of the language model and the conventional generative model, and utilizes the characteristic that the language model can naturally learn and use context information to improve the effect of structured data generation. The data generation method of this embodiment is applicable to structured data sets under any feature conditions, and the generated data set can replace the original data set as training data and be applied to other downstream tasks, avoiding the difficulty of downstream model training caused by data security and privacy protection issues and the problem of too few samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a schematic diagram of data processing in the two-stage structured data generation method that combines a language model and a generative model in an embodiment of the present invention;

[0037] Figure 2 is a flowchart of the two-stage structured data generation method that combines a language model and a generative model in an embodiment of the present invention;

[0038] Figure 3 is a flowchart of step S2 in an embodiment of the present invention;

[0039] Figure 4 is a flowchart of step S3 in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] In order to make the technical means, creative features, achieved purposes and effects of the present invention easy to understand, the two-stage structured data generation method that combines a language model and a generative model of the present invention will be specifically described below in conjunction with embodiments and drawings.

[0041] Figure 1 is a schematic diagram of data processing in the two-stage structured data generation method that combines a language model and a generative model in this embodiment, Figure 2 is a flowchart of the two-stage structured data generation method that combines a language model and a generative model in this embodiment.

[0042] As Figure 1 and Figure 2 shown, the method of this embodiment includes the following steps:

[0043] Step S1, calculate the information entropy of the features contained in each piece of original data, and divide each piece of original data into high-entropy data or low-entropy data based on the information entropy.

[0044] Step S2 (the first stage), based on the low-entropy data, generate corresponding low-entropy synthetic data through a generative model.

[0045] Step S3 (the second stage), based on the original data and the low-entropy synthetic data, generate corresponding high-entropy synthetic data through a language model, so as to obtain the final synthetic data.

[0046] The following will elaborate on each of the above steps in detail.

[0047] Step S1, calculate the information entropy of the features contained in each piece of original data, and divide each piece of original data into high-entropy data or low-entropy data based on the information entropy.

[0048] Among them, the original data can be the corresponding real data, such as real financial data. Specifically, given a structured original dataset D ( Figure 1 Original Data in it) composed of n pieces of original data, all features are respectively denoted as [X 1 , X 2 , …, X m , and the j-th feature of the i-th data is denoted as x ij . The information entropy of the k-th feature in the dataset is:

[0049]

[0050] For each piece of original data, based on the total information entropy of all features of this piece of original data, divide this piece of original data into "high-entropy data" or "low-entropy data":

[0051]

[0052] Among them, according to the characteristics of different datasets, the threshold threshold used to distinguish high-entropy data and low-entropy data is correspondingly different. As Figure 2 shown, for example, divide the feature columns of features X 1 , X 3 into low-entropy data, and divide the feature columns of features X 2 , X 4 into high-entropy data.

[0053] Step S2 (the first stage), based on the low-entropy data, generate corresponding low-entropy synthetic data through a generative model.

[0054] Among them, the generation model can adopt conventional generation models in the prior art, such as TVAE, CTGAN, TabDDPM, etc. In this embodiment, CTGAN is taken as an example for illustration. CTGAN learns the distribution of data through the GAN framework. In GAN, the generator will try to generate samples similar to the real data distribution, and the discriminator is responsible for distinguishing the generated data from the real data.

[0055] Figure 3 It is the flowchart of step S2 in this embodiment.

[0056] As Figure 3 shown, step S2 specifically includes the following sub-steps:

[0057] Step S2-1: Input all feature columns of multiple low-entropy data into the generation model. The generation model converts the discrete data among them into one-hot vectors, and normalizes the continuous data into combined vectors composed of one-hot vectors and scalars.

[0058] Specifically, for each continuous data column C i , use the variational Gaussian mixture model (VGM) to estimate the number of modes m i , and fit the Gaussian mixture model:

[0059]

[0060] In the formula, μ k is the weight of mode k, η k is the mean of k, and φ k is the standard deviation of mode k.

[0061] According to the calculated probability density, randomly sample from the modes of each value. Assume that mode k is selected, then c i,j is represented as a mode vector:

[0062] β i,j =[0,...,1,...,0]

[0063] That is, a one-hot vector with 1 at the k-th position and 0 at other positions.

[0064] And a normalized scalar:

[0065]

[0066] After the processing of step S2-1, whether the data in a row is discrete data or continuous data can be represented by vectors.

[0067] Step S2-2: Combine all one-hot vectors and combined vectors to obtain a "large vector", and use this "large vector" as the real data in the generative adversarial network to train the generative model, obtaining a trained generative model. The "large vector" is expressed as:

[0068]

[0069] In the formula, represents the concatenation operation of vectors, α i,j and β i,j are respectively the normalization scalar and one-hot vector of the continuous data column C i , and d i,j is the one-hot representation of the discrete data column D i .

[0070] Step S2-3: Generate a low-entropy feature column with similar statistical features to the input feature column through the trained generative model.

[0071] Step S2-4: Decode and denormalize the generated low-entropy feature column to restore it to the same format as the features in the original data, thereby obtaining low-entropy synthetic data.

[0072] Step S3 (the second stage): Based on the original data and the low-entropy synthetic data, generate corresponding high-entropy synthetic data through a language model, thereby obtaining the final synthetic data.

[0073] Intuitively, high-entropy data mostly consists of continuous variables or discrete variables with a relatively uniform distribution and a large number of categories. In this embodiment, a language model (also known as a large language model, LLM) is used to generate this part of the data. In this embodiment, the language model DistilGPT2 is adopted.

[0074] Figure 4 is the flowchart of step S3 in this embodiment.

[0075] As Figure 4 shown, step S3 specifically includes the following sub-steps:

[0076] Step S3-1: Encode all feature columns of the real data set D (i.e., the original data) into the form of "feature name + feature value" to obtain the encoded data set D 1 ,

[0077] Step S3-2: Fine-tune and train the language model through this encoded data set.

[0078] Among them, during the inference process of the autoregressive language model q, the output distribution can be expressed as:

[0079] z = q(ω 1 , …, ω k-1)

[0080] In the formula, ω 1 , …, ω k-1 represents the token sequence of the input text.

[0081] The language model outputs the probability that the token is ω at each position k according to the sequence of the input text as:

[0082]

[0083] In the formula, T is the sampling temperature, which affects the creativity of the generation of the language model. The higher the temperature, the more innovative the generated content.

[0084] Step S3-3: Perform the same encoding operation on the low-entropy synthetic data generated in the first stage, that is, encode all feature columns of the low-entropy synthetic data into the form of "feature name + feature value" to obtain the encoded low-entropy synthetic data.

[0085] Step S3-4: Use the encoded low-entropy synthetic data as the input above to fine-tune the language model, and generate high-entropy synthetic data in encoded form through the language model, so as to obtain synthetic data in encoded form including low-entropy synthetic data and high-entropy synthetic data, that is, data in the form of "feature name + feature value".

[0086] Among them, using the encoded low-entropy synthetic data as the input above can help the language model understand the potential relationship between low-entropy data and high-entropy data, and serve as the basis for generating high-entropy synthetic data, so as to organically combine the advantages of common generation models and language models and generate high-entropy synthetic data that more conforms to the target distribution.

[0087] Step S3-5: Decode the synthetic data in encoded form (data in the form of "feature name + feature value") back into tabular form so that it is restored to the same format as the features in the original data, thereby obtaining the final synthetic data (that is Figure 1 the Synthetic Data in

[0088] Functions and effects of the embodiment

[0089] According to the two-stage structured data generation method combining the language model and the generative model provided in this embodiment, the concept of information entropy is introduced to classify all the features of the original data, and then, in the first stage, low-entropy synthetic data is generated by the generative model, and in the second stage, the language model is fine-tuned with the original data, and the low-entropy synthetic data generated in the first stage is used as the input of the fine-tuned language model above to help the language model understand the potential association between low-entropy data and high-entropy data, so that the language model can better generate high-entropy synthetic data. That is, this method combines the advantages of the language model and the conventional generative model, and utilizes the characteristics of the language model that can naturally learn and use contextual information, thereby improving the effect of structured data generation. The data generation method of this embodiment is applicable to structured data sets of any characteristic conditions, and the generated data set can replace the original data set as training data and be applied to other downstream tasks, avoiding the difficulty of downstream model training caused by data security and privacy protection issues and the problem of too few samples.

[0090] In an embodiment, according to the characteristics of different data sets, the thresholds for distinguishing high entropy data from low entropy data can be set to different values ​​accordingly to better adapt to specific data sets and improve the generation efficiency and effect of the two types of data.

[0091] Furthermore, in the first stage, all data are encoded into vector representations, and the generated low-entropy synthetic data are subsequently decoded and denormalized to obtain a format consistent with the original features; in the second stage, all features are encoded into the form of "feature name + feature value", and the generated synthetic data are subsequently decoded to restore it to a tabular form. Therefore, there are no specific requirements for the data format of the original data, etc., and it can be applied to a variety of different data sets and has a wide range of applications.

[0092] <Comparative Example>

[0093] This comparative example provides two data generation methods in the prior art for comparison with the data generation method in the embodiment.

[0094] The two existing methods are data generation methods based on CTGAN and GReaT, respectively. These two models are used as baseline models to evaluate the two-stage model proposed in the embodiment.

[0095] In this comparative example, the adult dataset was used to train two baseline models and the two-stage model of the embodiment respectively. Five synthetic datasets equal in size to the original training set were sampled using these three models. Then, these synthetic datasets were used to train a random forest model for testing, predicting the values of the income feature column in the adult dataset, and evaluating the performance of each model through the original test set. The corresponding average Accuracy, ROC AUC value, and F1-score results were calculated as evaluation metrics. Table 1 below shows the test data of the three models.

[0096] Table 1 Machine learning efficiency test data table of three models

[0097]

[0098] As shown in the data in Table 1, the machine learning efficiency of the two-stage model of the embodiment exceeded that of the two baseline models in all three evaluation metrics, indicating that the synthetic data generated by the two-stage model of the embodiment was more in line with the data characteristics of the original dataset.

[0099] In addition, in this comparative example, the privacy protection capabilities of the data generation methods were also evaluated by calculating two evaluation metrics, the closest record distance (DCR) and the nearest neighbor distance ratio (NNDR), to ensure that the method of the embodiment would not disclose private information in the original data. Table 2 below shows the test data of the three models.

[0100] Table 2 Privacy protection test data table of three models

[0101] Metric Ours GReaT CTGAN DCR 0.642479903 0.607771264 0.545424987 NNDR 0.657366435 0.703837889 0.614687327

[0102] As shown in Table 2, taking DCR as the evaluation metric, the privacy protection performance of the two-stage model of the embodiment was the best among the three models; taking NNDR as the evaluation metric, the privacy protection performance of the two-stage model of the embodiment was the second best among the three models, indicating that the two-stage model and method of the embodiment had strong privacy protection capabilities.

[0103] Generally speaking, the data generation method of the embodiment was more superior than the traditional tabular data generation method CTGAN and the tabular data generation method GReaT based on language models.

[0104] The above embodiments are only used to illustrate the specific implementation manners of the present invention, and the present invention is not limited to the description scope of the above embodiments. Those skilled in the art should understand that the present invention is not restricted by the above embodiments. What is described in the above embodiments and the specification only illustrates the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A two-stage structured data generation method combining a language model and a generative model, characterized in that: The following steps are involved: Step S1, calculating the information entropy of the features contained in each piece of original data, and classifying each piece of the original data into high entropy data or low entropy data based on the information entropy; Step S2, based on the low entropy data, generating corresponding low entropy synthetic data through a generation model; Step S3, based on the original data and the low-entropy synthesized data, generating corresponding high-entropy synthesized data through a language model, so as to obtain final synthesized data.

2. The two-stage structured data generation method combining a language model and a generative model according to claim 1, characterized in that: in, In step S1, for a structured original data set D consisting of n pieces of original data and having m features, all of the features are [X1, X2, …, X m ], where the jth feature of the original data of the ith item is recorded as x ij , the information entropy of the kth feature in the data set D is: Based on the total information entropy of all the features of each piece of the original data, the piece of the original data is divided into the high entropy data or the low entropy data: In the formula, threshold is the threshold used to divide high entropy data from low entropy data, which is set according to the characteristics of the data set.

3. The two-stage structured data generation method combining a language model and a generative model according to claim 2, Features: Wherein, step S2 includes the following sub-steps: Step S2-1, inputting all feature columns of the plurality of low entropy data into the generative model, wherein the generative model converts the discrete data therein into a one-hot vector, and normalizes the continuous data therein into a combined vector composed of the one-hot vector and a scalar; Step S2-2, combining all the one-hot vectors and the combination vectors to obtain a large vector, and using the large vector as real data to train the generative model to obtain the trained generative model; Step S2-3, generating a low entropy feature column having similar statistical characteristics to the input feature column through the trained generation model; Step S2-4, decoding and denormalizing the generated low entropy feature column to restore it to a format consistent with the features in the original data, thereby obtaining the low entropy synthetic data.

4. The two-stage structured data generation method combining a language model and a generative model according to claim 3, characterized in that: in, In step S2-1, for each column of the continuous data, C i , use the variational Gaussian mixture model (VGM) to estimate the number of modes m i , and fit a Gaussian mixture model: In the formula, μ k is the weight of mode k, η k is the mean of k, φ k is the standard deviation of mode k, According to the calculated probability density, randomly sample from the mode of each value, and for the selected mode k, c i,j It is represented by a mode vector with the kth position being 1 and a normalized scalar, and the mode vector is represented by: β i,j ＝[0,...,1,...,0] The normalized scalar is expressed as: In step S2-2, the large vector is represented as: In the formula, represents the concatenation operation of vectors, α i,j and β i,j They are continuous data columns C i The normalized scalar and one-hot vector of , d i,j is a discrete data column D i The unique heat expression.

5. The two-stage structured data generation method combining a language model and a generative model according to claim 1, Features: Wherein, step S3 includes the following sub-steps: Step S3-1, encoding all feature columns of the original data to obtain an encoded data set; Step S3-2, fine-tuning the language model using the encoded data set; Step S3-3, encoding the low entropy synthesized data to obtain encoded low entropy synthesized data; Step S3-4, using the encoded low-entropy synthetic data as the input of the fine-tuned language model, generating encoded high-entropy synthetic data through the language model, thereby obtaining encoded synthetic data; Step S3-5, decoding the encoded synthesized data into the tabular form of the original data, and obtaining the final synthesized data.

6. The two-stage structured data generation method combining a language model and a generative model according to claim 5, characterized in that: in, In step S3-1, all feature columns of the original data are encoded into the form of feature name + feature value, thereby obtaining the encoded data set. In step S3-3, all feature columns of the low-entropy synthetic data are encoded into the form of feature name+feature value, thereby obtaining the encoded low-entropy synthetic data.

7. The two-stage structured data generation method combining a language model and a generative model according to claim 6, characterized in that: in, In step S3-2, during the inference process of the autoregressive language model q, the output distribution is expressed as: z=q(ω1,…,ω k-1 ) Where ω1,…,ω k-1 a sequence of tokens representing the input text, The language model outputs the probability of ω as a token at each position k according to the sequence of input text: Where T is the sampling temperature that affects the creativity of language model generation.

8. The two-stage structured data generation method combining a language model and a generative model according to claim 1, characterized in that: in, The generation model is any one of TVAE, CTGAN, and TabDDPM. The language model is DistilGPT2.

Images