A table data generation method based on pre-training and artificial inductive bias

Abstract

The present application relates to the technical field of data processing, and particularly relates to a table data generation method based on pre-training and artificial inductive bias. The method comprises the following steps: obtaining original table data to obtain a real data set; constructing a deep generation model for table data generation; training the deep generation model by using the real data set, and generating a synthetic data set by using the trained deep generation model; constructing a new model copy which has the same structure as the deep generation model, training the new model copy by using the synthetic data set until convergence is achieved, and extracting all weight parameters of the trained new model copy as artificial inductive bias; constructing a target model which has the same structure as the deep generation model, setting initial weights of the target model as the artificial inductive bias, and training and fine-tuning the target model by using the real data set. The method of the present application improves the diversity and distribution authenticity of the synthetic data.

Description

Technical Field

[0001] This invention relates to the fields of data processing and machine learning technology, and in particular to a method for generating tabular data based on pre-training and manual inductive bias. Background Technology

[0002] In numerous scientific fields such as environmental science, particle physics, and biomedicine, tabular data, as a fundamental form of organized information, is of great significance for building predictive models and conducting scientific exploration. In these application scenarios, predicting label columns based on existing feature columns is one of the core tasks. However, in many critical scenarios, obtaining large-scale, high-quality labeled data faces significant challenges, necessitating the use of data generation technologies to create more value from limited samples.

[0003] Currently, deep generative models, such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), have been widely used for the synthesis and augmentation of tabular data. Taking GANs as an example, their typical implementation involves a discriminator receiving samples from both the real and generated data distributions, and using cross-entropy loss to drive the generator to fit the real distribution. The specific training process usually includes two stages: first, training the GAN to Nash equilibrium using the source dataset, at which point the discriminator cannot distinguish between real and fake samples; then, directly injecting all the synthesized samples output by the generator into the training set to train the downstream model. On the other hand, classic transfer learning methods typically rely on large-scale datasets of the same modality, achieving knowledge transfer by loading pre-trained weights and fine-tuning only specific layers. Its core operations include: loading pre-trained model weights from a large dataset, freezing the basic feature extraction part, replacing the output layer with one that matches the target domain, and training only the newly added layers.

[0004] However, deep generative models heavily rely on large amounts of training data when synthesizing tabular data. When the number of real samples is limited (e.g., only a few hundred samples), the generator tends to replicate existing sample patterns rather than explore uncovered feature spaces, leading to decreased generation quality and problems such as overfitting and distribution bias. For transfer learning methods, it is generally required that the pre-training data and the target task be of the same modality (e.g., image-to-image), and transfer is achieved only through fine-tuning via frozen layers, making it difficult to provide effective structural priors for tabular data generation tasks. These shortcomings make it difficult for existing techniques to effectively capture the complex relationships between features when dealing with small sample tabular data, limiting the diversity and practicality of the generated data. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a method for generating tabular data based on pre-training and manual inductive bias, so as to improve the diversity and distribution accuracy of synthetic data under limited real data conditions.

[0006] To achieve the above objectives, this invention provides a method for generating tabular data based on pre-training and manual inductive bias, comprising the following steps: Obtain the original table data and preprocess it to obtain the real dataset; Build a deep generative model for generating tabular data; First training phase: Train the deep generative model using the real dataset, and use the trained deep generative model to generate a synthetic dataset; Second training phase: Construct a new model copy with the same structure as the deep generative model, train the new model copy using the synthetic dataset until convergence, and extract all weight parameters of the trained new model copy as manual inductive bias; Construct a target model with the same structure as the deep generative model, set the initial weights of the target model to the artificial inductive bias, and train and fine-tune the target model using the real dataset to obtain the final tabular data generation model.

[0007] Furthermore, the preprocessing includes: normalizing continuous features in the original table data, performing one-hot encoding on categorical features, and filling in missing values ​​using the K-nearest neighbor imputation method.

[0008] Furthermore, the deep generative model adopts either the CTGAN architecture or the VAE-BGM architecture.

[0009] Furthermore, the network structure of the deep generative model is fixed at 7 layers, with the first 6 layers using the LeakyReLU activation function and the last layer using the Sigmoid activation function.

[0010] Furthermore, the network structure of the deep generative model is fixed at 7 layers, with the first 6 layers using the LeakyReLU activation function and the last layer using the Sigmoid activation function.

[0011] Furthermore, when training and fine-tuning the target model using the real dataset, the following loss function is used for optimization: ; in, Samples of the real dataset input into the encoder; Let be the latent variables of the hidden encoding, and be the posterior distribution given by the encoder. Obtained by sampling from the middle; The reconstruction loss term is given by the encoder as the posterior distribution. Below, log-likelihood Expected value; For regularization (KL divergence); Distribution of latent variables; It is the prior distribution; The data likelihood given by the decoder; This is the Kullback-Leibler divergence, used to measure the difference between two distributions.

[0012] The optimization process employs the Adam optimizer, enables adaptive gradient clipping to avoid parameter space divergence, and the learning rate decays exponentially in each training round.

[0013] Furthermore, the method also includes: At each preset training epoch, a discriminative neural network is used to calculate the JS divergence between the generated data and the real data, and the training parameters are adjusted or a synthetic dataset is regenerated based on the fluctuation of the JS divergence.

[0014] Furthermore, the preset training rounds are 200 rounds; when the JS divergence fluctuation exceeds 15% in three consecutive verifications, the regularization coefficient is increased by 0.2 times or a synthetic dataset is regenerated.

[0015] Furthermore, the method also includes: When the standard deviation of the reconstruction loss of the target model exceeds 30% of its mean, the Adam optimizer is switched to RMSProp; when the JS divergence between the generated data and the real data exceeds 0.5, Gaussian noise is injected into the synthetic data and the synthetic dataset is regenerated.

[0016] To achieve the above objectives, this application also provides an electronic device, comprising: processor; A memory having stored thereon one or more computer program instructions that run on the processor; When the processor executes the computer instructions, it performs the tabular data generation method based on pre-training and manual inductive bias as described above.

[0017] To achieve the above objectives, this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, perform the steps of the tabular data generation method based on pre-training and manual inductive bias as described above.

[0018] The tabular data generation method based on pre-training and manual inductive bias provided by this invention significantly improves the generalization ability and generation quality of the tabular data generation model in data-scarce scenarios through a two-stage, dual-model coupling mechanism of "synthetic data pre-training - weight transfer - real data fine-tuning".

[0019] The tabular data generation method based on pre-training and manual inductive bias provided by this invention avoids the distribution shift problem caused by directly stacking and synthesizing samples in traditional data augmentation methods by transferring the manual inductive bias in the form of complete model weights.

[0020] The tabular data generation method based on pre-training and manual inductive bias provided by this invention enhances the training stability and robustness of the generated model by introducing similarity verification and anomaly handling mechanisms (calculating JS divergence, dynamically adjusting regularization coefficients, switching optimizers, injecting noise, etc.).

[0021] The tabular data generation method based on pre-training and manual inductive bias provided by this invention improves the model's compatibility and feature extraction efficiency for different types of tabular data through standardized preprocessing such as continuous feature normalization, one-hot encoding of categorical features, and K-nearest neighbor imputation for missing values. Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating the tabular data generation method based on pre-training and manual inductive bias according to Embodiment 1 of the invention. Figure 2 This is a schematic diagram comparing convergence performance. Detailed Implementation

[0023] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0024] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the invention. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the invention.

[0025] The term "comprising" and its variations as used in this invention are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0026] It should be noted that the concepts of "first" and "second" may be mentioned in this invention only to distinguish different devices, components or parts, and are not used to limit the order of the functions performed by these devices, components or parts or their interdependence.

[0027] It should be noted that the terms "one" and "multiple" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless explicitly stated otherwise in the context, they should be understood as "one or more". "Multiple" should be understood as two or more.

[0028] Example 1 This embodiment provides a method for generating tabular data based on pre-training and manual inductive bias, the method comprising: Step S101: Obtain the original table data and preprocess the original table data to obtain the real dataset; Retrieve the original table data, denoted as x. r Its dimension is the number of samples N multiplied by the number of features C. To ensure the compatibility and stability of model training, the original data is preprocessed as follows: For continuous features, the min-max normalization method is used to map their values ​​to the interval [-1, 1].

[0029] For categorical features, one-hot encoding is performed to expand each category into a binary feature vector.

[0030] For samples with missing values, the K-nearest neighbor imputation method is used to complete them. Specifically, the number of neighboring points is set to K=5. For the sample to be imputed, the Euclidean distance between it and all other samples in the complete feature space is calculated. The 5 closest samples are selected, and the mean (for continuous features) or mode (for categorical features) of the corresponding features of these samples are used as imputation values.

[0031] Step S102: Construct a deep generative model for generating tabular data; This embodiment constructs a deep generative model based on VAE-BGM as the initial model. This model consists of two parts: an encoder and a decoder, both of which are fully connected neural networks. The encoder receives preprocessed tabular data as input and maps it to the mean and log-variance of the latent variable z through multiple hidden layers; the decoder then reconstructs the original data from the latent variable z.

[0032] The encoder and decoder have the following network structure: six hidden layers, each containing 256 neurons, with ReLU activation. The output layer processes features according to their type: continuous features are activated linearly, while categorical features are activated using Softmax. The entire model uses variational lower bound (ELBO) as the optimization objective.

[0033] Step S103: First training phase: Train the deep generative model using the real dataset, and use the trained deep generative model to generate a synthetic dataset; Using the preprocessed original real dataset x r Train the VAE-BGM model constructed above. The training run consists of 1000 epochs, with the batch size determined based on the data size (typically 256). After training, the model should be able to generate synthetic data with a distribution similar to the original data. Use this model to generate at least 10,000 synthetic samples, denoted as the synthetic dataset x. g .

[0034] Step S104: Second training phase: Construct a new model copy with the same structure as the deep generative model, train the new model copy using the synthetic dataset until convergence, and extract all weight parameters of the trained new model copy as manual inductive bias; Construct a new model copy with the exact same structure as the initial model. Then, apply this copy model to the synthetic dataset x. g Training was conducted on the model for 500 epochs with a learning rate of 0.001 and a batch size of 256. Training terminated when the loss function value reached a stable state, i.e., the fluctuation of the loss value was less than 5% over 10 consecutive training epochs. After training, all weight parameters of the new model copy were extracted and denoted as the transfer weights θ. pt This weighting parameter serves as the "artificial inductive bias" described in this invention.

[0035] Step S105: Construct a target model with the same structure as the deep generative model, set the initial weights of the target model to the artificial inductive bias, and train and fine-tune the target model using the real dataset to obtain the final tabular data generation model.

[0036] Construct another target model with the same structure as the aforementioned model. The initial weights of this target model are not randomly initialized, but rather loaded with the transfer weights θ obtained in step S3. pt Subsequently, only the original, limited-size real dataset x was used. r The target model is then fine-tuned. This fine-tuning process is repeated 500 times. The final tabular data generation model is obtained after fine-tuning.

[0037] It should be noted that the following loss function is used for optimization during training: ; in, Samples of the real dataset input into the encoder; Let be the latent variables of the hidden encoding, and be the posterior distribution given by the encoder. Obtained by sampling from the middle; The reconstruction loss term is given by the encoder as the posterior distribution. Below, log-likelihood Expected value; For regularization (KL divergence); Distribution of latent variables; It is the prior distribution; The data likelihood given by the decoder; This is the Kullback-Leibler divergence, used to measure the difference between two distributions.

[0038] The optimization process employs the Adam optimizer with an initial learning rate of 0.001 and enables adaptive gradient clipping with a threshold of 1.0. The learning rate decays exponentially in each round at a rate of 0.01. The regularization coefficients are implicitly included in the KL term and do not require additional configuration.

[0039] In this embodiment, during the fine-tuning of the target model, a distribution similarity verification is performed every 200 training epochs. The specific verification method is as follows: Construct a three-layer discriminative neural network to evaluate the distributional differences between generated data and real data.

[0040] Input real sample x r And the samples H generated by the current target model g Calculate the JS divergence value. The formula for calculating JS divergence is: JS divergence is calculated by dividing the actual data distribution P... r With the generated data distribution P g Synthesize an intermediate distribution M (M equals the mean of both), and then output the average of the two KL divergences (KL divergence measures the distribution deviation). The target JS value should be close to the baseline value.

[0041] The target JS value should be as small as possible, ideally close to 0. If the JS divergence remains consistently high or fluctuates drastically, an adjustment mechanism will be triggered.

[0042] If the JS value fluctuates by more than 15% in three consecutive validations, the regularization coefficient is increased by 0.2 times or the composite set x is regenerated. g .

[0043] In this embodiment of the application, when abnormal parameters or performance degradation occur during training, the following adjustment strategy is automatically executed: If the standard deviation of the reconstruction loss of the target model exceeds 30% of its mean, the optimizer is switched to RMSProp with the momentum coefficient set to 0.9.

[0044] If the JS divergence value is abnormally high (e.g., exceeding 0.5), Gaussian noise of 0.05 standard deviations is injected into the current synthetic data, and then x is regenerated using the noisy synthetic data. g And continue training.

[0045] Example 2 This embodiment provides a method for generating tabular data based on pre-training and manual inductive bias. Based on Embodiment 1, this embodiment provides a detailed implementation of another deep generative model—CTGAN, and gives preferred training parameters and verification strategies.

[0046] This embodiment uses CTGAN as the basic generative model. CTGAN includes a generator G and a discriminator D. The generator receives random noise and a conditional vector, and outputs synthesized tabular data; the discriminator receives real data or generated data and outputs its true / false probabilities. To handle mixed-type features, CTGAN uses pattern normalization for continuous features and conditional generation for discrete features.

[0047] In this embodiment, the network depth of both the generator and discriminator is fixed at 7 layers. The first 6 layers use the LeakyReLU activation function with a negative slope coefficient of 0.2; the last layer (output layer) uses the Sigmoid activation function to constrain the output range to [0,1]. The number of neurons in each layer is adaptively determined based on the input feature dimension, typically set to 256. This 7-layer structure has been experimentally verified to achieve an optimal balance between generation quality and computational efficiency.

[0048] In this embodiment, during the first training phase, the number of training rounds is 1000; the number of generated synthetic samples is no less than 10,000 (specifically, it can be set to N×30 based on the original sample size N, but at least 10,000). During the second training phase, the number of training rounds is 500; the learning rate is 0.001; the batch size is 256; and the termination condition is that the loss function fluctuates by less than 5% for 10 consecutive rounds.

[0049] In this embodiment, the total loss function consists of the generator loss and the discriminator loss, with an L2 regularization term added, and the regularization coefficient λ = 0.01. The optimizer uses Adam, with an adaptive gradient pruning threshold of 1.0. The learning rate decays exponentially at a rate of 0.01.

[0050] In this embodiment, the JS divergence is calculated every 200 rounds. The discriminant neural network is a three-layer fully connected structure with 128 neurons per layer, and the output layer uses Sigmoid activation. The baseline value for JS divergence is set to 0.1. If the JS divergence exceeds 0.25 (i.e., fluctuation exceeds 15%) in three consecutive validations, an adjustment is performed.

[0051] When the standard deviation of the reconstruction loss exceeds 30% of the mean, in addition to switching to the RMSProp optimizer, the learning rate can be reduced to 0.5 times the original value. When the JS divergence exceeds 0.5, Gaussian noise with a standard deviation of 0.05 is injected.

[0052] To verify the technical effects of the present invention, such as Figure 2 As shown, a convergence performance comparison experiment was conducted with the basic linear model in the HSKC sparse data scenario (N=300). The method in this application (i.e., the propsed method in the figure) reduced the MSE (mean squared error) of the downstream regression task from 0.516 to 0.451. The results show that the synthetic data generated by this method effectively expands the dataset, resulting in smaller errors when predicting the target variable, and significantly improving its practical value.

[0053] Example 3 This embodiment provides a method for generating tabular data based on pre-training and manual inductive bias, combined with a specific example (the HSKC dataset of house sales in King County, Washington, USA). Figure 1 The flowchart shown below fully describes the working process of this invention: Dataset Description: The HSKC dataset contains 21,613 housing sales records from May 2014 to May 2015, with 20 features (including numerical and categorical features such as house area, number of bedrooms, number of bathrooms, floor level, view rating, building area, and neighborhood location). The target variable is the house price. To simulate a "scarce data" scenario, this example randomly selects N=300 records from the original dataset as the real training dataset x. r .

[0054] Data preprocessing (S1). Min-max normalization, one-hot encoding, and K-nearest neighbor imputation (K=5) are performed on 300 samples. After processing, a normalized real dataset with a dimension of 300× (after feature expansion) is obtained.

[0055] Model Construction (S2). This example uses the VAE-BGM architecture, with both the encoder and decoder being fully connected networks with 6 hidden layers and 256 neurons per layer. The activation function used is ReLU. The batch size is set to 256, the initial learning rate is 0.001, and the optimizer is Adam.

[0056] Two-stage training: Phase 1: Train the initial VAE-BGM model for 1000 rounds using 300 real-world datasets. After training, generate 10,000 synthetic datasets using this model. g ; Phase Two: Construct a new VAE-BGM model copy with the same structure, using only the generated 10,000 synthetic data points. g The replica model was trained for 500 epochs (learning rate 0.001, batch size 256). After training, all weight parameters θ of the replica model were extracted. pt As a form of artificial inductive bias.

[0057] Fine-tuning the target model. Construct a third VAE-BGM model (the target model) and set its initial weights to θ. pt Then, the target model was fine-tuned and trained for 500 epochs using only the original 300 real data points.

[0058] During training, the JS divergence between the generated data and the real data is calculated every 200 rounds. If the JS divergence fluctuates by more than 15% or exceeds 0.5, an anomaly response mechanism is triggered (such as switching optimizers or injecting noise).

[0059] After fine-tuning, the final tabular data generation model is obtained. This model can be used to generate any amount of synthetic home sales data to expand the training set or balance the data distribution.

[0060] Example 4 In this embodiment, an electronic device is also provided, including a processor and a memory. The memory is used to store non-transitory computer-readable instructions. The processor is used to execute the non-transitory computer-readable instructions, which, when executed by the processor, can perform one or more steps of the tabular data generation method based on pre-training and manual inductive bias described above. The memory and the processor can be interconnected via a bus system and / or other forms of connection mechanisms.

[0061] For example, the processor can be a central processing unit (CPU), a digital signal processor (DSP), or other processing units with data processing and / or program execution capabilities, such as a field-programmable gate array (FPGA); for example, the central processing unit (CPU) can be an x86 or ARM architecture.

[0062] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor may run one or more computer program modules to implement various functions of the electronic device. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0063] It should be noted that, in the embodiments of this application, the specific functions and technical effects of the electronic devices can be referred to the description of the collaborative filtering recommendation method based on spectral graph neural networks above, and will not be repeated here.

[0064] Example 5 In this embodiment, a computer-readable storage medium is also provided for storing non-transitory computer-readable instructions. For example, when the non-transitory computer-readable instructions are executed by a computer, one or more steps in the tabular data generation method based on pre-training and human inductive bias described above can be performed.

[0065] For example, this storage medium can be used in the aforementioned electronic device. For example, the storage medium can be the memory in the electronic device of Embodiment 2. For example, relevant descriptions of the storage medium can be found in the corresponding description of the memory in the electronic device of Embodiment 2, and will not be repeated here.

[0066] It should be noted that the storage medium (computer-readable medium) mentioned above in this application may be a computer-readable signal medium or a non-transitory computer-readable storage medium, or any combination of the two. A non-transitory computer-readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a non-transitory computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0067] In this application, a non-transitory computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a non-transitory computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0068] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0069] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.

[0070] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0071] The units described in the embodiments of this application can be implemented in software or hardware. The names of the units are not, in some cases, limiting the scope of the unit itself.

[0072] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), etc.

[0073] The above description is merely a partial embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0074] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in sequential order. Multitasking and parallel processing may be advantageous in certain environments. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this application. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0075] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for generating tabular data based on pre-training and manual inductive bias, characterized in that, Includes the following steps: Obtain the original table data and preprocess it to obtain the real dataset; Build a deep generative model for generating tabular data; First training phase: Train the deep generative model using the real dataset, and use the trained deep generative model to generate a synthetic dataset; Second training phase: Construct a new model copy with the same structure as the deep generative model, train the new model copy using the synthetic dataset until convergence, and extract all weight parameters of the trained new model copy as manual inductive bias; Construct a target model with the same structure as the deep generative model, set the initial weights of the target model to the artificial inductive bias, and train and fine-tune the target model using the real dataset to obtain the final tabular data generation model.

2. The tabular data generation method based on pre-training and manual inductive bias according to claim 1, characterized in that, The preprocessing includes: normalizing continuous features in the original table data, performing one-hot encoding on categorical features, and using K-nearest neighbor imputation to fill in missing values.

3. The tabular data generation method based on pre-training and manual inductive bias according to claim 1, characterized in that, The deep generative model adopts either the CTGAN architecture or the VAE-BGM architecture.

4. The tabular data generation method based on pre-training and manual inductive bias according to claim 1, characterized in that, The deep generative model has a fixed network structure of 7 layers, with the first 6 layers using the LeakyReLU activation function and the last layer using the Sigmoid activation function.

5. The tabular data generation method based on pre-training and manual inductive bias according to claim 1, characterized in that, When training and fine-tuning the target model using the real dataset, the following loss function is used for optimization: ； in, Samples of the real dataset input into the encoder; Let be the latent variables of the hidden encoding, and be the posterior distribution given by the encoder. Obtained by sampling from the middle; The reconstruction loss term is given by the encoder as the posterior distribution. Below, log-likelihood Expected value; For regularization (KL divergence); Distribution of latent variables; It is the prior distribution; The data likelihood given by the decoder; The divergence is Kullback-Leibler.

6. The tabular data generation method based on pre-training and manual inductive bias according to claim 1, characterized in that, The method further includes: At each preset training epoch, a discriminative neural network is used to calculate the JS divergence between the generated data and the real data, and the training parameters are adjusted or a synthetic dataset is regenerated based on the fluctuation of the JS divergence.

7. The tabular data generation method based on pre-training and manual inductive bias according to claim 6, characterized in that, The preset training rounds are 200 rounds; when the JS divergence fluctuation exceeds 15% in three consecutive validations, the regularization coefficient is increased by 0.2 times or a synthetic dataset is regenerated.

8. The tabular data generation method based on pre-training and manual inductive bias according to claim 7, characterized in that, The method further includes: When the standard deviation of the reconstruction loss of the target model exceeds 30% of its mean, the Adam optimizer is switched to RMSProp; when the JS divergence between the generated data and the real data exceeds 0.5, Gaussian noise is injected into the synthetic data and the synthetic dataset is regenerated.

9. An electronic device, characterized in that, include: processor; A memory having stored thereon one or more computer program instructions that run on the processor; When the processor executes the computer program instructions, it performs the tabular data generation method based on pre-training and manual inductive bias as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, It stores computer instructions, which, when executed by the processor, perform the steps of the tabular data generation method based on pre-training and manual inductive bias as described in any one of claims 1-8.

Images