Sample data generation method and device, equipment and storage medium

Abstract

The invention discloses a sample data generation method and device, equipment and a storage medium, and relates to the technical field of data processing, and the sample data generation method comprises the steps: obtaining original financial data, carrying out the logic constraint extraction of the original financial data, and obtaining financial data logic constraint information; generating initial financial sample data based on the financial data logic constraint information; determining logic consistency and sample authenticity of the initial financial sample data; and determining target financial sample data according to the logic consistency and the sample authenticity. The target financial sample data is determined according to the logic consistency and the sample authenticity of the initial financial sample data. Compared with an existing mode of generating the sample data through the generative adversarial network technology, the mode of the application can improve the logic consistency and authenticity of the generated sample data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to sample data generation methods, apparatus, devices and storage media. Background Technology

[0002] In fields such as financial risk control, credit scoring, fraud detection, and customer risk assessment, structured financial data is the core foundation for training risk identification models. Structured financial data typically contains multidimensional heterogeneous features, such as basic user information, transaction records, credit limits, behavioral characteristics, and device characteristics. These features are subject to complex logical constraints and business rules (such as "credit limit ≤ total assets", "repayment amount ≤ loan amount", "credit rating ∈ {A, B, C, D}", etc.).

[0003] However, in actual business operations, due to limitations in sample privacy, security, and compliance requirements, real data is often scarce or unevenly distributed (e.g., a low proportion of high-risk customer samples). This makes risk control models and credit scoring models prone to overfitting or sample bias during training, leading to a decline in risk identification effectiveness. Summary of the Invention

[0004] The main purpose of this application is to provide a sample data generation method, apparatus, device and storage medium, which aims to solve the technical problem of the scarcity of existing financial structured data samples used for risk control model training.

[0005] To achieve the above objectives, this application proposes a sample data generation method, which includes: Obtain raw financial data, extract logical constraints from the raw financial data, and obtain financial data logical constraint information; Initial financial sample data is generated based on the aforementioned financial data logical constraint information; Determine the logical consistency and sample authenticity of the initial financial sample data; The target financial sample data is determined based on the logical consistency and the authenticity of the samples.

[0006] Optionally, the step of generating initial financial sample data based on the financial data logical constraint information includes: The financial data logical constraint information is input into the latent space representation generation model, wherein the latent space representation generation model includes a preset encoder and a preset decoder; The latent space representation of the logical constraint information of the financial data is extracted through the preset encoder; Initial financial sample data is generated based on the latent space representation and the preset decoder.

[0007] Optionally, the step of determining the target financial sample data based on the logical consistency and the sample authenticity includes: Based on the logical consistency and the sample authenticity, the latent space representation generation model is backpropagated and trained to obtain a target latent space representation generation model that meets the preset training conditions. Target financial sample data is generated based on the target latent space representation generation model.

[0008] Optionally, the step of extracting the latent space representation of the financial data logical constraint information through a preset encoder includes: The preset encoder is pre-trained using information from the financial risk prediction task, enabling the preset encoder to learn high-dimensional semantic representations between features. Freeze the pre-training phase of the preset encoder and extract the latent space representation of the financial data logical constraint information through the preset encoder.

[0009] Optionally, the step of determining the logical consistency and sample authenticity of the financial sample data includes: The logical consistency of the financial sample data is determined by a logical consistency discrimination model, which is used to verify the logical consistency of the financial sample data based on pre-extracted logical constraint information of the financial data. The authenticity of the financial sample data is determined based on an authenticity discrimination model, which is used to distinguish between real samples and generated samples.

[0010] Optionally, the step of extracting logical constraints from the original financial data to obtain financial data logical constraint information includes: The original financial data is feature scaled to obtain feature-scaled original financial data; Association rule mining is performed on the original financial data after scaling the features to obtain conditional rules; Identify relevant feature pairs in the original financial data after feature scaling to obtain target rule pairs; The financial data logical constraint information is determined based on the target rule pair and the condition rule.

[0011] Optionally, the step of performing feature scaling on the original financial data to obtain feature-scaled original financial data includes: The original financial data is subjected to outlier detection and missing value completion to obtain the first original financial data; The discrete fields in the first raw financial data are categorically encoded to obtain the second raw financial data. The second original financial data is feature-scaled and standardized to obtain the feature-scaled original financial data.

[0012] Furthermore, to achieve the above objectives, this application also proposes a sample data generation apparatus, which includes: The acquisition module is used to acquire raw financial data, extract logical constraints from the raw financial data, and obtain financial data logical constraint information. A financial sample data generation module is used to generate financial sample data based on the financial data logical constraint information. The determination module is used to determine the logical consistency and sample authenticity of the financial sample data; The target financial sample data generation module is used to determine the target financial sample data based on the logical consistency and the sample authenticity.

[0013] In addition, to achieve the above objectives, this application also proposes a sample data generation device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the sample data generation method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the sample data generation method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the sample data generation method described above.

[0016] This application acquires raw financial data, extracts logical constraints from the raw financial data to obtain financial data logical constraint information; generates initial financial sample data based on the financial data logical constraint information; determines the logical consistency and sample authenticity of the initial financial sample data; and determines target financial sample data based on the logical consistency and sample authenticity. Because this application determines the target financial sample data based on the logical consistency and sample authenticity of the initial financial sample data, compared to existing methods of generating sample data using generative adversarial networks (GANs), the above method of this application can improve the logical consistency and authenticity of the generated sample data. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the method for generating sample data in Embodiment 1 of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the sample data generation method for this application. Figure 3 This is a schematic diagram of the module structure of the sample data generation device according to an embodiment of this application; Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the sample data generation method in this application embodiment.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of this application is as follows: acquiring raw financial data, extracting logical constraints from the raw financial data to obtain financial data logical constraint information; generating initial financial sample data based on the financial data logical constraint information; determining the logical consistency and sample authenticity of the initial financial sample data; and determining target financial sample data based on the logical consistency and sample authenticity. Since this application determines the target financial sample data based on the logical consistency and sample authenticity of the initial financial sample data, compared to existing methods of generating sample data using generative adversarial networks (GANs), the above method of this application can improve the logical consistency and authenticity of the generated sample data.

[0024] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or sample data generation device capable of performing the above functions. The following description uses a sample data generation device as an example to illustrate this embodiment and the subsequent embodiments.

[0025] Based on this, embodiments of this application provide a method for generating sample data, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the sample data generation method for this application.

[0026] In this embodiment, the sample data generation method includes the following steps: Step S10: Obtain raw financial data, extract logical constraints from the raw financial data, and obtain financial data logical constraint information; It should be noted that the raw financial data can be collected real financial data, which may include user basic information, transaction records, credit limits, behavioral characteristics, device characteristics, occupation, loan type, credit rating, place of residence, and other financial or credit data. The extraction of logical constraints from the raw financial data can be achieved by automatically analyzing field dependencies and logical relationships within the raw financial data to form a set of rules, i.e., the logical constraint information of the financial data. Implementation methods include, but are not limited to: conditional dependency extraction based on association rule mining (Apriori, FP-growth); detection of strongly correlated feature pairs based on mutual information; extraction of explicit rules based on logistic regression / decision tree models, such as "if loan amount > income × 5 - high-risk customer"; and automatic generation of logical templates based on statistical constraints (such as numerical range and co-occurrence probability). The final generated rules are stored in a computable form for subsequent module calls.

[0027] Furthermore, in order to obtain the logical constraint information of the financial data, step S10 may include: performing feature scaling on the original financial data to obtain feature-scaled original financial data; Association rule mining is performed on the original financial data after scaling the features to obtain conditional rules; Identify relevant feature pairs in the original financial data after feature scaling to obtain target rule pairs; The financial data logical constraint information is determined based on the target rule pair and the condition rule.

[0028] It should be noted that the feature scaling of the original financial data can be achieved through methods such as standardization (Z-score), normalization (Min-Max), or robust scaling to transform features with different dimensions and large differences in range into a unified numerical range, thereby eliminating the influence of dimensions and improving the model training effect and stability. The association rule mining of the feature-scaled original financial data can be performed by extracting conditional rules from the original financial data using Apriori or FP-growth algorithms. Identifying related feature pairs in the feature-scaled original financial data can be achieved by extracting pairwise feature sets with mutual relationships from the feature-scaled original financial data. Determining the logical constraint information of the financial data based on the target rule pairs and the conditional rules can be achieved by template storage and encoding of the target rule pairs and the conditional rules, that is, converting all statistical constraints (target rule pairs and conditional rules) into executable logical templates. This embodiment can automatically mine logical rules and dependencies between fields from the original financial data, forming a computable set of constraint functions to guide logical consistency judgment in the subsequent generation process, eliminating the reliance on manually set rules.

[0029] Furthermore, the step of performing feature scaling on the original financial data to obtain feature-scaled original financial data includes: The original financial data is subjected to outlier detection and missing value completion to obtain the first original financial data; The discrete fields in the first raw financial data are categorically encoded to obtain the second raw financial data. The second original financial data is feature-scaled and standardized to obtain the feature-scaled original financial data.

[0030] It should be noted that the outlier detection and missing value completion of the original financial data can be performed using Z-score, box plots, or KNN / MICE methods to repair and fill in the missing values, removing extreme outliers. The categorical variable encoding of discrete fields in the first original financial data can be performed using One-hot or Embedding encoding. The feature scaling and standardization of the second original financial data can be performed using Z-score or Min-Max normalization for continuous variables. The resulting normalized structured data matrix X_norm represents the feature-scaled original financial data.

[0031] In specific implementation, extracting logical constraints from the original financial data may include: 1. Association rule mining: Extracting conditional rules using Apriori or FP-growth algorithms; 2. Use mutual information analysis to identify highly correlated feature pairs. Specifically, the steps for detecting strongly correlated feature pairs based on mutual information are as follows: (1) Input data matrix: Read the preprocessed standardized data matrix (i.e. the original financial data after feature scaling) X_norm∈R^{n×d}, where n is the number of samples and d is the feature dimension; (2) Mutual information calculation: For any two features x_i, x_j, calculate the mutual information value I(x_i;x_j)=Σ_{x_i,x_j}p(x_i,x_j)log[p(x_i,x_j) / (p(x_i)p(x_j))]; Where p(x_i, x_j) is used to characterize the joint probability distribution of x_i and x_j; p(x_i) and p(x_j) are the marginal probability distributions of x_i and x_j, respectively; log is usually base 2 (unit: bits) or natural log e (unit: nats), depending on the scenario; (3) Normalization and screening: Normalize the mutual information of all feature pairs to [0, 1], set a threshold τ (e.g., 0.3), and screen out strongly correlated feature pairs with I(x_i;x_j)>τ; (4) Logical Relationship Determination: Perform monotonicity tests (such as Spearman correlation or fitted curve trend analysis) on the selected feature pairs to generate logical templates, such as "Income Increase - Credit Limit Increase"; (The "Logical Relationship Determination" is used to determine whether the directionality and monotonicity of the correlation conform to business logic. For example:) Increased revenue leads to increased credit limit (consistent with business logic) Increased debt ratio - decreased credit score (reasonable) However, if the model generates a result showing increased income but decreased credit limit, it violates common financial sense or risk control rules. (5) Rule output: Store the feature pairs that meet the conditions in the form of rules into the rule set R; This embodiment may also include model-driven rule extraction: logistic regression or decision tree extraction of explicit rules, such as "loan amount > income × 5 - high risk".

[0032] This embodiment may also include statistical constraint template generation: forming a constraint template based on the numerical range and the co-occurrence probability of values. The steps for automatically generating a logic template based on statistical constraints are as follows: (1) Calculate the statistical range of a single variable: Calculate the minimum, maximum, mean, and standard deviation of the continuous feature x_i to form the basic constraint x_i∈[mean minimum, mean maximum]; (This process establishes a reasonable range of values ​​for each individual feature, that is, it automatically generates logical constraint boundaries using the "statistical regularity" of the data to prevent unreasonable extreme values ​​or abnormal combinations from appearing in the output of the generated model. For example: income = -5000 yuan is incorrect; loan amount = 9e8 (900 million) is incorrect.) (2) Calculate the co-occurrence probability of values: Calculate the co-occurrence probability matrix P(a, b) for discrete feature combinations (such as "occupation category - loan type"); (In financial or credit data, there are often strong logical relationships between discrete fields (such as occupation, loan type, credit rating, place of residence, etc.).

[0033] For example: For civil servants, the most common type of loan is a mortgage. For college students, the most common type of loan is "consumer loan". "Unemployed individuals" are unlikely to receive "large business loans". (3) Generate constraints: If P(a, b) < ε (low co-occurrence probability), then the generated logic constraint "combination (a, b) is unreasonable" is applied. If P(a, b) > 1 - ε, then generate the logical constraint "combination (a, b) should appear"; (4) Template storage and encoding: All statistical constraints are converted into executable logic templates (JSON / XML format). Each template can be represented as T_k:(x_i∈R_i)∧(P(x_j, x_k)>ε); (template formula T_k:(x_i∈R_i)∧(P(x_j, x_k)>ε)); i ∈R i )∧(P(x j x k )>ε) represents a continuous-discrete hybrid logic constraint.

[0034] in: x i This indicates continuous characteristics (such as income, age, credit limit, etc.). R i This represents the statistically reasonable range of values ​​for this continuous characteristic. x j x k This represents discrete features (such as occupation category, loan type, etc.). P(x j x k The ) represents the co-occurrence probability of the discrete feature value combination in historical samples. ε is the co-occurrence probability threshold, used to distinguish between common combinations and anomalous combinations.

[0035] The symbol "∧" stands for logical AND, indicating that the template requires both conditions to be true simultaneously during execution. (5) Dynamic update mechanism: As new data enters the system, the statistics and templates are periodically recalculated and updated to ensure the adaptability of the logical template.

[0036] Output: Set of automated logic rules R.

[0037] Then, the rule set R is mapped to the mathematical constraint function set C={c1, c2, c3......cm} through the business logic constraint modeling module (M3).

[0038] Each rule ci is transformed into a logical function: c i (x) = {1, if the sample satisfies rule ci; 0, if the sample does not satisfy rule ci} Logical consistency score calculation: L_logic(x)=(1 / m)×Σc i (x) For continuous constraints, a smoothing function (such as Sigmoid) can be used to ensure differentiability and obtain the logical constraint information of financial data.

[0039] Step S20: Generate initial financial sample data based on the financial data logical constraint information; It should be noted that generating initial financial sample data based on the financial data logical constraint information can involve inputting the financial data logical constraint information into a preset latent space representation generation model, so that the latent space representation generation model generates data that conforms to the financial data logical constraint information, i.e., the initial financial sample data. The preset latent space representation generation model is pre-trained and is capable of first identifying the latent space representation in the financial data logical constraint information, and then generating sample data based on the latent space representation.

[0040] Step S30: Determine the logical consistency and sample authenticity of the initial financial sample data; It should be noted that determining the logical consistency and sample authenticity of the initial financial sample data can be achieved by logically judging the generated initial financial sample data through the logical constraint information of the financial data (this can be done using a logical consistency discrimination model, which assesses whether the generated sample conforms to the logical constraints of financial business and backpropagates the logical consistency signal to the latent space representation generation model, achieving logical-level credibility control), thus obtaining logical consistency. The authenticity of the initial financial sample data is then identified through an authenticity discrimination model to obtain an authenticity score, i.e., sample authenticity. The authenticity discrimination model can be a model used to identify whether data is genuine sample data.

[0041] Furthermore, in order to improve the logical consistency and sample authenticity of the target financial sample data, step S30 may include: determining the logical consistency of the financial sample data through a logical consistency discrimination model, wherein the logical consistency discrimination model is used to verify the logical consistency in the financial sample data based on pre-extracted financial data logical constraint information; The authenticity of the financial sample data is determined based on an authenticity discrimination model, which is used to distinguish between real samples and generated samples.

[0042] It should be noted that the logical consistency discrimination model, acting as the first discriminator D1, takes a sample as input and calculates its logical consistency score. If a sample violates logical constraints, the discriminator outputs a low confidence value and backpropagates the constraint signal to the generator (i.e., the latent space representation generation model), thereby forcing the generator to learn and generate samples that conform to business logic. The authenticity discrimination model, the second discriminator D2, adopts a traditional GAN ​​discriminator structure and is used to distinguish between real samples and generated samples. Its output represents the authenticity probability of a sample.

[0043] Step S40: Determine the target financial sample data based on the logical consistency and the sample authenticity.

[0044] It should be noted that determining the target financial sample data based on the logical consistency and the sample authenticity can be achieved by backpropagating the preset latent space representation generation model based on the logical consistency and the sample authenticity to obtain a preset latent space representation generation model that meets preset training conditions, and then generating the target financial sample data based on the preset latent space representation generation model that meets the preset training conditions. The preset training conditions can be the preset latent space representation generation model.

[0045] This embodiment acquires raw financial data, extracts logical constraints from the raw financial data to obtain financial data logical constraint information, generates initial financial sample data based on the financial data logical constraint information, determines the logical consistency and sample authenticity of the initial financial sample data, and determines target financial sample data based on the logical consistency and sample authenticity. Since this embodiment determines the target financial sample data based on the logical consistency and sample authenticity of the initial financial sample data, compared to existing methods of generating sample data using generative adversarial networks (GANs), the above method in this embodiment can improve the logical consistency and authenticity of the generated sample data.

[0046] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the sample data generation method of this application. Step S20 further includes the following steps: Step S201: Input the financial data logical constraint information into the latent space representation generation model, wherein the latent space representation generation model includes a preset encoder and a preset decoder; It should be noted that the latent space representation generation model includes: a pre-trained encoder module (i.e., a pre-set encoder): the encoder is pre-trained for the financial risk prediction task, learning high-dimensional semantic representations between various features; the encoder remains frozen during the generation training phase and is only used to extract latent representations; the latent space representation reflects both the features of the input data and carries semantic information related to the risk prediction task. A decoder module (i.e., a pre-set decoder): responsible for generating samples from the latent space representation; the training objectives include high-quality reconstruction and latent space semantic preservation; ensuring that the generated samples closely approximate real samples in terms of logical rationality and data distribution. Through this design, the generation model (i.e., the latent space representation generation model) can achieve both high-quality reconstruction and provide semantically rich latent space representations, providing reliable input for subsequent discrimination modules (including logical consistency discrimination models and authenticity discrimination models).

[0047] Step S202: Extract the latent space representation of the financial data logical constraint information through the preset encoder; It should be noted that the latent space representation of the financial data logical constraint information extracted by the preset encoder can be achieved by first pre-training the encoder for a financial risk prediction task to learn high-dimensional semantic representations between features, and then inputting the financial data logical constraint information to enable the preset encoder to extract the latent space representation of the financial data logical constraint information. The latent space representation maps the original high-dimensional data (such as financial transaction records, stock prices, user behavior, etc.) into a low-dimensional, hidden feature space through a neural network (such as an encoder). In this space, the key information of the data is compressed and reorganized to form a more compact and semantically meaningful expression. The financial risk prediction task is the process of identifying, assessing, and predicting various risks that may occur in financial activities (such as credit default, fraud, market volatility, liquidity crises, etc.).

[0048] Furthermore, in order for the preset encoder to better learn the latent space representation of the financial data logical constraint information, step S202 may include: pre-training the preset encoder with financial risk prediction task information, so that the preset encoder learns the high-dimensional semantic representation between each feature; Freeze the pre-training phase of the preset encoder and extract the latent space representation of the financial data logical constraint information through the preset encoder.

[0049] It should be noted that in this embodiment, the encoder is pre-trained for the financial risk prediction task to learn the high-dimensional semantic representation between each feature; then the encoder is frozen during the generation training stage (the stage of generating latent space representation) and is only used to extract latent representation; the latent space representation reflects both the features of the input data and carries semantic information related to the risk prediction task.

[0050] Step S203: Generate initial financial sample data based on the latent space representation and the preset decoder.

[0051] It should be noted that the generation of initial financial sample data based on the latent space representation and the preset decoder can be achieved by generating initial financial sample data based on the latent space representation using the preset decoder. The preset decoder is responsible for generating samples from the latent space representation; the training objectives include high-quality reconstruction and latent space semantic preservation; ensuring that the generated samples closely approximate real samples in terms of logical rationality and data distribution. Through this design, the generative model can achieve both high-quality reconstruction and provide semantically rich latent space representations, providing reliable input for the subsequent discrimination module.

[0052] Furthermore, in order to improve the quality of the generated target financial sample data, step S40 includes: performing backpropagation training on the latent space representation generation model based on the logical consistency and the sample authenticity to obtain a target latent space representation generation model that meets the preset training conditions. Target financial sample data is generated based on the target latent space representation generation model.

[0053] It should be noted that the pre-trained encoder in this implementation takes the following inputs: financial data logical constraint information X_norm; output: latent space vector z; structure: multi-layer Transformer or MLP; training: trained on financial risk prediction tasks (such as default prediction, fraud detection), optimizing the cross-entropy loss: L_enc=L_cls(y, Freezing strategy: Keep the data frozen during the generation stage and use it only for feature extraction; Function: Provide a latent space representation with business semantics.

[0054] Decoder: Input: latent space vector z; Output: generated samples Training objective: Reconstruction loss: L_recon = ||X_norm ||2; Semantic consistency loss: maximizes latent space similarity; function: generates structured samples that are logical and close to real data.

[0055] Logical consistency discriminator D1: Input generated samples ; Calculate the logical consistency score L_logic( If a sample violates the constraints, output a low confidence level; backpropagate the logic constraint signal to the generator. Authenticity Discriminator D2: Discriminates generated samples The difference in distribution between the real sample X_norm and the actual sample X_norm; output the probability of authenticity P_real; The steps for implementing the generator's overall loss balance mechanism are as follows: (1) Logical loss calculation: The logical consistency discriminator outputs a logical consistency score L_logic( ), representing the proportion of generated samples that satisfy the constraints; (2) Calculation of authenticity loss: The authenticity discriminator outputs the authenticity probability P_real of the generated sample. The adversarial loss is defined as L_adv^D2 = -E[log P_real( )]; (3) Construction of the comprehensive loss function: L_G=λ1·L_adv^D2+λ2·(1-L_logic), where λ1 and λ2 are dynamically adjusted through grid search or adaptive weighting; Where L_G is the generator's overall loss function, used to guide the generator to simultaneously optimize adversarial performance and logical consistency during training; L_adv^D2 is the adversarial loss for D2, derived from the discriminator D. 2， Function: To measure the similarity between generated samples and real samples, driving the generator to generate more realistic samples that are harder for the discriminator to distinguish; L_logic: Logical consistency score; Value range: [0, 1], the larger the value, the more the generated sample conforms to the business rules; In the loss function, 1-L_logic represents the penalty for logical inconsistency: the higher the logical consistency, the smaller this loss; λ1: Adversarial loss weight, controlling the importance of the "realism" target in generator training; It can be adjusted through grid search or adaptive methods to stabilize adversarial training; λ2: Logical consistency weight, controlling the importance of the "rule compliance" target in generator training; It can be automatically adjusted through grid search or dynamic weighting strategies to balance the realism of generated samples with logical constraints. The balancing update strategy is: if the logical constraint violation rate increases, increase λ2 to strengthen logical consistency; if the distribution of generated samples deviates from the real distribution, increase λ1 to strengthen realism optimization; Adversarial training loop: the generator, D1, and D2 are updated alternately until the comprehensive loss converges.

[0056] In the specific implementation, 1. Data preparation: The raw data contains credit records of 5,000 customers, including basic information (age, gender), financial information (monthly income, loan amount), historical repayment behavior, and device information. Through the data preprocessing module, missing values ​​are filled using KNN interpolation, categorical features are encoded using One-hot encoding, and continuous features are standardized using Z-score to obtain the normalized data matrix X_norm.

[0057] 2. Business logic rule extraction and constraint modeling Using the Apriori algorithm and mutual information analysis, rules such as "loan amount ≤ monthly income × 5" and "credit rating ∈ {A, B, C, D}" are extracted. The rule set R is then transformed into a constraint function set C, providing constraint signals for the logical consistency discriminator D1.

[0058] 3. Encoder pre-training Train the encoder on the financial risk prediction task and optimize the cross-entropy loss L_enc = L_cls(y, ).

[0059] The encoder is frozen after training and is only used for latent space representation extraction during the generation stage.

[0060] 4. Generator vs. LD-GAN Adversarial Training The generator receives the latent space vector z and outputs the generated samples. .

[0061] Logical consistency discriminator D1 calculates L_logic( The authenticity discriminator D2 calculates P_real.

[0062] Total loss for generator optimization: L_G = λ1·L_adv^D2 + λ2·(1-L_logic) After 2,000 iterations of training, the generated samples closely resemble real customer data in terms of both statistical features and logical constraints.

[0063] 4. Performance Verification: The credit risk model trained on the generated data showed a 5% improvement in AUC on the validation set, while the proportion of generated samples violating business rules was less than 1%. Experiments demonstrate that the data generated in this embodiment is both high-quality and consistent with financial logic, and can be used for model enhancement and supplementation for small-sample scenarios.

[0064] This embodiment inputs the logical constraint information of the financial data into a latent space representation generation model, wherein the latent space representation generation model includes a preset encoder and a preset decoder; the latent space representation of the logical constraint information of the financial data is extracted through the preset encoder; and initial financial sample data is generated based on the latent space representation and the preset decoder. This embodiment can automatically generate sample data for training risk control models and credit scoring models, improving the training efficiency of risk control models and credit scoring models.

[0065] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the sample data generation method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0066] This application also provides a sample data generation apparatus, please refer to... Figure 3 The sample data generation device includes: The acquisition module 10 is used to acquire raw financial data, extract logical constraints from the raw financial data, and obtain financial data logical constraint information. The financial sample data generation module 20 is used to generate financial sample data based on the financial data logical constraint information. The determination module 30 is used to determine the logical consistency and sample authenticity of the financial sample data; The target financial sample data generation module 40 is used to determine the target financial sample data based on the logical consistency and the sample authenticity.

[0067] This embodiment acquires raw financial data, extracts logical constraints from the raw financial data to obtain financial data logical constraint information, generates initial financial sample data based on the financial data logical constraint information, determines the logical consistency and sample authenticity of the initial financial sample data, and determines target financial sample data based on the logical consistency and sample authenticity. Since this embodiment determines the target financial sample data based on the logical consistency and sample authenticity of the initial financial sample data, compared to existing methods of generating sample data using generative adversarial networks (GANs), the above method in this embodiment can improve the logical consistency and authenticity of the generated sample data.

[0068] The sample data generation apparatus provided in this application, employing the sample data generation method described in the above embodiments, can solve the technical problem of scarce financial structured data samples used for risk control model training. Compared with the prior art, the beneficial effects of the sample data generation apparatus provided in this application are the same as those of the sample data generation method provided in the above embodiments, and other technical features in the sample data generation apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0069] This application provides a sample data generation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the sample data generation method in Embodiment 1 above.

[0070] The following is for reference. Figure 4 The diagram illustrates a structural schematic of a sample data generation device suitable for implementing embodiments of this application. The sample data generation device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The sample data generation device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0071] like Figure 4As shown, the sample data generation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the sample data generation device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the sample data generation device to communicate wirelessly or wiredly with other devices to exchange data. Although sample data generation devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0072] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0073] The sample data generation device provided in this application, employing the sample data generation method described in the above embodiments, can solve the technical problem of scarce financial structured data samples used for risk control model training. Compared with the prior art, the beneficial effects of the sample data generation device provided in this application are the same as those of the sample data generation method provided in the above embodiments, and other technical features of this sample data generation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0074] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0075] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0076] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the sample data generation method in the above embodiments.

[0077] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0078] The aforementioned computer-readable storage medium may be included in the sample data generation device; or it may exist independently and not assembled into the sample data generation device.

[0079] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Python, Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0080] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0081] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0082] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described sample data generation method, which can solve the technical problem of scarce financial structured data samples used for risk control model training. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the sample data generation method provided in the above embodiments, and will not be repeated here.

[0083] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the sample data generation method described above.

[0084] The computer program product provided in this application can solve the technical problem of scarce financial structured data samples used for training risk control models. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the sample data generation method provided in the above embodiments, and will not be repeated here.

[0085] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A method for generating sample data, characterized in that, The sample data generation method includes the following steps: Obtain raw financial data, extract logical constraints from the raw financial data, and obtain financial data logical constraint information; Initial financial sample data is generated based on the aforementioned financial data logical constraint information; Determine the logical consistency and sample authenticity of the initial financial sample data; The target financial sample data is determined based on the logical consistency and the authenticity of the samples.

2. The sample data generation method as described in claim 1, characterized in that, The step of generating initial financial sample data based on the financial data logical constraint information includes: The financial data logical constraint information is input into the latent space representation generation model, wherein the latent space representation generation model includes a preset encoder and a preset decoder; The latent space representation of the logical constraint information of the financial data is extracted through the preset encoder; Initial financial sample data is generated based on the latent space representation and the preset decoder.

3. The sample data generation method as described in claim 2, characterized in that, The step of determining the target financial sample data based on the logical consistency and the sample authenticity includes: Based on the logical consistency and the sample authenticity, the latent space representation generation model is backpropagated and trained to obtain a target latent space representation generation model that meets the preset training conditions. Target financial sample data is generated based on the target latent space representation generation model.

4. The sample data generation method as described in claim 2, characterized in that, The step of extracting the latent space representation of the financial data logical constraint information through a preset encoder includes: The preset encoder is pre-trained using information from the financial risk prediction task, enabling the preset encoder to learn high-dimensional semantic representations between features. Freeze the pre-training phase of the preset encoder and extract the latent space representation of the financial data logical constraint information through the preset encoder.

5. The sample data generation method according to any one of claims 1-4, characterized in that, The steps for determining the logical consistency and sample authenticity of the financial sample data include: The logical consistency of the financial sample data is determined by a logical consistency discrimination model, which is used to verify the logical consistency of the financial sample data based on pre-extracted logical constraint information of the financial data. The authenticity of the financial sample data is determined based on an authenticity discrimination model, which is used to distinguish between real samples and generated samples.

6. The sample data generation method according to any one of claims 1-4, characterized in that, The step of extracting logical constraints from the original financial data to obtain logical constraint information for the financial data includes: The original financial data is feature scaled to obtain feature-scaled original financial data; Association rule mining is performed on the original financial data after scaling the features to obtain conditional rules; Identify relevant feature pairs in the original financial data after feature scaling to obtain target rule pairs; The financial data logical constraint information is determined based on the target rule pair and the condition rule.

7. The sample data generation method as described in claim 6, characterized in that, The step of performing feature scaling on the original financial data to obtain feature-scaled original financial data includes: The original financial data is subjected to outlier detection and missing value completion to obtain the first original financial data; The discrete fields in the first raw financial data are categorically encoded to obtain the second raw financial data. The second original financial data is feature-scaled and standardized to obtain the feature-scaled original financial data.

8. A sample data generation device, characterized in that, The sample data generation device includes: The acquisition module is used to acquire raw financial data, extract logical constraints from the raw financial data, and obtain financial data logical constraint information. A financial sample data generation module is used to generate financial sample data based on the financial data logical constraint information. The determination module is used to determine the logical consistency and sample authenticity of the financial sample data; The target financial sample data generation module is used to determine the target financial sample data based on the logical consistency and the sample authenticity.

9. A sample data generation device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the sample data generation method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the sample data generation method as described in any one of claims 1 to 7.

Images