Test data generation method and device and electronic equipment

Abstract

The invention discloses a test data generation method and device and electronic equipment. The method relates to the field of big data, and comprises the following steps: receiving a data generation request sent by a user side, and analyzing the data generation request to obtain test scene information and test product information; determining a test index associated with the to-be-tested product according to the test product information, and inputting the test index and the test scene information into a test data generation model to obtain initial test data under the test index; and detecting the initial test data according to the test product information to obtain a detection result, and feeding back the initial test data to the user side under the condition that the detection result represents that the initial test data is not abnormal. Through the method and the device, the problems of relatively low accuracy and authenticity of the generated test data in related technologies are solved.

Description

Technical Field

[0001] This application relates to the field of big data, and more specifically, to a method, apparatus, and electronic device for generating test data. Background Technology

[0002] During the market risk stress testing process for financial products, financial institutions need to generate test data to simulate financial data under various possible market scenarios, thereby assessing the potential risks of the products under different market conditions. This process is crucial for ensuring the sound operation of financial institutions, meeting operational requirements, and taking timely and effective countermeasures in the event of market changes.

[0003] Currently, when generating test data, the common method is to manually determine the financial data that may appear in the scenario after the simulation is determined. However, the data generated in this way may have significant differences from real market data, such as data trends and data value ranges, thereby reducing the accuracy and authenticity of the test data, and consequently reducing the test accuracy.

[0004] There is currently no effective solution to the problem of low accuracy and authenticity of test data generated in related technologies. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, and electronic device for generating test data, in order to solve the problem of low accuracy and authenticity of generated test data in related technologies.

[0006] To achieve the above objectives, according to one aspect of this application, a method for generating test data is provided. The method includes: receiving a data generation request sent by a user terminal, parsing the data generation request to obtain test scenario information and test product information; determining test indicators associated with the product under test based on the test product information, and inputting the test indicators and test scenario information into a test data generation model to obtain initial test data under the test indicators; detecting the initial test data based on the test product information to obtain detection results, and if the detection results indicate that the initial test data is normal, feeding back the initial test data to the user terminal.

[0007] Optionally, determining the test indicators associated with the product under test based on the test product information includes: determining the business type to which the product under test belongs based on the test product information; obtaining preset indicators that affect business products under the business type, resulting in multiple preset indicators; and determining the multiple preset indicators as test indicators associated with the product under test.

[0008] Optionally, the test data generation model is obtained as follows: multiple historical product information sets, historical economic scenario information for each historical product within its sales cycle, and historical indicator data under preset indicators related to the historical product information are acquired; the historical economic scenario information, preset indicators, and historical indicator data under preset indicators for each historical product are determined as a sample data set, resulting in multiple sample data sets; the generator and discriminator of the conditional generative adversarial network (GAN) model are trained using the multiple sample data sets, and a loss function for the GAN model is constructed; if the loss value of the loss function is less than a preset threshold, the generator in the GAN model is determined as the test data generation model.

[0009] Optionally, the loss function for constructing the conditional generative adversarial network model includes: determining the discriminator's recognition result of the sample data as the first loss function term; determining the discriminator's recognition result of the generator's output as the second loss function term; and combining the first loss function term, the second loss function term, and the preset financial penalty term into a loss function.

[0010] Optionally, the method further includes: obtaining the number of times the test data generation model processes each sample data and the accuracy of the processing result obtained after each processing; if the number of processing times is greater than the number of processing times threshold and the accuracy is less than the accuracy threshold, determining the sample data as abnormal sample data and deleting the abnormal sample data from the training set.

[0011] Optionally, the initial test data is tested based on the test product information, and the test results include: determining the indicator value requirements of the test indicators based on the test product information, and using the indicator value requirements to test the initial test data; if the initial test data meets the indicator value requirements, the test result is determined to be without abnormality; if the initial test data does not meet the indicator value requirements, the test result is determined to be abnormal.

[0012] Optionally, feeding back the initial test data to the user terminal includes: obtaining the data receiving format of the user terminal and standardizing the initial test data according to the data receiving format; encapsulating the standardized initial test data to obtain the target test data, and feeding back the target test data to the user terminal.

[0013] To achieve the above objectives, according to another aspect of this application, a test data generation apparatus is provided. The apparatus includes: a parsing unit, configured to receive a data generation request sent by a user terminal, and parse the data generation request to obtain test scenario information and test product information; a first determining unit, configured to determine test indicators associated with the product to be tested based on the test product information, and input the test indicators and test scenario information into a test data generation model to obtain initial test data under the test indicators; and a detection unit, configured to detect the initial test data based on the test product information, obtain detection results, and, if the detection results indicate that the initial test data is normal, feed back the initial test data to the user terminal.

[0014] Optionally, the first determining unit includes: a first determining module, used to determine the business type to which the product under test belongs based on the test product information; a first obtaining module, used to obtain preset indicators that affect business products under the business type, and obtain multiple preset indicators; and a second determining module, used to determine the multiple preset indicators as test indicators associated with the product under test.

[0015] Optionally, the test data generation model is obtained through the following apparatus: a first acquisition unit, used to acquire multiple historical product information, historical economic scenario information of each historical product information within the product sales cycle, and historical indicator data under preset indicators related to the historical product information; a second determination unit, used to determine the historical economic scenario information, preset indicators, and historical indicator data under preset indicators of each historical product information as a sample data, thereby obtaining multiple sample data; a training unit, used to train the generator and discriminator of the conditional generative adversarial network model using multiple sample data, and construct the loss function of the conditional generative adversarial network model; and a third determination unit, used to determine the generator in the conditional generative adversarial network model as the test data generation model when the loss value of the loss function is less than a preset threshold.

[0016] Optionally, the training unit includes: a third determining module, used to determine the discriminator's recognition result of the sample data as the first loss function term; a fourth determining module, used to determine the discriminator's recognition result of the generator's output as the second loss function term; and a combining module, used to combine the first loss function term, the second loss function term, and the preset financial penalty term into a loss function.

[0017] Optionally, the device further includes: a second acquisition unit, used to acquire the number of times the test data generation model processes each sample data and the accuracy of the processing result obtained after each processing; and a fourth determination unit, used to determine that the sample data is abnormal sample data and delete the abnormal sample data from the training set when the number of processing is greater than the number threshold and the accuracy is less than the accuracy threshold.

[0018] Optionally, the detection unit includes: a fifth determining module, used to determine the indicator value requirements of the test indicators based on the test product information, and to use the indicator value requirements to detect the initial test data; a sixth determining module, used to determine that the test result is normal if the initial test data meets the indicator value requirements; and a seventh determining module, used to determine that the test result is abnormal if the initial test data does not meet the indicator value requirements.

[0019] Optionally, the detection unit includes: a second acquisition module, used to acquire the data receiving format of the user terminal and standardize the initial test data according to the data receiving format; and a feedback module, used to encapsulate the standardized initial test data to obtain target test data and feed the target test data back to the user terminal.

[0020] To achieve the above objectives, according to another aspect of this application, an electronic device is provided, the electronic device including a memory storing an executable program; and a processor for running the program, wherein the program executes the above-described method for generating test data during runtime.

[0021] To achieve the above objectives, according to another aspect of this application, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the above-described test data generation method.

[0022] In this embodiment, a method is adopted to receive and parse a data generation request sent by a user terminal to obtain test scenario information and test product information; determine the test indicators associated with the product to be tested based on the test product information, and input the test indicators and test scenario information into the test data generation model to obtain initial test data under the test indicators; detect the initial test data based on the test product information to obtain detection results, and if the detection results indicate that the initial test data is not abnormal, feed the initial test data back to the user terminal. By using the test data generation model to process the test scenario information and test product information, test data with a high similarity to real financial data that is related to the test product and test scenario is obtained, thus achieving the goal of accurately generating test data. This improves the accuracy and authenticity of test data and solves the technical problem of low accuracy and authenticity of generated test data in related technologies. Attached Figure Description

[0023] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0024] Figure 1A hardware block diagram of a computer terminal for implementing a method for generating test data is shown.

[0025] Figure 2 This is a flowchart of the test data generation method provided in Embodiment 1 of this application;

[0026] Figure 3 This is a schematic diagram of the structure of the model generated based on the test data provided in Embodiment 1 of this application;

[0027] Figure 4 This is a schematic diagram of the test data generation device provided in Embodiment 2 of this application;

[0028] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0029] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It should be noted that the test data generation method, apparatus, and electronic equipment defined in this disclosure can be used in the field of big data, or in any field other than big data. The application fields of the test data generation method, apparatus, and electronic equipment defined in this disclosure are not limited.

[0033] It should be noted that all information, user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) used in this application are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse use. If the user chooses to refuse, the process will proceed to the expert decision-making process. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface. After receiving consent from the aforementioned user or organization, the relevant information is obtained. Users can view the purpose of data use in real time through the authorization interface and have the right to withdraw authorization or delete data at any time. After the authorization is withdrawn, the system will terminate the relevant data processing within 24 hours.

[0034] The embodiments or examples disclosed herein are not exhaustive, but merely illustrative of some embodiments or examples, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment or example can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment or example can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment or example can be arbitrarily interchanged. Furthermore, optional methods or examples in a particular embodiment or example can be arbitrarily combined; moreover, embodiments or examples can be arbitrarily combined. For example, some or all steps of different embodiments or examples can be arbitrarily combined, and a particular embodiment or example can be arbitrarily combined with optional methods or examples of other embodiments or examples.

[0035] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0036] CGAN: Conditional Generative Adversarial Network.

[0037] Example 1

[0038] According to an embodiment of this application, an embodiment of a method for generating test data is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0039] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing a method for generating test data is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, processing devices such as microprocessors or programmable logic devices), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface, a universal serial bus port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0040] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0041] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the test data generation method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned test data generation method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0042] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0043] The display may be, for example, a touchscreen LCD display that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0044] Under the aforementioned operating environment, this application provides the following: Figure 2 The method for generating test data is shown. Figure 2 This is a flowchart of the test data generation method provided in Embodiment 1 of this application, as follows: Figure 2 As shown, the method includes:

[0045] Step S201: Receive the data generation request sent by the user terminal, parse the data generation request, and obtain the test scenario information and test product information.

[0046] It should be noted that the execution entity in this embodiment can be a test data generation system. This system can generate corresponding test data based on the data generation request input by the user, and can make the generated test data follow the same trend as historical financial data.

[0047] It should be noted that a data generation request can be an instruction issued by the user to generate a corresponding financial dataset based on a specific test scenario. Test scenario information can be parameters describing stress test or market simulation scenarios, including but not limited to extreme value settings for risk factors and the economic environment to be simulated. Test product information refers to information about the specific financial products being tested by the financial institution, such as product type, product composition, and related risk factors.

[0048] For example, when generating test data, the system first receives a data generation request. This request contains information about the scenario the user wishes to simulate, such as holidays for the coming year, and information about the test product, such as the deposit transaction status for the coming year. By parsing the request, the system clarifies the specific content of the simulated scenario and the product information, and can then generate the corresponding test data based on this information.

[0049] Step S202: Determine the test indicators associated with the product to be tested based on the test product information, and input the test indicators and test scenario information into the test data generation model to obtain the initial test data under the test indicators.

[0050] For example, after parsing the request, the system automatically identifies key indicators related to deposit business, such as user activity, product click-through rate, user holiday consumption data, and user holiday deposit business volume, and inputs these indicators along with the scenario information of specific holidays into a pre-trained test data generation model. The model generates data under each test indicator based on the test scenario information, thereby generating product information under the test scenario.

[0051] For example, suppose the model needs to test credit card products during a holiday promotion. In this scenario, it needs to generate consumer behavior data related to the holiday promotion. In this case, the model first generates a random noise vector to represent market uncertainty, and then fuses it with conditional information about the holiday promotion (such as the percentage increase in purchase frequency and expected changes in average purchase amount) to simulate possible changes in consumer behavior during the holiday promotion. For instance, the model generates a dataset containing the shopping records of 1000 users during the holiday promotion, along with their credit card usage.

[0052] Step S203: Detect the initial test data according to the test product information, obtain the test results, and if the test results indicate that the initial test data is normal, feed the initial test data back to the user terminal.

[0053] For example, after generating test data, it is necessary to check the test data. This may include checking whether the generated data format meets the requirements, whether the values ​​are within the preset range, and whether there is any contradictory data. If the check results show that the data is normal, i.e., the test data is reasonable, the system feeds back this initial test data to the user, allowing the user to perform corresponding test operations on the product based on the test data. This improves the accuracy of the test data by generating test data through the model, thereby improving the accuracy of subsequent product testing.

[0054] It should be noted that in this embodiment, the system generates corresponding test data based on the data generation request input by the user, so that the generated test data follows the same trend as historical financial data. This also improves the standardization and traceability of the data. By standardizing and regulating the test data generation process, each piece of test data can be traced, thereby improving the accuracy of the test data and avoiding the problem of inaccurate data caused by inconsistent data generation standards when data is generated manually.

[0055] The test data generation method provided in this application embodiment receives a data generation request sent by a user terminal, parses the request to obtain test scenario information and test product information; determines test indicators associated with the product under test based on the test product information, and inputs the test indicators and test scenario information into a test data generation model to obtain initial test data under the test indicators; detects the initial test data based on the test product information to obtain detection results, and feeds back the initial test data to the user terminal if the detection results indicate that the initial test data is normal. By processing the test scenario information and test product information through the test data generation model, test data with a high similarity to real financial data and a correlation with the test product and test scenario is obtained, achieving the goal of accurately generating test data. This improves the accuracy and authenticity of test data and solves the technical problem of low accuracy and authenticity of generated test data in related technologies.

[0056] Optionally, in the test data generation method provided in this application embodiment, determining the test indicators associated with the product under test based on the test product information includes: determining the business type to which the product under test belongs based on the test product information; obtaining preset indicators that affect business products under the business type, and obtaining multiple preset indicators; and determining the multiple preset indicators as test indicators associated with the product under test.

[0057] It should be noted that "test product information" refers to a detailed description of the deposit product to be tested, including product characteristics and target customer groups. "Product to be tested" refers to a product offered by a financial institution. "Business type" can refer to the type of product, such as a wealth management product. "Preset indicators" are key quantitative standards pre-set by the financial institution to evaluate the product's performance in specific scenarios. "Test indicators" are specific quantitative metrics used to test product performance, composed of multiple preset indicators, reflecting the product's expected performance in specific scenarios.

[0058] For example, after receiving the test product information, the system first determines the business type of the product to be tested based on the product description. For instance, if the test product is a deposit product, the system will classify it as a savings business.

[0059] After identifying the business type of the product to be tested, the system will retrieve preset indicators that affect business products of that business type from the financial institution's indicator database. For example, if the product is a deposit product, the indicators may include, but are not limited to, user consumption data, user deposit data, and user deposit transaction volume. This allows the system to test the deposit business based on the aforementioned indicator data, such as determining how to set the quantity or amount of gifts for the deposit business based on the test data.

[0060] Furthermore, once the preset indicators are determined, they can be identified as test indicators associated with the product under test, thereby generating data for each test indicator based on the test data generation model.

[0061] This embodiment improves the accuracy of test data generation and the usability of the data by determining test indicators based on product information.

[0062] To improve the accuracy of the model output, optionally, in the test data generation method provided in this application embodiment, the test data generation model is obtained in the following way: acquiring multiple historical product information, historical economic scenario information of each historical product information within the product sales cycle, and historical indicator data under preset indicators related to the historical product information; determining the historical economic scenario information, preset indicators, and historical indicator data under preset indicators of each historical product information as a sample data, thus obtaining multiple sample data; using multiple sample data to train the generator and discriminator of the conditional generative adversarial network model, and constructing the loss function of the conditional generative adversarial network model; if the loss value of the loss function is less than a preset threshold, determining the generator in the conditional generative adversarial network model as the test data generation model.

[0063] For example, the test data generation model can be a CGAN model. When building the model, it is necessary to generate a training set. Multiple historical product information can be obtained, and the real historical indicator data of each historical product information under the preset indicators within the product sales cycle, as well as the real historical economic scenario, can be obtained. This will determine it as a sample data, resulting in multiple sample data, and the dataset is composed of multiple sample data.

[0064] For example, historical product information can be product information of different types of financial products, and the historical economic scenarios can be the same, such as the occurrence of large-value shopping subsidy coupons in a certain period of the past. Real historical indicator data can be the data values ​​of various historical indicator data and the data change trends under that historical economic scenario.

[0065] It should be noted that when acquiring sample data, it is also necessary to perform integrity checks (handling missing values), reasonableness checks (identifying and correcting outliers), and consistency checks (ensuring that the data logic remains consistent across data sources) to improve the accuracy and validity of the data.

[0066] Furthermore, after obtaining the training set, the system uses the constructed sample data to train the generator and discriminator of the conditional generative adversarial network model. The training process follows the standard min-max game procedure of CGAN, but financial rules are added as a penalty term to the loss function. When updating the discriminator, no special processing is done. Real data from the training set is input into the discriminator, and the loss for the discriminator's judgment of "true and conditionally matched" is calculated. Then, data fabricated by the generator based on scene and product information from the training set is input into the discriminator, and the loss for the discriminator's judgment of "false" is calculated. The two losses are added together and backpropagated to update the discriminator parameters. When updating the generator, data fabricated by the generator based on scene and product information from the training set is input into the discriminator, and the loss for the discriminator's judgment of "true and conditionally matched" is calculated. The loss from the penalty term is added as the total loss, and the generator parameters are backpropagated again. The above training process is repeated until the model converges, thus completing the model training process.

[0067] Finally, the system continues to train the CGAN model until the loss function's value is less than a preset threshold. This condition signifies that the data generated by the model is of sufficient quality to simulate real market conditions and conforms to the business logic under preset metrics. Once this standard is met, the system designates the generator in the conditional generative adversarial network model as the test data generation model for subsequent data generation tasks.

[0068] It should be noted that, Figure 3 This is a schematic diagram of the structure of the model generated based on the test data provided in Embodiment 1 of this application, as shown below. Figure 3 As shown, the generator consists of: an input layer, a dimension increment layer, a product information embedding layer, a feature fusion layer, a temporal generation core layer, and an output layer, wherein:

[0069] The input layer takes random noise z and conditional information c as input. The random noise z is a vector z sampled from a standard normal distribution with dimensions [batch_size, noise_dim], representing the uncertainty of the market. The conditional information c is a scene vector c with dimensions [batch_size, condition_dim].

[0070] The dimension increment layer concatenates z and c according to their feature dimensions to obtain a fusion vector [batch_size, noise_dim + condition_dim]. This fusion vector is then mapped to a higher-dimensional tensor through a fully connected layer, and its shape is adjusted to become a time series seed [batch_size, sequence_length, hidden_dim]. sequence_length is the planned length of the time series (e.g., 252 trading days), and hidden_dim is the initial number of features at each time step.

[0071] The product information embedding layer takes product information as input and generates a product identifier sequence [sequence_length, num_assets]. For each time step, it needs to know which product it is, and obtains the embedding layer output [batch_size, sequence_length, embedding_dim] through the embedding matrix [num_assets, embedding_dim].

[0072] Further, by fusing the feature layers, the shape of the concatenated tensor is obtained as [batch_size, sequence_length, hidden_dim + embedding_dim]. It contains overall market state information (from noise and conditions) and individual identity information for each asset.

[0073] The Transformer decoder is further used in the core layer of time-series generation for the following reasons: 1) Masked self-attention ensures that when generating data at time step t, only information from steps 1 to t-1 is seen, which aligns with the causal relationship of sequence generation. It can effectively capture long-term dependencies; 2) Real market crises evolve gradually, with early, small signals gradually amplified through positive feedback. The Transformer decoder can model this complex, multi-time-scale propagation mechanism, thus obtaining a high-dimensional feature tensor output [batch_size, sequence_length, new_hidden_dim].

[0074] Finally, a fully connected layer maps new_hidden_dim to the target dimension. (If the goal is to generate a yield sequence of num_assets products, then the output size of this fully connected layer is num_assets).

[0075] Finally, the generator outputs x_fake[batch_size, sequence_length, num_assets].

[0076] The discriminator consists of: an input layer, an asset information embedding layer, a feature fusion layer, a temporal feature extraction layer, a conditional matching judgment layer, and an output layer, wherein:

[0077] The input layer takes into account the market path and condition information c under the product sequence. The market path under the product sequence is real data x_real or generated data x_fake, with a shape of [batch_size, sequence_length, num_assets]. The condition information c is a scene vector c with dimensions of [batch_size, condition_dim].

[0078] The asset information embedding layer includes the same asset embedding matrix [num_assets, embedding_dim] as the generator. Similarly, a product ID sequence is created and passed through the embedding layer to obtain the asset embedding tensor [batch_size, sequence_length, embedding_dim].

[0079] The feature fusion layer concatenates the input market path X with the asset embedding tensor to obtain a tensor of shape [batch_size, sequence_length, num_assets + embedding_dim]. When the discriminator analyzes market data, it knows which product category it is currently dealing with, thus enabling it to better learn the different behavioral patterns of different products.

[0080] The temporal feature extraction layer is used to input the fused tensor into a Transformer encoder. Since the discriminator can see complete, future information to judge the overall authenticity of the entire path, the Transformer encoder is used to output a complete sequence. The output of the last time step can be taken, or global average pooling can be performed on the outputs of all time steps to obtain a comprehensive, fixed-size path representation vector with shape [batch_size, feature_dim].

[0081] The condition matching decision layer is used to concatenate the path representation vector obtained by the Transformer encoder with the condition vector c to obtain [batch_size, feature_dim + condition_dim], and then the concatenated vector is passed through a fully connected network.

[0082] The output layer consists of a linear layer with two output neurons, coupled with a sigmoid activation function, where:

[0083] The first neuron outputs d_real, which represents the probability that the discriminator considers the input path to be real rather than generated.

[0084] The second neuron outputs d_match, which represents the probability that the discriminator considers the input path to match condition c.

[0085] During training, the expected values ​​for [d_real, d_match] are: for real data x_real, the output is close to [1, 1]; for generated data x_fake, the output is close to [0, 1] (if the condition matches) or [0, 0] (if the condition does not match).

[0086] It's important to note that on the validation set, the value of each penalty term is calculated from the generated data. If the value of a penalty term is consistently significantly higher than others, it indicates that the specific threshold set by the generator for that pattern is unreasonable. The adaptive weight allocation module dynamically adjusts the weights of each penalty term, automatically increasing their corresponding weights to enable more significant gradient descent. Conversely, if a term remains consistently low, its weight can be slightly reduced, allowing the model to focus more on optimizing other, more challenging constraints.

[0087] This embodiment uses historical data as sample data to train the model, enabling the model to generate test data with similar data trends and ranges to the historical data. This improves the similarity between the test data and the real data, thereby enhancing the accuracy and effectiveness of the test data.

[0088] Optionally, in the test data generation method provided in the embodiments of this application, constructing the loss function of the conditional generative adversarial network model includes: determining the discriminator's recognition result of the sample data as a first loss function term; determining the discriminator's recognition result of the generator's output as a second loss function term; and combining the first loss function term, the second loss function term, and a preset financial penalty term into a loss function.

[0089] For example, the system constructs a loss function for a conditional generative adversarial network model. The loss function consists of two parts: one part is based on the discriminator's judgment, which measures the fidelity of the generated data; the other part is a penalty term specific to the business scenario, which makes the generated data conform to the market logic under preset indicators.

[0090] First, we define the first loss function term, which measures the accuracy of the discriminator in identifying real sample data (i.e., a combination of historical product information, historical economic scenario information, and historical indicator data). During training, the first loss function term encourages the discriminator to learn to distinguish between real and generated data; that is, when given real sample data, the discriminator should output a higher probability of being true.

[0091] Next, the system defines a second loss function term, focusing on the quality of the generated data. The goal of this term is to ensure that the data generated by the generator can be recognized as real data by the discriminator. During training, the generator continuously attempts to optimize its output, making the generated data as close as possible to the distribution of real data, while also meeting the requirements of specific market scenarios.

[0092] The preset financial penalties include, but are not limited to, price non-negativity checks, inventory logic consistency checks, and user behavior rationality checks, making the generated data not only realistic but also consistent with actual business logic.

[0093] The loss function can be as follows:

[0094] V(D,G)=E[logD(X|C)]+E[log(1-D(G(z|C)))]+λ R(G);

[0095] Where V(D, G) is the loss function, and E[logD(X|C)] is the true data term of the discriminator (D), representing the expected logarithm of the true probability output by the discriminator (D) for true data (X) and given conditions (C). E[log(1-D(G(z|C)))] is the generated data term of the generator (G), representing the expected logarithm of the false probability output by the discriminator (D) for data (G(z|C)) generated by noise (z) and conditions (C). R(G) is a financial penalty term, used to ensure that the generated data (G(z|C)) output by the generator (G) is not only statistically similar to the true data, but also follows specific financial logic and rules. (R(G)) can be represented by a weighted sum of multiple sub-penalty terms, each reflecting one aspect of the financial properties of the generated data.

[0096] It should be noted that for the discriminator (D), the objective is to maximize V(D, G), that is, to maximize the sum of E[logD(X|C)] and E[log(1-D(G(z|C)))]. This means that the discriminator should be able to accurately distinguish between real data (X) and generated data (G(z|C)) in all cases. For the generator (G), the objective is to minimize V(D, G), that is, to minimize E[log(1-D(G(z|C)))] (equivalent to maximizing D(G(z|C)), i.e., to make the generated data look as realistic as possible), and to minimize the penalty term R(G).

[0097] This embodiment achieves the technical effect of improving the accuracy of the model's output results by adding a financial penalty term to the loss function and using adversarial training.

[0098] Optionally, in the test data generation method provided in the embodiments of this application, the method further includes: obtaining the number of times the test data generation model processes each sample data and the accuracy of the processing result obtained after each processing; if the number of processing times is greater than the number of processing times threshold and the accuracy is less than the accuracy threshold, determining that the sample data is abnormal sample data and deleting the abnormal sample data from the training set.

[0099] For example, during the training of a conditional generative adversarial network (GAN) model using multiple sample data, the system records the number of times the model processes each sample data and the accuracy of the processing result after each processing. The accuracy is calculated based on the degree to which the discriminator's judgment of the generated data matches the actual sample data. In specific implementation, the system creates a tracking log for each sample data, recording the number of iterations the model processes for that sample and the discrimination accuracy of the generated data after each iteration.

[0100] Furthermore, it is necessary to pre-set thresholds for the number of iterations and accuracy. The number of iterations threshold is determined based on the model's convergence speed and the complexity of the sample data. The accuracy threshold needs to be set according to business requirements and data quality requirements; it represents the minimum similarity that the model-generated data should achieve with the actual data. For example, the system might set a threshold of 100 iterations and an accuracy threshold of 90%, meaning the model should achieve an accuracy of 90% or higher in the generated data within no more than 100 iterations.

[0101] During model training, the system continuously monitors the number of times each sample data is processed and the accuracy of the processing results. When it detects that the number of times a sample data is processed exceeds a threshold, and the accuracy of its processing results is consistently lower than a threshold, the system marks that sample data as an anomalous sample data. Subsequently, the system automatically removes these anomalous sample data from the training set to prevent them from continuing to interfere with the model's training process, ensuring that the model learns high-quality, representative data features.

[0102] This embodiment improves the quality of the training set by removing outlier sample data, avoiding the negative impact of inefficient or erroneous samples on model training. Furthermore, removing outlier sample data also helps improve the efficiency and accuracy of model training, as the model can focus more resources and attention on samples that effectively enhance its generation and discrimination capabilities.

[0103] Optionally, in the test data generation method provided in this application embodiment, the initial test data is tested based on the test product information to obtain the test result, including: determining the indicator value requirements of the test indicators based on the test product information, and using the indicator value requirements to test the initial test data; if the initial test data meets the indicator value requirements, the test result is determined to be without abnormality; if the initial test data does not meet the indicator value requirements, the test result is determined to be abnormal.

[0104] For example, after obtaining the test data, it is also necessary to determine the required index values, such as whether each value of the test data is within a preset range, and whether the data format meets the requirements, such as whether it is a positive number.

[0105] After the indicator value requirements are determined, if the test data all meet the indicator value requirements, it indicates that the test data is not abnormal. If the initial test data does not meet the indicator value requirements, the test result is determined to be abnormal, and the test data needs to be regenerated or the operation and maintenance section needs to intervene to process the test data, thereby improving the accuracy and effectiveness of the data.

[0106] This embodiment achieves the technical effect of improving the accuracy and usability of test data by detecting the test data.

[0107] Optionally, in the test data generation method provided in the embodiments of this application, feeding back the initial test data to the user terminal includes: obtaining the data receiving format of the user terminal, and standardizing the initial test data according to the data receiving format; encapsulating the standardized initial test data to obtain target test data, and feeding back the target test data to the user terminal.

[0108] For example, after obtaining the test data, the system first retrieves the user's data reception format requirements from the user's end. This may include the field order of the data, data type, whether a specific encoding format is required, and data label requirements. For instance, the user may require the data to be submitted in a preset format, with each row of data arranged chronologically and column names including fields such as "date", "product_id", and "sales_volume".

[0109] Furthermore, the system begins standardization processing of the initial test data. This processing mainly includes: Data cleaning: removing null and outlier values ​​from the generated data to improve its completeness and rationality. Format conversion: standardizing the format of the initial test data according to the user's data reception format requirements, such as converting date fields to YYYY-MM-DD format, and number fields to floating-point or integer types. Encoding standardization: if the user has specific encoding requirements for the received data (such as UTF-8), the system will perform encoding conversion to unify all data to the same encoding standard.

[0110] Furthermore, after standardization, the system encapsulates the initial test data to form target test data. The encapsulation process includes data encryption, file compression, and adding metadata, resulting in encapsulated target test data. This encapsulated target test data is then securely transmitted to the user, ensuring secure data transmission.

[0111] This embodiment achieves the technical effect of improving the usability of test data by standardizing the test data.

[0112] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0113] Example 2

[0114] This application also provides a test data generation apparatus. It should be noted that the test data generation apparatus of this application can be used to execute the test data generation method provided in the above embodiments. The test data generation apparatus provided in this application will be described below.

[0115] According to an embodiment of this application, an apparatus for implementing the above-described method for generating test data is also provided. Figure 4 This is a schematic diagram of the test data generation device provided in Embodiment 2 of this application, as shown below. Figure 4 As shown, the device includes:

[0116] The parsing unit 41 is used to receive data generation requests sent by the user terminal, parse the data generation requests, and obtain test scenario information and test product information.

[0117] The first determining unit 42 is used to determine the test indicators associated with the product to be tested based on the test product information, and input the test indicators and test scenario information into the test data generation model to obtain the initial test data under the test indicators.

[0118] The detection unit 43 is used to detect the initial test data according to the test product information, obtain the detection results, and feed back the initial test data to the user terminal if the detection results indicate that the initial test data is normal.

[0119] The test data generation apparatus provided in this application embodiment receives a data generation request sent by a user terminal through a parsing unit 41, and parses the data generation request to obtain test scenario information and test product information; a first determining unit 42 determines the test indicators associated with the product to be tested based on the test product information, and inputs the test indicators and test scenario information into the test data generation model to obtain initial test data under the test indicators; a detection unit 43 detects the initial test data based on the test product information, obtains the detection result, and if the detection result indicates that the initial test data is normal, feeds back the initial test data to the user terminal. This method involves receiving and parsing data generation requests from the user terminal to obtain test scenario and test product information. Based on the test product information, it determines the test indicators associated with the product under test and inputs these indicators and test scenario information into a test data generation model to obtain initial test data under these indicators. The initial test data is then tested against the test product information to obtain the test results. If the test results indicate that the initial test data is normal, it is fed back to the user terminal. By processing the test scenario and test product information using the test data generation model, test data with a high degree of similarity to real financial data and a correlation with the test product and test scenario is obtained. This achieves the goal of accurately generating test data, thereby improving the accuracy and authenticity of test data and solving the technical problem of low accuracy and authenticity in generated test data in related technologies.

[0120] Optionally, in the test data generation apparatus provided in this application embodiment, the first determining unit 42 includes: a first determining module, used to determine the business type to which the product to be tested belongs based on the test product information; a first obtaining module, used to obtain preset indicators of business products under the business type, and obtain multiple preset indicators; and a second determining module, used to determine the multiple preset indicators as test indicators associated with the product to be tested.

[0121] Optionally, in the test data generation apparatus provided in this application embodiment, the test data generation model is obtained through the following apparatus: a first acquisition unit, used to acquire multiple historical product information, historical economic scenario information of each historical product information within the product sales cycle, and historical indicator data under preset indicators related to the historical product information; a second determination unit, used to determine the historical economic scenario information, preset indicators, and historical indicator data under preset indicators of each historical product information as a sample data, thereby obtaining multiple sample data; a training unit, used to train the generator and discriminator of the conditional generative adversarial network model using multiple sample data, and construct the loss function of the conditional generative adversarial network model; and a third determination unit, used to determine the generator in the conditional generative adversarial network model as the test data generation model when the loss value of the loss function is less than a preset threshold.

[0122] Optionally, in the test data generation apparatus provided in this application embodiment, the training unit includes: a third determining module, used to determine the discriminator's recognition result of the sample data as a first loss function term; a fourth determining module, used to determine the discriminator's recognition result of the generator's output result as a second loss function term; and a combining module, used to combine the first loss function term, the second loss function term, and the preset financial penalty term into a loss function.

[0123] Optionally, in the test data generation apparatus provided in the embodiments of this application, the apparatus further includes: a second acquisition unit, used to acquire the number of times the test data generation model processes each sample data and the accuracy of the processing result obtained after each processing; and a fourth determination unit, used to determine that the sample data is abnormal sample data and delete the abnormal sample data from the training set when the number of processing is greater than the number threshold and the accuracy is less than the accuracy threshold.

[0124] Optionally, in the test data generation device provided in this application embodiment, the detection unit 43 includes: a fifth determining module, used to determine the indicator value requirements of the test indicators based on the test product information, and use the indicator value requirements to detect the initial test data; a sixth determining module, used to determine that the detection result is no abnormality when the initial test data meets the indicator value requirements; and a seventh determining module, used to determine that the detection result is abnormal when the initial test data does not meet the indicator value requirements.

[0125] Optionally, in the test data generation apparatus provided in this application embodiment, the detection unit 43 includes: a second acquisition module, used to acquire the data receiving format of the user terminal and perform standardization processing on the initial test data according to the data receiving format; and a feedback module, used to encapsulate the standardized initial test data to obtain target test data and feed the target test data back to the user terminal.

[0126] It should be noted that the parsing unit 41, the first determining unit 42, and the detection unit 43 mentioned above correspond to steps S201 to S203 in Embodiment 1. The instances and application scenarios implemented by each of the above units and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be part of a device and can run in the computer terminal 10 provided in Embodiment 1.

[0127] Example 3

[0128] Embodiments of this application may provide an electronic device. Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 (Only one is shown) processor 1002, memory 1004, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0129] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0130] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0131] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0132] Example 4

[0133] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the test data generation method provided in Embodiment 1.

[0134] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0135] Embodiments of this application also provide a computer program product, which, when executed on a data processing device, is adapted to perform the steps of a method for generating test data.

[0136] Embodiments of this application also provide a computer-readable storage medium, which includes a stored executable program, wherein the executable program controls the device where the computer-readable storage medium is located to execute the above-described method for generating test data when it runs.

[0137] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0138] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0140] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0142] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0143] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for generating test data, characterized in that, include: Receive data generation requests sent by the user terminal, and parse the data generation requests to obtain test scenario information and test product information; Based on the test product information, determine the test indicators associated with the product to be tested, and input the test indicators and the test scenario information into the test data generation model to obtain the initial test data under the test indicators; The initial test data is tested based on the test product information to obtain test results. If the test results indicate that the initial test data is normal, the initial test data is fed back to the user terminal.

2. The method according to claim 1, characterized in that, The test indicators associated with the product under test are determined based on the test product information, including: Determine the business type of the product to be tested based on the test product information; Obtain preset indicators that affect business products under the aforementioned business type, thus obtaining multiple preset indicators; The plurality of preset indicators are determined as the test indicators associated with the product to be tested.

3. The method according to claim 1, characterized in that, The test data generation model is obtained in the following way: Acquire multiple historical product information, historical economic scenario information of each historical product information within the product sales cycle, and historical indicator data under preset indicators related to the historical product information; Each historical product information is defined as a sample data point, consisting of historical economic scenario information, preset indicators, and historical indicator data under the preset indicators. Multiple sample data points are obtained by identifying the historical economic scenario information, preset indicators, and historical indicator data under the preset indicators for each historical product information. The generator and discriminator of the conditional generative adversarial network model are trained using the multiple sample data, and the loss function of the conditional generative adversarial network model is constructed. If the loss value of the loss function is less than a preset threshold, the generator in the conditional generative adversarial network model is determined as the test data generation model.

4. The method according to claim 3, characterized in that, The loss function for constructing the conditional generative adversarial network model includes: The discriminant's identification result of the sample data is determined as the first loss function term; The discriminator's identification result of the generator's output is determined as the second loss function term; The first loss function term, the second loss function term, and the preset financial penalty term are combined to form the loss function.

5. The method according to claim 3, characterized in that, The method further includes: The number of times the test data generation model processes each sample data and the accuracy of the processing result obtained after each processing are obtained. If the number of processing attempts exceeds a threshold and the accuracy is less than a threshold, the sample data is determined to be abnormal sample data, and the abnormal sample data is deleted from the training set.

6. The method according to claim 1, characterized in that, The initial test data is tested based on the test product information, and the test results include: The required values ​​of the test indicators are determined based on the test product information, and the initial test data are tested using the required values. If the initial test data meets the required index value, the test result is determined to be without abnormality; If the initial test data does not meet the required index value, the test result is determined to be abnormal.

7. The method according to claim 1, characterized in that, Feeding the initial test data back to the user terminal includes: Obtain the data receiving format of the user terminal, and standardize the initial test data according to the data receiving format; The standardized initial test data is encapsulated to obtain target test data, and the target test data is fed back to the user terminal.

8. A test data generation device, characterized in that, include: The parsing unit is used to receive data generation requests sent by the user terminal, and parse the data generation requests to obtain test scenario information and test product information; The first determining unit is used to determine the test indicators associated with the product to be tested based on the test product information, and input the test indicators and the test scenario information into the test data generation model to obtain the initial test data under the test indicators; The detection unit is used to detect the initial test data according to the test product information, obtain the detection result, and if the detection result indicates that the initial test data is normal, feed the initial test data back to the user terminal.

9. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the test data generation method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, executes the method for generating test data according to any one of claims 1 to 7.

Images