Simulation data generation method and device, equipment and storage medium

Abstract

The invention discloses a simulation data generation method and device, equipment and a storage medium, and relates to the technical field of deep learning, and the method comprises the steps: constructing an adversarial network generator based on a preset image data feature extraction model, a preset sequence data feature extraction model and a preset multi-modal fusion layer, constructing an initial adversarial network model based on the adversarial network generator and a preset adversarial network discriminator; generating a model training strategy based on a preset regularization strategy, and constructing a target loss function based on a data training task corresponding to the target simulation data field; and obtaining training data corresponding to the target simulation data field, inputting the training data into the initial adversarial network model, and training the initial adversarial network model based on a model training strategy and the target loss function to obtain a simulation data generation model, so as to generate target simulation data corresponding to the target simulation field based on the simulation data generation model. In this way, simulation data meeting the use requirements of specific application scenes can be generated.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a simulation data generation method, device, equipment and storage medium. Background Art

[0002] With the development of artificial intelligence technology, especially the progress of deep learning, Generative Adversarial Networks (GANs) have achieved remarkable results in the fields of image generation, speech synthesis, natural language processing, etc. GANs are composed of two neural networks: the generator and the discriminator. The generator is responsible for generating new samples similar to the training data from random noise, while the discriminator attempts to distinguish these generated samples from the real training data. Through the adversarial training of the two, the generator can gradually learn to generate high-quality simulated data.

[0003] However, in some application scenarios, such as financial risk assessment, medical and health analysis, the traditional GAN ​​architecture may not meet specific requirements, such as data privacy protection, quality and diversity control of generated data, etc. Therefore, a new data simulation model based on generative adversarial neural network is needed to solve these problems. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a simulation data generation method, device, equipment and storage medium, which can generate simulation data that meets the use requirements of specific application scenarios. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a method for generating simulation data, comprising:

[0006] Constructing an adversarial network generator based on a preset image data feature extraction model, a preset sequence data feature extraction model, and a preset multimodal fusion layer, and constructing an initial adversarial network model based on the adversarial network generator and a preset adversarial network discriminator;

[0007] Generate a model training strategy based on a preset regularization strategy, and construct a target loss function based on the data training task corresponding to the target simulation data domain;

[0008] The training data corresponding to the target simulation data domain is obtained and input into the initial adversarial network model, and then the initial adversarial network model is trained based on the model training strategy and the target loss function to obtain a simulation data generation model, so as to generate the target simulation data corresponding to the target simulation domain based on the simulation data generation model.

[0009] Optionally, the constructing an adversarial network generator based on a preset image data feature extraction model, a preset sequence data feature extraction model and a preset multimodal fusion layer includes:

[0010] A convolutional neural network is constructed based on a transposed convolutional layer, a residual block, and an attention mechanism to obtain a preset image data feature extraction model, and a preset sequence data feature extraction model is constructed based on a bidirectional long short-term memory network and a transformer structure;

[0011] An adversarial network generator is constructed based on the preset image data feature extraction model, the preset sequence data feature extraction model and the preset multimodal fusion layer; the preset multimodal fusion layer is used to fuse the features extracted by the preset image data feature extraction model and the preset sequence data feature extraction model to obtain target multimodal features.

[0012] Optionally, constructing an initial adversarial network model based on the adversarial network generator and a preset adversarial network discriminator includes:

[0013] A multi-scale feature extraction mechanism is added to the initial adversarial network discriminator to obtain a preset adversarial network discriminator, and an initial adversarial network model is constructed based on the adversarial network generator and the preset adversarial network discriminator.

[0014] Optionally, the generating a model training strategy based on a preset regularization strategy includes:

[0015] During the model training process, gradient penalty term technology, differential privacy technology, specific loss terms and feature matching technology are set to generate a model training strategy.

[0016] Optionally, the step of constructing a target loss function based on a data training task corresponding to a target simulation data domain includes:

[0017] Acquire a data training task corresponding to the target simulation data field, and generate a specific task loss function of the adversarial network generator based on the data training task;

[0018] Constructing a generator loss function of the adversarial network generator based on the specific task loss function and the cross entropy loss function, and constructing a discriminator loss function corresponding to the preset adversarial network discriminator based on the real data loss function and the generated data loss function;

[0019] The target loss function is determined using the generator loss function and the discriminator loss function.

[0020] Optionally, the method further includes:

[0021] A corresponding target constraint is set based on the target simulation data domain, and the target constraint is added to the target loss function.

[0022] Optionally, the acquiring training data corresponding to the target simulation data domain and inputting the training data into the initial adversarial network model, and then training the initial adversarial network model based on the model training strategy and the target loss function to obtain a simulation data generation model, includes:

[0023] Initializing the weights of the adversarial network generator and the preset adversarial network discriminator in the initial adversarial network model through a preset pre-training model to obtain an initialized adversarial network model;

[0024] The training data corresponding to the target simulation data field is obtained and input into the adversarial network model, and the adversarial network model is trained based on the model training strategy and the target loss function to obtain a simulation data generation model.

[0025] In a second aspect, the present application discloses a simulation data generating device, comprising:

[0026] A model building module, used to build an adversarial network generator based on a preset image data feature extraction model, a preset sequence data feature extraction model and a preset multimodal fusion layer, and to build an initial adversarial network model based on the adversarial network generator and a preset adversarial network discriminator;

[0027] A loss function determination module, used to generate a model training strategy based on a preset regularization strategy, and to construct a target loss function based on a data training task corresponding to a target simulation data domain;

[0028] A simulation data generation module is used to obtain the training data corresponding to the target simulation data field and input it into the initial adversarial network model, and then train the initial adversarial network model based on the model training strategy and the target loss function to obtain a simulation data generation model, so as to generate the target simulation data corresponding to the target simulation field based on the simulation data generation model.

[0029] In a third aspect, the present application discloses an electronic device, comprising:

[0030] Memory, used to store computer programs;

[0031] A processor is used to execute the computer program to implement the aforementioned simulation data generation method.

[0032] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program implements the aforementioned simulation data generation method when executed by a processor.

[0033] It can be seen that in this application, an adversarial network generator is constructed based on a preset image data feature extraction model, a preset sequence data feature extraction model and a preset multimodal fusion layer, and an initial adversarial network model is constructed based on the adversarial network generator and the preset adversarial network discriminator; a model training strategy is generated based on a preset regularization strategy, and a target loss function is constructed based on the data training task corresponding to the target simulation data field; the training data corresponding to the target simulation data field is obtained and input into the initial adversarial network model, and then the initial adversarial network model is trained based on the model training strategy and the target loss function to obtain a simulation data generation model, so as to generate the target simulation data corresponding to the target simulation field based on the simulation data generation model. That is, by integrating the data in the target simulation data field and constructing a multimodal adversarial network with image data feature and sequence data feature extraction functions, the generated simulation data can be made closer to the application scenarios of the real world; and by training the model through the preset regularization strategy generation model training strategy, the diversity and coverage of the generated data can be improved, thereby improving the quality of the generated simulation data. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0035] Figure 1 A flow chart of a simulation data generation method disclosed in this application;

[0036] Figure 2 A schematic diagram of a generator and a discriminator in a specific data simulation model disclosed in this application;

[0037] Figure 3 A flow chart of a specific simulation data generation method disclosed in this application;

[0038] Figure 4 A schematic diagram of the structure of a simulation data generating device disclosed in this application;

[0039] Figure 5 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] In application fields such as financial risk assessment and medical health analysis, the traditional GAN ​​architecture may not meet specific requirements, such as data privacy protection, quality and diversity control of generated data, etc. Therefore, this application will specifically introduce a simulation data generation method.

[0042] See also Figure 1 As shown, the embodiment of the present application discloses a simulation data generation method, including:

[0043] Step S11: constructing an adversarial network generator based on a preset image data feature extraction model, a preset sequence data feature extraction model and a preset multimodal fusion layer, and constructing an initial adversarial network model based on the adversarial network generator and a preset adversarial network discriminator.

[0044] In this embodiment, the adversarial network generator is constructed based on the preset image data feature extraction model, the preset sequence data feature extraction model and the preset multimodal fusion layer, including: constructing a convolutional neural network based on a transposed convolutional layer, a residual block and an attention mechanism to obtain a preset image data feature extraction model, and constructing a preset sequence data feature extraction model based on a bidirectional long short-term memory network and a transformer structure; constructing an adversarial network generator based on the preset image data feature extraction model, the preset sequence data feature extraction model and the preset multimodal fusion layer; the preset multimodal fusion layer is used to fuse the features extracted by the preset image data feature extraction model and the preset sequence data feature extraction model to obtain the target multimodal features.

[0045] In this embodiment, Figure 2As shown in the figure, the generator adopts a variety of structural designs. For the preset image data, it is different from the ordinary convolutional neural network (CNN) structure. It uses transposed convolutional layers to perform upsampling operations to restore the spatial resolution of the image. In addition, residual blocks and attention mechanisms are combined to improve the details and clarity of the generated images. By adding residual connections between certain layers of the generator, the network is allowed to directly learn the residual between the input and output, rather than the complete input-output mapping. In this way, it not only speeds up the training process, but also improves the accuracy and details of the generated data. Specifically, jump connections are added between multiple convolutional blocks of the generator, so that the input of each layer can directly skip several layers and then be added to the subsequent output, thereby maintaining the smooth transmission of information. For sequence data For time series data (such as speech, text, etc.), choose a recurrent neural network (RNN) or a long short-term memory network (LSTM) structure to better capture the temporal dependencies in the data. In order to better capture the forward and backward time series features, the present invention adopts a bidirectional long short-term memory network. For long sequence data, a transformer architecture is adopted to make full use of the good parallelism and information capture ability of the self-attention mechanism when processing long sequence data. If it is necessary to process multimodal data, such as image + text, this embodiment designs a multimodal generator, which contains different sub-networks for different types of data, and combines them through a fusion layer to extract features from data of different modalities.

[0046] In this embodiment, the construction of the initial adversarial network model based on the adversarial network generator and the preset adversarial network discriminator includes: adding a multi-scale feature extraction mechanism to the initial adversarial network discriminator to obtain the preset adversarial network discriminator, and constructing the initial adversarial network model based on the adversarial network generator and the preset adversarial network discriminator. Specifically, a multi-scale feature extraction mechanism is used in the discriminator to capture information at different levels. For example, feature maps at multiple resolution levels are used to train the discriminator. In addition to the basic true and false classification tasks, some auxiliary tasks are added to the discriminator, such as classification, regression, etc., so that not only the learning difficulty of the discriminator can be increased, but also the inherent structure of the data can be better understood. In addition, in order to make full use of the information in the data, a multi-task learning framework can be adopted to allow the discriminator to perform multiple tasks at the same time, thereby improving its comprehensive performance ability.

[0047] Step S12: Generate a model training strategy based on a preset regularization strategy, and construct a target loss function based on a data training task corresponding to a target simulation data domain.

[0048] In this embodiment, iterative training is performed by minimizing the loss function of the generator and maximizing the loss function of the discriminator. Specifically, the goal of the generator is to maximize the misclassification rate of the discriminator for the generated samples, while the goal of the discriminator is to maximize the correct classification rate of the real samples and the generated samples. Finally, the target loss function is obtained. Among them, the generation model training strategy based on the preset regularization strategy includes: setting the gradient penalty term technology, differential privacy technology, setting specific loss terms and feature matching technology in the model training process to generate the model training strategy. Specifically, a gradient penalty term can be added during the training process to limit the gradient norm of the discriminator to alleviate the problem of mode collapse; by matching the statistics of the real data and the generated data, the generator learns not only the distribution of the data, but also the high-level features of the data; by introducing specific loss terms to control the diversity and coverage of the generated data, such as using a distance metric to ensure the difference between the generated data points; using differential privacy technology, adding noise to blur the influence of the training data during the training process, which can be achieved by modifying the loss function or the gradient update rule.

[0049] In this embodiment, the target loss function is constructed based on the data training task corresponding to the target simulation data field, including: obtaining the data training task corresponding to the target simulation data field, and generating the specific task loss function of the adversarial network generator based on the data training task; constructing the generator loss function of the adversarial network generator based on the specific task loss function and the cross entropy loss function, and constructing the discriminator loss function corresponding to the preset adversarial network discriminator based on the real data loss function and the generated data loss function; using the generator loss function and the discriminator loss function to determine the target loss function. Specifically, the loss function of the generator usually includes two parts: cross entropy loss and loss of a specific task. Cross entropy loss is to measure the authenticity of the generated samples by using cross entropy loss. The specific task loss is to add classification loss, regression loss, etc. according to the application scenario to ensure that the generated data can meet the needs of a specific task. The loss function of the discriminator is also composed of two parts: the loss of real data and the loss of generated data. The loss of real data is used to maximize the probability of the discriminator correctly classifying real data and generated data. The loss of generated data is used to minimize the misclassification rate of the discriminator to generated data. In addition, corresponding target constraints can be set based on the target simulation data domain, and the target constraints can be added to the target loss function. That is, a set of constraints are defined to ensure that the generated data meets specific attributes or restrictions; by adding specific items to the loss function, the generator is guided to generate data with specific attributes. For example, for medical imaging data, anatomical constraints can be added to the loss function to ensure that the generated imaging data conforms to the human anatomical structure. For example, the anatomical correctness of the generated image can be evaluated by a pre-trained segmentation network, and the evaluation results can be fed back to the generator to improve the quality of the generated image. In the financial risk assessment scenario, financial indicators and economic models can be integrated to ensure that the generated data conforms to specific economic logic. For example, a trend prediction model based on historical data can be added to guide the generator to generate more reasonable financial data. In the autonomous driving scenario, road rules and traffic regulations can be added as constraints to ensure that the generated simulated driving environment conforms to actual driving conditions. For example, a path planning algorithm can be used to guide the generator to generate a road layout that conforms to traffic rules.

[0050] Step S13: Obtain the training data corresponding to the target simulation data field and input it into the initial adversarial network model, and then train the initial adversarial network model based on the model training strategy and the target loss function to obtain a simulation data generation model, so as to generate the target simulation data corresponding to the target simulation field based on the simulation data generation model.

[0051] In this embodiment, the training data corresponding to the target simulation data domain is obtained and input into the initial adversarial network model, and then the initial adversarial network model is trained based on the model training strategy and the target loss function to obtain a simulation data generation model, including: initializing the weights of the adversarial network generator and the preset adversarial network discriminator in the initial adversarial network model by a preset pre-training model to obtain the initialized adversarial network model; obtaining the training data corresponding to the target simulation data domain and inputting it into the adversarial network model, and training the adversarial network model based on the model training strategy and the target loss function to obtain the simulation data generation model. That is, the weights of the generator and the discriminator are initialized by using the pre-training model to obtain the initialized adversarial network model, and the pre-training model includes but is not limited to convolutional neural networks, recurrent neural networks, Transformers, etc. During the training process, the update frequency of the generator and the discriminator is dynamically adjusted according to the performance of the discriminator to maintain the balance of training. In addition, the best hyperparameter combination is selected by grid search and Bayesian optimization methods to obtain the optimal model performance. After the simulation data generation model is obtained, the simulation data generation model may be used to generate target simulation data corresponding to the target simulation field.

[0052] In addition, in order to cope with changes in data distribution, the simulation data production model in this embodiment has an online learning mechanism that can update itself when new data arrives. In order to reduce the computational cost and storage space of the model, the simulation data production model is compressed using model pruning, quantization and other technologies. During the model operation process, the performance of the model is monitored in real time to promptly identify potential problems and make necessary adjustments.

[0053] It can be seen that in this embodiment, Figure 3As shown, an adversarial network generator is constructed based on a preset image data feature extraction model, a preset sequence data feature extraction model and a preset multimodal fusion layer, and an initial adversarial network model is constructed based on the adversarial network generator and the preset adversarial network discriminator; a model training strategy is generated based on a preset regularization strategy, and a target loss function is constructed based on the data training task corresponding to the target simulation data field; the training data corresponding to the target simulation data field is obtained and input into the initial adversarial network model, and then the initial adversarial network model is trained based on the model training strategy and the target loss function to obtain a simulation data generation model, so as to generate the target simulation data corresponding to the target simulation field based on the simulation data generation model. That is, by integrating the data in the target simulation data field and constructing a multimodal adversarial network with image data feature and sequence data feature extraction functions, the generated simulation data can be made closer to the application scenarios of the real world; and by training the model through the preset regularization strategy generation model training strategy, the diversity and coverage of the generated data can be improved, thereby improving the quality of the generated simulation data.

[0054] refer to Figure 4 The embodiment of the present application also discloses a simulation data generating device, including:

[0055] A model building module 11 is used to build an adversarial network generator based on a preset image data feature extraction model, a preset sequence data feature extraction model and a preset multimodal fusion layer, and to build an initial adversarial network model based on the adversarial network generator and a preset adversarial network discriminator;

[0056] A loss function determination module 12, for generating a model training strategy based on a preset regularization strategy, and constructing a target loss function based on a data training task corresponding to a target simulation data domain;

[0057] The simulation data generation module 13 is used to obtain the training data corresponding to the target simulation data field and input it into the initial adversarial network model, and then train the initial adversarial network model based on the model training strategy and the target loss function to obtain a simulation data generation model, so as to generate the target simulation data corresponding to the target simulation field based on the simulation data generation model.

[0058] In this embodiment, by integrating data in the target simulation data field and constructing a multimodal adversarial network with image data feature and sequence data feature extraction functions, the generated simulation data can be made closer to the application scenarios of the real world; and by training the model through a preset regularization strategy to generate a model training strategy, the diversity and coverage of the generated data can be improved, thereby improving the quality of the generated simulation data.

[0059] In some specific embodiments, the model building module 11 may specifically include:

[0060] A feature extraction model construction unit, used to construct a convolutional neural network based on a transposed convolutional layer, a residual block and an attention mechanism to obtain a preset image data feature extraction model, and to construct a preset sequence data feature extraction model based on a bidirectional long short-term memory network and a transformer structure;

[0061] A generator construction unit is used to construct an adversarial network generator based on the preset image data feature extraction model, the preset sequence data feature extraction model and the preset multimodal fusion layer; the preset multimodal fusion layer is used to fuse the features extracted by the preset image data feature extraction model and the preset sequence data feature extraction model to obtain target multimodal features.

[0062] In some specific embodiments, the model building module 11 may specifically include:

[0063] The adversarial network construction unit is used to add a multi-scale feature extraction mechanism to the initial adversarial network discriminator to obtain a preset adversarial network discriminator, and to construct an initial adversarial network model based on the adversarial network generator and the preset adversarial network discriminator.

[0064] In some specific embodiments, the loss function determination module 12 may specifically include:

[0065] The regularization strategy determination unit is used to set the gradient penalty term technology, differential privacy technology, specific loss terms and feature matching technology during the model training process to generate a model training strategy.

[0066] In some specific embodiments, the loss function determination module 12 may specifically include:

[0067] A first loss function determination unit, used for acquiring a data training task corresponding to a target simulation data domain, and generating a specific task loss function of the adversarial network generator based on the data training task;

[0068] A second loss function determination unit, used to construct a generator loss function of the adversarial network generator based on the specific task loss function and the cross entropy loss function, and to construct a discriminator loss function corresponding to the preset adversarial network discriminator based on the real data loss function and the generated data loss function;

[0069] An objective function determination unit is used to determine an objective loss function using the generator loss function and the discriminator loss function.

[0070] In some specific embodiments, the simulation data generating device may further include:

[0071] The constraint condition adding module is used to set corresponding target constraints based on the target simulation data field and add the target constraints to the target loss function.

[0072] In some specific embodiments, the simulation data generating module 13 may specifically include:

[0073] A model initialization unit, used to initialize the weights of the adversarial network generator and the preset adversarial network discriminator in the initial adversarial network model through a preset pre-training model to obtain an initialized adversarial network model;

[0074] A model training unit is used to obtain training data corresponding to the target simulation data field and input it into the adversarial network model, and train the adversarial network model based on the model training strategy and the target loss function to obtain a simulation data generation model.

[0075] Furthermore, the present application also discloses an electronic device. Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be considered as any limitation on the scope of use of the present application.

[0076] Figure 5 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the simulation data generation method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0077] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0078] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0079] The operating system 221 is used to manage and control the hardware devices on the electronic device 20 and the computer program 222, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the simulation data generation method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0080] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the simulation data generation method disclosed above. The specific steps of the method can refer to the corresponding contents disclosed in the above embodiments, and will not be repeated here.

[0081] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0082] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0083] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0084] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0085] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for generating simulation data, characterized in that: include: Constructing an adversarial network generator based on a preset image data feature extraction model, a preset sequence data feature extraction model, and a preset multimodal fusion layer, and constructing an initial adversarial network model based on the adversarial network generator and a preset adversarial network discriminator; Generate a model training strategy based on a preset regularization strategy, and construct a target loss function based on the data training task corresponding to the target simulation data domain; The training data corresponding to the target simulation data domain is obtained and input into the initial adversarial network model, and then the initial adversarial network model is trained based on the model training strategy and the target loss function to obtain a simulation data generation model, so as to generate the target simulation data corresponding to the target simulation domain based on the simulation data generation model.

2. The simulation data generation method according to claim 1, characterized in that: The adversarial network generator is constructed based on a preset image data feature extraction model, a preset sequence data feature extraction model and a preset multimodal fusion layer, including: A convolutional neural network is constructed based on a transposed convolutional layer, a residual block, and an attention mechanism to obtain a preset image data feature extraction model, and a preset sequence data feature extraction model is constructed based on a bidirectional long short-term memory network and a transformer structure; An adversarial network generator is constructed based on the preset image data feature extraction model, the preset sequence data feature extraction model and the preset multimodal fusion layer; the preset multimodal fusion layer is used to fuse the features extracted by the preset image data feature extraction model and the preset sequence data feature extraction model to obtain target multimodal features.

3. The simulation data generation method according to claim 1, characterized in that: The constructing of an initial adversarial network model based on the adversarial network generator and the preset adversarial network discriminator includes: A multi-scale feature extraction mechanism is added to the initial adversarial network discriminator to obtain a preset adversarial network discriminator, and an initial adversarial network model is constructed based on the adversarial network generator and the preset adversarial network discriminator.

4. The simulation data generation method according to claim 1, characterized in that: The generating model training strategy based on the preset regularization strategy includes: During the model training process, gradient penalty term technology, differential privacy technology, specific loss terms and feature matching technology are set to generate a model training strategy.

5. The simulation data generation method according to claim 1, characterized in that: The target loss function is constructed based on the data training task corresponding to the target simulation data field, including: Acquire a data training task corresponding to the target simulation data field, and generate a specific task loss function of the adversarial network generator based on the data training task; Constructing a generator loss function of the adversarial network generator based on the specific task loss function and the cross entropy loss function, and constructing a discriminator loss function corresponding to the preset adversarial network discriminator based on the real data loss function and the generated data loss function; The target loss function is determined using the generator loss function and the discriminator loss function.

6. The method for generating simulation data according to claim 5, characterized in that: Also includes: A corresponding target constraint is set based on the target simulation data domain, and the target constraint is added to the target loss function.

7. The simulation data generation method according to claim 1, characterized in that: The step of obtaining the training data corresponding to the target simulation data domain and inputting the training data into the initial adversarial network model, and then training the initial adversarial network model based on the model training strategy and the target loss function to obtain a simulation data generation model includes: Initializing the weights of the adversarial network generator and the preset adversarial network discriminator in the initial adversarial network model through a preset pre-training model to obtain an initialized adversarial network model; The training data corresponding to the target simulation data field is obtained and input into the adversarial network model, and the adversarial network model is trained based on the model training strategy and the target loss function to obtain a simulation data generation model.

8. A simulation data generating device, characterized in that: include: A model building module, used to build an adversarial network generator based on a preset image data feature extraction model, a preset sequence data feature extraction model and a preset multimodal fusion layer, and to build an initial adversarial network model based on the adversarial network generator and a preset adversarial network discriminator; A loss function determination module, used to generate a model training strategy based on a preset regularization strategy, and to construct a target loss function based on a data training task corresponding to a target simulation data domain; A simulation data generation module is used to obtain the training data corresponding to the target simulation data field and input it into the initial adversarial network model, and then train the initial adversarial network model based on the model training strategy and the target loss function to obtain a simulation data generation model, so as to generate the target simulation data corresponding to the target simulation field based on the simulation data generation model.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the simulation data generating method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the simulation data generating method according to any one of claims 1 to 7.

Images