Simulation data generation method and device and storage medium

Abstract

The invention discloses a simulation data generation method and device and a storage medium. The simulation data generation method comprises the steps that a to-be-simulated data table and fields contained in the to-be-simulated data table are determined based on a simulation prompt text; generating task execution nodes based on fields contained in the to-be-simulated data tables, and determining an execution sequence between the task execution nodes based on the dependency relationship between the to-be-simulated data tables; generating a to-be-executed task in combination with each task execution node and the execution sequence between the task execution nodes; according to the method, the to-be-executed task is executed, and the data simulation result is obtained based on the execution result of the to-be-executed task, so that complex business logic can be split into independent or associated task execution nodes, the simulation data generation process is optimized, the data generation cost is reduced, and the data generation speed is increased; and a large amount of simulation data can be generated only according to the simulation prompt text, so that the method has the data generation capability of'no middle life ', and is suitable for various business scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a simulation data generation method, device and storage medium. BACKGROUND

[0002] Effective training of a machine learning model relies on a large amount of sample data, and the quality of the sample data affects the training effect of the machine learning model. However, in order to protect user personal privacy and sensitive information, large-scale data collection is often not possible, and techniques such as data augmentation, synthetic data generation, and data labeling tools are generally used to improve the richness and diversity of the data, thereby ensuring sufficient data support for model training.

[0003] However, data augmentation has problems such as model training overfitting, data information loss, uncontrollable enhancement quality, and high computational cost; synthetic data generation has low timeliness, authenticity, and reliability, and the calculation process is complex, the scene diversity is insufficient, and the demand for computing resources is high; data labeling tools have inconsistent labeling quality, high computational cost, poor privacy protection effect, and limited application scenarios. SUMMARY

[0004] To solve the above technical problems, the present application provides at least a simulation data generation method, device and storage medium to generate high-quality data.

[0005] The first aspect of the present application provides a simulation data generation method, the method comprising: obtaining a simulation prompt text, determining a to-be-simulated data table and fields contained in the to-be-simulated data table based on the simulation prompt text; generating a task execution node based on the fields contained in the to-be-simulated data table, the task execution node being used to generate data of the fields in the to-be-simulated data table, and determining an execution order between each task execution node based on a dependency relationship between each to-be-simulated data table; combining each task execution node and the execution order between each task execution node to generate a to-be-executed task; executing the to-be-executed task, and obtaining a data simulation result based on an execution result of the to-be-executed task.

[0006] In an embodiment, determining the to-be-simulated data table and the fields contained in the to-be-simulated data table based on the simulation prompt text comprises: performing keyword extraction on the simulation prompt text to obtain initial keywords; calculating a similarity between the initial keywords and standard words pre-stored in a factor library, selecting standard words satisfying a preset condition based on the similarity, and obtaining a factor text; determining a data table and fields matched with the factor text to obtain the to-be-simulated data table and the fields contained in the to-be-simulated data table.

[0007] In an embodiment, the data table and the field matched with the element text are determined to obtain the to-be-simulated data table and the field contained in the to-be-simulated data table, including: taking the data table and the field matched with the element text as an initial data table and a field contained in the initial data table; performing intent recognition on the simulation prompt text, and taking the data table and the field matched with the intent recognition result as a supplementary data table and a field contained in the supplementary data table; determining the data table relied on by the to-be-simulated data table and the supplementary data table based on a preset dependency relationship between the data tables, to obtain a basic data table and a field contained in the basic data table; and combining the initial data table and the field contained in the initial data table, the supplementary data table and the field contained in the supplementary data table, and the basic data table and the field contained in the basic data table to obtain the to-be-simulated data table and the field contained in the to-be-simulated data table.

[0008] In an embodiment, the to-be-executed task is executed, and a data simulation result is obtained based on an execution result of the to-be-executed task, including: selecting sample corpus from a sample corpus library based on the simulation prompt text to obtain target corpus; performing keyword extraction on the target corpus to obtain corpus keywords; and filling the corpus keywords into fields of the to-be-simulated data table as field values to obtain the data simulation result.

[0009] In an embodiment, the target corpus is obtained by selecting sample corpus from a sample corpus library based on the simulation prompt text, including: performing intent recognition on the simulation prompt text to obtain an intent recognition result; and selecting sample corpus matched with the intent recognition result from the sample corpus library to obtain the target corpus.

[0010] In an embodiment, the to-be-executed task is further used to indicate a data type that needs to be simulated; and the corpus keywords are filled into the fields of the to-be-simulated data table as field values to obtain the data simulation result, including: filling the corpus keywords into the fields of the to-be-simulated data table as field values to obtain initial data; calling a constructor matched with the data type based on the data type that needs to be simulated indicated by the to-be-executed task; wherein the constructor is used to generate data of a specified data type; and inputting the initial data into the constructor, and taking data of the data type output by the constructor as the data simulation result.

[0011] In an embodiment, the number of to-be-executed tasks is a plurality; the to-be-executed tasks are executed, and a data simulation result is obtained based on an execution result of the to-be-executed tasks, including: detecting whether there is a task execution node for simulating the same field between each to-be-executed task, and dividing the to-be-executed tasks having the task execution node for simulating the same field into a same associated task set; for each to-be-executed task in the associated task set, extracting the same field for unified simulation and other fields for separate simulation to obtain the data simulation result.

[0012] In an embodiment, after the task to be executed is executed and the data simulation result is obtained based on the execution result of the task to be executed, the method further comprises: determining a target database matched with the data simulation result based on a data type to which the data simulation result belongs, to obtain the target database; and calling a database interface corresponding to the target database to write the data simulation result into the target database.

[0013] The second aspect of the present application provides a simulation data generation device, the device comprising: a table and field determination module configured to obtain a simulation prompt text, determine a table to be simulated and fields contained in the table to be simulated based on the simulation prompt text; a node generation module configured to generate a task execution node based on the fields contained in the table to be simulated, the task execution node being configured to generate data of the fields in the table to be simulated and determine an execution order between each task execution node based on a dependency relationship between each table to be simulated; a task generation module configured to generate a task to be executed in combination with each task execution node and the execution order between each task execution node; and a task execution module configured to execute the task to be executed and obtain a data simulation result based on an execution result of the task to be executed.

[0014] The third aspect of the present application provides an electronic device comprising a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the simulation data generation method described above.

[0015] The fourth aspect of the present application provides a computer readable storage medium having program instructions stored thereon, the program instructions being configured to be executed by a processor to implement the simulation data generation method described above.

[0016] The above scheme determines the table to be simulated and the fields contained in the table to be simulated based on the simulation prompt text, generates the task execution node based on the fields contained in the table to be simulated and determines the execution order between each task execution node based on the dependency relationship between each table to be simulated, generates the task to be executed in combination with each task execution node and the execution order between each task execution node, executes the task to be executed, and obtains the data simulation result based on the execution result of the task to be executed, which can split the complex business logic into independent or associated task execution nodes, ensures the clarity of each step in the data simulation process, optimizes the process of generating simulation data, reduces the cost of data generation, improves the speed of data generation, and only needs to generate a large amount of simulation data according to the simulation prompt text, has the data generation ability of "creating something out of nothing", and is suitable for various business scenarios.

[0017] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present application. BRIEF DESCRIPTION OF DRAWINGS

[0018] The drawings incorporated in and forming a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0019] Figure 1 is a schematic diagram of a scheme implementation environment shown by an exemplary embodiment of the present application;

[0020] Figure 2 is a flow chart of a simulation data generation method shown by an exemplary embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a visualization page of simulation data generation shown by an exemplary embodiment of the present application;

[0022] Figure 4 is a block diagram of a simulation data generation apparatus shown by an exemplary embodiment of the present application;

[0023] Figure 5 is a structural schematic diagram of an electronic device shown by an exemplary embodiment of the present application;

[0024] Figure 6 is a structural schematic diagram of a computer readable storage medium shown by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0025] The scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0026] In the following description, specific details are set forth in order to provide a thorough understanding of the present application. However, persons having ordinary skill in the art will appreciate that the present application can be practiced without the specific details.

[0027] The term "and / or" herein is merely an associated information of associated objects, and means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally means that the front and rear associated objects are in an "or" relationship. In addition, "multiple" herein means two or more than two. In addition, the term "at least one" herein means any one of multiple or any combination of at least two of multiple, for example, including at least one of A, B and C can mean including any one or more elements selected from the set consisting of A, B and C.

[0028] The simulation data generation method provided by the embodiments of the present application will be described below.

[0029] Reference is made to Figure 1 , Figure 1is a schematic diagram of a scheme implementation environment shown by an example embodiment of the present application. The scheme implementation environment can include a terminal 110, a server 120 and a database 130, which are communicatively connected with each other.

[0030] The number of terminals 110 can be one or more. The terminal 110 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto.

[0031] The server 120 can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc.

[0032] In an example, the terminal 110 sends a simulation prompt text to the server 120, and the server 120 can perform simulation data generation processing according to the simulation prompt text obtained from the terminal 110 to obtain a data simulation result. Of course, the server 120 can store the data simulation result locally, return it to the terminal 110 or transmit it to other terminals, such as storing the data simulation result to the database 130.

[0033] In an example, the terminal 110 has a client of a target application installed and running therein, such as an application providing a simulation data generation function. Based on the target application, simulation data is generated to obtain a data simulation result. The server 120 can be a background server of the target application, configured to provide background services for the client of the target application.

[0034] The simulation data generation method provided by the embodiments of the present application can be executed by the terminal 110, such as a client of a target application installed and running in the terminal 110, or by the server 120, or by the terminal 110 and the server 120 in cooperation.

[0035] Please refer to Figure 2 , Figure 2 is a flowchart of a simulation data generation method shown by an example embodiment of the present application. The simulation data generation method can be applied to Figure 1The method is shown in the implementation environment and is specifically performed by the server in the implementation environment. It should be understood that the method can also be applicable to other exemplary implementation environments and be specifically performed by devices in other implementation environments, and the present embodiment does not limit the implementation environment to which the method is applicable.

[0036] As Figure 2 The simulation data generation method includes at least steps S210 to S240, which are described in detail as follows.

[0037] Step S210: Obtain a simulation prompt text, and determine a to-be-simulated data table and fields contained in the to-be-simulated data table based on the simulation prompt text.

[0038] The simulation prompt text refers to a text used to assist in simulation data generation. For example, the simulation prompt text can be "find a vehicle registered for mortgage in A place".

[0039] For example, a terminal displays a visual page for simulation data generation. In response to a text input operation detected by the visual page, the terminal obtains input text corresponding to the text input operation, takes the input text as the simulation prompt text, and transmits the simulation prompt text to a server through a preset application programming interface (API).

[0040] Of course, in addition to the manner of obtaining the simulation prompt text, other manners of obtaining the simulation prompt text can also be used, such as pre-setting a prompt text template and generating the simulation prompt text by randomly filling the prompt text template. The present application does not limit the specific generation manner of the simulation prompt text.

[0041] After obtaining the simulation prompt text, the simulation prompt text is subjected to text analysis, and a data table required in a data simulation process and fields contained in the data table are determined according to an analysis result of the simulation prompt text, to obtain the to-be-simulated data table and the fields contained in the to-be-simulated data table.

[0042] For example, the current simulation prompt text is "find a vehicle registered for mortgage in A place". Through text analysis of the simulation prompt text, it can be known that the data table required in the data simulation process includes a vehicle mortgage information table and a vehicle attribute table, the vehicle attribute table contains fields such as vehicle type and color, and the vehicle mortgage information table contains fields such as location and time.

[0043] Step S220: Generate a task execution node based on the fields contained in the to-be-simulated data table, the task execution node being used to generate data of the fields in the to-be-simulated data table, and determine an execution order between each task execution node based on a dependency relationship between each to-be-simulated data table.

[0044] The task execution node represents a specific data generation step for generating data of a field in the to-be-simulated data table.

[0045] Optionally, each task execution node can include one or more to-be-simulated data tables required in the data simulation process, such as an entity attribute table, and / or an entity location information table, and other related data tables.

[0046] Optionally, each task execution node can also define the fields contained in the to-be-simulated data table. For example, the task execution node for generating data of a field in the entity attribute table can define each field to facilitate subsequent field data simulation.

[0047] Optionally, each task execution node can also define the business system to which the to-be-simulated data table belongs, such as a vehicle product management system or a traffic management system, to determine the data source and application scenario.

[0048] Optionally, each task execution node can also define the storage system corresponding to the to-be-simulated data table, such as a relational database or a data lake, to indicate the storage location of each to-be-simulated data table, facilitating subsequent data extraction and processing.

[0049] The to-be-simulated data tables can have a dependency relationship. Specifically, a change in data in a to-be-simulated data table can cause a change in data in another to-be-simulated data table, indicating that the two to-be-simulated data tables have a dependency relationship.

[0050] For example, the vehicle attribute table corresponding to a vehicle records the basic attribute information of the vehicle, and the vehicle mortgage information table contains part of the fields in the vehicle attribute table (such as vehicle model). A change in data in the vehicle attribute table can cause a change in data in the vehicle mortgage information table, so the vehicle mortgage information table depends on the vehicle attribute table.

[0051] For to-be-simulated data tables with a dependency relationship, in order to ensure the data integrity and accuracy of each data table, the execution order between each task execution node needs to be determined according to the dependency relationship between each to-be-simulated data table.

[0052] For example, the vehicle mortgage information table depends on the vehicle attribute table, so the vehicle attribute table needs to be generated before the vehicle mortgage information table. That is, the task execution node for generating data of a field in the vehicle attribute table needs to be executed first to ensure the data integrity and accuracy of the vehicle attribute table.

[0053] Step S230: Generate to-be-executed tasks in combination with each task execution node and the execution order between each task execution node.

[0054] According to the all task execution nodes and the execution sequence between the task execution nodes determined in the above embodiments, the final to-be-executed task is generated.

[0055] The number of the generated to-be-executed tasks can be one or more, and the to-be-executed tasks determine the information such as data tables and fields required by the simulation, such as the to-be-executed task for generating the vehicle attribute table and the to-be-executed task for generating the vehicle mortgage information table.

[0056] Step S240: executing the to-be-executed task, and obtaining the data simulation result based on the execution result of the to-be-executed task.

[0057] The to-be-executed task explicitly indicates the specific data generation steps required to be executed through the task execution nodes, and the execution sequence between the data generation steps through the execution sequence between the task execution nodes.

[0058] The to-be-executed task is executed, and the execution result of the to-be-executed task is obtained, and the data simulation result is obtained.

[0059] The simulation prompt text is used to determine the to-be-simulated data table and the fields contained in the to-be-simulated data table, the task execution nodes are generated according to the fields contained in the to-be-simulated data table, the execution sequence between each task execution node is determined based on the dependency relationship between each to-be-simulated data table, the to-be-executed task is generated by combining each task execution node and the execution sequence between each task execution node, the data simulation result is obtained by executing the to-be-executed task, the complex business logic is split into independent or associated task execution nodes, the clarity of each step in the data simulation process is ensured, the process of generating simulation data is optimized, the cost of data generation is reduced, the speed of data generation is improved, and a large amount of simulation data can be generated only according to the simulation prompt text, which has the data generation ability of "creating something out of nothing" and is suitable for various business scenarios.

[0060] Next, some embodiments of the present application will be described in detail.

[0061] In some embodiments, the step S210 of determining the to-be-simulated data table and the fields contained in the to-be-simulated data table based on the simulation prompt text includes the following steps S211 to S213.

[0062] Step S211: keyword extraction is performed on the simulation prompt text to obtain initial keywords.

[0063] The keywords in the simulation prompt text are extracted, and the extracted keywords are used as initial keywords.

[0064] The keyword extraction can be based on statistics, and / or a graph model, and / or a deep learning method, etc.

[0065] For example, a neural network model with text analysis function is pre-trained, the simulation prompt text is input into the neural network model, the keywords output by the neural network model are obtained, and the initial keywords are obtained. The neural network model can be a Transformer model or a BERT (Bidirectional Encoder Representations from Transformers) model, and the present application does not limit this.

[0066] Step S212: Calculate the similarity between the initial keywords and the standard words pre-stored in the element library, select the standard words whose similarity meets the preset condition, and obtain the element text.

[0067] Since the extracted initial keywords may not match the real business scenario, the initial keywords need to be standardized to facilitate the execution of subsequent steps.

[0068] Specifically, the similarity between the initial keywords and the standard words pre-stored in the element library is calculated, and the standard words whose similarity meets the preset condition are selected to obtain the element text corresponding to the simulation prompt text.

[0069] The standard word refers to a word used in a business scenario, which conforms to the industry standard.

[0070] Among them, it can be judged whether the similarity is greater than the preset similarity threshold, and the standard word greater than the preset similarity threshold is selected as the standard word whose similarity meets the preset condition; it can also be the standard word with the largest similarity, which is selected as the standard word whose similarity meets the preset condition, and the present application does not limit this.

[0071] For example, the similarity between the initial keywords and the standard words pre-stored in the element library is calculated using a pre-trained generative pre-trained transformer (GPT), and the standard words whose similarity meets the preset condition are selected. After selecting the standard words for each initial keyword, the element text is obtained by combining all the selected standard words.

[0072] For example, the current simulation prompt text is "find the vehicle registered for mortgage in A", and the initial keywords obtained by keyword extraction may include "mortgage registration", "A", and "vehicle". Through similarity matching, the standard word corresponding to "mortgage registration" is "mortgage registration".

[0073] Step S213: Determine the data table and field matched with the element text, and obtain the to-be-simulated data table and the fields contained in the to-be-simulated data table.

[0074] After obtaining the element text, the element text is converted into data tables and fields required in the data simulation process to obtain a to-be-simulated data table and fields contained in the to-be-simulated data table.

[0075] Specifically, the element text is converted into corresponding entities (such as vehicles, locations, etc.), attributes (which refer to basic information of the entities), relationships (which refer to association relationships between the entities), labels (such as mortgage registration, use period, etc.), and the like, to obtain data tables and fields matched with the element text.

[0076] For example, the current simulation prompt text is "find a vehicle registered for mortgage in A location", and the element text obtained after standardization includes "mortgage registration", "A location", and "vehicle". After conversion, the JSON (JavaScript Object Notation) text is obtained as follows:

[0077]

[0078] That is, the data tables matched with the element text include a vehicle attribute table and a vehicle mortgage information table.

[0079] The data tables and fields matched with the element text can be directly used as the to-be-simulated data table and the fields contained in the to-be-simulated data table. The data tables and fields determined directly according to the simulation prompt text can be incomplete, and therefore, the data tables and fields can be further supplemented.

[0080] Exemplarily, the data tables and fields matched with the element text are used as initial data tables and fields contained in the initial data tables; intent recognition is performed on the simulation prompt text, and data tables and fields matched with the intent recognition result are used as supplementary data tables and fields contained in the supplementary data tables; based on preset dependency relationships between the data tables, data tables on which the to-be-simulated data tables and the supplementary data tables depend are determined to obtain basic data tables and fields contained in the basic data tables; and the initial data tables and the fields contained in the initial data tables, the supplementary data tables and the fields contained in the supplementary data tables, and the basic data tables and the fields contained in the basic data tables are combined to obtain the to-be-simulated data table and the fields contained in the to-be-simulated data table.

[0081] For example, the intent recognition result is obtained by directly performing intent recognition on the simulation prompt text and / or performing intent recognition on the element text corresponding to the simulation prompt text.

[0082] The intent recognition result is used to further determine whether other data tables are required in the data simulation process, to obtain the supplementary data tables and the fields contained in the supplementary data tables.

[0083] For example, the intent recognition result is obtained by analyzing semantic information related to each entity and association relationship information between the entities in the simulation prompt text.

[0084] For example, the current simulation prompt text is "find the vehicle registered for mortgage in A", the initial data table determined by the element text includes the vehicle attribute table and the vehicle mortgage information table, the initial intent obtained by semantic analysis is "need to construct the mortgage information of the vehicle", further analysis obtains the intent recognition result "need to construct the mortgage information of the vehicle in A and the use information corresponding to the vehicle and the like", and according to the intent recognition result, the supplementary data table includes the vehicle use information table. Wherein, the field corresponding to the supplementary data table can be obtained according to the intent recognition result, or the default field is obtained as the field corresponding to the supplementary data table.

[0085] By extracting the intent in the simulation prompt text, fine data simulation is realized, and the accuracy and diversity of the data simulation result are improved.

[0086] In addition, according to the preset dependency relationship between each data table, the data table on which the to-be-simulated data table and the supplementary data table depend is further determined, and the basic data table and the field contained in the basic data table are obtained.

[0087] For example, the vehicle attribute table depends on the vehicle technical parameter table, so the vehicle technical parameter table is taken as the basic data table. Wherein, the field corresponding to the basic data table can be obtained according to the intent recognition result, or the default field is obtained as the field corresponding to the basic data table.

[0088] The initial data table and the field contained in the initial data table, the supplementary data table and the field contained in the supplementary data table, the basic data table and the field contained in the basic data table are comprehensively obtained, and all the to-be-simulated data tables and the fields contained in these to-be-simulated data tables are obtained.

[0089] Then, after determining the to-be-simulated data table and the field contained in the to-be-simulated data table according to the above embodiment, the corresponding to-be-executed task is generated, and the to-be-executed task is executed to generate the simulation data.

[0090] In some embodiments, the number of to-be-executed tasks in step S240 is multiple; the to-be-executed task is executed, and the data simulation result is obtained based on the execution result of the to-be-executed task, including the following steps S2411 to S2412.

[0091] Step S2411: detecting whether there is a task execution node for simulating the same field between each to-be-executed task, and dividing the to-be-executed task with the task execution node for simulating the same field into the same associated task set.

[0092] Step S2412: for each to-be-executed task in the associated task set, the same field is extracted for unified simulation, and other fields are simulated separately to obtain the data simulation result.

[0093] The to-be-executed tasks are divided into associated tasks and non-associated tasks.

[0094] When there is simulation of the same field between multiple to-be-executed tasks, it is determined that the to-be-executed tasks are associated with each other, and are divided into the same associated task set to extract the same field for unified simulation.

[0095] For example, there is simulation of the same field between the data table 1 and the data table 2, such as simulation of the entity identifier and the timestamp, at this time, the same format and standard are used for the same field, and the consistency between the finally filled field values is ensured, and unified simulation is realized.

[0096] Of course, other fields are simulated separately to improve the diversity of data.

[0097] And the to-be-executed tasks without the same field are divided into non-associated tasks, and the fields in the non-associated tasks are also simulated separately.

[0098] The above embodiments can ensure the consistency and accuracy of the finally simulated data by unified simulation of the same field, can improve the flexibility of data simulation by independent simulation of different fields, can improve the robustness of simulated data generation, and each to-be-executed task can be simulated based on its own attribute characteristics and business requirements to ensure processing efficiency, which is conducive to adaptive data generation in different scenarios.

[0099] In some embodiments, the to-be-executed task is executed in step S240, and a data simulation result is obtained based on the execution result of the to-be-executed task, including the following steps S2421 to S2423.

[0100] Step S2421: based on the simulation prompt text, selecting a sample corpus from a sample corpus library to obtain a target corpus.

[0101] The sample corpus library contains multiple sample corpora.

[0102] According to the simulation prompt text, the sample corpus is selected from the sample corpus library, and the selected sample corpus is used as the target corpus.

[0103] For example, if the intent recognition result is “need to construct the mortgage information of the vehicle within the preset time and the use information corresponding to the vehicle”, the sample corpus library can be screened out from the sample corpus library. The sample corpus related to the description of the vehicle use information is used as the target corpus.

[0104] For example, if the intent recognition result is “need to construct the mortgage information of the vehicle within the preset time and the use information corresponding to the vehicle”, the sample corpus library can be screened out from the sample corpus library. The sample corpus related to the description of the vehicle use information is used as the target corpus.

[0105] Step S2422: keyword extraction is performed on the target corpus to obtain corpus keywords.

[0106] The keyword extraction manner of the target corpus is similar to the keyword extraction manner of the simulation prompt text, and is not described herein.

[0107] Step S2423: the corpus keywords are filled as field values into the fields of the to-be-simulated data table to obtain a data simulation result.

[0108] For example, the to-be-simulated data table is a vehicle use information table, sample corpora related to vehicle use information description are filtered out from the sample corpus library, and vehicle use life, annual inspection records and other corpus keywords are extracted. The data simulation result is obtained by filling the corpus keywords as field values of the corresponding fields in the vehicle use information and filling the corpus keywords into the fields of the to-be-simulated data table.

[0109] The field values can be flexibly filled through the sample corpora, and the diversity of the generated simulation data is improved.

[0110] Optionally, in addition to the field value filling according to the sample corpora in the above embodiments, default values can also be directly filled for some fields, for example, the vehicle production location is set as the address xxx by default, and the type of the vehicle is set as a small car by default. In combination with the field values extracted from the sample corpora and the field values set by default, an information-complete and expandable data table is constructed.

[0111] In some embodiments, the to-be-executed task is also used to indicate a data type that needs to be simulated; the corpus keywords are filled as field values into the fields of the to-be-simulated data table to obtain a data simulation result, including: the corpus keywords are filled as field values into the fields of the to-be-simulated data table to obtain initial data; a constructor matched with the data type is called based on the data type that needs to be simulated indicated by the to-be-executed task; and the data belonging to the data type output by the constructor is input into the constructor, and the data is taken as the data simulation result.

[0112] The constructor is used to generate data of a specified data type.

[0113] Different constructors can be set according to the specific needs of application scenarios, such as the type of a database where the data simulation result is finally stored and / or the data type to which the data simulation result belongs. The constructor can be flexibly set, and developers can dynamically extend the constructor according to actual business needs to improve the adaptability and scalability of the system.

[0114] For example, the constructor can define the basic attributes of the task to be executed, such as the task name, task type, execution time, etc. of the task to be executed; the constructor can also define methods, such as generation methods related to the task to be executed, and / or execution methods related to the task to be executed, and / or state detection of the task to be executed, and / or data storage methods, etc. That is, the overall process of data simulation can be managed through the constructor.

[0115] The data type to be simulated is obtained, such as vehicle attribute information, and / or vehicle mortgage information, and / or vehicle use information, etc.

[0116] Different constructors can generate simulation data of different data types, such as a vehicle attribute information constructor, a vehicle mortgage information constructor, and / or a vehicle use information constructor, etc.

[0117] The initial data is integrated and / or data type conversion is performed through the constructor to obtain the final data simulation result.

[0118] In some embodiments, after step S240, the method further includes steps S250 to S260.

[0119] Step S250: based on the data type to which the data simulation result belongs, determining a database matched with the data simulation result to obtain a target database.

[0120] Step S260: calling a database interface corresponding to the target database to write the data simulation result into the target database.

[0121] According to the data simulation result obtained by the above-mentioned embodiments, an API corresponding to the target database is called to perform data writing, for example, image data is written into cloud storage, structured data is written into a relational database, and unstructured data is written into a file.

[0122] Since data simulation may result in multiple data tables of one type, multiple data tables can be integrated into one base table to ensure integration of various data sources, such as unified storage of data from different data sources through standardized interfaces. In addition, when each data table is output, accurate field value mapping is performed on each field to ensure that the corresponding field value of each field is found. Further, the data format or data type of the data table from different data sources is different, and the data format or data type of the data table from multiple data sources can also be converted to ensure that multiple data tables loaded into the base table can meet the expected format and type, thereby avoiding errors caused by format or type mismatch.

[0123] In the above embodiments, by integrating multiple types of databases, the storage requirements of various types of data can be flexibly met, large-scale video data storage and retrieval can be supported, high throughput and low latency access speed can be ensured, real-time video stream processing and archiving functions can be realized, subsequent playback and analysis are facilitated, and the combination of relational databases and non-relational databases can effectively manage various structured data and unstructured data generated during the simulation data generation process. Various query languages can also be supported to provide fast data retrieval and analysis capabilities, including complex queries and statistical report generation.

[0124] Optionally, in order to show the user the data generation situation, the number of data generations can also be counted every N seconds, and the number of data generations can be written into the database for calling and showing by the user-side visualization page.

[0125] By way of example, please refer to Figure 3 , Figure 3 is a schematic diagram of a visualization page for simulation data generation according to an example embodiment of the present application, as shown in Figure 3 The visualization page provides an input interface for simulation prompt text, and the visualization page can provide a data type selection button to select the data type to be simulated.

[0126] In addition, the visualization page can also display the simulation data generation log, which records the execution of each step of the data simulation. The running state, historical events and key operations of the simulation data generation task can be detected in real time, and the required information can be quickly obtained. In addition, log retrieval can also be provided to support keyword search, time range filtering and multi-condition combination query of the simulation data generation log, etc., to help users quickly locate related logs and improve the efficiency of fault diagnosis and data analysis.

[0127] The visualization page can also display the data simulation result. When displaying the data simulation result, icons, statistical data, and visualization dashboards can be used to display the data simulation result. For example, the statistical results of the data simulation result can be displayed, such as the number of generated vehicle attribute information, the number of vehicle mortgage information, the number of vehicle use information, etc.

[0128] The visualization page can also extract data from the data simulation result according to the set standard or demand rule, such as selecting the data with the highest matching degree with the intent recognition result from the data simulation result as the display sample, and displaying the display sample in the visualization page to select representative data as the display sample for in-depth analysis and testing to ensure the effectiveness and scientificity of the generated data.

[0129] The simulation data generation method provided in the application determines a to-be-simulated data table and fields contained in the to-be-simulated data table based on a simulation prompt text; generates a task execution node based on the fields contained in the to-be-simulated data table, and determines an execution order between each task execution node based on a dependency relationship between each to-be-simulated data table; generates a to-be-executed task in combination with each task execution node and the execution order between each task execution node; executes the to-be-executed task, and obtains a data simulation result based on an execution result of the to-be-executed task. The complex business logic can be split into independent or associated task execution nodes, the clarity of each step in the data simulation process is ensured, the process of generating simulation data is optimized, the cost of data generation is reduced, the speed of data generation is improved, a large amount of simulation data can be generated only according to the simulation prompt text, the data generation capability of "creating something out of nothing" is possessed, and the method is suitable for various business scenarios.

[0130] Figure 4 is a block diagram of a simulation data generation device shown in an exemplary embodiment of the application. As shown in the figure, the exemplary simulation data generation device 400 includes: Figure 4

[0131] a table and field determination module 410, configured to acquire a simulation prompt text, and determine a to-be-simulated data table and fields contained in the to-be-simulated data table based on the simulation prompt text;

[0132] a node generation module 420, configured to generate a task execution node based on the fields contained in the to-be-simulated data table, the task execution node being used to generate data of the fields in the to-be-simulated data table, and determine an execution order between each task execution node based on a dependency relationship between each to-be-simulated data table;

[0133] a task generation module 430, configured to generate a to-be-executed task in combination with each task execution node and the execution order between each task execution node;

[0134] a task execution module 440, configured to execute the to-be-executed task, and obtain a data simulation result based on an execution result of the to-be-executed task.

[0135] It should be noted that the simulation data generation device provided in the above embodiments and the simulation data generation method provided in the above embodiments belong to the same concept, and the specific manner in which each module and unit performs operations has been described in detail in the method embodiments, which will not be described here. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above, which is not limited here.

[0136] Please refer to Figure 5 , Figure 5 ​is a structural schematic diagram of an embodiment of an electronic device of the present application. The electronic device 500 comprises a memory 501 and a processor 502, the processor 502 being configured to execute program instructions stored in the memory 501 to implement the steps in any of the above-mentioned embodiments of the method for generating simulation data. In a specific implementation scenario, the electronic device 500 can include but is not limited to a microcomputer, a server, and in addition, the electronic device 500 can also include a notebook computer, a tablet computer and other mobile devices, which are not limited herein.

[0137] Specifically, the processor 502 is configured to control itself and the memory 501 to implement the steps in any of the above-mentioned embodiments of the method for generating simulation data. The processor 502 can also be referred to as a central processing unit (CPU). The processor 502 can be an integrated circuit chip having a processing capability of signals. The processor 502 can also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like. In addition, the processor 502 can be implemented by an integrated circuit chip together.

[0138] Please refer to Figure 6 , Figure 6 is a structural schematic diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 600 stores program instructions 610 capable of being executed by a processor, the program instructions 610 being configured to implement the steps in any of the above-mentioned embodiments of the method for generating simulation data.

[0139] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For the sake of brevity, they will not be described here.

[0140] The above description of each embodiment tends to emphasize the differences between each embodiment, and the same or similar parts can be mutually referred to. For the sake of brevity, they will not be described here.

[0141] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other manners. For example, the division of the apparatus embodiments described above is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0142] In addition, each function unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of a software function unit. If the integrated unit is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various other media that can store program codes.

Claims

1. A method for generating simulation data, characterized in that: The method comprises: Acquire a simulation prompt text, and determine a data table to be simulated and fields contained in the data table to be simulated based on the simulation prompt text; Generate task execution nodes based on the fields contained in the data table to be simulated, wherein the task execution nodes are used to generate data of the fields in the data table to be simulated, and determine the execution order between each task execution node based on the dependency relationship between each data table to be simulated; Generate a task to be executed by combining each task execution node and the execution order between each task execution node; The task to be executed is executed, and a data simulation result is obtained based on the execution result of the task to be executed.

2. The method according to claim 1, characterized in that The determining of the data table to be simulated and the fields contained in the data table to be simulated based on the simulation prompt text includes: Extracting keywords from the simulation prompt text to obtain initial keywords; Calculating the similarity between the initial keyword and the standard words pre-stored in the element library, selecting the standard words whose similarity meets the preset conditions, and obtaining the element text; The data table and fields matching the element text are determined, and the data table to be simulated and the fields contained in the data table to be simulated are obtained.

3. The method according to claim 2, characterized in that The step of determining a data table and fields that match the element text, and obtaining a data table to be simulated and fields contained in the data table to be simulated, includes: The data table and fields matching the element text are used as the initial data table and the fields contained in the initial data table; Performing intent recognition on the simulation prompt text, and using the data table and fields that match the intent recognition result as the supplementary data table and the fields contained in the supplementary data table; Based on the preset dependency relationship between the various data tables, determining the data tables on which the data table to be simulated and the supplementary data table depend, and obtaining a basic data table and the fields contained in the basic data table; The data table to be simulated and the fields contained in the data table to be simulated are obtained by combining the initial data table and the fields contained in the initial data table, the supplementary data table and the fields contained in the supplementary data table, and the basic data table and the fields contained in the basic data table.

4. The method according to claim 1, wherein The executing the task to be executed and obtaining a data simulation result based on the execution result of the task to be executed includes: Based on the simulation prompt text, sample corpus is selected from a sample corpus to obtain target corpus; Performing keyword extraction on the target corpus to obtain corpus keywords; The corpus keywords are filled as field values ​​into the fields of the data table to be simulated to obtain data simulation results.

5. The method according to claim 4, characterized in that The step of selecting a sample corpus from a sample corpus based on the simulation prompt text to obtain a target corpus includes: Performing intent recognition on the simulation prompt text to obtain an intent recognition result; A sample corpus that matches the intention recognition result is selected from the sample corpus to obtain the target corpus.

6. The method according to claim 4, characterized in that The task to be executed is also used to indicate the type of data to be simulated; the corpus keywords are filled as field values ​​into the fields of the data table to be simulated to obtain data simulation results, including: Filling the corpus keywords as field values ​​into the fields of the to-be-simulated data table to obtain initial data; Based on the data type that needs to be simulated indicated by the task to be executed, calling a constructor that matches the data type; wherein the constructor is used to generate data of the specified data type; The initial data is input into the constructor, and the data belonging to the data type output by the constructor is used as a data simulation result.

7. The method according to claim 1, characterized in that There are multiple tasks to be executed; and executing the tasks to be executed and obtaining data simulation results based on the execution results of the tasks to be executed includes: Detect whether there are task execution nodes for simulating the same field between each to-be-executed task, and divide the to-be-executed tasks that have task execution nodes for simulating the same field into the same associated task set; For each task to be executed in the associated task set, the same fields are extracted for unified simulation, and other fields are simulated separately to obtain data simulation results.

8. The method according to claim 1, characterized in that After executing the task to be executed and obtaining a data simulation result based on the execution result of the task to be executed, the method further includes: Based on the data type of the data simulation result, determining a database matching the data simulation result to obtain a target database; The database interface corresponding to the target database is called to write the data simulation result into the target database.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, and the processor is used to execute program instructions stored in the memory to implement the steps in the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program instructions, and the program instructions can be executed by a processor to implement the steps in the method according to any one of claims 1 to 8.

Images