Automatic data filling method, data filling tool, equipment and storage medium
By a configuration module parsing the configuration file, the data generation module generates target fill data, and the data output module outputs data to the specified location, solving the problem of low automation efficiency of existing data fill tools and achieving efficient and flexible automatic data fill.
Patent Information
- Application Number
- CN202510130117.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-05
- Publication Date
- 2025-06-10
AI Technical Summary
The existing data filling tools are inefficient in automation, and require users to manually enter information, and the filling rules are single, resulting in low data filling efficiency.
Provides a data automatic filling method, which obtains task configuration information by parsing the configuration file through the configuration module. The data generation module generates target filling data based on the configuration information, and the data output module outputs data to a designated location.
It reduces the workload of users to manually configure fill rules, improves the flexibility and ease of use of data fill tools, and realizes automated data fill.
Smart Images
Figure CN120124604A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of data generation, and particularly to a method for automatically filling data, a data filling tool, a device, and a storage medium. Background Art
[0002] A data filling tool is software or a tool used to automatically generate and fill data. It is used to quickly generate a large amount of test data or simulation data in a database and automatically create data that conforms to specific rules and patterns. Currently, most data filling tools are static filling tools. Such tools usually use preset algorithms and rules to automatically generate and fill into specified positions based on existing data or information input by users. However, such tools often require users to manually input some information, and the filling rules are relatively single, resulting in low automation efficiency of data filling. Summary of the Invention
[0003] This application provides a method for automatically filling data, a data filling tool, a device, and a storage medium, which can achieve automated data filling. The technical solution is as follows:
[0004] According to one aspect of this application, a method for automatically filling data is provided. The method is applied to a data filling tool, and the data filling tool includes a configuration module, a data generation module, and a data output module. The method includes:
[0005] Parsing a configuration file through the configuration module to obtain task configuration information, where the task configuration information at least includes data rules, data volume, output format, and output position of the data to be filled;
[0006] Reading the task configuration information from the configuration module through the data generation module;
[0007] Generating target filling data based on the task configuration information through the data generation module, and sending the target filling data to the data output module;
[0008] Receiving the target filling data through the data output module, and writing the target filling data to the target output position.
[0009] According to another aspect of this application, a data filling tool is provided. The data filling tool includes a configuration module, a data generation module, and a data output module:
[0010] The configuration module is used to parse a configuration file to obtain task configuration information, where the task configuration information at least includes data rules, data volume, output format, and output position of the data to be filled;
[0011] The data generation module is configured to read the task configuration information from the configuration module;
[0012] The data generation module is further configured to generate target filling data based on the task configuration information, and send the target filling data to the data output module;
[0013] The data output module is configured to receive the target filling data and write the target filling data to the target output location.
[0014] According to one aspect of the present application, there is provided an electronic device, including: a processor and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to execute the data automatic filling method as described above.
[0015] According to another aspect of the present application, there is provided a non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to cause the computer to execute the data automatic filling method as described above.
[0016] According to another aspect of the present application, there is provided a computer program product, the computer program product including computer instructions, the computer instructions being stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the above data automatic filling method.
[0017] The beneficial effects brought by the technical solution provided by the embodiments of the present application at least include:
[0018] By providing a data filling toolkit including a configuration module, a data generation module, and a data output module, the configuration module parses the configuration file provided by the user to obtain task configuration information, such as data rules, data volume, output format, and output location, etc.; and then the data generation module automatically generates target filling data based on the task configuration information, and the data output module outputs the target filling data to the target output location. A data filling tool that can parse configuration information and automatically generate target filling data is provided, thereby reducing the workload of the user manually configuring filling rules and improving the flexibility and usability of the data filling tool. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In the following description of the exemplary embodiments in conjunction with the drawings, more details, features, and advantages of the present application are disclosed. In the drawings:
[0020] Figure 1 A flowchart of a data automatic filling method according to an exemplary embodiment of the present application is shown;
[0021] Figure 2Shows a flowchart of another method for automatic data filling according to an exemplary embodiment of the present application;
[0022] Figure 3 Shows a flowchart of another method for automatic data filling according to an exemplary embodiment of the present application;
[0023] Figure 4 Is a schematic structural diagram of a data filling tool provided by an embodiment of the present application;
[0024] Figure 5 Shows a structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present application. Detailed implementation manners
[0025] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not used to limit the protection scope of the present application.
[0026] It should be understood that the various steps recorded in the method embodiments of the present application can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this regard.
[0027] The term "including" and its variants used herein are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first" and "second" mentioned in the present application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions executed by these devices, modules or units. It should be noted that the modifications of "one" and "multiple" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more". The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only for illustrative purposes and are not used to limit the scope of these messages or information.
[0028] The solutions of the present invention are described below with reference to the accompanying drawings, and the technical solutions provided by the embodiments of the present application are described in detail through specific embodiments and their application scenarios.
[0029] In view of the problem of low automation efficiency of data filling under the related art static filling tool, the embodiment of the present application provides a data filling tool. By using this data filling tool, automated data filling can be achieved, and it is not necessary for the user to manually configure the data information to be filled, reducing the user's manual configuration operation.
[0030] Please refer to Figure 1 , which shows a flowchart of a data automatic filling method according to an exemplary embodiment of the present application. This method is described by taking its application to a data filling tool as an example. As Figure 1 shown, the method includes:
[0031] Step 101, parse the configuration file through the configuration module to obtain task configuration information, where the task configuration information at least includes the data rule, data volume, output format, and output location of the data to be filled.
[0032] The data filling tool provided by the embodiment of the present application includes a configuration module, a data generation module, and a data output module. Among them, the configuration module is an important component in the data filling tool, responsible for parsing the configuration information provided by the user and generating a unified data structure for other modules to use. For example, by parsing the CREATE TABLE statement, the configuration module automatically extracts content such as table field information, type, and filling rules, forming an object structure convenient for subsequent module processing. The parsed field information may include default values, auto-increment rules, and range limits, etc., which are further applied by the data generation module, thereby reducing the workload of the user's manual configuration of filling rules and improving the flexibility and usability of the tool.
[0033] In a possible implementation manner, the user can configure the data table structure and field rules (or data rules) in the configuration file in advance. The field rules may include field type, default value, auto-increment rule, value range, data volume, output format, and output location. Then the data filling tool reads the configuration file and parses the configuration file through the configuration module to extract relevant task configuration information. Exemplarily, the task configuration information at least includes the data rule (or field rule), data volume, output format, and output location of the data to be filled. Optionally, the configuration file provides diverse data filling options, supporting diverse configurations of data rules (field rules), including default values, auto-increment values (including auto-increment of VARCHAR type), random value filling within a range, to meet richer test scenarios; in addition, the configuration file also provides customization of the data volume, for example, 10 filling data can be generated; it also provides configuration of the data output format, and provides configuration of the number of files for saving the generated filling data, that is, in the case of a large amount of filling data, it can be configured to save the generated filling data in multiple files in a sharded manner.
[0034] Step 102, the data generation module reads task configuration information from the configuration module.
[0035] Among them, the data generation module receives the configuration information in the configuration module and generates data in a specified format as needed. It supports generating SQL insert statements or.csv file content, and fills the generated data into the asynchronous channel chan for use by the output module. For rules such as auto-increment primary keys, the data generation module uses the methods of data splitting and multi-channel concurrency to generate data, greatly improving the generation speed and efficiency, supporting large-scale data generation tasks, and ensuring that all generated data complies with the rules and constraints in the configuration.
[0036] After the configuration module extracts task configuration information from the configuration file, the data generation module reads the task configuration information from the configuration module, and then creates a data generation task and a data output task according to the task configuration information, so as to complete the generation of the target filling data by executing the data generation task and the data output task.
[0037] Step 103, the data generation module generates target filling data based on the task configuration information and sends the target filling data to the data output module.
[0038] Among them, the data output module is responsible for receiving the chan channel data from the data generation module and performing output operations. It can choose to write the generated target filling data into the database through a MySQL connection according to the task configuration information, or write the data into a.csv file that complies with RFC 180 specifications.
[0039] Corresponding to the data generation module, based on the read task configuration information, it generates the indicated target filling data and sends the target Tianshan Red data to the data output module.
[0040] Step 104, the data output module receives the target filling data and writes the target filling data to the target output location.
[0041] After the data output module receives the target filling data, it can write the target filling data to the target output location according to the output location indicated by the task configuration information. Specifically, the data output module monitors the chan channel from the data generation module in real time and outputs the target filling data to the target output location. The data output module decides whether to import the data into the MySQL database or write it to a.csv file according to the task configuration. When choosing to import into the database, the data output module connects to MySQL and executes the INSERT statement into the database to achieve real-time data insertion; if output to a.csv file, the data output module writes the data into a.csv file that complies with the RFC 4180 specification. If it is expected that the amount of data to be generated is large, the number of generated files can also be configured. At this time, the.csv file will be sharded according to the configuration to avoid a single file being too large.
[0042] In summary, the embodiment of the present application provides a data automatic filling method: by providing a data filling toolkit including a configuration module, a data generation module, and a data output module, the configuration module parses the configuration file provided by the user to obtain task configuration information, which includes data rules, data volume, output format, output location, etc.; furthermore, the data generation module automatically generates target filling data based on the task configuration information, and the data output module outputs the target filling data to the target output location. A data filling tool that can parse configuration information and automatically generate target filling data is provided, thereby reducing the workload of users manually configuring filling rules and improving the flexibility and usability of the data filling tool.
[0043] Please refer to Figure 2 , which shows a flowchart of another data automatic filling method according to an exemplary embodiment of the present application. This method is described by taking its application to a data filling tool as an example. As Figure 2 shown, this method includes:
[0044] Step 201, parse the configuration file through the configuration module to obtain task configuration information, where the task configuration information includes at least data rules, data volume, output format, and output location of the data to be filled.
[0045] Step 202, read the task configuration information from the configuration module through the data generation module.
[0046] Among them, the data filling tool further includes a task management and scheduling module. The task management and scheduling module is the core scheduling component of the data filling tool, and automatically generates and manages data generation tasks and data output tasks through the task configuration information. The task management and scheduling module can schedule multiple tasks to execute in parallel, such as setting multiple.csv file output tasks, adjusting task priorities and concurrency numbers according to requirements, and supporting more refined configurations.
[0047] In a possible implementation, after the configuration module parses the configuration file to obtain task configuration information, the task management and scheduling module can read the task configuration information from the configuration module, generate at least one data filling task according to the user requirements indicated by the task configuration information. The data filling task includes a data generation task and a data output task, and then send the data filling task to the data generation module and the data output module. The data generation task and the data output task can specify the amount of data to be generated, the output format, the target database, etc. The task management and scheduling module sorts and assigns priorities to the data generation task and the data filling task according to the task configuration information. Optionally, the task management and scheduling module also generates a unique task ID for each data generation task and data output task to ensure the accuracy of task scheduling and status tracking.
[0048] Optionally, after receiving the data generation task, the data generation task includes the task configuration information.
[0049] Step 203, generate target filling data based on data rules, output format and data volume through the data generation module, and send the target filling data to the data output module.
[0050] Optionally, the data generation module generates the target filling data that meets the requirements line by line according to the data rules, output format and data volume. Specifically, if the output format indicates generating SQL data, the data generation module generates the target filling data in the form of INSERT statements based on the data rules and data volume; if the output format indicates generating a CSV file, the data generation module generates the target filling data in the form of a string array based on the data rules and data volume.
[0051] To improve performance, the data generation module runs in an asynchronous mode, and the data is transmitted through the chan channel. The data generation module dynamically creates multiple chan channels as needed to generate the target filling data in parallel; it is especially suitable for cases where the auto-increment primary key can be sharded, which can avoid concurrent problems due to the auto-increment of the primary key and maintain high performance. The data generation module continuously fills the generated target filling data into the chan for the data output module to read in real time.
[0052] Step 204, receive the target filling data through the data output module, and write the target filling data to the target output location.
[0053] The corresponding data output module receives the target filled data output by the data generation module and writes the target filled data to the target output location based on the output location in the task configuration information. Specifically, if the target output location is a MySQL database, the data output module writes the target filled data in the form of an INSERT statement to the MySQL database; if the target output location is a CSV file, the data output module writes the target filled data in the form of a string array to the CSV file.
[0054] Considering that the generated target filled data may have a large amount of data, in order to avoid a single file being too large, it is necessary to store the target filled data in slices. Optionally, writing the target filled data in the form of a string array to a CSV file by the data output module may further include steps 204B1 to 204B3.
[0055] Step 204B1, if the target data volume of the target filled data is greater than the preset data volume, determine the target number of files based on the target data volume.
[0056] Step 204B2, create CSV files corresponding to the target number of files.
[0057] Step 204B3, slice the target filled data according to the target number of files, and write the sliced target filled data into the CSV files.
[0058] In a possible implementation manner, it is automatically determined by the data filling tool whether slicing storage is required. By comparing whether the target data volume of the target filled data is greater than the preset data volume, if it is greater than the preset data volume, the target number of files to be sliced can be determined based on the target data volume, and then CSV files corresponding to the target number of files are created, the target filled data is sliced according to the target number of files, and the sliced target filled data is written into each CSV file.
[0059] Exemplarily, the preset data volume can be 100 entries. If the target data volume (data volume) indicated by the task configuration information is 500 entries and the target data volume is greater than the preset data volume, it is determined that the generated target filled data needs to be stored in slices. Assuming that each CSV file stores 100 entries, the target number of sliced files can be determined to be 5.
[0060] Optionally, the target number of files can also be configured manually, that is, the user has pre-configured the target number of files in the configuration file. The data output module obtains the target number of files from the parsed task configuration information, and can create CSV files corresponding to the target number of files when outputting the CSV file.
[0061] In this embodiment, by parsing the configuration module, the data volume, output format, and output location indicated by the task configuration information can be obtained, enabling the data generation module to generate corresponding target filling data based on the task configuration information, and the data output module to output the target filling data to the target output location based on the task configuration information, so as to achieve automated data generation and data output; moreover, when outputting to the target output location, multi-format output is supported, and it is possible to support data import into a MySQL database or generate a CSV file compliant with the RFC 4180 standard, realizing cross-database data import.
[0062] To improve the management efficiency of data filling tasks, in addition to the above-mentioned configuration module, task management and scheduling module, data generation module, and data output module, the data filling tool further includes a progress monitoring module to display the data filling progress of the current data filling task to the user in real time.
[0063] Please refer to Figure 3 , which shows a flowchart of another data automatic filling method according to an exemplary embodiment of the present application. This method is described by taking its application to a data filling tool as an example. As Figure 3 shown, this method includes:
[0064] Step 301, parse the configuration file through the configuration module to obtain task configuration information, where the task configuration information at least includes the data rules, data volume, output format, and output location of the data to be filled.
[0065] Step 302, read the task configuration information from the configuration module through the data generation module.
[0066] Step 303, generate target filling data based on the task configuration information through the data generation module, and send the target filling data to the data output module.
[0067] Step 304, receive the target filling data through the data output module, and write the target filling data to the target output location.
[0068] The implementation manners of steps 301 to 304 can refer to the above embodiments, and will not be elaborated herein.
[0069] Step 305, obtain the task status information from the data generation module and the data output module through the progress monitoring module.
[0070] Among them, the data filling tool further includes a progress monitoring module, which real-time tracks the execution status of each data filling task, summarizes the generated and output progress data, and provides feedback information such as clear task completion status, current speed, and estimated completion time. By comprehensively presenting the status data of the generation and output tasks, the user can understand the execution progress of the task in real time.
[0071] In a possible implementation, the data filling tool can obtain the task status information of each data filling task from the data generation module and the data output module through the progress monitoring module; for each data filling task, the task status information at least includes the generated data volume of the target filling data that the data generation module has currently generated, the current generation speed, and the current number of processed rows; the output data volume of the target filling data that the data output module has currently output, the current output speed, and the current number of processed rows, etc.
[0072] Step 306, predict the task completion time based on the task status information through the progress monitoring module.
[0073] After obtaining the task status information, the progress monitoring module can calculate the processing speed of the data filling task according to the generated data volume, the output data volume, the current generation speed, and the current output speed in the task status information, and then predict the task completion time of the data filling task according to the processing speed and the target data volume of the target filling data to be generated.
[0074] Step 307, display the task completion time through the progress monitoring module.
[0075] In order to enable the user to clearly understand the task completion status of the data filling task, the task completion time can be displayed through the progress monitoring module. Optionally, the completion percentage of the data filling task can also be determined according to the generated data volume, the output data volume, and the target data volume, and the completion percentage can be displayed while displaying the task completion time.
[0076] Exemplarily, the progress monitoring module can display the task completion time: it is expected to take another 3 minutes, and the completion percentage: 85%.
[0077] In this embodiment, in order to improve the management efficiency of the data filling task, in addition to the above-mentioned configuration module, task management and scheduling module, data generation module, and data output module, the data filling tool further includes a progress monitoring module, which predicts the task completion time and the completion percentage through the task status information obtained from the data generation module and the data output module, so as to display the data filling progress of the current data filling task to the user in real time.
[0078] In a possible implementation, the overall operation logic of the data filling tool is as follows (the data filling tool is composed of a configuration module, a task management and scheduling module, a data generation module, a data output module, and a progress monitoring module):
[0079] After the tool is started, the configuration module first loads the configuration information provided by the user and parses the database table structure and field rules (such as field types, default values, auto-increment rules, value ranges, etc.). When parsing the CREATE TABLE statement, the configuration module automatically extracts the table structure, converts the field information into a unified data structure object for subsequent modules to directly reference, thus reducing the user's manual configuration work.
[0080] Subsequently, the task management and scheduling module reads the task configuration information, generates a corresponding task list according to the user's needs, including data generation tasks and data output tasks. The tasks can specify the amount of data to be generated, the output format, the target database, etc. The scheduling module sorts and assigns priorities to the tasks according to the configuration information. The task management and scheduling module generates a unique task ID for each task to ensure the accuracy of task scheduling and status tracking.
[0081] After the task management and scheduling module is started, it sends the configured tasks to the data generation module. The data generation module generates the required data line by line according to the field rules. When generating SQL data, it generates data rows in the form of INSERT statements; if the output is a.csv file, it generates data rows in the form of string arrays.
[0082] To improve performance, the data generation module runs in an asynchronous mode, and the data is transmitted through the chan channel. The data generation module dynamically creates multiple chan channels as needed to generate data in parallel, which is especially suitable for cases where sharding is possible, such as auto-increment primary keys. It neither has concurrency problems due to primary key auto-increment nor sacrifices high performance. The data generation module continuously fills the generated data into the chan for the data output module to read in real time.
[0083] The data output module monitors the chan channel from the data generation module in real time and outputs the data to the target location. The data output module decides whether to import the data into the MySQL database or write it to a.csv file according to the task configuration. When choosing to import into the database, the data output module connects to MySQL and executes the INSERT statement into the database to achieve real-time data insertion; if the output is to a.csv file, the data output module writes the data to a.csv file that conforms to the RFC 4180 specification. If it is expected that the amount of data to be generated is large, the number of generated files can also be configured. In this case, the.csv file will be sharded according to the configuration to avoid a single file being too large.
[0084] After the task starts, the progress monitoring module continuously collects and generates status information of the output task. The progress monitoring module obtains information such as the data volume, current processing speed, and number of processed lines from the data generation module and the data output module, summarizes the execution status of the task, and periodically prints the progress to the logging system. By calculating the processing speed, the progress monitoring module can predict the task completion time, and users can view the task progress in real time to ensure the controllability and transparency of the data filling task.
[0085] After all tasks are completed, the data filling tool records information such as the final status of the task, the filled data volume, and the execution time in the log file, generating a complete task execution report for subsequent reference. The log file contains detailed filling rules and the execution status of each task, facilitating problem troubleshooting and result verification.
[0086] Please refer to Figure 4 , which is a schematic structural diagram of a data filling tool provided by an embodiment of the present application. Exemplarily, as Figure 4 shown, the data filling tool 400 includes a configuration module 401, a data generation module 402, and a data output module 403.
[0087] The configuration module 401 is used to parse the configuration file to obtain task configuration information, and the task configuration information at least includes data rules, data volume, output format, and output location of the data to be filled;
[0088] The data generation module 402 is used to read the task configuration information from the configuration module;
[0089] The data generation module 402 is further used to generate target filling data based on the task configuration information and send the target filling data to the data output module;
[0090] The data output module 403 is used to receive the target filling data and write the target filling data to the target output location.
[0091] Optionally, the data generation module 402 is further used to:
[0092] Generate the target filling data based on the data rules, the output format, and the data volume.
[0093] Optionally, the data generation module 402 is further used to:
[0094] If the output format indicates generating SQL data, generate the target filling data in the form of an INSERT statement based on the data rules and the data volume;
[0095] If the output format indicates generating a CSV file, generate the target filled data in the form of a string array based on the data rules and the data volume.
[0096] Optionally, the data output module 403 is further configured to:
[0097] If the target output location is a MySQL database, write the target filled data in the form of the INSERT statement to the MySQL database;
[0098] If the target output location is the CSV file, write the target filled data in the form of the string array to the CSV file.
[0099] Optionally, the data output module 403 is further configured to:
[0100] If the target data volume of the target filled data is greater than the preset data volume, determine the target number of files based on the target data volume;
[0101] Create the CSV files corresponding to the target number of files;
[0102] Slice the target filled data according to the target number of files, and write the sliced target filled data into the CSV files.
[0103] Optionally, the data filling tool further includes a progress monitoring module, and the progress monitoring module is configured to:
[0104] Obtain task status information from the data generation module and the data output module;
[0105] Predict the task completion time based on the task status information;
[0106] Display the task completion time.
[0107] Optionally, the data filling tool further includes a task management and scheduling module, and the task management and scheduling module is configured to:
[0108] Generate at least one data filling task based on the task configuration information;
[0109] Send the data filling task to the data generation module and the data output module.
[0110] In summary, the embodiments of the present application provide a data automatic filling method: by providing a data filling toolkit including a configuration module, a data generation module, and a data output module, the configuration module parses a configuration file provided by a user to obtain task configuration information, such as data rules, data volume, output format, and output location; then, the data generation module automatically generates target filling data based on the task configuration information, and the data output module outputs the target filling data to a target output location. A data filling tool that can parse configuration information and automatically generate target filling data is provided, thereby reducing the workload of manual configuration of filling rules by users and improving the flexibility and usability of the data filling tool.
[0111] An exemplary embodiment of the present application further provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, it is used to cause the electronic device to execute the data automatic filling method according to the embodiments of the present application.
[0112] An exemplary embodiment of the present application further provides a non-transitory computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor of a computer, it is used to cause the computer to execute the data automatic filling method according to the embodiments of the present application.
[0113] An exemplary embodiment of the present application further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, it is used to cause the computer to execute the data automatic filling method according to the embodiments of the present application.
[0114] Reference Figure 5 , the block diagram of the electronic device 500 that can be used as the server or client of the present application will now be described. It is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.
[0115] As Figure 5As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to computer programs stored in a read-only memory (ROM) 502 or computer programs loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0116] A plurality of components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 can be any type of device capable of inputting information into the electronic device 500. The input unit 506 can receive input digital or character information and generate key signal inputs related to user settings and / or function controls of the electronic device. The output unit 507 can be any type of device capable of presenting information and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 can include, but is not limited to, magnetic disks and optical discs. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.
[0117] Optionally, a single-channel electroencephalogram (EEG) signal acquisition module (not shown in the figure) is also provided in the electronic device 500. It is used to acquire EEG signals and transmit the EEG signals to a signal processor of the electronic device 500 for EEG signal processing.
[0118] The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 executes the various methods and processes described above. For example, in some embodiments, Figure 1 , Figure 2 , Figure 3The method shown above can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 500 via ROM 502 and / or communication unit 509. In some embodiments, computing unit 501 may be configured to execute Figure 1 , Figure 2 , Figure 3 the method shown above.
[0119] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.
[0120] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0121] As used in this application, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., a disk, an optical disc, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.
[0122] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).
[0123] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0124] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship to each other.
Claims
1. A method for automatic data filling, characterized in that: The method is applied to a data filling tool, the data filling tool includes a configuration module, a data generation module and a data output module, and the method includes: Parsing the configuration file by the configuration module to obtain task configuration information, wherein the task configuration information at least includes data rules, data volume, output format and output location of the data to be filled; Reading the task configuration information from the configuration module through the data generation module; Generate target filling data based on the task configuration information through the data generation module, and send the target filling data to the data output module; The target filling data is received by the data output module, and the target filling data is written into a target output location.
2. The method according to claim 1, characterized in that The step of generating target filling data based on the task configuration information by the data generation module includes: The target filling data is generated by the data generation module based on the data rule, the output format and the data amount.
3. The method according to claim 1, characterized in that The step of generating the target filling data based on the data rule, the output format and the data amount by the data generation module includes: If the output format indicates generating SQL data, the target filling data is generated in the form of an INSERT statement by the data generating module based on the data rule and the data amount; If the output format indicates generating a CSV file, the target filling data is generated in the form of a string array by the data generating module based on the data rule and the data amount.
4. The method according to claim 3, characterized in that Writing the target filling data into the target output position comprises: If the target output location is a MySQL database, writing the target fill data in the form of the INSERT statement into the MySQL database through the data output module; If the target output location is the CSV file, the target filling data in the string array form is written into the CSV file through the data output module.
5. The method according to claim 4, characterized in that Writing the target filling data in the string array form into the CSV file through the data output module includes: If the target data volume of the target filling data is greater than the preset data volume, determining the target number of files based on the target data volume; Create the CSV file corresponding to the target number of files; The target filling data is segmented according to the number of target files, and the segmented target filling data is written into the CSV file.
6. The method according to any one of claims 1 to 5, characterized in that: The data filling tool further includes a progress monitoring module, and the method further includes: Acquiring task status information from the data generation module and the data output module through the progress monitoring module; Predicting the task completion time based on the task status information by the progress monitoring module; The task completion time is displayed through the progress monitoring module.
7. The method according to any one of claims 1 to 5, characterized in that: The data filling tool also includes a task management and scheduling module, and the method also includes: Generate at least one data filling task based on the task configuration information through the task management and scheduling module; The data filling task is sent to the data generating module and the data output module through the task management and scheduling module.
8. A data filling tool, characterized in that: The data filling tool includes a configuration module, a data generation module and a data output module: The configuration module is used to parse the configuration file to obtain task configuration information, wherein the task configuration information at least includes data rules, data volume, output format and output location of the data to be filled; The data generation module is used to read the task configuration information from the configuration module; The data generation module is further used to generate target filling data based on the task configuration information, and send the target filling data to the data output module; The data output module is used to receive the target filling data and write the target filling data into a target output location.
9. An electronic device, comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the data automatic filling method according to any one of claims 1-7.
10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the data automatic filling method according to any one of claims 1-7.