Data writing method, device, storage medium and system

Abstract

The embodiment of the present invention provides a data writing method, device, storage medium and system, the method comprising: obtaining at least two target data to be written; storing the at least two target data in a preset batch entity group to generate batch entity group data; wherein the batch entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one-to-one, and the number of the sub-processing entity group data is the same as the number of the target data; writing each sub-processing entity group data in the batch entity group data into at least two target database tables based on at least two preset configuration parameters; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data. The scheme of the embodiment of the present invention can write multiple target data into multiple database tables, realize batch processing of target data, and improve data writing efficiency.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing technology, and in particular, to a data writing method, device, storage medium and system. Background Art

[0002] As the scale of banking business continues to expand, the amount of data that needs to be processed is also increasing. At the same time, various business systems are undergoing distributed system transformation. Bank business systems are facing the problem of how to use a batch processing framework suitable for distributed systems to quickly and effectively complete data processing. Therefore, there is an urgent need for a flexible, easy-to-use, highly scalable batch processing framework with high performance to process large amounts of information to improve development efficiency and operation efficiency. Spring Batch is a lightweight, fully functional batch processing framework based on the Spring framework. It provides a wealth of plug-and-play components to quickly build lightweight and robust parallel processing applications, including logging / tracing, transaction management, job processing statistics, job restart, skip and resource management functions, so that batch application developers only need to focus on business processing, and the interaction of the underlying framework is handled by Spring Batch. Robust parallel processing applications can be quickly built, effectively improving the development efficiency of batch applications.

[0003] The RPW (Reader-Processor-Writer) batch processing mode provided by Spring Batch is very suitable for solving the most common batch processing scenarios in banking business systems, that is, reading data from one or more files or database tables, performing relevant business logic processing, and then writing it to one or more files or database tables. However, the RPW batch processing mode of SpringBatch only supports reading data from a single database table and then writing it to the corresponding single database table. Summary of the invention

[0004] The embodiments of the present invention provide a data writing method, device, storage medium and system, which can write multiple target data into multiple database tables, realize batch processing of the target data, and improve the data writing efficiency.

[0005] In a first aspect, an embodiment of the present invention provides a data writing method, comprising:

[0006] Obtain at least two pieces of target data to be written;

[0007] The at least two target data are stored in a preset batch entity group to generate batch entity group data; wherein the batch entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one by one, and the number of the sub-processing entity group data is the same as the number of the target data;

[0008] Each sub-processing entity group data in the batch processing entity group data is written into at least two target database tables based on at least two preset configuration parameters; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data.

[0009] In a second aspect, an embodiment of the present invention further provides a data writing device, including:

[0010] A target data acquisition module, used to acquire at least two pieces of target data to be written;

[0011] A batch processing entity group data generating module, used for storing the at least two target data in a preset batch processing entity group to generate batch processing entity group data; wherein the batch processing entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one by one, and the number of the sub-processing entity group data is the same as the number of the target data;

[0012] The data writing module is used to write each sub-processing entity group data in the batch processing entity group data into at least two target database tables based on at least two preset configuration parameters; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data.

[0013] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a data writing method as provided in an embodiment of the present invention.

[0014] In a fourth aspect, an embodiment of the present invention provides a data writing system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the data writing method provided in the embodiment of the present invention is implemented.

[0015] The data writing scheme provided by the embodiment of the present invention obtains at least two target data to be written; stores the at least two target data in a preset batch entity group to generate batch entity group data; wherein the batch entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one by one, and the number of the sub-processing entity group data is the same as the number of the target data; based on at least two preset configuration parameters, each sub-processing entity group data in the batch entity group data is written into at least two target database tables; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data. Through the technical scheme provided by the embodiment of the present invention, multiple target data can be written into multiple database tables, batch processing of target data can be realized, and data writing efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a data writing method provided by an embodiment of the present invention;

[0017] Figure 2 is a schematic diagram of a data writing process in another embodiment of the present invention;

[0018] Figure 3 is a structural schematic diagram of a data writing device in another embodiment of the present invention;

[0019] Figure 4 It is a structural schematic diagram of a data writing system in another embodiment of the present invention. DETAILED DESCRIPTION

[0020] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0021] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0022] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0023] It should be noted that the concepts such as "first" and "second" mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0024] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0025] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0026] In the related technology, in the Spring Batch framework, the following methods can be used to implement database table writing: 1. In RPW mode, multiple Steps can be used to solve the problem of reading data from a database table and then writing it to one or more database tables. 2. In RPW mode, use Processor to write to a database table. 3. In RPW mode, a combined writing component is used to combine multiple writing components together to implement multi-database table writing. 4. In Tasklet mode, developers need to program to implement the reading and writing logic of the database table.

[0027] However, in RPW mode, multiple steps are used to solve the problem of reading data from one or more files or database tables and then writing it to one or more database tables. This requires repeated data reading and processing, which leads to resource waste and longer batch processing time. In RPW mode, the method of using Processor to write to the database table can only write one piece of data processed by the processor to the database table, and cannot write data in batches, which greatly increases the data processing time. In RPW mode, a method of combining write components is adopted to combine multiple write components together to implement a method of writing to multiple database tables. Its flexibility is limited. Each batch processing scenario requires the implementation of a combiner to ultimately achieve the writing of multiple database tables. In Tasklet mode, developers are required to program to implement the database table reading and writing logic, which increases the complexity of development and is extremely error-prone.

[0028] Figure 1 This is a flow chart of a data writing method provided by an embodiment of the present invention. The embodiment of the present invention is applicable to the case of writing data. The method can be executed by a data writing device, which can be composed of hardware and / or software and can generally be integrated in a data writing system. Figure 1 As shown, the method specifically comprises the following steps:

[0029] Step 110, obtaining at least two pieces of target data to be written.

[0030] The target data may be data directly read from one or more files or database tables, or data read from other terminal devices. It should be noted that the embodiment of the present invention does not limit the source of the target data.

[0031] Optionally, obtaining at least two target data to be written includes: reading at least two original data from at least one file or database table through a reader; and processing the at least two original data through a processor to generate target data. Exemplarily, based on the RPW batch processing mode provided by the Spring Batch framework, at least two original data can be read from one or more files or database tables through a reader, and then the reader sends the read original data to a processor, and the processor processes the original data and uses the processed original data as the target data to be written. The processing operation of the processor on the original data may include related operations such as adding, deleting, and changing.

[0032] Step 120, storing the at least two target data in a pre-set batch entity group to generate batch entity group data; wherein the batch entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one by one, and the number of the sub-processing entity group data is the same as the number of the target data.

[0033] The batch entity group is a data structure for storing entity objects to be operated, and the structure contains multiple sub-entity groups. Each sub-entity group can contain multiple entity objects with the same structured query language (SQL).

[0034] In an embodiment of the present invention, at least two pieces of target data are stored in a batch entity group to generate batch entity group data. It is understandable that at least two pieces of target data are assembled into the batch entity group. Specifically, the batch entity group includes a plurality of sub-processing entity groups, and the specific operation of storing at least two pieces of target data in the batch entity group may include: storing each piece of target data in a sub-processing entity group in the batch entity group, generating sub-processing entity group data, and using a set consisting of each sub-processing entity group data as the batch entity group data.

[0035] Step 130, writing each sub-processing entity group data in the batch processing entity group data into at least two target database tables based on at least two preset configuration parameters; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data.

[0036] In the embodiment of the present invention, at least two preset configuration parameters are obtained, wherein the preset configuration parameters can be understood as configuration parameters for writing sub-processing entity group data. Optionally, the number of preset configuration parameters is the same as the number of sub-processing entity group data, so as to implement a one-to-one writing operation on the sub-processing entity group data based on the preset configuration parameters.

[0037] Exemplarily, the preset configuration parameters include data source information, SQL statements to be executed, and a setter for executing the SQL statements; wherein the data source information is used to indicate the write location of the sub-processing entity group data. It can be understood that each preset configuration parameter includes data source information, SQL statements to be executed, and a setter for executing the SQL statements. wherein, according to the data source information, it can be determined into which database the corresponding sub-processing entity group data is specifically written. The setter for executing the SQL statement may include itemPreparedStatementSetter and itemSqlParameterSourceProvider. wherein itemPreparedStatementSetter is a setter for precompiled database execution statement parameters applied to Placeholder form SQL. wherein Placeholder form SQL refers to SQL in which variables in SQL are replaced with "?" as placeholders. itemSqlParameterSourceProvider is a setter for precompiled database execution statement parameters applied to NamedParam form SQL. wherein NamedParam form SQL refers to SQL in which variables in SQL are replaced with ": attribute name" as placeholders. The advantage of this setting is that it uses a mixed form of SQL and supports both Placeholder and NamedParam JDBC protocol SQL data writing by setting itemPreparedStatementSetter or itemSqlParameterSourceProvider in sequence.

[0038] In an embodiment of the present invention, the corresponding sub-processing entity group data in the batch entity group data is analyzed and processed based on each preset configuration parameter, and the analyzed sub-processing entity group data is written into the corresponding target database table. Among them, the preset configuration parameters and the sub-processing entity group data are in a one-to-one processing relationship. Exemplarily, the target database table to which the sub-processing entity group data is to be written is determined by the source data information in the preset configuration parameters, and whether the target database table is a sub-table and the corresponding sub-table key are determined according to the SQL statement to be executed in the preset configuration parameters, and the logical table in the SQL statement is replaced with a physical table; and according to the SQL type contained in the SQL statement to be executed, the itemPreparedStatementSetter or itemSqlParameterSourceProvider in the form of Placeholder and NamedParam are sorted in order. Then, the SQL statement to be executed is traversed in order, and the target data contained in the sub-processing entity group is written into the corresponding target database table through the SQL batch execution interface in the template method of the database operation.

[0039] The data writing method provided by the embodiment of the present invention obtains at least two target data to be written; stores the at least two target data in a preset batch entity group to generate batch entity group data; wherein the batch entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one by one, and the number of the sub-processing entity group data is the same as the number of the target data; based on at least two preset configuration parameters, each sub-processing entity group data in the batch entity group data is written into at least two target database tables; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data. Through the technical solution provided by the embodiment of the present invention, multiple target data can be written into multiple database tables, batch processing of target data can be realized, and data writing efficiency can be improved.

[0040] In some embodiments, before writing each sub-processing entity group data in the batch entity group data into at least two target database tables based on at least two preset configuration parameters, it also includes: creating an initial multi-database table write component based on a multi-database table write component creator; writing the at least two preset configuration parameters into the initial multi-database table write component based on the multi-database table write component creator to generate a target multi-database table write component; writing each sub-processing entity group data in the batch entity group data into at least two target database tables based on at least two preset configuration parameters, including: running the target multi-database table write component to write each sub-processing entity group data in the batch entity group data into at least two target database tables through at least two preset configuration parameters in the target multi-database table write component.

[0041] Among them, the multi-database table writing component (MultiTableJdbcBatchItemWriter) is a customized data batch writing executor for multiple database tables. The multi-database table writing component creator (MultiTableJdbcBatchItemWriterBuilder) is a creator that creates a multi-database table writing component by setting different parameters.

[0042] In an embodiment of the present invention, an initial multi-database table write component can be created by a multi-database table write component creator in the Spring Batch framework. At least two preset configuration parameters are obtained, and the at least two preset configuration parameters obtained are written into the initial multi-database table write component by the multi-database table write component creator. That is, the multi-database table write component creator assembles the initial multi-database table write component with preset configuration parameters such as data source information, SQL statements to be executed, and SQL statement setters, thereby achieving the purpose of creating a target multi-database table write component. The target multi-database table write component is run through the SpringBatch framework, that is, when the target multi-database table write component is called up by the SpringBatch framework, each sub-processing entity group data in the batch entity group data is written into at least two target database tables through at least two preset configuration parameters in the target multi-database table write component. Among them, before writing the data into the target database table, the target multi-database table write component can be used to perform basic checks such as SQL quantity and SQL legality on each sub-processing entity group data in the batch entity group data. When the check passes, the sub-processing entity group data is written into at least two target database tables.

[0043] In some embodiments, the at least two target data are stored in a preset batch entity group to generate batch entity group data, including: numbering the at least two target data in sequence; writing the numbered at least two target data into the preset batch entity group to generate batch entity group data. Specifically, the at least two target data can be numbered in sequence, and the at least two numbered target data can be written into each sub-processing entity group in the preset batch entity group to generate at least two sub-processing entity group data, and the set consisting of the at least two sub-processing entity group data is used as the batch entity group data. Among them, each sub-processing entity group data contained in the batch entity group data is numbered data. The advantage of such a setting is that each sub-processing entity group data can be written in the order of numbering based on at least two preset configuration parameters to ensure that the data writing operation can be carried out in an orderly manner.

[0044] In some embodiments, before the at least two preset configuration parameters are written into the initial multi-database table write component based on the multi-database table write component creator to generate a target multi-database table write component, the method further includes: numbering the at least two preset configuration parameters in sequence; writing the at least two preset configuration parameters into the initial multi-database table write component based on the multi-database table write component creator to generate a target multi-database table write component, including: writing the at least two numbered preset configuration parameters into the initial multi-database table write component based on the multi-database table write component creator to generate a target multi-database table write component; running the target multi-database table write component to write each sub-processing entity group data in the batch processing entity group data into at least two target database tables through at least two preset configuration parameters in the target multi-database table write component, including: running the target multi-database table write component to write the sub-processing entity group data in the batch processing entity group data having the same number as the preset configuration parameter into at least two target database tables respectively through at least two preset configuration parameters in the target multi-database table write component.

[0045] In an embodiment of the present invention, at least two preset configuration parameters are obtained and numbered, and the multi-database table writing component creator writes the at least two numbered configuration parameters into the initial multi-database table writing component to generate a target multi-database table writing component. It can be understood that the target multi-database table writing component contains at least two numbered preset configuration parameters. When the target multi-database table writing component is called by the Spring Batch framework, the target multi-database table writing component writes the sub-processing entity data with the same sequence number as the preset configuration parameter into the corresponding target database table.

[0046] For example, Figure 2 A schematic diagram of a data writing process provided by an embodiment of the present invention. Figure 2 The process of writing data to multiple database tables can be understood based on the Spring Batch framework, so I will not go into details here.

[0047] Figure 3 FIG. 1 is a schematic diagram of a data writing device provided by another embodiment of the present invention. Figure 3 As shown, the device includes: a target data acquisition module 310, a batch entity group data generation module 320 and a data writing module 330. Among them,

[0048] A target data acquisition module 310 is used to acquire at least two pieces of target data to be written;

[0049] The batch entity group data generating module 320 is used to store the at least two target data in a preset batch entity group to generate batch entity group data; wherein the batch entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one by one, and the number of the sub-processing entity group data is the same as the number of the target data;

[0050] The data writing module 330 is used to write each sub-processing entity group data in the batch processing entity group data into at least two target database tables based on at least two preset configuration parameters; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data.

[0051] The data writing device provided by the embodiment of the present invention obtains at least two target data to be written; stores the at least two target data in a preset batch entity group to generate batch entity group data; wherein the batch entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one by one, and the number of the sub-processing entity group data is the same as the number of the target data; based on at least two preset configuration parameters, each sub-processing entity group data in the batch entity group data is written into at least two target database tables; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data. Through the technical solution provided by the embodiment of the present invention, multiple target data can be written into multiple database tables, batch processing of target data can be realized, and data writing efficiency can be improved.

[0052] Optionally, the device further comprises:

[0053] A writing component creation module, used for creating an initial multi-database table writing component based on a multi-database table writing component creator before writing each sub-processing entity group data in the batch processing entity group data into at least two target database tables based on at least two preset configuration parameters;

[0054] A configuration parameter writing module, configured to write the at least two preset configuration parameters into the initial multi-database table writing component based on the multi-database table writing component creator, and generate a target multi-database table writing component;

[0055] The data writing module comprises:

[0056] The data writing unit is used to run the target multi-database table writing component to write each sub-processing entity group data in the batch processing entity group data into at least two target database tables through at least two preset configuration parameters in the target multi-database table writing component.

[0057] Optionally, the batch entity group data generation module is used to:

[0058] Numbering the at least two target data in sequence;

[0059] At least two numbered target data are written into a preset batch entity group to generate batch entity group data.

[0060] Optionally, also include:

[0061] A configuration parameter numbering module, used for sequentially numbering the at least two preset configuration parameters before writing the at least two preset configuration parameters into the initial multi-database table writing component to generate a target multi-database table writing component based on the multi-database table writing component creator;

[0062] The configuration parameter writing module is used to:

[0063] Based on the multi-database table writing component creator, at least two numbered preset configuration parameters are written into the initial multi-database table writing component to generate a target multi-database table writing component;

[0064] The data writing unit is used for:

[0065] The target multi-database table writing component is run to write the sub-processing entity group data with the same number as the preset configuration parameter in the batch processing entity group data into at least two target database tables respectively through at least two preset configuration parameters in the target multi-database table writing component.

[0066] Optionally, the target data acquisition module is used to:

[0067] Read at least two pieces of original data from at least one file or database table through a reader;

[0068] The processor performs processing operations on the at least two original data to generate target data.

[0069] Optionally, the preset configuration parameters include data source information, an SQL statement to be executed, and a setter for executing the SQL statement; wherein the data source information is used to indicate a writing location for sub-processing entity group data.

[0070] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not fully described in the embodiments of the present invention, please refer to the methods provided by all the above embodiments of the present invention.

[0071] An embodiment of the present invention further provides a storage medium containing computer executable instructions, and when the computer executable instructions are executed by a computer processor, they are used to execute the data writing method provided by the embodiment of the present invention.

[0072] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDRRAM, SRAM, EDORAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0073] Of course, the computer executable instructions of a storage medium including computer executable instructions provided by an embodiment of the present invention are not limited to the data writing operations described above, and can also execute related operations in the data writing method provided by any embodiment of the present invention.

[0074] An embodiment of the present invention provides a data writing system, into which the data writing device provided by the embodiment of the present invention can be integrated. Figure 4The data writing system 400 may include a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor. When the processor 402 executes the computer program, the data writing method described in the embodiment of the present invention is implemented.

[0075] The data writing system provided in the embodiment of the present invention obtains at least two target data to be written; stores the at least two target data in a preset batch entity group to generate batch entity group data; wherein the batch entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one by one, and the number of the sub-processing entity group data is the same as the number of the target data; based on at least two preset configuration parameters, each sub-processing entity group data in the batch entity group data is written into at least two target database tables; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data. Through the technical solution provided in the embodiment of the present invention, multiple target data can be written into multiple database tables, batch processing of target data can be realized, and data writing efficiency can be improved.

[0076] The data writing device, storage medium and data writing system provided in the above embodiments can execute the data writing method provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects of executing the method. For technical details not described in detail in the above embodiments, please refer to the data writing method provided in any embodiment of the present invention.

[0077] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data writing method, characterized in that: include: Obtain at least two pieces of target data to be written; The at least two target data are stored in a preset batch entity group to generate batch entity group data; wherein the batch entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one by one, and the number of the sub-processing entity group data is the same as the number of the target data; Writing each sub-processing entity group data in the batch processing entity group data into at least two target database tables based on at least two preset configuration parameters; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data; Before writing each sub-processing entity group data in the batch processing entity group data into at least two target database tables based on at least two preset configuration parameters, the method further includes: Create an initial multi-database table writing component based on the multi-database table writing component creator; Based on the multi-database table writing component creator, the at least two preset configuration parameters are written into the initial multi-database table writing component to generate a target multi-database table writing component; Writing each sub-processing entity group data in the batch processing entity group data into at least two target database tables based on at least two preset configuration parameters includes: The target multi-database table writing component is run to write each sub-processing entity group data in the batch processing entity group data into at least two target database tables through at least two preset configuration parameters in the target multi-database table writing component.

2. The method according to claim 1, characterized in that: The at least two target data are stored in a preset batch entity group to generate batch entity group data, including: Numbering the at least two target data in sequence; At least two numbered target data are written into a preset batch entity group to generate batch entity group data.

3. The method according to claim 2, characterized in that Before the at least two preset configuration parameters are written into the initial multi-database table writing component based on the multi-database table writing component creator to generate the target multi-database table writing component, the method further includes: Numbering the at least two preset configuration parameters in sequence; The at least two preset configuration parameters are written into the initial multi-database table writing component based on the multi-database table writing component creator to generate a target multi-database table writing component, including: Based on the multi-database table writing component creator, at least two numbered preset configuration parameters are written into the initial multi-database table writing component to generate a target multi-database table writing component; Running the target multi-database table writing component to write each sub-processing entity group data in the batch processing entity group data into at least two target database tables through at least two preset configuration parameters in the target multi-database table writing component includes: The target multi-database table writing component is run to write the sub-processing entity group data with the same number as the preset configuration parameter in the batch processing entity group data into at least two target database tables respectively through at least two preset configuration parameters in the target multi-database table writing component.

4. The method according to claim 1, characterized in that: Get at least two pieces of target data to be written, including: Read at least two pieces of original data from at least one file or database table through a reader; The processor performs processing operations on the at least two original data to generate target data.

5. The method according to any one of claims 1 to 4, characterized in that: The preset configuration parameters include data source information, SQL statements to be executed, and a setter for executing the SQL statements; wherein the data source information is used to indicate the writing location of the sub-processing entity group data.

6. A data writing device, characterized in that: include: A target data acquisition module, used to acquire at least two pieces of target data to be written; A batch processing entity group data generating module, used for storing the at least two target data in a preset batch processing entity group to generate batch processing entity group data; wherein the batch processing entity group data includes at least two sub-processing entity group data, the sub-processing entity group data corresponds to the target data one by one, and the number of the sub-processing entity group data is the same as the number of the target data; A data writing module, used for writing each sub-processing entity group data in the batch processing entity group data into at least two target database tables based on at least two preset configuration parameters; wherein the number of the preset configuration parameters is the same as the number of the sub-processing entity group data; Wherein, the device further comprises: A writing component creation module, used for creating an initial multi-database table writing component based on a multi-database table writing component creator before writing each sub-processing entity group data in the batch processing entity group data into at least two target database tables based on at least two preset configuration parameters; A configuration parameter writing module, configured to write the at least two preset configuration parameters into the initial multi-database table writing component based on the multi-database table writing component creator, and generate a target multi-database table writing component; The data writing module comprises: The data writing unit is used to run the target multi-database table writing component to write each sub-processing entity group data in the batch processing entity group data into at least two target database tables through at least two preset configuration parameters in the target multi-database table writing component.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, the data writing method as described in any one of claims 1 to 5 is implemented.

8. A data writing system, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the data writing method according to any one of claims 1 to 5 is implemented.

Images