Batch operation processing method and device, equipment, medium and database system

Abstract

The invention provides a batch operation processing method and device, electronic equipment, a storage medium and a database system. According to the embodiment of the invention, the batch processing flow can be firstly obtained, and then the database batch operation in the batch processing flow is executed through the database matched with the processing type of the database batch operation in sequence. Wherein after the database batch operation executed first is executed, the database batch operation executed later and the database batch operation executed first are determined to correspond to different types of databases, and the database batch operation executed later depends on the processing result of the database batch operation executed first. And calling a cloud data transmission synchronization tool, and synchronizing a processing result corresponding to the previously executed database batch operation to a database corresponding to the later executed database batch operation. By adopting the scheme, the database batch operation efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a batch operation processing method, device, electronic device, storage medium, and database system. Background Art

[0002] In the actual application of databases, many industries need to process large quantities of data, which is also called database batch processing. The basic process of batch data processing can be to read batch data from the database first, process it by calculation, and then output it in the form of graphics or data files.

[0003] Take the banking industry as an example. When banks perform batch processing of databases at the end of the day, the banking industry needs to process a large number of transactions, involving a wide range of data. The batch processing of different transactions also has a front-to-back dependency relationship. For these data, more batch processing may be required, and the processing logic used for different batch processing may also be different. If these batch operations are processed in the same database, it may cause problems such as long batch processing time and unstable database batch operations. Therefore, it is necessary to configure a more appropriate data processing link for this type of data batch processing. Summary of the invention

[0004] The embodiments of the present application provide a batch operation processing method, device, electronic device, storage medium and database system to solve one or more of the above-mentioned technical problems.

[0005] In a first aspect, an embodiment of the present application provides a batch operation processing method, including:

[0006] Acquire a batch processing flow, wherein the batch processing flow includes a plurality of database batch operations executed sequentially, wherein the plurality of database batch operations correspond to a plurality of batch processing types for accessing different types of databases;

[0007] The database batch operations in the batch processing flow are executed sequentially through databases matching the batch processing type of the database batch operations, wherein after the database batch operation executed earlier is completed, when it is determined that the database batch operation executed later corresponds to databases of different types as the database batch operation executed earlier, and the database batch operation executed later depends on the processing result of the database batch operation executed earlier, the processing result corresponding to the database batch operation executed earlier is synchronized to the database corresponding to the database batch operation executed later by calling a cloud data transmission synchronization tool.

[0008] In a second aspect, an embodiment of the present application provides a batch operation processing method, including:

[0009] Acquire a batch processing flow, wherein the batch processing flow includes a plurality of database batch operations executed sequentially, wherein the plurality of database batch operations include transaction batch processing operations and analysis batch processing operations corresponding to accessing different types of databases;

[0010] The database batch operations in the batch processing flow are executed sequentially through databases that match the processing type of the database batch operations, wherein after the transaction batch processing operation executed first is completed, when it is determined that the analysis batch processing operation executed later corresponds to a different type of database than the transaction batch processing operation executed first, and the analysis batch processing operation executed later depends on the processing result of the transaction batch processing operation executed first, a cloud data transmission synchronization tool is called to synchronize the processing result corresponding to the transaction batch processing operation executed first to the database corresponding to the report batch processing operation executed later.

[0011] In a third aspect, an embodiment of the present application provides a database system, including multiple databases for processing database batch operations of different batch processing types, and a process management node;

[0012] The process management node includes:

[0013] A process acquisition module, used to acquire a batch processing process, wherein the batch processing process includes a plurality of database batch operations executed sequentially, and the plurality of database batch operations correspond to a plurality of batch processing types for accessing different types of databases;

[0014] A database scheduling module is used to execute the database batch operations in the batch processing flow in sequence through databases that match the batch processing type of the database batch operations, wherein after the database batch operation executed earlier is completed, if it is determined that the database batch operation executed later corresponds to a different type of database than the database batch operation executed earlier, and the database batch operation executed later depends on the processing result of the database batch operation executed earlier, the processing result corresponding to the database batch operation executed earlier is synchronized to the database corresponding to the database batch operation executed later by calling a cloud data transmission synchronization tool.

[0015] In a fourth aspect, an embodiment of the present application provides a batch operation processing device, including:

[0016] A process acquisition module, used to acquire a batch processing process, wherein the batch processing process includes a plurality of database batch operations executed sequentially, and the plurality of database batch operations correspond to a plurality of batch processing types for accessing different types of databases;

[0017] A database scheduling module is used to execute the database batch operations in the batch processing flow in sequence through databases that match the batch processing type of the database batch operations, wherein after the database batch operation executed earlier is completed, if it is determined that the database batch operation executed later corresponds to a different type of database than the database batch operation executed earlier, and the database batch operation executed later depends on the processing result of the database batch operation executed earlier, the processing result corresponding to the database batch operation executed earlier is synchronized to the database corresponding to the database batch operation executed later by calling a cloud data transmission synchronization tool.

[0018] In a fifth aspect, an embodiment of the present application provides a batch operation processing device, including:

[0019] A processing flow acquisition module, used to acquire a batch processing flow, wherein the batch processing flow includes a plurality of database batch operations executed sequentially, wherein the plurality of database batch operations include transaction batch processing operations and analysis batch processing operations corresponding to accessing different types of databases;

[0020] A batch operation execution module is used to execute the database batch operations in the batch processing flow in sequence through databases that match the processing type of the database batch operations, wherein after the transaction batch processing operation executed first is completed, when it is determined that the analysis batch processing operation executed later corresponds to a different type of database than the transaction batch processing operation executed first, and the analysis batch processing operation executed later depends on the processing result of the transaction batch processing operation executed first, a cloud data transmission synchronization tool is called to synchronize the processing result corresponding to the transaction batch processing operation executed first to the database corresponding to the report batch processing operation executed later.

[0021] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods described above when executing the computer program.

[0022] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the methods described above is implemented.

[0023] Compared with the related art, this application has the following advantages:

[0024] According to the embodiment of the present application, the database batch operation in the batch processing process can be executed by first obtaining the batch processing process, and then sequentially passing through the database matching the processing type of the database batch operation. Among them, after the database batch operation executed first is completed, when it is determined that the database batch operation executed later corresponds to a different type of database as the database batch operation executed earlier, and the database batch operation executed later depends on the processing result of the database batch operation executed earlier, the cloud data transmission synchronization tool is called to synchronize the processing result corresponding to the database batch operation executed earlier to the database corresponding to the database batch operation executed later. In the above process, different database batch operation processing tasks are split into different databases for execution, which can avoid the problem of long processing time caused by different batch operations affecting each other when using the same database to process different types of database batch operations. At the same time, using the cloud data transmission synchronization tool to synchronize data between different databases in the process of executing different batch operation processing tasks can ensure the consistency of data in different databases. Therefore, the above scheme can improve the efficiency of database batch operations.

[0025] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0027] Figure 1 A schematic diagram showing an application example of a batch operation processing solution provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram showing a processing flow of an application example of a batch operation processing solution provided in an embodiment of the present application;

[0029] Figure 3 A flow chart of a batch operation processing method provided in an embodiment of the present application is shown;

[0030] Figure 4 A flowchart of another batch operation processing method provided in an embodiment of the present application is shown;

[0031] Figure 5A structural block diagram of a database system provided in an embodiment of the present application is shown;

[0032] Figure 6 A structural block diagram of a batch operation processing device provided in an embodiment of the present application is shown;

[0033] Figure 7 A structural block diagram of another batch operation processing device provided in an embodiment of the present application is shown; and

[0034] Figure 8 A block diagram of an electronic device used to implement an embodiment of the present application is shown. DETAILED DESCRIPTION

[0035] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0036] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0037] In the related technologies prior to this application, taking the banking industry's use of databases for batch data processing as an example, batch data is generally retrieved from the same set of databases and closed-loop processed to ensure that the data flow link is as short as possible and that batch data with dependencies before and after are simply scheduled.

[0038] However, due to the large number of transactions that the banking industry needs to handle, the batch operation processing logic of different transactions is different. If batch data is retrieved from the same database and different batch operations are performed, it may cause problems such as long processing time and unstable database batch operations. For example, data belonging to bank transaction batches are mostly transaction-related data, generally data of heavy transaction processing transactions, such as daily outgoing and incoming amounts, etc., for which only simple summary statistics are required; while data belonging to bank report batches are generally data of heavy analysis and processing transactions, which often require complex analysis and calculation, involving a relatively complex calculation process. When batch operations are performed on transaction batches and report batches, when both batch operations call data from the same database, the two operations may affect each other. For example, when the database type called is a transaction database, the transaction batch may be completed quickly during the processing, but the processing time of the report batch will increase exponentially, which will seriously affect the progress of the database's end-of-day batch processing and the stability of the database's batch operations.

[0039] In addition, batch operations on different transactions in the banking industry may have a front-end dependency relationship. How to retrieve batch data from the database while ensuring the accuracy of the retrieved data requires configuring a more appropriate data processing link and calling logic for this type of data batch operation processing.

[0040] In view of this, embodiments of the present application provide a batch operation processing method, device, electronic device, storage medium and database system to solve all or part of the above-mentioned technical problems.

[0041] The embodiment of the present application can obtain a batch processing flow, and then sequentially pass through the database that matches the processing type of the database batch operation (such as the aforementioned transaction batch and report batch) to execute the database batch operation in the batch processing flow. Among them, the batch processing flow may include multiple database batch operations executed sequentially, and the multiple database batch operations may correspond to multiple batch processing types for accessing different types of databases. When the database batch operation executed first is completed, if it is determined that the database batch operation executed later corresponds to a different type of database than the database batch operation executed first, and the database batch operation executed later depends on the processing result of the database batch operation executed first, the cloud data transmission synchronization tool can be called to synchronize the processing result corresponding to the database batch operation executed first to the database corresponding to the database batch operation executed later.

[0042] Figure 1 It is a schematic diagram of an application example for implementing the batch operation processing scheme of the embodiment of the present application. In the credit service of the banking industry, taking the batch operation processing of transaction batch data and report batch data at the same time as an example, the bank will perform end-of-day batch processing on the transaction batch data and report batch data every day. The end-of-day batch processing can refer to an accounting reconciliation behavior performed by the bank system on the counter accounts every day, which is used for interest calculation, reconciliation, etc. Since the bank core system includes four basic subsystems: customer information subsystem, service processing subsystem, account processing subsystem, and accounting report subsystem, among which the account processing subsystem is generally completed in batch mode, it can be recorded as a batch processing subsystem, and the end-of-day batch processing can be the batch processing behavior of the batch processing subsystem before the accounting day switch.

[0043] like Figure 1As shown, compared with the traditional method of calling batch data in the same set of databases, in the embodiment of the present application, multiple databases can be created. When performing different types of batch operations, batch data can be called from these different databases respectively. For example, two databases can be created, namely the credit service transaction master database (transaction database) and the credit service analysis database (analysis database), wherein the transaction database can be used to collect and store bank credit service data, is the main database, and has all the functions of the database such as reading, writing, deleting, and modifying data; the analysis database can be a database used to generate accounting reports, etc., is a slave database, can be set to read-only form, and the data therein cannot be changed to ensure the data security in the analysis database. The transaction database can be deployed on the local client of the bank, so that the bank can collect and store transaction data in a timely manner; and the analysis database can be deployed in the cloud to save local computing and storage resources. Of course, both the transaction database and the analysis database can be deployed locally or in the cloud, and this application does not impose any restrictions on this.

[0044] When adding, deleting, or modifying related data in the transaction database, such as adding fields, deleting fields, modifying field length, modifying field type, or modifying field name, you can use the cloud-based data transmission synchronization tool (DTS, Data Transmission Service) to transfer the related data to the analysis database, so that the data in the analysis database is consistent with the data in the transaction database.

[0045] Assume that the bank needs to process six groups of data in batches today, namely batch 0, batch 1, batch 2, batch 3, batch 4, and batch 5. Batch 0, batch 1, batch 2, and batch 5 are all transaction batches, and batch 3 and batch 4 are report batches. Batch 5 is dependent on the batch operation processing results of batch 4, and batch 4 is dependent on the batch operation processing results of batch 3. And so on. That is, in order, in batch 1 to batch 5, each database batch operation (batch) executed later depends on the processing results of the database batch operation executed earlier. Therefore, for batch 0 to batch 5, the obtained batch processing flow is processed from batch 0 to batch 5 in order.

[0046] The batch processing operations described above can be uniformly scheduled and processed through batch processing flow scheduling, that is, by obtaining the batch processing flow, the database batch operations in the batch processing flow can be executed in sequence through the databases matching the processing type of the database batch operation. The batch processing flow may include multiple database batch operations executed sequentially, and the multiple database batch operations may correspond to multiple batch processing types for accessing different types of databases.

[0047] For example, the database that matches the four groups of database batch operations, batch 0, batch 1, batch 2, and batch 5, which belong to the transaction batch, is the transaction database. Then, the database batch operations can be performed on batch 0, batch 1, and batch 2 in sequence through the transaction database. However, since the data batch operation on batch 5 needs to rely on the processing result of batch 4, and the database that matches batch 4 is the analysis database, batch 4 and batch 5 correspond to different databases, so it is necessary to perform multiple database batch operations by accessing multiple batch processing types of different types of databases. Similarly, the databases corresponding to batch 3 and batch 2 are also different, and it is also necessary to perform multiple database batch operations by accessing multiple batch processing types of different types of databases. Of course, in the above processing process, it can be determined by reading the timestamp of the transaction operation log in the corresponding database whether the batch data operation processing of batch 2 or batch 4 has been completed. After it is determined that batch 2 has been processed by reading the timestamp in the transaction database, the analysis database can be called to start processing batch 3; when it is determined that batch 4 has been processed by reading the timestamp in the analysis database, the data in the transaction database can be called to start processing batch 5.

[0048] In the above scheme, the original batch processing flow of calling the same transaction database to perform database batch operations on batches 0 to 5 in sequence is changed. By adding an analysis database, which carries the complex data statistical analysis of the report batch, the batch data operations of batches 3 and 4 belonging to the report batch are separated from the original mode of calling the same transaction database as the transaction batch, and converted to calling the new analysis database to perform data batch operations on batches 3 and 4, thereby realizing the decoupling of "transaction-analysis", which can prevent the batch data operations of transaction batches and report batches from affecting each other when using a single transaction database to process analysis reports, resulting in excessive processing time.

[0049] Correspondingly, the scheduling flowchart of batch process scheduling can be found in Figure 2 .like Figure 2As shown, after the bank's end-of-day batch processing begins, the transaction database can be called through batch process scheduling, and batch 0, batch 1, and batch 2 can be processed in batches. Then, the analysis database can be called to perform batch operations on batches 3 and 4 in order. Before batch operations are performed on batches 3 and 4, since batch 3 and batch 2 correspond to different databases, the processing of batch 3 depends on the processing result of batch 2. Therefore, after batch 2 is processed in batches, it can be determined whether the batch dependency logs of the transaction database and the analysis database are synchronized. If they are completed (yes), batch 3 is further processed in batches. If not completed (no), it is necessary to return to batch 2 and query the processing result of batch 2 again to determine whether the batch dependency logs of the two databases are synchronized. After batch 4 in the analysis database is processed, the transaction database can be called again to perform database batch operations on batch 5. Correspondingly, after batch 4 is processed, it can also be determined whether the batch dependency logs of the transaction database and the analysis database are synchronized. If they are completed (yes), batch 5 is further processed. After batch 5 is processed, the bank's end-of-day batch process ends.

[0050] Among them, when the previously executed database batch operation is completed, the batch dependency log can be synchronized by calling the cloud data transmission synchronization tool, obtaining the first timestamp of the operation transaction log of the previously executed database batch operation, and synchronizing the operation transaction log to the database corresponding to the database batch operation executed later. Further, by obtaining the second timestamp of the currently synchronized operation transaction log, it is determined whether the first timestamp is consistent with the second timestamp. When it is determined that the first timestamp is consistent with the second timestamp, it is determined that the synchronization of the above operation transaction log is completed, that is, the synchronization of the batch dependency log is completed. Of course, the data transmission synchronization tool called in the above process, this application does not impose any restrictions on the type of data transmission synchronization tool.

[0051] The execution subject of the embodiments of the present application may be an application, service, instance, software-based functional module, virtual machine (VM), container or cloud server, etc., or a hardware device (such as a server or terminal device) or hardware chip (such as CPU, GPU, FPGA, NPU, AI acceleration card or DPU) with data processing function. The device for implementing batch operation processing can be deployed on the computing device of the application party that provides the corresponding service or on a cloud computing platform that provides computing power, storage and network resources. The mode of providing services to the outside world by the cloud computing platform may be IaaS (Infrastructure as a Service), PaaS (Platform as a Service), SaaS (Software as a Service) or DaaS (Data as a Service). Among them, the data transmission synchronization tool can be deployed on the cloud and function calls can be made through the API interface.

[0052] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0053] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0054] The present application embodiment provides a batch operation processing method, such as Figure 3 FIG. 1 is a flow chart of a batch operation processing method 300 according to an embodiment of the present application. The method 300 may include:

[0055] In step S301, a batch processing flow is obtained. The batch processing flow may include a plurality of database batch operations executed sequentially. The plurality of database batch operations correspond to a plurality of batch processing types for accessing different types of databases.

[0056] The batch processing flow involved in the embodiments of the present application may be a batch processing flow determined based on the dependency relationship between the database batch operation executed earlier and the database batch operation executed later, and may include multiple database batch operations executed sequentially. Among them, the database accessed by the database batch operation executed earlier may be of a different type from the database accessed by the database batch operation executed later, and the batch processing type to which the database batch operation executed earlier belongs may also be different from the batch processing type to which the database batch operation executed later belongs.

[0057] For example, the batch processing type of the database batch operation executed first is the transaction batch in the bank credit service; the batch processing type of the database batch operation executed later is the report batch in the bank credit service, so the transaction batch and report batch here are different batch processing types. If the data of the transaction batch is stored in the transaction database, and the data of the report batch is stored in the analysis database, then when the batch processing flow includes both the transaction batch and the report batch batch, when performing the database batch operation according to the batch processing flow, it is necessary to execute both the database batch operation in the transaction database and the database batch operation in the analysis database, that is, to execute multiple database batch operations in sequence.

[0058] Accordingly, the above-mentioned database batch operation may include transaction batch processing operations and analysis batch processing operations respectively executed based on the database, wherein the transaction batch processing operation may be a batch processing operation corresponding to accessing the transaction database, and the analysis batch processing operation may be a batch processing operation corresponding to accessing the analysis database.

[0059] Of course, in addition to the above-mentioned transaction database and analysis database, multiple other databases may also be included, each of which may store data corresponding to multiple batch processing types. These databases can be deployed locally or in the cloud, all locally or in the cloud, or partly locally and partly in the cloud. The types of these databases may be relational databases, distributed databases, document databases, key-value databases, search engines, graph databases, etc., and this application does not impose any restrictions on this.

[0060] Different databases may be of different types. For example, a database deployed locally may be a distributed database, and a database deployed in the cloud may be a relational database; or a database deployed locally may be an IBM DB2 database in a relational database, and a database deployed in the cloud may be a MySQL database in a relational database. Accordingly, different databases may also be of the same type, and this application does not impose any restrictions on this.

[0061] Compared with the traditional method of calling batch data from the same database, in the embodiment of the present application, multiple databases can be created, and when different types of batch operations are performed, batch data can be called from these different databases respectively. That is, the means of integrating heterogeneous databases can be adopted, and the computing power of the cloud can be used to undertake the functions of part of the database batch operation, thereby improving the efficiency of database batch operations and ensuring the stability of database batch operations.

[0062] For example, taking the end-of-day batch processing of bank credit services as an example, you can create an IBM DB2 relational database locally and set the database as the master database, which has all the database functions such as reading, writing, deleting, and modifying data. This database can be recorded as a "transaction database"; at the same time, you can create a relational database AnalyticDB PostgreSQL (ADB database) with real-time analysis functions in the cloud. This database can be recorded as an "analysis database" and can be set to a slave database and read-only mode. Of course, you can also not set the database as a slave database while ensuring the data security of the database, and make the database have the same functions as the master database, including reading, writing, deleting, and modifying data.

[0063] Among them, the ADB database engine deployed in the cloud can perform real-time analysis and processing on the ADB database in the cloud. The engine can be composed of two major components: coordination nodes and computing nodes. The coordination node can be responsible for global transaction management, global metadata storage, SQL statement parsing, rewrite optimization execution plan generation and scheduling, etc. The computing node can include the execution engine and storage engine, etc. The coordination node and the computing node can ensure high availability through dual copies, and provide linear expansion of computing and storage resources through horizontal and vertical expansion. At the same time, the ADB database engine can synchronize, process, query and analyze data. In the data synchronization stage, data can be written into the ADB database in real time or loaded in batches to form the ODS (Operational Data Model) layer; data processing can process the ODS layer data in the ADB database to form the CDM (Common Data Model) and ADS (Application Data Service) layers. Typical operations may include INSERT INTO SELECT (insert selection operation), CREATE TABLE AS (create table operation), etc.; data query and analysis can query and analyze the data in the ADB database according to demand, or provide it for consumption and processing by downstream systems, including interactive analysis, BI reports, data applications, etc. The ADB database can support high-concurrency and low-latency multi-dimensional range queries through features such as storage engine index sorting, and can also support complex analysis of multi-table association aggregation of large amounts of data through vectorized execution engines.

[0064] Accordingly, the transaction database and analysis database can be synchronized through the cloud data transmission synchronization tool so that the data stored in the two databases remain consistent, so as to ensure the consistency and accuracy of the data when multiple database batch operations are performed on the data in different types of databases according to the acquired batch processing flow. Among them, the data transmission synchronization tool used can be DTS (Data Transmission Service), DataX (heterogeneous data source offline synchronization tool), external table parallel import and other data transmission synchronization tools, and this application does not impose any restrictions on this.

[0065] For example, DTS (Data Transmission Service) can synchronize and migrate data between heterogeneous databases, and can also provide real-time data streaming services. DTS supports data interaction between relational databases (RDBMS), non-relational databases (NoSQL), data multidimensional analysis (OLAP) and other data sources, integrating data synchronization, migration, subscription, integration, and processing to build a secure, scalable, and highly available data architecture. Each module within DTS has a master-slave architecture to ensure high system availability.

[0066] In one possible implementation, a data tool may be called to monitor data definition statements in a database of a first target type, and when changes in field information in the data definition statements are detected, the changed field information may be synchronized to a database of a second target type that depends on the database of the first target type.

[0067] Among them, the data tool can be a data transmission synchronization tool, or it can be other data tools that can monitor data definition statements, and this application does not impose any restrictions on this. Data definition statements can be code statements at the database structure level, which can be used to create, remove, and delete database objects, such as DDL statements (Data Definition Language) in the database. Common DDL statements may include create, alter, drop, and truncate. Correspondingly, when adding, modifying, or deleting data to the database, the field information in the DDL statements in the database will also change accordingly. The DDL statement formats in different types of databases are different. By parsing the DDL code of the database of the first target type (which can be the source database during data synchronization), the DDL code of the database of the first target type is converted into a DDL code that conforms to the syntax of the database of the second target type (which can be the target database during data synchronization), data in different types of databases can be synchronized.

[0068] That is, in the above process, a data tool can be called to monitor the data definition statements DDL statements in the database of the first target type (which can be the transaction database in the above embodiment), and by capturing the field information in the DDL statements, it can be monitored whether the data in the database has changed. When the field information in the DDL statements is captured to be changed, the changed field information can be synchronized to the database of the second target type (which can be the analysis database in the above embodiment) that depends on the database of the first target type.

[0069] Taking the data transmission synchronization tool DTS as an example, the data synchronization migration work of DTS can include three stages, namely structure migration, full data migration and incremental data migration. In the structure migration stage, DTS can recreate the data structure in the target database before migrating the data. For migration between heterogeneous databases, DTS can parse the DDL code of the source database, convert the code into the syntax of the target database, and then recreate the structure objects in the target database. In the full data migration stage, DTS can migrate all the existing data of the source database to the target database, and when the source database remains in operation, data can be continuously updated during the migration process. In the incremental data migration stage, DTS can use the incremental data reading module to obtain the full data, and the incremental data reading module will be activated when the migration starts. In the full data migration stage, the incremental data will be parsed, reformatted and stored on the DTS server. When the full data migration is completed, DTS can retrieve the stored incremental data, reformat and apply the data updates to the target database, and can continue this process until all ongoing data changes are copied to the target database, so that the source database and the target database are fully synchronized.

[0070] For example, when the transaction model information in the transaction database changes, such as adding fields, deleting fields, modifying field length, modifying field type, modifying field name, etc., since such changes correspond to changes in DDL at the database level, you can use the cloud-based data transmission synchronization tool DTS, and use the DDL change perception capability in DTS to achieve automatic real-time synchronization between the transaction database and the analysis database.

[0071] In step S302, database batch operations in the batch processing flow are executed in sequence through databases that match the batch processing type of the database batch operations, wherein after the database batch operation executed earlier is completed, if it is determined that the database batch operation executed later corresponds to databases of different types as the database batch operation executed earlier, and the database batch operation executed later depends on the processing result of the database batch operation executed earlier, the processing result corresponding to the database batch operation executed earlier is synchronized to the database corresponding to the database batch operation executed later by calling a cloud data transmission synchronization tool.

[0072] In the embodiment of the present application, different batch operation processing tasks can be split into different databases for execution. At the same time, in the process of executing different batch operation processing tasks, different databases can be monitored and data between different databases can be synchronized. The method of synchronizing data can be to call a cloud data transmission synchronization tool. As mentioned above, the data transmission synchronization tool can use a synchronization tool such as DTS.

[0073] For example, when the databases that match the batch processing type of the database batch operation belong to two different types of databases, after the first database batch operation is completed, and the database batch operation executed later depends on the processing result of the first database batch operation, the processing result corresponding to the first database batch operation can be synchronized to the database corresponding to the database batch operation executed later by calling the cloud data transmission synchronization tool. For details, see Figure 1 , Figure 2 The examples of batch processing 2 and batch processing 3 are not described here. This application does not have any restrictions on the data structures stored in different types of databases.

[0074] Optionally, a database may correspond to multiple sub-databases of the same type, for example, it may have a dependency structure such as a master database and a slave database. That is, when the databases that match the batch processing type of the database batch operation belong to the same type, if the matched databases belong to the same type but do not belong to the same database, for example, they belong to different sub-databases corresponding to the same type of database, since different sub-databases need to be used for database batch operations, at this time, even if the two sub-databases belong to the same database type, the processing results corresponding to the previously executed database batch operation can be synchronized to the database corresponding to the later executed database batch operation by calling the cloud data transmission synchronization tool in the above manner.

[0075] In a possible implementation, the above-mentioned method of executing database batch operations in the batch processing flow in sequence through databases that match the processing type of the database batch operations can be to first distribute the database tasks corresponding to the database batch operations to the databases that match the batch processing type of the database batch operations, and then, after determining that the database batch operations executed earlier are completed, determining that the database batch operations executed later and the database batch operations executed earlier correspond to databases of different types, and that the database batch operations executed later depend on the processing results of the database batch operations executed earlier, call the cloud data transmission synchronization tool to synchronize the processing results corresponding to the database batch operations executed earlier to the database corresponding to the database batch operations executed later.

[0076] The database batch operation processing of this solution may include two stages, namely, the task distribution stage and the task monitoring stage. In the task distribution stage, the database tasks corresponding to the database batch operation may be distributed to the database matching the batch processing type of the database batch operation, and the batch processing process of the database batch operation may be used for unified scheduling processing, that is, the database type corresponding to the database batch operation may be determined by obtaining the batch processing process, and the corresponding database batch operation tasks may be assigned to the database of the corresponding type for execution.

[0077] In some embodiments, in the above process, the database tasks corresponding to the database batch operations are distributed to the databases that match the batch processing type of the database batch operations. The batch processing flow can be split into multiple database batch operations, and the batch processing type of the database batch operations and at least one previously executed database batch operation that corresponds to the dependency are marked. Before each database batch operation is executed, a corresponding database task is generated according to the database batch operation, and the database tasks are distributed to the databases that match the batch processing type, so that each database executes the corresponding database task.

[0078] The time for generating corresponding database tasks according to the database batch operations and distributing the database tasks to the databases matching the batch processing type can be before all database batch operations, or after the database batch operations executed earlier are completed, the database tasks are distributed to the databases matching the batch processing type corresponding to the database batch operations executed later. As long as the distribution is before the database executes the corresponding database tasks, this application does not impose any restrictions on this.

[0079] In some embodiments, the above process calls the cloud data transmission synchronization tool to synchronize the processing results corresponding to the previously executed database batch operation to the database corresponding to the subsequently executed database batch operation. The method may be to first synchronize the operation transaction log of the previously executed database batch operation to the database corresponding to the subsequently executed database batch operation, and then obtain the first timestamp of the currently synchronized operation transaction log through the cloud data transmission synchronization tool. When it is determined that the first timestamp is consistent with the second timestamp of the operation transaction log of the previously executed database batch operation, it is determined that the synchronization of the operation transaction log is completed.

[0080] Correspondingly, the time of obtaining the first timestamp of the currently synchronized operation transaction log through the cloud data transmission synchronization tool can be obtained at a time node at a fixed time interval or at a random time node, and this application does not impose any limitation on this.

[0081] That is, the synchronization point of the cloud data transmission synchronization tool DTS can be used to determine whether to enable the subsequent database batch operation. Figure 1 As shown, the batch process scheduling can be used to call the DTS API to read the batch 2 run end time. When the previously executed database batch operation (batch 2) is processed in the transaction database, the transaction database will generate an operation transaction log. Each operation transaction log will generate a site, which records information related to the operation transaction log, including a timestamp. The timestamp can then be extracted from the site to mark the generation time of the operation transaction log in the transaction database. The timestamp corresponding to the batch run end time of batch 2 in the transaction database can be recorded as the "second timestamp".

[0082] When the first timestamp obtained by the cloud data transmission synchronization tool is consistent with the second timestamp, it can be explained that the operation transaction log of the database batch operation executed first (that is, all operation transaction logs of the database batch operation on batch 2) has been fully synchronized to the database (analysis database) corresponding to the database batch operation executed later (that is, batch 3). At this time, it can be determined that the synchronization of the operation transaction log is completed, and then the database batch operation for batch 3 can be started on the analysis database.

[0083] Correspondingly, before calling the cloud-based data transmission synchronization tool to synchronize the processing results corresponding to the previously executed database batch operation to the database corresponding to the subsequently executed database batch operation, the above-mentioned process of executing the database batch operations in the batch processing flow in sequence through the databases matching the processing types of the database batch operations may also include: when the previously executed database batch operation is completed, calling the cloud-based data transmission synchronization tool to obtain the second timestamp of the operation transaction log of the previously executed database batch operation.

[0084] The present application also provides a batch operation processing method, such as Figure 4 The flowchart of a batch operation processing method 400 according to an embodiment of the present application is shown. The method 400 may include:

[0085] In step S401, a batch processing flow is obtained. The batch processing flow may include multiple database batch operations executed sequentially. The multiple database batch operations may include transaction batch processing operations and analysis batch processing operations corresponding to accessing different types of databases.

[0086] In step S402, database batch operations in the batch processing flow are executed in sequence through databases that match the processing type of the database batch operations. After the transaction batch processing operation executed first is completed, if it is determined that the analysis batch processing operation executed later and the transaction batch processing operation executed first correspond to databases of different types, and the analysis batch processing operation executed later depends on the processing result of the transaction batch processing operation executed first, a cloud data transmission synchronization tool is called to synchronize the processing result corresponding to the transaction batch processing operation executed first to the database corresponding to the report batch processing operation executed later.

[0087] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a database system, such as Figure 5 FIG. 5 is a structural block diagram of a database system 500 according to an embodiment of the present application.

[0088] The database system may include multiple databases for processing database batch operations of different batch processing types, such as database 1, database 2, or a transaction database, an analysis database, etc. The database system may also include a process management node to execute the batch operation processing method provided in the embodiment of the present application.

[0089] The process management nodes may include:

[0090] A process acquisition module, used to acquire a batch processing process, wherein the batch processing process includes a plurality of database batch operations executed sequentially, and the plurality of database batch operations correspond to a plurality of batch processing types for accessing different types of databases;

[0091] A database scheduling module is used to execute the database batch operations in the batch processing flow in sequence through databases that match the batch processing type of the database batch operations, wherein after the database batch operation executed earlier is completed, if it is determined that the database batch operation executed later corresponds to a different type of database than the database batch operation executed earlier, and the database batch operation executed later depends on the processing result of the database batch operation executed earlier, the processing result corresponding to the database batch operation executed earlier is synchronized to the database corresponding to the database batch operation executed later by calling a cloud data transmission synchronization tool.

[0092] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a batch operation processing device. Figure 6 FIG. 6 is a block diagram of a batch operation processing device 600 according to an embodiment of the present application. The device 600 may include:

[0093] A process acquisition module 601 is used to acquire a batch processing process, wherein the batch processing process includes a plurality of database batch operations executed sequentially, and the plurality of database batch operations correspond to a plurality of batch processing types for accessing different types of databases;

[0094] The database scheduling module 602 is used to execute the database batch operations in the batch processing process in sequence through the databases that match the batch processing type of the database batch operations, wherein, after the database batch operation executed earlier is completed, if it is determined that the database batch operation executed later corresponds to a different type of database than the database batch operation executed earlier, and the database batch operation executed later depends on the processing result of the database batch operation executed earlier, the processing result corresponding to the database batch operation executed earlier is synchronized to the database corresponding to the database batch operation executed later by calling the cloud data transmission synchronization tool.

[0095] In a possible implementation, the database scheduling module 602 may include:

[0096] A database task distribution submodule, used to distribute the database tasks corresponding to the database batch operation to a database matching the batch processing type of the database batch operation;

[0097] A database type determination submodule, used to determine, after the previously executed database batch operation is completed, that the subsequently executed database batch operation corresponds to a different type of database than the previously executed database batch operation, and that the subsequently executed database batch operation depends on the processing result of the previously executed database batch operation;

[0098] The synchronization tool calling submodule is used to call the cloud data transmission synchronization tool to synchronize the processing results corresponding to the previously executed database batch operation to the database corresponding to the subsequently executed database batch operation.

[0099] In some embodiments, the above database task distribution submodule may include:

[0100] A batch operation splitting unit, used to split the batch processing flow into multiple database batch operations, and mark the batch processing type of the database batch operation and at least one previously executed database batch operation on which it depends;

[0101] The database task execution unit is used to generate corresponding database tasks according to the database batch operations before executing each database batch operation, and distribute the database tasks to databases matching the batch processing type so that each database executes the corresponding database task.

[0102] In some embodiments, the synchronization tool calling submodule may include:

[0103] An operation transaction log synchronization unit, used to synchronize the operation transaction log of the previously executed database batch operation to the database corresponding to the subsequently executed database batch operation;

[0104] A first timestamp acquisition unit, configured to acquire a first timestamp of a currently synchronized operation transaction log through the cloud data transmission synchronization tool;

[0105] The timestamp consistency determination unit is used to determine the end of synchronization of the operation transaction log when it is determined that the first timestamp is consistent with the second timestamp of the operation transaction log of the previously executed database batch operation.

[0106] In a possible implementation, before calling the cloud data transmission synchronization tool to synchronize the processing result corresponding to the previously executed database batch operation to the database corresponding to the subsequently executed database batch operation, the database scheduling module 602 may further include:

[0107] The second timestamp acquisition submodule is used to call the cloud data transmission synchronization tool to obtain the second timestamp of the operation transaction log of the previously executed database batch operation when the previously executed database batch operation is completed.

[0108] In a possible implementation, the above device may further include:

[0109] A data definition statement monitoring module, used for calling a data tool to monitor data definition statements in a database of the first target type;

[0110] The changed field information synchronization module is used to synchronize the changed field information to a database of a second target type that depends on the database of the first target type when a change in the field information in the data definition statement is captured.

[0111] In a possible implementation, the database batch operation includes a transaction batch processing operation and an analysis batch processing operation respectively executed based on the database.

[0112] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a batch operation processing device. Figure 7 FIG. 1 is a block diagram of a batch operation processing device 700 according to an embodiment of the present application. The device 700 may include:

[0113] A processing flow acquisition module 701 is used to acquire a batch processing flow, wherein the batch processing flow includes a plurality of database batch operations executed sequentially, wherein the plurality of database batch operations include transaction batch processing operations and analysis batch processing operations corresponding to accessing different types of databases;

[0114] The batch operation execution module 702 is used to execute the database batch operations in the batch processing process in sequence through the databases that match the processing type of the database batch operations, wherein after the transaction batch processing operation executed first is completed, when it is determined that the analysis batch processing operation executed later corresponds to a different type of database than the transaction batch processing operation executed first, and the analysis batch processing operation executed later depends on the processing result of the transaction batch processing operation executed first, a cloud data transmission synchronization tool is called to synchronize the processing result corresponding to the transaction batch processing operation executed first to the database corresponding to the report batch processing operation executed later.

[0115] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0116] Figure 8 FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 8 As shown, the electronic device includes: a memory 801 and a processor 802. The memory 801 stores a computer program that can be run on the processor 802. When the processor 802 executes the computer program, the method in the above embodiment is implemented. The number of the memory 801 and the processor 802 can be one or more.

[0117] The electronic device also includes:

[0118] The communication interface 803 is used to communicate with external devices and perform data exchange transmission.

[0119] If the memory 801, the processor 802 and the communication interface 803 are implemented independently, the memory 801, the processor 802 and the communication interface 803 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0120] Optionally, in a specific implementation, if the memory 801, the processor 802 and the communication interface 803 are integrated on a chip, the memory 801, the processor 802 and the communication interface 803 can communicate with each other through an internal interface.

[0121] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0122] An embodiment of the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method provided in any embodiment of the present application is implemented.

[0123] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0124] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the present application.

[0125] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0126] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0127] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0128] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0129] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0130] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0131] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.

[0132] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0133] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0134] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A batch operation processing method, comprising: Acquire a batch processing flow, wherein the batch processing flow includes a plurality of database batch operations executed sequentially, wherein the plurality of database batch operations correspond to a plurality of batch processing types for accessing different types of databases; The database batch operations in the batch processing flow are executed sequentially through databases matching the batch processing type of the database batch operations, wherein after the database batch operation executed earlier is completed, when it is determined that the database batch operation executed later corresponds to databases of different types as the database batch operation executed earlier, and the database batch operation executed later depends on the processing result of the database batch operation executed earlier, the processing result corresponding to the database batch operation executed earlier is synchronized to the database corresponding to the database batch operation executed later by calling a cloud data transmission synchronization tool.

2. The method according to claim 1, wherein: The executing of the database batch operation in the batch processing flow sequentially through the databases matching the processing type of the database batch operation comprises: Distributing the database tasks corresponding to the database batch operation to a database matching the batch processing type of the database batch operation; After the previously executed database batch operation is completed, determining that the subsequently executed database batch operation corresponds to a different type of database than the previously executed database batch operation, and that the subsequently executed database batch operation depends on a processing result of the previously executed database batch operation; The cloud data transmission synchronization tool is called to synchronize the processing result corresponding to the previously executed database batch operation to the database corresponding to the subsequently executed database batch operation.

3. The method according to claim 2, wherein: The step of distributing the database tasks corresponding to the database batch operation to a database matching the batch processing type of the database batch operation includes: Splitting the batch processing flow into multiple database batch operations, and marking the batch processing type of the database batch operation and at least one previously executed database batch operation that corresponds to the dependency; Before each database batch operation is executed, a corresponding database task is generated according to the database batch operation, and the database task is distributed to databases matching the batch processing type so that each database executes the corresponding database task.

4. The method according to claim 2, wherein: The calling of the cloud data transmission synchronization tool to synchronize the processing result corresponding to the previously executed database batch operation to the database corresponding to the subsequently executed database batch operation includes: Synchronize the operation transaction log of the database batch operation executed previously to the database corresponding to the database batch operation executed later; Obtaining a first timestamp of a currently synchronized operation transaction log through the cloud data transmission synchronization tool; When it is determined that the first timestamp is consistent with the second timestamp of the operation transaction log of the previously executed database batch operation, it is determined that the synchronization of the operation transaction log is completed.

5. The method according to claim 4, wherein: Before the calling of the cloud data transmission synchronization tool to synchronize the processing result corresponding to the previously executed database batch operation to the database corresponding to the subsequently executed database batch operation, the executing of the database batch operation in the batch processing flow sequentially through the databases matching the processing type of the database batch operation further includes: When the previously executed database batch operation is completed, the cloud data transmission synchronization tool is called to obtain the second timestamp of the operation transaction log of the previously executed database batch operation.

6. The method according to claim 1, wherein: The method further comprises: Invoking a data tool to monitor data definition statements in a database of the first target type; In the case where a change in the field information in the data definition statement is captured, the changed field information is synchronized to a database of a second target type that depends on the database of the first target type.

7. The method according to claim 1, wherein: The database batch operations include transaction batch processing operations and analysis batch processing operations respectively executed based on the database.

8. A batch operation processing method, comprising: Acquire a batch processing flow, wherein the batch processing flow includes a plurality of database batch operations executed sequentially, wherein the plurality of database batch operations include transaction batch processing operations and analysis batch processing operations corresponding to accessing different types of databases; The database batch operations in the batch processing flow are executed sequentially through databases that match the processing type of the database batch operations, wherein after the transaction batch processing operation executed first is completed, when it is determined that the analysis batch processing operation executed later corresponds to a different type of database than the transaction batch processing operation executed first, and the analysis batch processing operation executed later depends on the processing result of the transaction batch processing operation executed first, a cloud data transmission synchronization tool is called to synchronize the processing result corresponding to the transaction batch processing operation executed first to the database corresponding to the report batch processing operation executed later.

9. A database system comprising a plurality of databases for processing database batch operations of different batch processing types, and a process management node; The process management node includes: A process acquisition module, used to acquire a batch processing process, wherein the batch processing process includes a plurality of database batch operations executed sequentially, and the plurality of database batch operations correspond to a plurality of batch processing types for accessing different types of databases; A database scheduling module is used to execute the database batch operations in the batch processing flow in sequence through databases that match the batch processing type of the database batch operations, wherein after the database batch operation executed earlier is completed, if it is determined that the database batch operation executed later corresponds to a different type of database than the database batch operation executed earlier, and the database batch operation executed later depends on the processing result of the database batch operation executed earlier, the processing result corresponding to the database batch operation executed earlier is synchronized to the database corresponding to the database batch operation executed later by calling a cloud data transmission synchronization tool.

10. A batch operation processing device, comprising: A process acquisition module, used to acquire a batch processing process, wherein the batch processing process includes a plurality of database batch operations executed sequentially, and the plurality of database batch operations correspond to a plurality of batch processing types for accessing different types of databases; A database scheduling module is used to execute the database batch operations in the batch processing flow in sequence through databases that match the batch processing type of the database batch operations, wherein after the database batch operation executed earlier is completed, if it is determined that the database batch operation executed later corresponds to a different type of database than the database batch operation executed earlier, and the database batch operation executed later depends on the processing result of the database batch operation executed earlier, the processing result corresponding to the database batch operation executed earlier is synchronized to the database corresponding to the database batch operation executed later by calling a cloud data transmission synchronization tool.

11. A batch operation processing device, comprising: A processing flow acquisition module, used to acquire a batch processing flow, wherein the batch processing flow includes a plurality of database batch operations executed sequentially, wherein the plurality of database batch operations include transaction batch processing operations and analysis batch processing operations corresponding to accessing different types of databases; A batch operation execution module is used to execute the database batch operations in the batch processing flow in sequence through databases that match the processing type of the database batch operations, wherein after the transaction batch processing operation executed first is completed, when it is determined that the analysis batch processing operation executed later corresponds to a different type of database than the transaction batch processing operation executed first, and the analysis batch processing operation executed later depends on the processing result of the transaction batch processing operation executed first, a cloud data transmission synchronization tool is called to synchronize the processing result corresponding to the transaction batch processing operation executed first to the database corresponding to the report batch processing operation executed later.

12. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

13. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images