Event processing method, apparatus, device, and storage medium
By judging the table partitioning status and event type of the upstream database during the data migration process, and flexibly handling DDL and DML events, the problem of table structure changes during data synchronization is solved, and efficient data synchronization is achieved.
Patent Information
- Application Number
- CN202210398903.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-15
- Publication Date
- 2026-02-10
- Estimated Expiration
- 2042-04-15
AI Technical Summary
In existing technologies, during data migration, the different execution order of DDL events in the various tables of the upstream database leads to changes in the merged table structure of the downstream database, affecting data synchronization.
By obtaining data update events from the upstream database, we can determine the table partitioning status and event type, flexibly handle DDL and DML events, ensure table structure compatibility, adjust the event processing order, and avoid synchronization conflicts.
It enables flexible adjustment of data update event handling methods after data migration, reduces data synchronization latency, and improves the applicability and efficiency of data synchronization.
Smart Images
Figure CN114691651B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer, and particularly relates to an event processing method and device, equipment and a storage medium. BACKGROUND
[0002] Data migration refers to a process of migrating data stored in an upstream database to a downstream database. Data migration in a database often includes full data migration at a certain time and incremental data migration after the time. During the full data migration process, the data of the sub-tables in the upstream database is written into a union table in the downstream database.
[0003] After the full data migration, the data definition language (DDL) and the data manipulation language (DML) corresponding to the data update of the upstream database are synchronized to the downstream database, such as the DDL and DML corresponding to the incremental data of the upstream database are synchronized to the downstream database.
[0004] However, the specific event content and execution order of the DDL events corresponding to each sub-table in the upstream database may not be the same. If the DDL events corresponding to each sub-table in the upstream database are synchronized to the downstream database according to the execution time order of the DDL events, the table structure of the data union table in the downstream database may change and affect data synchronization. SUMMARY
[0005] The embodiments of the present application provide an event processing method, device, equipment and storage medium, which can flexibly process the data update events corresponding to the sub-tables in the upstream database, and have high applicability.
[0006] In a first aspect, the embodiments of the present application provide an event processing method, which comprises:
[0007] obtaining a data update event when a target sub-table in an upstream database is updated;
[0008] in response to the target sub-table not being in a preset state and the data update event not being executed for a downstream database, determining that the data update event includes a data definition language (DDL) event or a data manipulation language (DML) event;
[0009] in response to the data update event including the DML event, executing the data update event for the downstream database;
[0010] In response to the aforementioned data update events, including the aforementioned DDL events, the compatibility status of the target table structure before the data update of the target upstream sub-table with the table structures of other upstream sub-tables in the aforementioned upstream database is determined, and the aforementioned data update events are processed based on the aforementioned compatibility status.
[0011] Secondly, embodiments of this application provide an event processing apparatus, the apparatus comprising:
[0012] The event acquisition module is used to acquire data update events when the target upstream table in the upstream database is updated.
[0013] The event determination module is used to determine whether the data update event includes a Data Definition Language (DDL) event or a Data Manipulation Language (DML) event in response to the above-mentioned upstream partition table not being in a preset state and not executing the above-mentioned data update event for the downstream database.
[0014] The event handling module is used to respond to the aforementioned data update events, including the aforementioned DML events, and to execute the aforementioned data update events for the aforementioned downstream databases;
[0015] The aforementioned event handling module is used to respond to the aforementioned data update events, including the aforementioned DDL events, determine the compatibility status between the target table structure before the data update of the target upstream sub-table and the table structures of other upstream sub-tables in the aforementioned upstream database, and process the aforementioned data update events based on the aforementioned compatibility status.
[0016] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, which are interconnected;
[0017] The memory is used to store computer programs;
[0018] The processor is configured to execute the event handling method provided in the embodiments of this application when the computer program is invoked.
[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that is executed by a processor to implement the event handling method provided in embodiments of this application.
[0020] In scenarios where data update events corresponding to data updates of upstream tables in the upstream database are synchronized to the downstream database after a full data migration, the event handling method provided in this application embodiment can flexibly adjust the handling method of data update events according to the execution status of data update events and the table structure compatibility status of each data table in the upstream database. This achieves data synchronization between the upstream and downstream databases while reducing data synchronization latency, making it highly applicable. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 This is a flowchart illustrating the event handling method provided in an embodiment of this application;
[0023] Figure 2 This is a schematic diagram of the event processing based on a coordinator and an executor provided in an embodiment of this application;
[0024] Figure 3 This is a schematic diagram of the structure of the event processing device provided in the embodiments of this application;
[0025] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0027] The event handling method provided in this application embodiment can be applied to the process of synchronizing the data update events of each upstream sub-table of the upstream database to the downstream database after the full data of the upstream database has been migrated to the downstream database, so as to process the aforementioned data update events.
[0028] During the full data migration process, the sharded data in the upstream database will be written into the merged table in the downstream database.
[0029] The data update events in the embodiments of this application include DDL events or DML events.
[0030] DDL (Data Definition Language) is used to manipulate objects and their attributes. These objects include the database itself, as well as database objects such as tables and views. The specific methods by which DDL manages and defines these objects and attributes are reflected in operations such as Create, Drop, and Alter. Taking tables as an example: Create creates a table, Alter modifies its fields, and Drop deletes it.
[0031] DML is used to manipulate the data contained in database objects; that is, the unit of operation is a record. For example, DML can be used to insert a record into a data table, delete one or more records from a data table, or modify the contents of existing records in a table.
[0032] See Figure 1 , Figure 1 This is a flowchart illustrating the event handling method provided in an embodiment of this application. Figure 1 As shown, the event handling method provided in this application embodiment may include the following steps:
[0033] Step S11: Obtain the data update event when the target upstream sub-table in the upstream database is updated.
[0034] The target upstream table can be any upstream table in the upstream database, and the upstream database can be one database or multiple databases, without any restrictions.
[0035] The data update event can be a binlog event, or other related data, logs, etc. used to characterize the upstream database's DDL or DML events during the data update process; there are no restrictions here.
[0036] In the case where the data update event is a binlog event, the upstream database must support the binlog protocol.
[0037] Here, binlog events refer to binary logs, which are generated when an upstream database updates data and converts the executed DDL or DML statements into binary log format for subsequent data replication or recovery. In the embodiments of this application, the data update event (such as a binlog event) can be a single DDL event or a complete DML event corresponding to each upstream table in the binary log.
[0038] Step S12: In response to the target upstream partition not being in a preset state and not executing a data update event for the downstream database, determine whether the data update event includes a DDL event or a DML event.
[0039] In some feasible implementations, after obtaining the data update event when the target upstream table (i.e., any upstream table) of the upstream database is updating data, it can be determined whether the target upstream table is in a preset state and whether the data update event is executed for the downstream database.
[0040] When the target upstream sub-table is in a preset state, it means that the table structure of the target upstream sub-table before data update (hereinafter referred to as the target table structure for ease of description) is incompatible (conflicting) with the table structure of any other upstream sub-table in the upstream database, and at least one other upstream sub-table in the upstream database has not performed the same data update operation; otherwise, the target upstream sub-table is not in a preset state.
[0041] In other words, for a target upstream sub-table in the upstream database, if the target table structure before the data update is incompatible with the table structure of at least one other upstream sub-table in the upstream database, and at least one upstream sub-table in the upstream database has not received the same data update event, then the target upstream sub-table is marked as a preset state; otherwise, the target upstream sub-table is not in a preset state.
[0042] Specifically, if the target upstream partition is not in a preset state and the data update event is not executed for the downstream database, the DDL event or DML event included in the data update event corresponding to the target upstream partition can be determined. That is, the data update event corresponding to the target upstream partition is determined to be a DDL event or a DML event, and then different event processing methods are adopted for the specific event content of the data update event corresponding to the target upstream partition.
[0043] Optionally, if the target upstream table is in a preset state, that is, if the target table structure before the data update of the target upstream table is incompatible with the table structure of at least one other upstream table in the upstream database, and at least one upstream table in the upstream database has not received the same data update event, the data update event corresponding to the target upstream table is skipped, that is, the DDL event or DML event corresponding to the target upstream table is not executed.
[0044] Optionally, if the target upstream table is in a preset state, that is, if the target table structure before the data update is performed is incompatible with the table structure of at least one other upstream table in the upstream database, and at least one upstream table in the upstream database has not received the same data update event, then if the same data update event has been performed on the downstream database before, the data update event corresponding to the target upstream table can also be skipped and there is no need to perform it again.
[0045] Step S13: In response to the data update event, including the DML event, execute a data update event for the downstream database.
[0046] In some feasible implementations, if the target upstream partition table is not in a preset state and the data update event is not executed for the downstream database, then if the data update event corresponding to the target upstream partition table is a DML event, the DML event can be executed directly for the downstream database.
[0047] If the target upstream partition is not in a preset state and the data update event is not executed for the downstream database, and the data update event corresponding to the target upstream partition is a DML event, then the same data operation can be performed on the data merge table corresponding to the target upstream partition in the downstream database based on the DML event.
[0048] Step S14: In response to data update events, including DDL events, determine the compatibility status of the target table structure before the data update with the table structures of other upstream tables in the upstream database, and process the data update event based on the compatibility status.
[0049] In some feasible implementations, if the target upstream table is not in a preset state and the data update event is not executed for the downstream database, and the data update event corresponding to the target upstream table is a DDL event, the data update event (i.e., the DDL event) can be processed based on the compatibility status between the target table structure before the data update and the table structure of other upstream tables in the upstream database.
[0050] Specifically, if the target table structure corresponding to the target upstream sub-table is compatible with the table structures of other upstream sub-tables in the upstream database, the union of the target table structure with the table structures of other upstream sub-tables in the upstream database (hereinafter referred to as the first table structure union for ease of description) and the union of the target upstream sub-table structure after data update with the table structures of other upstream sub-tables in the upstream database (hereinafter referred to as the second table structure union for ease of description) can be further determined.
[0051] Among them, the union of the table structure of the target upstream sub-table before data update and the first table structure of other upstream sub-tables in the upstream database can also be regarded as the table structure of the data merge table corresponding to the current target upstream sub-table in the downstream database.
[0052] Furthermore, after determining the union of the first and second table structures corresponding to the target upstream sub-table, it can be determined whether the union of the first and second table structures is the same. If the union of the first and second table structures is different, that is, the union of the table structures of the target upstream sub-table and other upstream sub-tables in the upstream database changes as the target upstream sub-table's data is updated, then a data update event is executed for the downstream database. That is, based on the DDL event, the same operation is performed on the corresponding data merge table of the target upstream sub-table in the downstream data, such as changing the fields of the data table or renaming a column.
[0053] If the union of the first table structure and the union of the second table structure are the same, that is, the union of the table structures of the target upstream partition table and other upstream partition tables in the upstream database has not changed with the data update of the target upstream partition table, then the data update event can be skipped, that is, the corresponding DDL event is not executed for the downstream database.
[0054] Optionally, if the target table structure corresponding to the target upstream sub-table is incompatible with the table structure of at least one other upstream sub-table in the upstream database, it is determined whether the other upstream sub-tables in the upstream database have performed the same data update operation, that is, whether the other upstream sub-tables in the upstream database have received the same data update event (DDL event).
[0055] If at least one upstream table in the upstream database has not performed the same data update operation, that is, if at least one upstream table in the upstream database has not received the same data update event (DDL event), then the target upstream table is marked as a preset state and the data update event (DDL event) is skipped.
[0056] If all upstream tables in the aforementioned database have undergone the same data update operation, meaning that each upstream table in the upstream database has received the same data update event (DDL event), then the data update operation can be performed on the downstream database. This means that based on the DDL event, the same operation can be performed on the corresponding merged table of the target upstream table in the downstream data, such as changing the fields of the data table or renaming a column.
[0057] Specifically, for a given DDL event (hereinafter referred to as the target DDL event for ease of description), when the target DDL event is first obtained from the upstream database, if the table structure of the upstream partition corresponding to the DDL event before executing the target DDL event is incompatible with the table structure of at least one other upstream partition in the upstream database, then the upstream partition is marked as a preset state. If a target DDL event is received again afterward, and the table structure of the upstream partition corresponding to this DDL event after executing the target DDL event is incompatible with the table structure of at least one other upstream partition in the upstream database, and there are still upstream partitions in the upstream database that have not executed the target DDL event, then the upstream partition corresponding to this target DDL event is still marked as a preset state. This process continues until all upstream partitions in the upstream database have executed the target DDL event, at which point the DDL event can be executed on the corresponding merged table in the downstream database.
[0058] Since data update events (such as binlog events) are typically written to update log files (such as binlog files) in the order of their commits, the order in which data update events are retrieved for any upstream table in the upstream database corresponds to the commit order of the data update events. Therefore, when an upstream table corresponding to the same DDL event is marked as a preset state, the commit order of the DDL can be determined, which in turn determines the specific location of the DDL event in the update log file (such as the binlog file). After all upstream tables in the upstream database have received or executed the DDL event, the preset states of the upstream tables corresponding to the DDL event can be cleared, and the specific location of the DDL event in the binlog text can be determined. This allows for the further execution of the DDL and subsequent DML events on the merged tables in the downstream database.
[0059] The event handling method provided in this application embodiment can be executed by an independent logic component, or by two logic components: a coordinator and an executor.
[0060] In this system, an upstream database can correspond to an executor, and a data merge table in a downstream database can correspond to at least one executor and a coordinator.
[0061] For details regarding the execution of the event handling method provided in this application's embodiments based on the actuator and coordinator, please refer to... Figure 2 , Figure 2 This is a schematic diagram illustrating the event processing based on a coordinator and executor, provided in an embodiment of this application. For example... Figure 2 As shown
[0062] Step 1: After the executor starts, check whether each upstream table in the upstream database has received and executed the same DDL event.
[0063] Step 2: If all upstream tables in the upstream database have received and executed the same DDL event, then clear the conflict state (i.e., the preset state) of each upstream table corresponding to the DDL event, and start executing the corresponding data merge table in the downstream database, as well as the DDL event and subsequent DML events, starting from the first time the coordinator receives the DDL event.
[0064] Step 3: If all upstream tables in the upstream database have received and executed the same DDL event, then obtain the new binlog event (data update event) from the upstream database.
[0065] Step 4: After the executor obtains the binlog event, it determines whether the upstream partition table corresponding to the binlog event is in a conflict state.
[0066] Step 5: If the upstream partition table corresponding to the binlog event is in a conflict state, skip the binlog event and re-execute step 1.
[0067] Step 6: Obtain other binlog events. If the upstream partition corresponding to the binlog event is not in a conflict state, determine whether the binlog event has been executed for the downstream database. If the binlog event has already been executed for the downstream database, skip the binlog event and re-execute Step 1.
[0068] Step 7: If the binlog event has not been executed against the downstream database, it is necessary to determine whether the binlog event is a DDL event or a DML event. If the binlog event is a DML event, proceed to step 8; if the binlog event is a DDL event, proceed to step 9.
[0069] Step 8: Execute the binlog event downstream, that is, execute the binlog event on the downstream database.
[0070] Step 9: After receiving the DDL event, send the DDL event to the coordinator and send the table structure after the corresponding upstream table partitioning DDL event to the coordinator.
[0071] Step 10: The coordinator determines whether the table structure of the upstream partition table after executing the DDL event conflicts with the table structure of other upstream partition tables in the upstream database. If there is a conflict, proceed to step 11; otherwise, proceed to step 12.
[0072] Step 11: The coordinator determines whether all upstream tables in the upstream database have received the DDL event. If all upstream tables in the upstream database have received the DDL event, the coordinator notifies the executor to execute Step 2. If at least one upstream table in the upstream database has not received the DDL event, the executor skips the DDL event and marks that upstream table as conflicted.
[0073] Step 12: Determine whether the union of the table structure of the upstream table after executing the DDL event and the table structures of other upstream tables in the upstream database has changed compared to the union of the table structure of the upstream table before executing the DDL event and the table structures of other upstream tables in the upstream database. If it has changed, notify the executor to execute the DDL for the downstream database. If it has not changed, notify the executor to skip the DDL event.
[0074] In scenarios where DDL and DML events corresponding to data updates in the upstream database are synchronized to the downstream database after a full data migration, the event processing method provided in this application embodiment allows for data synchronization between the upstream and downstream databases even when the table structures of the upstream sub-tables in the upstream database are different and the execution order of the DDL events corresponding to the upstream sub-tables is different. This avoids synchronization interruptions caused by coordinating the synchronization order of various DDLs, reduces data synchronization latency, and has high applicability.
[0075] See Figure 3 , Figure 3 This is a schematic diagram of the event processing device provided in an embodiment of this application. The event processing device provided in an embodiment of this application includes:
[0076] Event acquisition module 31 is used to acquire data update events when the target upstream sub-table is updated in the upstream database;
[0077] Event determination module 32 is used to determine, in response to the above-mentioned upstream partition table not being in a preset state and not executing the above-mentioned data update event for the downstream database, whether the above-mentioned data update event includes a Data Definition Language (DDL) event or a Data Manipulation Language (DML) event.
[0078] Event handling module 33 is used to respond to the data update event, including the DML event, and execute the data update event for the downstream database.
[0079] The event handling module 33 is used to respond to the data update event, including the DDL event, to determine the compatibility status between the target table structure before the data update of the target upstream sub-table and the table structure of other upstream sub-tables in the upstream database, and to process the data update event based on the compatibility status.
[0080] In some feasible implementations, the event handling module 33 described above is used for:
[0081] In response to the fact that the target table structure is compatible with the table structures of other upstream sub-tables in the upstream database, the first table structure union of the target table structure and the table structures of other upstream sub-tables in the upstream database, and the second table structure union of the target upstream sub-table after data update and the table structures of other upstream sub-tables in the upstream database are determined.
[0082] Based on the union of the first table structure and the union of the second table structure, the above data update events are processed.
[0083] In some feasible implementations, the event handling module 33 described above is used for:
[0084] If the union of the first table structure and the union of the second table structure are the same, skip the data update event.
[0085] In response to the difference between the union of the first table structure and the union of the second table structure, the aforementioned data update event is executed for the aforementioned downstream database.
[0086] In some feasible implementations, the event handling module 33 described above is further used for:
[0087] In response to the fact that the target table structure is incompatible with the table structure of at least one other upstream sub-table in the upstream database, and that at least one upstream sub-table in the upstream database has not performed the data update operation corresponding to the data update event, the data update event is skipped, and the target upstream sub-table is marked as the preset state.
[0088] In some feasible implementations, the event handling module 33 described above is further used for:
[0089] In response to the fact that the target table structure is incompatible with the table structure of at least one other upstream sub-table in the upstream database, and all upstream sub-tables in the upstream database have performed the data update operation corresponding to the data update event, the preset state marked on the target upstream sub-table is cleared, and the data update event is executed for the downstream database.
[0090] In some feasible implementations, the event handling module 33 described above is further used for:
[0091] In response to the upstream sub-table of the target being in the aforementioned preset state, the aforementioned data update event is skipped.
[0092] In some feasible implementations, the event handling module 33 described above is further used for:
[0093] If the target upstream table is not in the preset state, and the data update event corresponding to any other upstream table in the upstream database has been executed for the downstream database before the data update event corresponding to the target upstream table is received, the data update event is skipped.
[0094] In specific implementation, the aforementioned event handling device can execute the above-described actions through its built-in functional modules. Figure 1 The implementation methods provided for each step are detailed in the above-mentioned implementation methods, and will not be repeated here.
[0095] See Figure 4 , Figure 4 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 4As shown, the electronic device 400 in this embodiment may include: a processor 401, a network interface 404, and a memory 405. Furthermore, the electronic device 400 may also include: a user interface 403, and at least one communication bus 402. The communication bus 402 is used to enable communication between these components. The user interface 403 may include a display screen and a keyboard; optionally, the user interface 403 may also include a standard wired interface or a wireless interface. The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 404 may be a high-speed RAM or non-volatile memory (NVM), such as at least one disk storage device. The memory 405 may optionally be at least one storage device located remotely from the processor 401. Figure 4 As shown, the memory 405, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.
[0096] exist Figure 4 In the illustrated electronic device 400, the network interface 404 provides network communication functionality; the user interface 403 is primarily used to provide an input interface for the user; and the processor 401 can be used to call the device control application stored in the memory 405 to achieve:
[0097] Retrieve data update events when the target upstream table in the upstream database is updated;
[0098] In response to the fact that the upstream partition of the target is not in the preset state and the above data update event is not executed for the downstream database, it is determined that the above data update event includes a Data Definition Language (DDL) event or a Data Manipulation Language (DML) event.
[0099] In response to the aforementioned data update events, including the aforementioned DML events, the aforementioned data update events are executed for the aforementioned downstream databases;
[0100] In response to the aforementioned data update events, including the aforementioned DDL events, the compatibility status of the target table structure before the data update of the target upstream sub-table with the table structures of other upstream sub-tables in the aforementioned upstream database is determined, and the aforementioned data update events are processed based on the aforementioned compatibility status.
[0101] In some feasible implementations, the processor 401 described above is used for:
[0102] In response to the fact that the target table structure is compatible with the table structures of other upstream sub-tables in the upstream database, the first table structure union of the target table structure and the table structures of other upstream sub-tables in the upstream database, and the second table structure union of the target upstream sub-table after data update and the table structures of other upstream sub-tables in the upstream database are determined.
[0103] Based on the union of the first table structure and the union of the second table structure, the above data update events are processed.
[0104] In some feasible implementations, the processor 401 described above is used for:
[0105] If the union of the first table structure and the union of the second table structure are the same, skip the data update event.
[0106] In response to the difference between the union of the first table structure and the union of the second table structure, the aforementioned data update event is executed for the aforementioned downstream database.
[0107] In some feasible implementations, the processor 401 is further configured to:
[0108] In response to the fact that the target table structure is incompatible with the table structure of at least one other upstream sub-table in the upstream database, and that at least one upstream sub-table in the upstream database has not performed the data update operation corresponding to the data update event, the data update event is skipped, and the target upstream sub-table is marked as the preset state.
[0109] In some feasible implementations, the processor 401 is further configured to:
[0110] In response to the fact that the target table structure is incompatible with the table structure of at least one other upstream sub-table in the upstream database, and all upstream sub-tables in the upstream database have performed the data update operation corresponding to the data update event, the preset state marked on the target upstream sub-table is cleared, and the data update event is executed for the downstream database.
[0111] In some feasible implementations, the processor 401 is further configured to:
[0112] In response to the upstream sub-table of the target being in the aforementioned preset state, the aforementioned data update event is skipped.
[0113] In some feasible implementations, the processor 401 is further configured to:
[0114] If the target upstream table is not in the preset state, and the data update event corresponding to any other upstream table in the upstream database has been executed for the downstream database before the data update event corresponding to the target upstream table is received, the data update event is skipped.
[0115] It should be understood that in some feasible implementations, the processor 401 described above may be a central processing unit (CPU), which may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.
[0116] In specific implementation, the aforementioned electronic device 400 can perform the above-described actions through its built-in functional modules. Figure 1 The implementation methods provided for each step are detailed in the above-mentioned implementation methods, and will not be repeated here.
[0117] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to implement... Figure 1 The methods provided in each step are detailed in the implementation methods provided in the above steps, and will not be repeated here.
[0118] The aforementioned computer-readable storage medium can be an internal storage unit of the event processing apparatus or electronic device provided in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device. The aforementioned computer-readable storage medium can also include magnetic disks, optical disks, read-only memory (ROM), or random access memory (RAM), etc. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the electronic device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
[0119] This application provides a computer program product, which includes a computer program or computer instructions, and the computer program or computer instructions are executed by a processor. Figure 1 The methods provided for each step in the process.
[0120] The terms "first," "second," etc., used in the claims, description, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or electronic device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or electronic devices. References to "embodiment" herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The presentation of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and / or" as used in this application's description and appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0121] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.
[0122] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.
Claims
1. An event handling method, characterized in that, The method includes: Retrieve data update events when the target upstream table in the upstream database is updated; In response to the target upstream sub-table being in a preset state, the data update event is skipped; In response to the target upstream table not being in the preset state and not executing the data update event for the downstream database, it is determined that the data update event includes a Data Definition Language (DDL) event or a Data Manipulation Language (DML) event; wherein, the target upstream table being in the preset state indicates that the table structure of the target upstream table before the data update is incompatible with the table structure of at least one other upstream table in the upstream database, and at least one upstream table in the upstream database has not received the same data update event; otherwise, the target upstream table is not in the preset state. In response to the data update event, including the DML event, the data update event is executed for the downstream database; In response to the data update event, including the DDL event, determine the compatibility status between the target table structure before the data update of the target upstream table and the table structures of other upstream tables in the upstream database, and determine whether to execute the data update event for the downstream database or skip the data update event based on the compatibility status; The step of determining whether to execute the data update event or skip the data update event for the downstream database based on the compatibility status includes: In response to the fact that the target table structure is incompatible with the table structure of at least one other upstream sub-table in the upstream database, and that at least one upstream sub-table in the upstream database has not performed the data update operation corresponding to the data update event, the data update event is skipped, and the target upstream sub-table is marked as the preset state; In response to the fact that the target table structure is incompatible with the table structure of at least one other upstream sub-table in the upstream database, and all upstream sub-tables in the upstream database have performed the data update operation corresponding to the data update event, the preset state marked on the target upstream sub-table is cleared, and the data update event is executed for the downstream database.
2. The method according to claim 1, characterized in that, The step of determining whether to execute the data update event or skip the data update event for the downstream database based on the compatibility status includes: In response to the fact that the target table structure is compatible with the table structures of other upstream sub-tables in the upstream database, a first table structure union of the target table structure and the table structures of other upstream sub-tables in the upstream database is determined, as well as a second table structure union of the target upstream sub-table structure after data update and the table structures of other upstream sub-tables in the upstream database. Based on the union of the first table structure and the union of the second table structure, it is determined whether to execute the data update event for the downstream database or skip the data update event.
3. The method according to claim 2, characterized in that, The step of determining whether to execute the data update event or skip the data update event for the downstream database based on the union of the first table structure and the union of the second table structure includes: If the union of the first table structure and the union of the second table structure are the same, the data update event is skipped. In response to the fact that the union of the first table structure and the union of the second table structure are different, the data update event is executed for the downstream database.
4. The method according to claim 1, characterized in that, The method further includes: If the target upstream table is not in the preset state, and the data update event corresponding to any other upstream table in the upstream database has been executed for the downstream database before the data update event corresponding to the target upstream table is received, the data update event is skipped.
5. An event processing device, characterized in that, The device includes: The event acquisition module is used to acquire data update events when the target upstream table in the upstream database is updated. The event handling module is used to skip the data update event in response to the target upstream sub-table being in a preset state; The event determination module is used to determine, in response to the target upstream table not being in the preset state and not executing the data update event for the downstream database, whether the data update event includes a Data Definition Language (DDL) event or a Data Manipulation Language (DML) event; wherein, the target upstream table being in the preset state indicates that the table structure of the target upstream table before the data update is incompatible with the table structure of at least one other upstream table in the upstream database, and at least one upstream table in the upstream database has not received the same data update event; otherwise, the target upstream table is not in the preset state. The event handling module is configured to execute the data update event for the downstream database in response to the data update event, including the DML event. The event handling module is configured to, in response to the data update event including the DDL event, determine the compatibility status of the target table structure before the data update of the target upstream table with the table structures of other upstream tables in the upstream database, and based on the compatibility status, determine whether to execute the data update event for the downstream database or skip the data update event. The event handling module is specifically configured to: in response to the target table structure being incompatible with the table structure of at least one other upstream sub-table in the upstream database, and at least one upstream sub-table in the upstream database not having performed the data update operation corresponding to the data update event, skip the data update event and mark the target upstream sub-table as the preset state; in response to the target table structure being incompatible with the table structure of at least one other upstream sub-table in the upstream database, and all upstream sub-tables in the upstream database having performed the data update operation corresponding to the data update event, clear the preset state marked on the target upstream sub-table and execute the data update event for the downstream database.
6. An electronic device, characterized in that, It includes a processor and a memory, which are interconnected; The memory is used to store computer programs; The processor is configured to perform the method as described in any one of claims 1 to 4 when the computer program is invoked.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the method of any one of claims 1 to 4.