Data synchronization method and system of redundant network, electronic equipment and storage medium

Abstract

The invention discloses a data synchronization method and system of a redundant network, electronic equipment and a storage medium, and belongs to the technical field of data synchronization and distribution.The method comprises the steps that in response to an operation instruction received by any equipment node for target data in any database, a target equipment node and a target database corresponding to the target data are determined; acquiring data operation record information of equipment nodes except the target equipment node, and current data in a target database on the target equipment node; and based on the operation instruction, the data operation record information and / or the current data, executing an operation of performing data synchronization on the target database on each equipment node. According to the scheme, a single-point fault can be avoided, so that the fault-tolerant capability and reliability of data synchronization of the distributed system in a redundant network environment can be remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data synchronization and distributed systems, in particular to a data synchronization method and system for a redundant network, an electronic device and a storage medium. BACKGROUND

[0002] At present, the common data synchronization method in the database is mainly applied to the non-redundant network environment. In this scenario, in order to ensure data consistency, any operation (such as addition, deletion, and modification) on the data usually needs to be successfully executed in real time on all device nodes or more than half of the device nodes, and then it is determined that the data operation is completed. Although this method can guarantee strong consistency, it has very high requirements for the stability and real-time performance of the network environment. Once there is network delay or partial device node failure, it will directly affect the efficiency and success rate of data operation, and even cause system service interruption.

[0003] However, the above method is difficult to adapt to the needs of the redundant network environment. In the redundant network, device nodes are connected through multiple independent network paths, and have higher availability and fault tolerance. However, the traditional synchronization mechanism cannot fully utilize the advantages of the redundant network, and still relies on the real-time cooperation of all or most device nodes, resulting in poor fault tolerance of data synchronization in the redundant network environment. SUMMARY

[0004] The present application provides a data synchronization method and system for a redundant network, an electronic device and a storage medium, aiming to solve the problem of poor fault tolerance of data synchronization in the related art in the redundant network environment.

[0005] In a first aspect, a data synchronization method for a redundant network is provided, wherein the redundant network is configured with a plurality of independent networks and device nodes, each device node is independently connected with each network, and each device node is connected with each database participating in data synchronization. The method comprises: In response to any device node receiving an operation instruction on target data in any database, determining a target device node and a target database corresponding to the target data; Obtaining data operation record information of device nodes other than the target device node, and current data in the target database on the target device node; Based on the operation instruction, the data operation record information and / or the current data, performing data synchronization operation on the target database on each device node.

[0006] In some embodiments, the operation of performing data synchronization on the target database on each device node based on the operation instruction, the data operation record information and / or the current data comprises: The data operation record information is used to query the data in the current data to determine the query result of the target data; Based on the operation instruction, the data operation record information, and / or the query result, the operation instruction is executed in the target database on the target device node to obtain the updated target database; Based on the updated target database, a data synchronization operation is performed on the target database on each device node.

[0007] In some embodiments, the step of executing the operation instruction in the target database on the target device node based on the operation instruction, the data operation record information, and / or the query result to obtain an updated target database includes: If the query result indicates that the target data does not exist, and the type of the operation instruction is deletion, then the target database on the target device node is determined to be the updated target database. If the query result indicates that the target data does not exist, and the type of the operation instruction is either an add or update operation, the target database on the target device node is added or updated based on the operation instruction to obtain the updated target database.

[0008] In some embodiments, the step of executing the operation instruction in the target database on the target device node based on the operation instruction, the data operation record information, and / or the query result to obtain an updated target database includes: If the query result indicates that the target data exists, and the type of the operation instruction is deletion, addition, or update, the time information of the target data and the data operation record information corresponding to the target data is used to obtain a time comparison result. Based on the time comparison result, the operation instruction is executed in the target database on the target device node to obtain the updated target database.

[0009] In some embodiments, executing the operation instruction in the target database on the target device node based on the time comparison result to obtain an updated target database includes: Let the time information of the data operation record information corresponding to the target data be the record time information; If the recorded time information is earlier than the time information of the target data, and the type of the operation instruction is a new or update type, the data operation record information corresponding to the target data is deleted, and a new operation is performed on the target database on the target device node based on the operation instruction to obtain the updated target database. If the recorded time information is later than the time information of the target data, and the type of the operation instruction is a new type or an update type, the target database on the target device node is updated based on the operation instruction to obtain the updated target database.

[0010] In some embodiments, executing the operation instruction in the target database on the target device node based on the time comparison result to obtain an updated target database includes: Let the device node containing the data operation record information corresponding to the target data be the recording device node; If the recorded time information is equal to the time information of the target data, the priorities of the recording device node and the target device node are compared to obtain a priority comparison result. Based on the priority comparison result, the target database on the target device node is updated according to the operation instruction to obtain the updated target database.

[0011] In some embodiments, prior to the step of responding to any device node receiving an operation instruction for target data in any database, the method includes: Real-time monitoring of log information on each device node for each database to determine the data operation records to be synchronized on each device node; Each device node sends the corresponding data operation record information to be synchronized to other device nodes through any network that is in a normal connection state.

[0012] In some embodiments, after sending the corresponding data operation record information to be synchronized to device nodes other than itself, the method further includes: If any device node fails to send or receive any of the aforementioned data operation record information, determine the data operation record information that failed to synchronize and the corresponding device node. Determine the location information of the database where the data corresponding to the data operation record information of the synchronization failure is located; The data operation record information and corresponding location information of the synchronization failure are sent to the corresponding device node.

[0013] In some embodiments, after executing the operation instruction in the target database on the target device node based on the operation instruction, the data operation record information, and / or the query result to obtain the updated target database, the process includes: If the operation instruction fails to be executed in the target database on the target device node, operation failure information and the location information corresponding to the target data are obtained; The operation failure information and the location information are fed back to the target device node.

[0014] Secondly, a data synchronization system for a redundant network is also provided. The redundant network is configured with several independent networks and databases, and several device nodes are independently connected to each network and each database. The system includes: The operation instruction monitoring module is used to respond to any device node receiving an operation instruction for target data in any database, and to determine the target device node and the target database corresponding to the target data. The data acquisition module is used to acquire data operation record information of device nodes other than the target device node, as well as the current data in the target database on the target device node; The data synchronization module is used to perform data synchronization operations on the target database on each device node based on the operation instructions, the data operation record information, and / or the current data.

[0015] Thirdly, an electronic device is also provided, including a memory and a processor, wherein a computer program is stored on the memory, and when executed by the processor, the computer program implements the steps of any one of the methods described above.

[0016] Fourthly, a computer-readable storage medium is also provided, on which a computer program is stored, the computer program being loaded by a processor to perform the steps of any of the methods described above.

[0017] Beneficial effects: Using the method in this embodiment, users can perform data operations on any normal device node in a redundant network, achieving redundancy of access points and avoiding single points of failure. Even if some nodes or the network fails, the entire system can still provide data services, thereby significantly improving the fault tolerance of data synchronization in a distributed system under redundant network environments. Based on operation instructions, data operation record information, and / or current data, data synchronization operations are performed on the target database on each device node. This ensures that even under network fluctuations or concurrent writes, the data on all nodes eventually converges to a consistent and correct state, effectively preventing data synchronization process chaos and data loss, thus improving the reliability of data synchronization. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a data synchronization method for a redundant network provided in an exemplary embodiment of this application; Figure 2 This is a schematic diagram of a redundant network provided by an exemplary embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] "A and / or B" includes the following three combinations: A only, B only, and a combination of A and B.

[0023] The use of "applies to" or "configured to" in this application implies open and inclusive language, which does not exclude the applicability to or configuration to devices performing additional tasks or steps. Additionally, the use of "based on" implies openness and inclusivity, because processes, steps, calculations, or other actions "based on" one or more of the stated conditions or values ​​may in practice be based on additional conditions or values ​​beyond those stated.

[0024] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0025] The data synchronization method for redundant networks proposed in this application can achieve efficient and reliable data synchronization in redundant networks. It supports confirmation upon successful operation of a single device node and ultimately achieves data consistency through asynchronous synchronization and conflict resolution strategies.

[0026] On one hand, this embodiment provides a data synchronization system for a redundant network, applied to a redundant network configured with several independent networks and several independent device nodes. In this redundant network, multiple switches are used to create multiple separate network environments. Each independent network has one switch deployed, and the switches are not interconnected. This embodiment employs a multi-network redundancy topology for networking. Each device node has multiple independent network ports, each independently connected to each network through one port, forming a fully connected network topology. Simultaneously, each device node is connected to various databases in a database set, which contains several databases participating in data synchronization.

[0027] Furthermore, each database (such as MariaDB or SQL database) on each device node in the system is configured with a log file (such as a binary log (binlog)) to record all database operation information; each database table that needs to be synchronized has a primary key, where the primary key refers to the field or combination of fields in the database table used to uniquely identify a record; each database table participating in the synchronization has a time information field, for example, the data type is DATETIME(6), and the default value is current_timestamp(6), which means that when a new data is inserted into the database but the value of this time field is not explicitly specified, the database system will automatically fill in the current system time (also accurate to microseconds) as the default value of this field. Among them, DATETIME(6) is a high-precision timestamp data type, and 6 means that it supports a precision of up to microseconds. It is easy to understand that the time information field will be automatically updated as the data in the database is updated, that is, when the database inserts or updates a piece of data, the time information field of that piece of data is automatically written to the current system time.

[0028] Each device node in the system is equipped with a network monitoring module, a log listening module, an operation command monitoring module, a data acquisition module, and a data synchronization module.

[0029] The network monitoring module is used to monitor the connection status of this device node with other device nodes on each independent network in real time, and record network status information such as normal connection and disconnection to ensure the real-time and traceability of data synchronization.

[0030] The log monitoring module is used to monitor the log information in the database log files in real time, extracting various data operation records of the database tables that need to be synchronized. Simultaneously, it sends these data operation records to other device nodes through any network with a normal connection. If any device node receives a data operation record transmission failure message, it writes that data operation record to the database log file. Each device node has an independent database log folder to record data operation records for each database, and updates the data in the log folder at preset time intervals. It also records the log position information of the currently monitored failed transmission operation in the log information stream. When the log monitoring module starts, it begins listening for log information from the position corresponding to this log position information.

[0031] The operation instruction monitoring module is used to respond to any device node receiving an operation instruction for target data in any database, and to determine the target device node and the target database corresponding to the target data.

[0032] The data acquisition module is used to acquire data operation record information from device nodes other than the target device node, as well as the current data in the target database on the target device node. Based on the data operation type corresponding to the data operation record information and the primary key information in the corresponding database table, it queries whether the target data corresponding to that primary key information currently exists in the target database on the target device node. Then, based on the received data operation record information, the current data in the target database on the target device node, and preset conflict resolution rules, it determines whether to execute the operation instruction in the target database on the target device node, and how to execute the operation instruction to synchronize data in the target databases on each device node.

[0033] The data synchronization module is used to perform data synchronization operations on the target database on each device node based on operation instructions, data operation record information and / or current data.

[0034] Based on this, the data synchronization method for redundant networks provided in this embodiment, such as Figure 1 As shown, the method mainly includes the following steps: Step S100: In response to any device node receiving an operation instruction for target data in any database, determine the target device node and the target database corresponding to the target data.

[0035] Specifically, when any device node in the redundant network receives a data operation command from a user, such as an add, delete, or update command, the device node that received the command is identified as the target device node. Simultaneously, based on the target data indicated by the operation command, the database to which that data belongs is determined, which is the target database for the synchronization operation.

[0036] Step S200: Obtain data operation record information of device nodes other than the target device node, as well as the current data in the target database on the target device node.

[0037] Specifically, after identifying the target device node and the target database, the system queries the recent data operation records of all other device nodes in the network, excluding the target device node. These data operation records refer to the add, delete, and modify operation records captured by each device node listening to the binary logs of its respective database. Then, the system reads the current data in the target database on the target device node, that is, the data contained in the target database at the current moment.

[0038] Step S300: Based on the operation instructions, data operation record information and / or current data, perform data synchronization operation on the target database on each device node.

[0039] Specifically, the system uses data operation log information to query the data in the current database, specifically the primary key information, to check if the target data corresponding to that primary key exists in the current database on the target device node. This determines the query result for the target data. Based on the query result, and according to preset conflict resolution rules, the system decides whether to execute the operation command in the target database on the target device node, and how to execute the operation command to synchronize data across the target databases on each device node. The preset conflict resolution rules are based on the timestamps of the target data in the target database on the target device node and the data operation log information received from other device nodes, as well as the device node priority, and are used to define the operation command conversion or execution rules.

[0040] In this embodiment, the method allows users to perform data operations on any normal device node in a redundant network, achieving redundancy of access points and avoiding single points of failure. Even if some nodes or the network fail, the entire system can still provide data services, significantly improving the fault tolerance of distributed systems in redundant network environments. Based on operation instructions, data operation record information, and / or current data, data synchronization operations are performed on the target database on each device node. This ensures that even under network fluctuations or concurrent writes, the data on all nodes eventually converges to a consistent and correct state, effectively preventing data synchronization process chaos and data loss, thereby improving the reliability of data synchronization.

[0041] In some embodiments, step S300 involves performing a data synchronization operation on the target database on each device node based on operation instructions, data operation record information, and / or current data, including: Step S310: Use the data operation record information to query the data in the current data and determine the query results for the target data.

[0042] Specifically, after the target device node obtains data operation record information from other device nodes, it first performs a query verification in the target database of the target device node. Specifically, based on the primary key contained in the received operation record information, it queries the target device node's database to determine whether a corresponding data entry exists, thus ascertaining the current state of the target data on the target device node—that is, whether the target data exists or not.

[0043] Step S320: Based on the operation instructions, data operation record information and / or query results, execute the operation instructions in the target database on the target device node to obtain the updated target database.

[0044] Specifically, the final execution method for the operation instruction is determined by considering the type of the operation instruction, the operation record information received from other device nodes, and the query results. The types of operation instructions include, but are not limited to, add, update, or delete. For example, if the query result indicates the existence of the target data and the operation instruction is a delete, the operation instruction is executed directly on the target database on the target device node, resulting in an updated target database. If the query result indicates the existence of the target data and the operation instruction is an add or update, the latest timestamp in the operation record of the target data in the target database on the target device node is compared with the latest timestamps of the data operation records corresponding to the target data on other device nodes. Based on the comparison result, the operation instruction is used to perform an add operation on the target database on the target device node, resulting in an updated target database.

[0045] Step S330: Based on the updated target database, perform data synchronization operations on the target database on each device node.

[0046] Specifically, after the target device node successfully executes the operation command and updates its target device node database, it immediately initiates a data synchronization process with other device nodes in the network. This is achieved through the target device node's log monitoring module, which captures log records generated by the operation in real time. These records are then encapsulated into data operation record information and sent to every device node except itself via a currently connected network path. Upon receiving this record, the data acquisition modules of other device nodes execute the corresponding operation commands in their target device node's database according to preset conflict resolution rules, ensuring that the data on all device nodes ultimately matches the updated state of the target device node. If real-time synchronization fails, the operation record is stored in the corresponding device node's failure data file, and the data synchronization module retryes it later, thus ensuring data consistency across all device nodes.

[0047] In this embodiment, when multiple device nodes modify the same data almost simultaneously, the system automatically retains the latest modifications based on the order of timestamps, without manual intervention. This ensures data accuracy while maintaining high system processing performance, achieving efficient concurrency control and automated conflict resolution. Furthermore, by embedding exception handling logic throughout the entire process, this embodiment allows the system to tolerate temporary network partitions or node failures. Once the connection is restored, synchronization operations can continue, guaranteeing the final completion of data synchronization. This enables the system to adapt to complex and ever-changing distributed environments, thereby enhancing its robustness.

[0048] In some embodiments, step S320 involves executing an operation instruction in the target database on the target device node based on the operation instruction, data operation record information, and / or query results to obtain an updated target database, including: Step S321: If the query result indicates that the target data does not exist and the operation instruction type is deletion, determine that the target database on the target device node is the updated target database.

[0049] Specifically, if the system confirms through querying that the target data pointed to by the operation command does not exist in the database of the target device node, and the received operation command type is deletion, then it is determined to be an invalid or redundant operation. Deleting a non-existent piece of data is meaningless and may cause errors. Therefore, the deletion operation command is ignored, and no modification is made to the database of the target device node. At this time, the database on the target device node remains unchanged and is directly identified as the updated target database. Simultaneously, the system returns a success or ignore notification to the command source to avoid unnecessary retries.

[0050] Step S322: If the query result indicates that the target data does not exist, and the operation instruction is of the add or update type, perform the add or update operation on the target database on the target device node based on the operation instruction to obtain the updated target database.

[0051] Specifically, if the system finds that the target data does not exist in the target database on the target device node, and the received operation instruction is of the add type, it indicates that the requirements of the operation instruction perfectly match the current database status. Therefore, the add operation instruction is executed directly, inserting the primary key, current timestamp, and other data contained in the instruction as a completely new record into the target database. Upon successful execution, this data is added to the target device node database, resulting in an updated target database and an updated database version.

[0052] If, after system querying, it is confirmed that the target data does not exist in the target database on the target device node, and the received operation instruction is of the update type, it indicates that there is an inconsistency in data synchronization among the device nodes (e.g., other device nodes have the data, but this device node has not yet synchronized it), or that the instruction itself has a logical problem. To resolve this conflict and ensure eventual data consistency, the type of this operation instruction is changed to an insert operation instruction. Then, the insert operation instruction is executed, inserting the primary key and new timestamp, among other data content carried by the insert operation instruction, into the target database of the target device node as a new record. In this way, an update operation that might have failed is successfully transformed into an insert operation, resulting in an updated target database containing this new data. This effectively compensates for the missing data and achieves the effect of data fault tolerance.

[0053] In some embodiments, step S320 involves executing an operation instruction in the target database on the target device node based on the operation instruction, data operation record information, and / or query results to obtain an updated target database, including: Step S323: If the query result indicates that the target data exists and the operation instruction is of the deletion, addition, or update type, execute the operation instruction in the target database on the target device node to obtain the updated target database.

[0054] Specifically, after the system queries and confirms that the target data pointed to by the operation instruction already exists in the target database on the target device node, and the operation instruction is of the deletion, addition, or update type, the system retrieves the time information field of the target data in the target database on the target device node and compares it precisely with the value of the time information field carried in the received operation record information to obtain the time comparison result, that is, to determine whether the timestamp in the operation record information is earlier, later, or equal to the timestamp of the target data in the target database on the target device node.

[0055] Step S324: Based on the time comparison results, execute the operation command in the target database on the target device node to obtain the updated target database.

[0056] Specifically, as an example, let the time information of the data operation record information corresponding to the target data be the recording time information. If the time comparison result shows that the recording time information is earlier than the time information of the target data, and the type of operation instruction is deletion, it means that the deletion operation corresponding to the data operation record information corresponding to the target data is an old operation. In this case, the deletion operation instruction is discarded without processing, and the execution success information is directly returned to the target device node. If the time comparison result shows that the recorded time information is earlier than the target data time information, and the operation instruction type is a new type, it indicates that the target data in the target database on other device nodes is old. Therefore, the data operation record information corresponding to the target data is deleted. At the same time, since the target data exists in the target database on the target device node, the operation instruction is adjusted to an update type operation instruction, and the update type operation instruction is executed in the target database on the target device node to obtain the updated target database. If the time comparison result shows that the recorded time information is earlier than the target data time information, and the operation instruction type is update, it means that the target data in the target database on other device nodes is old. In this case, the data operation record information corresponding to the target data is deleted, and the operation instruction is executed in the target database on the target device node to obtain the updated target database.

[0057] As another exemplary example, if the time comparison result shows that the recorded time information is later than the target data's time information, and the operation instruction type is deletion, it indicates that the deletion operation corresponding to the data operation record information of the target data is a new operation. In this case, the deletion operation instruction is directly converted into the corresponding SQL statement. Based on the primary key provided in the operation record information, the corresponding data record is located and deleted in the target database on the target device node. After successful execution, the data will be removed from the target device node database on the target device node, resulting in an updated target database.

[0058] If the time comparison result shows that the recorded time information is later than the target data time information, and the operation instruction type is the addition type, the operation instruction needs to be executed to overwrite the older target data in the target database on the target device node. At the same time, since the target data exists in the target database on the target device node, the operation instruction is adjusted to the update type operation instruction, and the update type operation instruction is executed in the target database on the target device node to obtain the updated target database. If the time comparison result shows that the recorded time information is later than the target data time information, and the operation instruction type is update, the operation instruction needs to be executed to overwrite the older target data in the target database on the target device node. Therefore, the operation instruction is executed in the target database on the target device node to obtain the updated target database.

[0059] As another exemplary example, when the time comparison result shows that the recorded time information is equal to the target data time information, it indicates that it is impossible to determine the age of the target data corresponding to different device nodes by the order of time. Therefore, a preset device node priority arbitration mechanism is activated, making the device node where the data operation record information corresponding to the target data is located the recording device node. Based on the unique priority configured for each device node in advance, the priorities of the recording device node and the target device node are compared to obtain the priority comparison result, that is, to determine whether the priority of the recording device node is higher or lower than the priority of the target device node.

[0060] Then, if the operation instruction is a delete operation, based on the priority comparison result, an update operation is performed on the target database on the target device node to obtain the updated target database. For example, if the record device node has a higher priority, the delete operation instruction is discarded without processing, and a success message is directly returned to the target device node; if the target device node has a higher priority, the delete operation instruction is converted into a corresponding SQL statement, and the corresponding data record is located and deleted in the target database on the target device node based on the primary key provided in the operation record information. After successful execution, the data is removed from the target device node database on the target device node, resulting in the updated target database.

[0061] When the operation instruction is of the add or update type, based on the priority comparison result, an update operation is performed on the target database on the target device node according to the operation instruction to obtain the updated target database. For example, if the record device node has a higher priority, the operation instruction is adjusted to an update operation instruction, and the update operation instruction is executed on the target database on the target device node to obtain the updated target database; if the target device node has a higher priority, the data operation record information corresponding to the target data is deleted, and an update operation instruction is executed on the target database on the target device node to obtain the updated target database.

[0062] It is readily understood that the execution operation instructions mentioned in the embodiments of this application, as well as the execution of deletion, addition, or update operation instructions, refer to the conversion of the corresponding operation instructions into corresponding SQL statements and the execution of the corresponding SQL statements. If the SQL statement executes successfully, a success message is returned to the target device node; otherwise, a failure message is returned to the target device node, and no log entries are generated during the execution of the SQL statement.

[0063] In this embodiment, when target data exists in the target database on the target device node, the time information is used to determine the age of the target data corresponding to different device nodes. The device node priority arbitration mechanism and the type of operation instruction are used to coordinate the discarding, conversion or execution rules of the operation instruction, so as to accurately operate on the data on the target device node and ensure the synchronization of the target database updates on each device node.

[0064] In some embodiments, before any device node receives an operation instruction for target data in any database in step S100, step S400 is included, specifically including: Step S410: Monitor the log information of each device node to each database in real time to determine the data operation record information to be synchronized on each device node.

[0065] Specifically, for each device node, the log monitoring module monitors the log information of each database on the corresponding device node in real time. Once an edit operation is performed on data in the database, the module can immediately capture the detailed information of these operations from the logs and identify this detailed information as the data operation record information to be synchronized by each device node. The data operation record information includes the database table to which the operated data belongs, the primary key, the operation type (INSERT, UPDATE, DELETE), and a key time information field (such as a timestamp), which serves as the basic data unit for subsequent data synchronization.

[0066] Step S420: Each device node sends the corresponding data operation record information to be synchronized to other device nodes through any network that is in a normal connection state.

[0067] Specifically, for each device node, the network monitoring module continuously monitors its connection status to each independent network through its various independent network ports. It determines whether the connection to other device nodes in the network is normal or interrupted by sending detection packets. The network monitoring module also dynamically maintains a list of normal network connections, serving as a direct basis for device nodes to select data transmission channels. This ensures that data operation logs are sent through currently available and stable network paths, thereby preventing information loss due to network failures.

[0068] Once the log monitoring module of the device node captures new data operation record information, it immediately initiates the transmission process. During transmission, the log monitoring module queries the list of normal network connections provided by the network monitoring module and arbitrarily selects a currently available network path. Then, the device node transmits the newly generated data operation record information to be synchronized to every other device node in the network topology except itself through this selected normal network. This achieves real-time, multi-channel broadcasting of operation records, ensuring that all device nodes receive data change information almost simultaneously, laying the foundation for final data consistency. If transmission fails, the data operation record information is written to a temporary data file for subsequent retransmission until the device node receives successful transmission information from all other device nodes except itself.

[0069] In some embodiments, after sending the corresponding data operation record information to be synchronized to a device node other than itself in step S420, the method further includes step S430, which specifically includes: Step S431: If any device node fails to send or receive any data operation record information, determine the data operation record information that failed to synchronize and the corresponding device node.

[0070] Specifically, the system continuously monitors the results of each send and receive operation during data synchronization. When a device node (denoted as the sender node) attempts to send data operation log information to another device node (denoted as the receiver node) via a normal network, if the sender node does not receive a success confirmation from the receiver node within a predetermined time, or if the receiver node fails to process the received information, then the synchronization operation is considered a failure. Once a failure occurs, the specific data operation log information for the failed synchronization, as well as the target device node corresponding to this synchronization failure (i.e., the device node where the information should have been delivered but failed), are recorded, and these are explicitly marked as failed tasks requiring remediation.

[0071] Step S432: Determine the location information of the database where the data corresponding to the data operation record information of the synchronization failure is located.

[0072] Specifically, to accurately retry failed synchronization tasks later, the precise location of the failed synchronization operation in the log stream is recorded. For example, for the sending device node, the unique location identifier corresponding to the failed data operation record in its database log sequence is recorded. Simultaneously, the system creates an independent temporary data file for each receiving device node that needs synchronization, corresponding to a specific time period. In the event of a real-time synchronization failure, the record location information of the failed synchronization operation, as well as the time period in which the synchronization operation occurred, is recorded, and this data operation record information is written to the corresponding temporary data file of the receiving device node for that time period. Therefore, the location information here includes the log location information of the failed synchronization operation in the log stream, and the storage location information of the target device node in the temporary data file corresponding to the time period on the receiving device node, together ensuring accurate location of the failed operation record during retries. It is worth noting that the data in the database on each device node is updated according to a preset time interval, and the temporary data file is used to store data operation record information of failed synchronization operations within one or more time intervals.

[0073] Step S433: Send the data operation record information and corresponding location information of the synchronization failure to the corresponding device node.

[0074] Specifically, the data synchronization module checks the temporary data folders created for each other device node on the sending device node in real time or periodically. Based on the recorded location information, it reads the data operation records of previously failed synchronization attempts stored in these files in chronological order. Then, the data synchronization module attempts to resend these failed operation records to the receiving device node. During transmission, it selects the currently connected and functional network path. This retry process for failed synchronization operations is repeated until the failed operation is successfully sent and a success confirmation is received from the receiving device node, thus ensuring that all operations are eventually synchronized to all device nodes, achieving data consistency across all device nodes.

[0075] In some embodiments, after executing the operation instruction in the target database on the target device node based on the operation instruction, data operation record information, and / or query results in step S320 to obtain the updated target database, step S500 is included, specifically including: Step S510: If the operation command fails to be executed in the target database on the target device node, obtain the operation failure information and the location information corresponding to the target data.

[0076] Specifically, when the data acquisition module of the target device node converts the received data operation record information into specific database editing instructions and executes them in the database on the target device node, if the database editing instruction fails to execute, the data acquisition module immediately captures this failure exception. Subsequently, it generates explicit operation failure information, which at least includes the result status of the failed execution. Simultaneously, it records the location information corresponding to this failed operation instruction.

[0077] Step S520: Send the operation failure information and location information back to the target device node.

[0078] Specifically, after determining the operation failure information and location information, the target device node immediately sends this failure feedback information back to the target device node that initially sent the data operation record information through a currently available and normally connected network path. This feedback process is performed in real time, with the purpose of promptly informing the sender that a synchronization attempt has failed. Upon receiving this failure feedback, the target device node knows that its previously sent operation command was blocked on the target device node. It then writes the operation record information into a temporary data file specific to that target device node, thereby triggering the data synchronization module to asynchronously retry until synchronization is successful. Finally, it deletes the synchronized temporary data file to free up storage space.

[0079] As an exemplary example, based on Figure 2 The redundant network shown details the specific implementation process of the method proposed in this application. This redundant network adopts a fully connected network topology, including switch 1, switch 2, and switch 3, as well as device node 1, device node 2, device node 3, and device node 4. Each device node has three independent network ports, each connected to one of the three switches, forming three independent networks. The network consisting of switch 1 and the four device nodes is defined as network 1, the network consisting of switch 2 and the four device nodes is defined as network 2, and the network consisting of switch 3 and the four device nodes is defined as network 3.

[0080] All four device nodes in this redundant network run MariaDB databases, all are configured to enable binlog logging, all database tables that need to be synchronized have primary keys, and all have a new time information field self_insert_update_date_time added. The corresponding SQL statements are as follows: ALTER TABLE `remote_object` ADD COLUMN `self_insert_update_date_ time` DATETIME(6) NULL DEFAULT current_timestamp(6) ON UPDATE current_timestamp(6); ALTER TABLE `remote_object` ADD INDEX `self_insert_update_date_ time`(`self_insert_update_date_time`).

[0081] Each device node in this redundant network runs a network monitoring module, a log listening module, a data acquisition module, and a data synchronization module, each of which is an independent program. Taking device node 2 as an example, the content executed by each module program is as follows: 1) Network monitoring module Monitor the network connectivity of device node 2 and device node 1 on network 1, network 2, and network 3 in real time, and record the networks with normal connections; do the same for device node 2, device node 3, and device node 4.

[0082] 2) Log monitoring module Monitor the binlog of the MariaDB database on device node 2 in real time, obtain the data operation record information of the database table that needs to be synchronized, and send it to device node 1, device node 3 and device node 4.

[0083] Taking sending data to device node 1 as an example, first select a network from those networks where device node 1 is normally connected. Under this network, send the data operation log information to device node 1. If a success message is received from device node 1, it indicates that the data synchronization to device node 1 was successful. Otherwise, it is considered a failure to synchronize data to device node 1. In this case, the data operation log information needs to be written to a temporary data file on device node 2 to store the failed synchronization data. For example, the folder path for storing the failed synchronization data is / data / node1, and the temporary data file path is / data / node1 / 2025070109.data, where 2025070109 represents 9:00 AM on July 1, 2025. The data operation log information within the corresponding time period needs to be written to the temporary data file within the corresponding time period.

[0084] The process of sending to device node 3 and device node 4 is the same as the process of sending to device node 1, and will not be described again here.

[0085] The current binlog position information is recorded in real time and written to a temporary data file. When the log monitoring module starts, it reads the file to get the previous binlog position information and starts listening to the binlog from that position.

[0086] 3) Data Acquisition Module Receive data operation record information sent from device node 1, device node 3, and device node 4, and complete the following processing steps, taking the receipt of data operation record information sent from device node 1 as an example.

[0087] a. Parse the received data operation record information to determine whether the information is a deletion, addition, or update operation; b. When the operation instruction is a delete, add, or update operation, retrieve the primary key information of the data from this information, and query whether the data exists in the MariaDB database of this device node based on the primary key information; c. If the data does not exist and the operation instruction is a delete operation instruction, the delete operation instruction will not be executed, and the execution success information will be returned directly to device node 1. If the data does not exist and the operation instruction is an add or update operation instruction, the values ​​of all fields will be obtained from the information, and the operation instruction will be converted into the corresponding SQL statement. This statement will be executed in the MariaDB database of this device node. If the execution is successful, a success information will be returned to device node 1; otherwise, a failure information will be returned to device node 1. d. If the data exists, compare the value of self_insert_update_date_time in the received information with the value of self_insert_update_date_time in the data of this device node; e. If the value of self_insert_update_date_time in the received information is small, it means that the data in the received information is old data. Discard the received information, do not process it, and directly return a success message to device node 1. f. If the value of self_insert_update_date_time in the received information is large, and the operation instruction is a delete operation instruction, retrieve the values ​​of all fields and the primary key information from the information, convert the operation instruction into the corresponding SQL statement, execute it in the MariaDB database of this device node, and if the execution is successful, return a success message to device node 1; otherwise, return a failure message to device node 1. If the operation instruction is an add or update operation instruction, retrieve the values ​​of all fields and the primary key information from the information, generate an update SQL statement, execute it in the MariaDB database of this device node, and if the execution is successful, return a success message to device node 1; otherwise, return a failure message to device node 1. g. If the value of self_insert_update_date_time in the received information is equal to the value of self_insert_update_date_time in the same data of this device node, it means that the newness of the target data corresponding to different device nodes cannot be determined by the time sequence. Therefore, the operation instruction is executed according to the preset device node priority. The operation instruction is only executed when device node 1 has the highest priority.

[0088] The processing steps for receiving information from device node 3 and device node 4 are the same as above and will not be repeated here.

[0089] 4) Data synchronization module The folder paths on this device node used to store data from synchronization failures are shown in the binlog monitoring module above. The corresponding folder paths for device node 1, device node 3, and device node 4 are / data / node1, / data / node3, and / data / node4, respectively.

[0090] Taking the folder / data / node1 on device node 1 as an example, the system monitors changes to files in / data / node1 in real time, reads temporary data files in chronological order, obtains data operation records of synchronization failures, selects a network from those with normal connections for device node 1, and sends information to device node 1 under that network. If a success message is received from device node 1, the data synchronization to device node 1 is successful; otherwise, it is considered a failure and needs to be retried until a success message is received from device node 1. If every line of the temporary data file has been read and successfully sent, and the temporary data file is not from the current time period, the synchronized temporary data file is deleted. The system records the current log position information, and starts listening to logs from this position when the log listening module starts.

[0091] This embodiment also provides an electronic device, such as... Figure 3 As shown, it includes a memory and a processor. In a specific example, the memory stores a computer program, which, when executed by the processor, implements the method of any of the above embodiments.

[0092] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps of any of the methods in the above embodiments.

[0093] In the embodiments of this application, the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0094] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0095] The above provides a detailed description of a data synchronization method, system, electronic device, and storage medium for a redundant network provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data synchronization method for a redundant network, characterized in that, The redundant network is configured with several independent network and device nodes. Each device node is independently connected to its respective network and to each database participating in data synchronization. The method includes: In response to any device node receiving an operation instruction for target data in any database, the target device node and the target database corresponding to the target data are determined. Obtain data operation record information of device nodes other than the target device node, as well as the current data in the target database on the target device node; Based on the operation instructions, the data operation record information, and / or the current data, perform data synchronization operations on the target database on each device node.

2. The data synchronization method for redundant networks according to claim 1, characterized in that, The operation of synchronizing the target database on each device node based on the operation instruction, the data operation record information, and / or the current data includes: The data operation record information is used to query the data in the current data to determine the query result of the target data; Based on the operation instruction, the data operation record information, and / or the query result, the operation instruction is executed in the target database on the target device node to obtain the updated target database; Based on the updated target database, a data synchronization operation is performed on the target database on each device node.

3. The data synchronization method for redundant networks according to claim 2, characterized in that, The step of executing the operation instruction in the target database on the target device node based on the operation instruction, the data operation record information, and / or the query result to obtain an updated target database includes: If the query result indicates that the target data does not exist, and the type of the operation instruction is deletion, then the target database on the target device node is determined to be the updated target database. If the query result indicates that the target data does not exist, and the type of the operation instruction is either an add or update operation, the target database on the target device node is added or updated based on the operation instruction to obtain the updated target database.

4. The data synchronization method for redundant networks according to claim 2, characterized in that, The step of executing the operation instruction in the target database on the target device node based on the operation instruction, the data operation record information, and / or the query result to obtain an updated target database includes: If the query result indicates that the target data exists, and the type of the operation instruction is deletion, addition, or update, the time information of the target data and the data operation record information corresponding to the target data is used to obtain a time comparison result. Based on the time comparison result, the operation instruction is executed in the target database on the target device node to obtain the updated target database.

5. The data synchronization method for redundant networks according to claim 4, characterized in that, Based on the time comparison result, the operation instruction is executed in the target database on the target device node to obtain the updated target database, including: Let the time information of the data operation record information corresponding to the target data be the record time information; If the recorded time information is earlier than the time information of the target data, and the type of the operation instruction is a new or update type, the data operation record information corresponding to the target data is deleted, and a new operation is performed on the target database on the target device node based on the operation instruction to obtain the updated target database. If the recorded time information is later than the time information of the target data, and the type of the operation instruction is a new type or an update type, the target database on the target device node is updated based on the operation instruction to obtain the updated target database.

6. The data synchronization method for redundant networks according to claim 5, characterized in that, Based on the time comparison result, the operation instruction is executed in the target database on the target device node to obtain the updated target database, including: Let the device node containing the data operation record information corresponding to the target data be the recording device node; If the recorded time information is equal to the time information of the target data, the priorities of the recording device node and the target device node are compared to obtain a priority comparison result. Based on the priority comparison result, the target database on the target device node is updated according to the operation instruction to obtain the updated target database.

7. The data synchronization method for redundant networks according to claim 1, characterized in that, Prior to the response of any device node receiving an operation instruction for target data in any database, the procedure includes: Real-time monitoring of log information on each device node for each database to determine the data operation records to be synchronized on each device node; Each device node sends the corresponding data operation record information to be synchronized to other device nodes through any network that is in a normal connection state.

8. The data synchronization method for redundant networks according to claim 7, characterized in that, After sending the corresponding data operation record information to be synchronized to device nodes other than itself, the process also includes: If any device node fails to send or receive any of the aforementioned data operation record information, determine the data operation record information that failed to synchronize and the corresponding device node. Determine the location information of the database where the data corresponding to the data operation record information of the synchronization failure is located; The data operation record information and corresponding location information of the synchronization failure are sent to the corresponding device node.

9. The data synchronization method for redundant networks according to claim 2, characterized in that, After executing the operation instruction in the target database on the target device node based on the operation instruction, the data operation record information, and / or the query result to obtain the updated target database, the process includes: If the operation instruction fails to be executed in the target database on the target device node, operation failure information and the location information corresponding to the target data are obtained; The operation failure information and the location information are fed back to the target device node.

10. A data synchronization system for a redundant network, characterized in that, The redundant network is configured with several independent networks and databases, and several device nodes are independently connected to each network and each database. The system includes: The operation instruction monitoring module is used to respond to any device node receiving an operation instruction for target data in any database, and to determine the target device node and the target database corresponding to the target data. The data acquisition module is used to acquire data operation record information of device nodes other than the target device node, as well as the current data in the target database on the target device node; The data synchronization module is used to perform data synchronization operations on the target database on each device node based on the operation instructions, the data operation record information, and / or the current data.

11. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to perform the steps of the method as described in any one of claims 1-9.

Images