A redundant framework design method based on real-time data synchronization of hierarchical systems

Abstract

A redundant framework design method based on real-time data synchronization of a hierarchical system relates to large-scale monitoring system software in the field of industrial automation. The method uses logical data partitioning and thread-level redundant hot-spotting to improve the real-time and reliability of data synchronization between system levels. The large-scale industrial monitoring system platform processes data separately according to business logic, divides the full range of data into several data blocks according to the connection density between the data, and a corresponding number of communication modules are responsible for synchronization management respectively. When the two layers of the system platform are synchronized, for each independent logical data block, there are two channels, one main and one backup, for real-time data interaction. The data content of the main and backup channels is completely consistent. The main and backup data channels and dynamic token authority management provide thread-level redundant data protection measures, ensure zero-delay switching of data channels and the effectiveness of data writing, and greatly improve the system synchronization reliability.

Description

Technical Field

[0001] The present invention relates to large-scale monitoring system software in the field of industrial automation. It is a reliable redundant technical means for real-time synchronization of large-scale data under a hierarchical distributed network architecture. It is suitable for various automated industrial monitoring industries such as comprehensive monitoring, power monitoring, environment and equipment monitoring, fire monitoring, coal mining, metallurgy, and petroleum. Background Art

[0002] Large-scale industrial monitoring system platforms often use a hierarchical structure due to the complexity of the network framework. This article takes the subway integrated monitoring system as an example. The data platform of the subway integrated monitoring system can be divided into two levels: the center layer and the station layer. In specific projects, the data collection work of the system is mainly completed by the station layer system that is closer to the field-level equipment, but the center layer system may also have some data collection interfaces. At this time, the data synchronization work between the center layer and the station layer is extremely important. In order to ensure data reliability and real-time performance, multi-point redundant connections are usually used between the center layer and the station layer. At both ends of data synchronization, the center layer deploys multiple (≥2) servers that undertake synchronization transactions. Similarly, the station layer also has at least two servers participating in data synchronization communication. A data synchronization communication module is deployed on each participating server, but only one synchronization module or one server is actually performing synchronization tasks at the same time between each level, and the remaining synchronization modules are in hot standby state and are managed by the fault tolerance function of the system platform. The fault-tolerant management system of the monitoring data platform selects a synchronization master server from the center layer and the station layer according to the priority setting. The synchronization process on it is allowed to enter the normal working process, and all other server synchronization processes can only be in hot standby waiting state. This not only ensures process redundancy, but also saves software and hardware resources. This design is stable but still has disadvantages. In this mode, only one data channel between two processes of two servers is communicating data between the system levels at the same time. Once the channel fails, whether it is switching the address channel or reselecting a hot standby waiting process to enter the working state, it takes time to rebuild the data channel. Although the reconstruction time is likely to be very short, there is still a risk of data loss. Figure 1 As shown, the black solid line channel represents the synchronous channel that is working at a certain moment, and the dark dotted line arrow represents the backup channel that may be established after the red channel fails. Summary of the invention

[0003] In view of the design defects of process-level redundant hot standby in the prior art, the purpose of the present invention is to use the method of logical data blocking and thread-level redundant hot standby to improve the real-time and reliability of data synchronization between system levels.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is: a redundant framework design method based on real-time data synchronization of a hierarchical system, comprising:

[0005] Data distributed synchronization steps: The monitoring system data platform processes data separately according to business logic, and divides the full range of data into several independent logical data blocks according to the connection density between the data. The corresponding number of communication modules are responsible for synchronization management respectively;

[0006] Thread-level redundancy protection steps: The master and standby synchronization processes of the middle layer and the station layer open the communication thread and communicate normally. There are two communication channels exchanging data for a certain independent logical data block between the two layers of the monitoring system data platform at the same time. In order to ensure the validity of the data, only the receiving end of the master data synchronization channel processes the data, and the receiving end of the standby data synchronization channel does not process the data, but only confirms the received data;

[0007] Main and standby process switching protection steps: When the data block system levels are synchronized in normal working status, station server 1 first initiates a connection request to central server 1 according to the node priority. After the center responds, the two parties establish a connection, and the channel obtains the permission to read and write the real-time library (see dynamic token design for details). The main synchronization channel starts to work normally. At the same time, station server 2 and central server n also establish a backup synchronization channel corresponding to the data block, n≥2; once the main channel fails, the main channel process releases the database operation permission, the backup channel obtains the permission to process data and becomes the main channel at the same time. At this time, the original synchronization main channel attempts to self-repair from the failure and establish a stable synchronization backup channel.

[0008] Furthermore, the security device node list of the synchronization peer system layer is registered on the synchronization hardware nodes of the intermediate layer and the station layer. Each synchronization process must first load the device node list when it is started. After receiving the connection request, the communication server synchronization module will check whether the IP address requesting the connection is the IP on the node list. If not, it will be regarded as an illegal request and will not be responded to. The communication client synchronization module determines the device priority of initiating the connection request based on the loading order of the nodes in the list. When a client module enters the main or backup synchronization mode, once the currently maintained network channel fails, it will switch to the next node according to the node priority.

[0009] Furthermore, the fault self-repair includes:

[0010] When the original main synchronization module on the station server 1 fails, since there are only two server nodes at the station level and the main synchronization module is already working on server 2, module A will prompt an alarm message to warn the operation staff, and try to recover itself by restarting server 1. If the recovery is invalid and still cannot work normally, a high-level alarm will be issued to notify the operation technicians to find a solution; if the recovery is successful, the alarm recovery information will be prompted and the process will be waited for;

[0011] When the original main synchronization module on the central server 1 fails, since there are multiple server nodes to choose from in the central layer and the main synchronization module is already working on server N, the central servers 1 and N are eliminated. The original main synchronization module working on the station server 1 selects the central server in its own safety device list according to the node priority to initiate a connection request until the backup synchronization channel is successfully re-established.

[0012] Furthermore, in the thread-level redundancy protection step, a dynamic token permission is used to determine which of the two channels is the master and which is the backup. Only the process that obtains the token permission can perform write operations on the system library, specifically including:

[0013] First, when the data distributed synchronization step is completed, a corresponding token status point is established in the system for each logical data block to be synchronized, which is used to record the current real-time permission status of the token. The token can only be obtained by the upper-level servers at both ends of the synchronization. The lower-level servers can know whether they have the authority to process data through the token status bit of the message. The initial state of the token point is that the token is not occupied. When the first synchronization module process of the upper-level server is initialized, the point status is read. If the token is not occupied, the process takes the token and writes its own server serial number into the token, telling other subsequent processes that the token has been occupied. The process that obtains the token is the synchronization main process, which has the authority to read and write the system real-time library. In this way, the second synchronization module that is started can only automatically become a backup synchronization process.

[0014] When the main process finds that it cannot work normally, it must immediately return the token and restore the token point status to unoccupied. The system platform pushes the token point status change notification to the standby process in real time. The standby process then obtains the token and has the authority to read and write the system real-time library, and becomes the main process. The original main process recovers from the failure by switching channels or restarting and enters the standby process status.

[0015] Compared with the prior art, the present invention provides a reliable and effective solution to the most complex inter-layer system synchronization problem of large-scale industrial monitoring systems. By breaking data into small pieces, the risk of large-scale data synchronization is reduced, and the real-time effectiveness of system data is improved; the main and standby data channels and dynamic token authority management provide thread-level redundant data protection measures, ensuring zero-delay switching of data channels and the effectiveness of data writing, greatly improving system synchronization reliability, and increasing system product stability and competitiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of hierarchical synchronization under the fault-tolerant mechanism in the prior art.

[0017] Figure 2 Schematic diagram of the data block distributed synchronization mechanism of this embodiment.

[0018] Figure 3 This is a schematic diagram of the normal state of hierarchical synchronization of the data block system of this embodiment.

[0019] Figure 4 The following is a schematic diagram of the message structure definition of this embodiment. DETAILED DESCRIPTION

[0020] In order to better reflect the creativity of the present invention, the technical solution of the present invention is further explained below in conjunction with the accompanying drawings and specific embodiments.

[0021] Large-scale industrial monitoring system platforms often use a hierarchical structure due to the complexity of the network framework. This embodiment uses the subway integrated monitoring system as an example. The subway integrated monitoring system data platform can be divided into two levels: the central layer and the station layer. In specific projects, the system's data collection work is mainly completed by the station layer system that is closer to the field-level equipment, but the central layer system may also have some data collection interfaces. At this time, the data synchronization work between the central layer and the station layer becomes extremely important.

[0022] This embodiment uses a redundant framework design method based on hierarchical system real-time data synchronization, including: (i) breaking data into parts to achieve distributed synchronization

[0023] In the data synchronization of large data system platforms, although there is module (process level) redundancy protection, the data is single-channel. Once there is a communication interruption, it is still possible to cause the loss of full-range monitoring data. At present, the integration level of large-scale industrial monitoring system platforms is getting higher and higher, and the types of data are complex and diverse. It is completely possible to process the data separately according to the business logic, and divide the full range of data into several data blocks according to the connection tightness between the data. The corresponding number of communication modules are responsible for synchronization management. The processes of each module do not affect each other, and each is responsible for its own data. If a synchronization process fails, it will not affect the normal operation of other processes (data blocks). At the same time, the pressure of single-channel data communication can be relieved by connecting multiple modules to multiple channels, which not only increases reliability but also improves real-time performance.

[0024] In addition, the hardware at each level of the monitoring system must be redundantly configured. Taking the subway integrated monitoring system as an example, the central layer is configured with at least 4 servers, and each station is also configured with at least 2 servers. All data synchronization modules are evenly distributed to each server according to the number of hardware.

[0025] like Figure 2As shown in the figure, each data block has a main synchronization module and a backup synchronization module to ensure its functional reliability. According to the specific project configuration, the main service modules corresponding to each data block are evenly distributed to the servers at each layer of the system platform. The main and backup synchronization tasks of the same data block must be undertaken by different hardware nodes in the system layer to maximize the process-level redundancy reliability.

[0026] (II) More reliable communication thread-level redundancy protection

[0027] Although the data is divided into blocks, the risk is reduced, but for a single data block, once the main module fails, the standby module has not been able to switch from waiting to working state in time, and there is still a risk of data loss. In order to solve this problem, the present invention implements thread-level redundancy, and the main and standby synchronization processes between the two layers both open the communication thread and communicate normally. There are two communication channels for exchanging data at the same time for a certain data block between the two layers of the integrated monitoring system platform. Of course, in order to ensure the validity of the data, only the main data synchronization channel receiving end processes the data, and the standby data channel receiving end does not process the data, but only confirms the received data. In this way, redundant protection at the communication thread level is provided for the data, and the resource burden on the server is minimal, and only the network load is increased. With the capacity of the industrial monitoring backbone network of 10 ...

[0028] The list of secure device nodes of the synchronization peer system layer is registered on the synchronization hardware nodes of each layer of the system. Each synchronization process must first load the device node list when it starts. After receiving the connection request, the communication server synchronization module (upper-layer synchronization node) will check whether the IP address requesting the connection is an IP on the node list. If not, it will be considered an illegal request and will not be responded to. The communication client synchronization module (lower-layer synchronization node) determines the device priority of the initiating connection request based on the loading order of the nodes in the list. When a client module enters the main or backup synchronization mode, once the currently maintained network channel fails, it will switch to the next node according to the node priority.

[0029] (III) More reliable active / standby process switching protection

[0030] like Figure 3As shown, when a data block is synchronized between system levels in normal working state, station server 1 first initiates a connection request to central server 1 according to the node priority. After the center responds, the two parties establish a connection, and the channel obtains the permission to read and write the real-time library (see dynamic token design for details). The main synchronization channel starts to work normally. At the same time, station server 2 and central server n also establish a backup synchronization channel corresponding to the data block. Once the main channel fails, the main channel process releases the database operation permission, and the backup channel obtains the permission to process the data and turns into the main channel at the same time. At this time, the original synchronization main channel tries to self-repair from the failure and establish a stable synchronization backup channel. Here, the fault repair strategy needs to be discussed in categories according to the different occurrences of the failure.

[0031] Possible fault 1: the original main synchronization module (we refer to it as module A) on the station server 1 fails. Since there are generally only two server nodes at the station level, the main synchronization module is already working on server 2. Therefore, module A prompts an alarm message to warn the operation staff, and tries to recover itself by restarting server 1. If the recovery is invalid and it still cannot work normally, a high-level alarm is issued to notify the operation technicians to find a solution; if the recovery is successful, the alarm recovery information is prompted and the process wait state is entered.

[0032] Fault possibility 2: the original main synchronization module on the central server 1 (we refer to it as module C) fails. Since there are multiple server nodes to choose from in the central layer, and the main synchronization module is already working on server N, the central servers 1 and N are eliminated. Module A working on station server 1 selects the central server in its own safety device list according to the node priority to initiate a connection request until the backup synchronization channel is successfully re-established.

[0033] Dynamic tokens ensure unique read and write

[0034] When the two layers of the system platform are synchronized, for each independent logical data block, there are two channels, one main and one backup, for real-time data exchange. The data content of the main and backup channels is completely consistent. The only difference is that the data receiver of the backup channel does not need to process the data packet content. This design uses dynamic token permissions to determine which of the two channels is the main and which is the backup. Only the process that obtains the token permission can perform write operations on the system library. First, after the synchronization data is divided into blocks, a corresponding token status point is set up in the system for each data block to be synchronized to record the current token real-time permission status. The token can only be obtained by the upper-layer servers at both ends of the synchronization. The lower-layer server can know whether it has the permission to process data through the token status bit of the message. The initial state of the token point is set as token unoccupied (0). When the first synchronization module process of the upper-layer server is initialized, the state of the point is read. If the token is not occupied, the process takes the token and writes its own server serial number into the token to tell other subsequent processes that the token has been occupied. The process that obtains the token is the synchronization master process and has the permission to read and write the system real-time library. In this way, the second synchronization module that has been started can only automatically become a backup synchronization process.

[0035] When the main process finds that it cannot work normally, it must immediately return the token and restore the token point status to unoccupied. The system platform pushes the token point status change notification to the standby process in real time. The standby process then obtains the token and has the authority to read and write the system real-time library, and becomes the main process. The original main process recovers from the failure by switching channels or restarting, and enters the standby process status.

[0036] like Figure 4 As shown in the figure, the fourth byte of each frame message is the token identifier. The data sender fills in this byte according to whether the channel has the token authority. The data receiver only needs to check this identifier to know whether this frame message needs to be processed. In this way, the master-slave synchronization process can achieve zero-delay seamless switching when necessary, and ensure that only one process will update the system real-time library at the same time.

[0037] The above embodiments are only the best embodiments disclosed according to the technical solution of the present invention, and the protection scope of the present invention cannot be limited thereto. Any changes made on the basis of the technical solution in accordance with the technical ideas proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A redundant framework design method based on real-time data synchronization of a hierarchical system, characterized in that: include: Data distributed synchronization steps: The monitoring system data platform processes data separately according to business logic, and divides the full range of data into several independent logical data blocks according to the connection density between the data. The corresponding number of communication modules are responsible for synchronization management respectively; Thread-level redundancy protection steps: The master and standby synchronization processes of the middle layer and the station layer open the communication thread and communicate normally. There are two communication channels exchanging data for a certain independent logical data block between the two layers of the monitoring system data platform at the same time. In order to ensure the validity of the data, only the receiving end of the master data synchronization channel processes the data, and the receiving end of the standby data synchronization channel does not process the data, but only confirms the received data; Main and standby process switching protection steps: When the data block system levels are synchronized in normal working state, station server 1 first initiates a connection request to central server 1 according to the node priority. After the center responds, the two parties establish a connection, and the channel obtains the permission to read and write the real-time library. The main synchronization channel starts to work normally. At the same time, station server 2 and central server n also establish a backup synchronization channel corresponding to the data block, n≥2; once the main channel fails, the main channel process releases the database operation permission, and the backup channel obtains the permission to process data and turns into the main channel at the same time. At this time, the original synchronization main channel tries to self-repair from the failure and establish a stable synchronization backup channel; The fault self-repair comprises: When the original master synchronization module on the station server 1 fails, since there are only two server nodes at the station level and the master synchronization module is already working on the station server 2, the original master synchronization module on the station server 1 fails and prompts an alarm message to warn the operation staff, and tries to recover itself by restarting the station server 1. If the recovery is invalid and still cannot work normally, a high-level alarm is issued to notify the operation technicians to find a solution; if the recovery is successful, the alarm recovery information is prompted and the process wait state is entered; When the original main synchronization module on the central server 1 fails, since there are multiple server nodes to choose from in the central layer and the main synchronization module is already working on the central server N, the central servers 1 and N are eliminated, and the original main synchronization module working on the station server 1 selects the central server in its own safety device list according to the node priority to initiate a connection request until the backup synchronization channel is successfully re-established.

2. The redundant framework design method based on hierarchical system real-time data synchronization according to claim 1 is characterized in that: On the synchronization hardware nodes of the middle layer and the station layer, a list of secure device nodes of the synchronization peer system layer is registered. Each synchronization process must first load the device node list when it is started. After receiving a connection request, the communication server synchronization module will check whether the IP address requesting a connection is an IP on the node list. If not, it will be considered an illegal request and will not be responded to. The communication client synchronization module determines the device priority of initiating a connection request based on the loading order of the nodes in the list. When a client module enters the main or backup synchronization mode, once the currently maintained network channel fails, it will switch to the next node based on the node priority.

3. The redundant framework design method based on hierarchical system real-time data synchronization according to claim 1 is characterized in that: In the thread-level redundancy protection step, dynamic token permissions are used to determine which of the two channels is the master and which is the backup. Only the process that obtains the token permission can perform real-time write operations on the system library, specifically including: First, when the data distributed synchronization step is completed, a corresponding token status point is set up in the system for each logical data block to be synchronized, which is used to record the current token real-time permission status. The token can only be obtained by the upper-layer servers at both ends of the synchronization. The lower-layer servers can know whether they have the permission to process data through the token status bit of the message. The initial state of the token point is that the token is not occupied. When the first synchronization module process of the upper-layer server is initialized, the point status is read. If the token is not occupied, the process takes the token and writes its own server serial number into the token, telling other subsequent processes that the token has been occupied. The process that obtains the token is the synchronization main process, which has the permission to read and write the system real-time library. In this way, the second synchronization module that has been started can only automatically become a standby synchronization process; When the main process finds that it cannot work normally, it must immediately return the token and restore the token point status to unoccupied. The system platform pushes the token point status change notification to the standby process in real time. The standby process then obtains the token and has the authority to read and write the system real-time library, and becomes the main process. The original main process recovers from the failure by switching channels or restarting and enters the standby process status.

Images