A distributed industrial SCADA system measurement data processing method based on time series library

Abstract

The present invention discloses a distributed industrial SCADA system measurement data processing method based on a time series library, comprising: adopting an election mechanism to elect a leader node and a follower node from a distributed application deployment server cluster; adopting a heartbeat detection method to maintain normal communication between the leader node and each follower node; making the leader node adopt a virtual node partition mapping method combined with a hash algorithm to perform virtual partition division, and evenly distribute the virtual nodes to each time series database node; according to the received measurement data read / write request, using the measurement point ID as a keyword to determine the virtual partition in the cache, locate the corresponding physical time series library server, and thus perform measurement data read / write on the physical time series library server. The present invention can provide a unified interface and dynamic expansion service for measurement data read / write, and improve the real-time performance of data read / write.

Description

Technical Field

[0001] The invention relates to a distributed industrial SCADA system measurement data processing method based on a time series library, belonging to the technical field of data processing. Background Art

[0002] In the actual application of industrial SCADA systems, many applications require multi-dimensional data, resulting in many data collection points. At the same time, due to the actual needs of the application, the collected data usually needs to be retained for a long time for query, analysis and statistics. On the other hand, the collection of many data requires high real-time performance and needs to maintain a high collection frequency, so the data in the database is bound to increase. Therefore, under the premise of ensuring system stability and normal access, it will inevitably face the problem of processing large amounts of data.

[0003] At present, the storage of industrial SCADA systems mostly stays in relational databases. Relational databases such as Oracle have gradually failed to meet the requirements of system acquisition and monitoring for read and write performance in the big data scenario. Another part uses time series databases for storage. Although it meets the requirements of read and write performance, it is a bottleneck in terms of storage capacity because the supplier of the time series database is not open source for cluster deployment and only opens storage on a single node. Another part uses a big data solution similar to HBase. The read and write performance of this solution is between the relational database and the time series database. It supports dynamic expansion, but does not have the function of simple SQL statement query. It needs to introduce other middleware deployment, and the deployment and problem location are highly complex. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a distributed industrial SCADA system measurement data processing method based on a timing library, which can provide a unified interface and dynamic expansion service for measurement data reading / writing, thereby improving the real-time performance of data reading / writing.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] A distributed industrial SCADA system measurement data processing method based on a time series library, the method comprising:

[0007] Deploy distributed applications to multiple timing library servers in the same network segment for operation;

[0008] An election mechanism is used to elect the leader node and follower node from the distributed application deployment server cluster, and the status of the leader node and each follower node is updated accordingly according to the election results;

[0009] The heartbeat detection method is used to maintain normal communication between the Leader node and each Follower node to form a service cluster;

[0010] The Leader node uses the virtual node partition mapping method combined with the hash algorithm to perform virtual partition division, and evenly distributes the virtual nodes to each time series database node;

[0011] Store the partition mapping information of the virtual node into the cache and synchronize it to the Follower node;

[0012] According to the received measurement data read / write request, the virtual partition is determined in the cache with the measurement point ID as the keyword, and the corresponding physical timing library server is located according to the determined virtual partition, and the measurement data is read / written to the physical timing library server.

[0013] Furthermore, the method of electing a leader node and a follower node from a distributed application deployment server cluster using an election mechanism includes:

[0014] Enable all distributed application deployment server nodes to initiate voting and generate ballot information containing elements required for the election;

[0015] Receive ballot information initiated by other distributed application deployment server nodes, compare the ballot information according to a predetermined comparison rule, determine the master node for this round of election, and update the ballot information of other distributed application deployment server nodes;

[0016] Count whether more than half of the distributed application deployment server nodes have received the same ballot information:

[0017] If yes, the election is complete, and the master node of this round of election is used as the final Leader node, and the non-master nodes in other clusters are used as slave nodes and Follower nodes. At the same time, the status of the Leader node is updated to Leading, and the status of the Follower node is updated to Following;

[0018] Otherwise, the next round of voting is initiated again until more than half of the distributed application deployment server nodes receive the same ballot information.

[0019] Furthermore, the elements required for the election include at least SID and TID; the SID represents the machine number of the distributed application deployment server; the TID represents the transaction number in the election process, and the default initialization value is 0;

[0020] The comparison rules include:

[0021] Prioritize TID comparison: If TIDs are different, the node with the larger TID is elected as the Leader node; if TIDs are the same, SIDs are further compared and the node with the larger SID is elected as the Leader node.

[0022] Furthermore, the method of using the heartbeat detection method to maintain normal communication between the Leader node and each Follower node includes:

[0023] The Leader node performs session timeout checks at set intervals within the preset session lifecycle and only receives measurement data read / write requests within the session lifecycle.

[0024] Furthermore, the method in which the leader node uses the partition mapping method of the virtual node combined with the hash algorithm to divide the virtual partitions includes:

[0025] Initialize the partitions of the virtual nodes and determine the maximum scalable virtual node;

[0026] Count the distributed application deployment servers according to the SID, and evenly distribute the subsets of virtual nodes according to the counting results;

[0027] The SID is the machine ID of the distributed application deployment server.

[0028] Furthermore, the method of determining a virtual partition in the cache using the measurement point ID as a keyword and locating the corresponding physical timing library server according to the determined virtual partition includes:

[0029] Get the query SQL statement and measurement point ID, and calculate the hash partition value;

[0030] According to the hash partition value, query the hash value partition range of the virtual node assigned by the Leader node in the cache and retrieve the ID of the corresponding Leader node.

[0031] Furthermore, the method for reading / writing measurement data on the physical timing library server includes:

[0032] The distributed application deployment server node uses the SQL statement and measurement point ID in the read / write request as input parameters and uses JDBC to connect to the timing library to perform measurement data read / write operations.

[0033] Furthermore, the method of storing the partition mapping information of the virtual node into a cache and synchronizing it to the Follower node includes:

[0034] When starting and dynamically increasing or decreasing distributed application deployment server nodes, initiate the virtual node allocation process and store the updated partition information in the cache;

[0035] The updated partition information is obtained from the cache through periodic polling, and the updated partition information is synchronized to each Follower node.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The method provided by the embodiment of the present invention is as follows:

[0038] At the deployment level, an election mechanism (Fast Leader Election) is adopted to provide unified scheduling services for time series databases;

[0039] At the data operation level, it provides a unified interface service, uses the hash algorithm (CRC16) and the partition mapping (partition-hash) method of the virtual node for access, accurately locates the physical node where the data is located, and uses the query performance of the time series library to improve the real-time performance of data acquisition;

[0040] In terms of data expansion, combined with the election mechanism (Fast Leader Election), hash algorithm (CRC16), and partition mapping (partition-hash) method of virtual nodes described above, when adding or deleting physical nodes, re-election is carried out and the data migration of virtual nodes is minimized and controllable based on the mapping relationship between virtual nodes and physical nodes, so as to realize dynamic addition and deletion of nodes.

[0041] The method provided in the embodiment of the present invention can be used for distributed read / write operations of industrial production measurement data, providing strong support for enterprise data governance, data analysis and production operation status perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is an architecture diagram based on which the method provided according to an embodiment of the present invention relies.

[0043] Figure 2 is a method flow chart of an election mechanism provided according to an embodiment of the present invention;

[0044] Figure 3 is a flow chart of a method for heartbeat detection provided according to an embodiment of the present invention;

[0045] Figure 4 is a flow chart of a method for partition mapping of virtual nodes provided according to an embodiment of the present invention;

[0046] Figure 5 It is a flow chart of a method for storing partition mapping information of a virtual node in a cache and synchronizing it to a Follower node according to an embodiment of the present invention;

[0047] Figure 6 is a flow chart of a method for reading / writing measurement data provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0049] The distributed industrial SCADA system measurement data processing method based on the time series library provided in the embodiment of the present invention can be based on Figure 1 The implementation of the architecture diagram shown includes the following steps:

[0050] Step 1: Deploy the distributed application to multiple timing library servers in the same network segment for operation;

[0051] Step 2: Use the election mechanism to elect the leader node and follower node from the distributed application deployment server cluster, and update the status of the leader node and each follower node accordingly according to the election results;

[0052] There are two server roles in the distributed application deployment server: Leader and Follower. If a leader election is performed, at least two machines are required. If the leader fails, only one machine is left and the election cannot be performed. In the embodiment of the present invention, a server cluster consisting of three machines is taken as an example.

[0053] like Figure 2 FIG. 1 is a flowchart of a method for an election mechanism provided according to an embodiment of the present invention. The specific election process is as follows:

[0054] Step 201: All distributed application deployment server nodes initiate voting to generate ballot information containing elements required for the election:

[0055] Since it is just started, each distributed application deployment server node will initiate voting as the leader node by default, and each vote contains the most basic elements required for the election, such as SID and TID. The SID represents the machine number of the distributed application deployment server; the TID represents the transaction number in the election process, and the default initialization value is 0. For example: SID is 1, TID is 0, which can be expressed as (1, 0). Since each distributed application deployment server node initiates voting according to its priority as the leader node, the default ballot generated by the distributed application deployment server node server1 is (1, 0), and the ballot generated by the distributed application deployment server node server2 is (2, 0), and so on.

[0056] Step 202: receiving ballot information initiated by other distributed application deployment server nodes, comparing the ballot information according to a predetermined comparison rule, determining the master node for this round of election, and updating the ballot information of other distributed application deployment server nodes;

[0057] The votes sent by other distributed application deployment server nodes must go through a series of verifications, such as whether they are votes for this round of voting, whether they are votes from distributed application deployment server nodes in the Looking state, where Looking means the waiting state, standby state, or waiting for election state. If the verification passes, it is compared with the vote information of the current distributed application deployment server node. In the embodiment of the present invention, the comparison rules are as follows:

[0058] ① Prioritize comparison of TIDs. If TIDs are different, select the distributed application deployment server node with the larger TID as the leader node;

[0059] ② If the TIDs of two votes are the same, the SIDs are further compared, and the distributed application deployment server node where the vote with the larger SID is located is used as the leader node by default;

[0060] For example, when server1 receives the vote from server2, both votes are from the first round of voting, so the TID is 0. After further comparison of SID, server1's SID is 1, while server2's SID is 2, which is greater than its own SID. Therefore, server2 should be the leader node. Therefore, server1 will update its own vote to (2,0) based on server2's vote. In the next round of voting, the vote information sent by server1 is (2,0).

[0061] Step 203: Count whether more than half of the distributed application deployment server nodes have received the same ballot information:

[0062] If yes, the election is complete, and the master node of this round of election is used as the final Leader node, and the non-master nodes in other clusters are used as slave nodes and Follower nodes. At the same time, the status of the Leader node is updated to Leading, and the status of the Follower node is updated to Following;

[0063] Otherwise, the next round of voting is initiated again until more than half of the distributed application deployment server nodes receive the same ballot information.

[0064] After each round of voting, all votes will be counted to determine whether more than half of the distributed application deployment server nodes have received the same voting information.

[0065] For the current server1 and server2, the election process is completed only if the votes of the two are the same. If the total number of distributed application deployment server nodes is an odd number, only (total number of distributed application deployment server nodes + 1) / 2 distributed application deployment server nodes need to receive the same votes. As long as server1 compares the votes and sends the vote information (2,0), the election is completed.

[0066] Once the election is completed and a Leader node is generated, each distributed application deployment server node will update its own status: if it is a Follower, the corresponding status is changed to Following; if it is a Leader, the corresponding status is changed to Leading.

[0067] Step 3: Use the heartbeat detection method to maintain normal communication between the Leader node and each Follower node to form a service cluster;

[0068] The connection between the Follower node and the Leader node is based on a TCP persistent connection. The client connects to the server’s default port 10031, which is a session.

[0069] like Figure 3 As shown in the figure, from the first connection establishment, the Follower node starts the session life cycle, and the Follower node initiates a heartbeat request to the Leader node. Each session can be set with a timeout. The Leader node periodically performs a session timeout check during operation, and performs a session timeout check at the set time interval. The default value of the time interval can be 1000 milliseconds. The Follower node will send a request (including read / write) or a ping request to the Follower node within the session timeout range to complete the session activation, thereby maintaining the validity of the session.

[0070] Step 4: The Leader node uses the virtual node partition mapping method combined with the hash algorithm to perform virtual partition division, and evenly distributes the virtual nodes to each time series database node; specifically, the following steps are included:

[0071] Step 401: Initialize the partitions of the virtual nodes and determine the maximum scalable virtual node. Figure 4 As shown, if 1024 virtual nodes are initialized, it means that the maximum number of scalable virtual nodes is 1024. In the embodiment of the present invention, a 16-bit CRC check algorithm is used, that is:

[0072] partition-hash = CRC16(key) mod 1024;

[0073] Step 402: Count the distributed application deployment servers according to the SID, and evenly distribute the subsets of virtual nodes according to the counting results; see Figure 4 ,If there are four distributed application deployment servers, namely DB1, DB2, DB3, and DB4 (they can be sorted according to SID in actual operation), then the 1024 virtual nodes can be evenly divided into four groups, namely 0~255, 256~511, 512~767, and 768~1023, and allocated to DB1, DB2, DB3, and DB4 respectively.

[0074] Step 5: Store the partition mapping information of the virtual node into the cache and synchronize it to the Follower node;

[0075] like Figure 5 As shown, the Leader node stores the partition mapping information of the virtual node in the cache, and the synchronization module obtains the updated partition information from the cache through periodic polling. The synchronization module is used to compare the partition information (i.e., the mapping information in the figure). When an update occurs, the partition information is synchronized to the Follower node through broadcasting. When starting and dynamically increasing or decreasing the distributed application deployment server nodes, the Leader node will initiate the virtual node allocation process and store the updated partition information in the cache so that the synchronization module can poll regularly.

[0076] Step 6: According to the received measurement data read / write request, the measurement point ID is used as a keyword to determine the virtual partition in the cache, and the corresponding physical timing library server is located according to the determined virtual partition; Figure 6 As shown, specifically:

[0077] Get the query SQL statement and measurement point ID, and calculate the hash partition value;

[0078] According to the hash partition value, query the hash value partition range of the virtual node assigned by the Leader node in the cache and retrieve the ID of the corresponding Leader node.

[0079] Step 7: Reading / writing measurement data from the physical timing library server, specifically including:

[0080] The distributed application deployment server node uses the SQL statement and measurement point ID in the read / write request as input parameters and uses JDBC to connect to the timing library to perform measurement data read / write operations.

[0081] The present invention proposes a distributed industrial SCADA system measurement data processing method based on a time series library. Aiming at the characteristics of a large number of measurement points, a high acquisition frequency, and a large amount of collected data, the present invention realizes the distributed deployment and dynamic expansion / reduction capabilities based on the time series library. Aiming at the characteristics of distributed deployment, a unified interface for operating the time series database is provided. Practical applications show that the application can effectively improve the storage capacity of industrial production measurement data, and has high availability and dynamic expansion capabilities.

[0082] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A distributed industrial SCADA system measurement data processing method based on a time series library, characterized in that: The method comprises: Deploy distributed applications to multiple timing library servers in the same network segment for operation; An election mechanism is used to elect the leader node and follower node from the distributed application deployment server cluster, and the status of the leader node and each follower node is updated accordingly according to the election results; The heartbeat detection method is used to maintain normal communication between the Leader node and each Follower node to form a service cluster; The leader node adopts a partition mapping method of a virtual node combined with a hash algorithm to perform virtual partition division, including: initializing the partition of the virtual node and determining the maximum scalable virtual node; counting the distributed application deployment servers according to the SID, and evenly distributing the subsets of the virtual nodes according to the counting result; wherein the SID is the machine number of the distributed application deployment server; Evenly distribute virtual nodes to each time series database node; Store the partition mapping information of the virtual node into the cache and synchronize it to the Follower node; According to the received measurement data read / write request, the virtual partition is determined in the cache with the measurement point ID as the keyword, and the corresponding physical timing library server is located according to the determined virtual partition, and the measurement data is read / written to the physical timing library server.

2. The distributed industrial SCADA system measurement data processing method based on the timing library according to claim 1 is characterized in that: The method for electing a leader node and a follower node from a distributed application deployment server cluster using an election mechanism includes: Enable all distributed application deployment server nodes to initiate voting and generate ballot information containing elements required for the election; Receive ballot information initiated by other distributed application deployment server nodes, compare the ballot information according to a predetermined comparison rule, determine the master node for this round of election, and update the ballot information of other distributed application deployment server nodes; Count whether more than half of the distributed application deployment server nodes have received the same ballot information: If yes, the election is complete, and the master node of this round of election is used as the final Leader node, and the non-master nodes in other clusters are used as slave nodes and Follower nodes. At the same time, the status of the Leader node is updated to Leading, and the status of the Follower node is updated to Following; Otherwise, the next round of voting is initiated again until more than half of the distributed application deployment server nodes receive the same ballot information.

3. The distributed industrial SCADA system measurement data processing method based on the timing library according to claim 2 is characterized in that: The elements required for the election include at least SID and TID; the SID represents the machine number of the distributed application deployment server; the TID represents the transaction number in the election process, and the default initialization value is 0; The comparison rules include: Prioritize TID comparison: If TIDs are different, the node with the larger TID is elected as the Leader node; if TIDs are the same, SIDs are further compared and the node with the larger SID is elected as the Leader node.

4. The distributed industrial SCADA system measurement data processing method based on the time series library according to claim 1 is characterized in that: The method of using the heartbeat detection method to maintain normal communication between the Leader node and each Follower node includes: The Leader node performs session timeout checks at set intervals within the preset session lifecycle and only receives measurement data read / write requests within the session lifecycle.

5. The distributed industrial SCADA system measurement data processing method based on the timing library according to claim 1 is characterized in that: The method of determining a virtual partition in a cache using a measurement point ID as a keyword and locating a corresponding physical timing library server according to the determined virtual partition includes: Get the query SQL statement and measurement point ID, and calculate the hash partition value; According to the hash partition value, query the hash value partition range of the virtual node assigned by the Leader node in the cache and retrieve the ID of the corresponding Leader node.

6. The distributed industrial SCADA system measurement data processing method based on the time series library according to claim 5 is characterized in that: The method for reading / writing measurement data on the physical timing library server includes: The distributed application deployment server node uses the SQL statement and measurement point ID in the read / write request as input parameters and uses JDBC to connect to the timing library to perform measurement data read / write operations.

7. The distributed industrial SCADA system measurement data processing method based on the time series library according to claim 1 is characterized in that: The method of storing the partition mapping information of the virtual node into the cache and synchronizing it to the Follower node includes: When starting and dynamically increasing or decreasing distributed application deployment server nodes, initiate the virtual node allocation process and store the updated partition information in the cache; The updated partition information is obtained from the cache through periodic polling, and the updated partition information is synchronized to each Follower node.

Images