Power SCADA (supervisory control and data acquisition) message bus service method, system and equipment based on distributed architecture and medium
By optimizing the power SCADA system through distributed architecture and Kafka and Raft protocols, the problems of poor scalability and low real-time performance are solved, efficient data transmission and reliability are achieved, and large-scale power equipment access and real-time monitoring are supported.
Patent Information
- Application Number
- CN202510583591.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-07
- Publication Date
- 2025-09-16
AI Technical Summary
The existing power SCADA system has problems such as poor scalability, low real-time performance and insufficient data reliability, and cannot meet the real-time requirements and data integrity needs of the power system.
The electric power SCADA message bus service method based on distributed architecture is adopted. The message routing path is dynamically allocated through the improved consistent hashing algorithm. The Kafka distributed publish-subscribe system and Raft protocol are combined to realize data storage and synchronization. It supports high concurrency processing and load balancing, and monitors the node status through the heartbeat mechanism to ensure system reliability.
The single-node message throughput reaches 800,000 messages per second, the end-to-end delay is ≤1.5ms, and it supports online expansion to 100+ nodes. The message delivery success rate during master-slave switching is ≥99.999%, meeting the real-time and reliability requirements of the power system.
Smart Images

Figure CN120658742A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of power system automation and industrial communication technology, and in particular to a power SCADA message bus service method, system, equipment and medium based on a distributed architecture. Background Art
[0002] Current power SCADA (Supervisory Control and Data Acquisition) systems generally use centralized message-based middleware (such as ActiveMQ and RabbitMQ) to implement device communication and data distribution. However, this system suffers from the following technical drawbacks:
[0003] Single point of failure and performance bottleneck: Under the centralized architecture, a single message server needs to handle data forwarding for all devices. When the number of connected devices exceeds 5,000 nodes, the message delay exceeds 100ms, which cannot meet the real-time requirements of power system protection actions (typical requirement ≤20ms).
[0004] Insufficient protocol compatibility: Traditional middleware lacks optimized parsing capabilities for power industry protocols (such as IEC 60870-5-104 and DNP3), requiring the additional deployment of protocol conversion gateways, increasing system complexity and cost.
[0005] Data consistency risk: Message loss or duplicate delivery may occur during the active / standby switching process, affecting the integrity of alarm events and telemetry data.
[0006] Poor horizontal scalability, unable to adapt to the exponential growth of smart grid devices;
[0007] The data loss rate is higher than 0.1%, which does not meet the requirements of the "DL / T 860 Substation Communication Standard";
[0008] End-to-end transmission delay exceeds 500ms, making it difficult to meet real-time fault response requirements.
[0009] In response to the problems of poor scalability, low real-time performance and insufficient data reliability in the existing technology, the present invention proposes a distributed message bus service architecture based on Kafka. Summary of the Invention
[0010] In view of the above-mentioned problems, the present invention is proposed.
[0011] Therefore, the technical problem solved by the present invention is: how to solve the problems of poor scalability, low real-time performance and insufficient data reliability in the prior art.
[0012] To solve the above technical problems, the present invention provides the following technical solutions: a power SCADA message bus service method based on a distributed architecture, which includes the following steps: collecting real-time operating data from power equipment; generating corresponding message topics according to data types and regions, and Broker nodes dynamically allocating message routing paths through hash rings to avoid single-point bottlenecks; distributing and synchronizing the collected message data; SCADA applications subscribing to message data according to preset topics, and performing monitoring, alarming or analysis operations according to topic types, and realizing parallel processing and load balancing through a partitioning mechanism; monitoring the status of each Broker node through a periodic heartbeat mechanism, and displaying message throughput, transmission delay and node status information in real time in the monitoring system.
[0013] As a preferred solution of the distributed architecture-based power SCADA message bus service method described in the present invention, the power equipment includes a remote terminal unit, an intelligent electronic device or a smart meter, and data is transmitted through multiple communication protocols.
[0014] As a preferred solution of the distributed architecture-based power SCADA message bus service method described in the present invention, the Broker node dynamically allocates message routing paths through a hash ring, including an improved consistent hashing algorithm that dynamically calculates hash weights based on the real-time load of the node CPU, memory, and network bandwidth, expressed as:
[0015]
[0016] Among them, W i is the hash weight, α, β, γ are adjustable coefficients, CPU usage is the node CPU usage, Memory usage is the node memory usage, Bandwidth free Bandwidth is the available bandwidth value of the current node. total is the total bandwidth value of the current node.
[0017] As a preferred solution of the electric power SCADA message bus service method based on a distributed architecture described in the present invention, the collected message data is distributedly stored and synchronously processed, including writing high-frequency data into a memory database and compressing low-frequency data into a distributed file system; synchronizing message logs between multiple Broker nodes through a consistency protocol to ensure consistency of message data between nodes; in the event of a failure of the master Broker node, electing a new master node through a consistency protocol and restoring the continuity of the message queue.
[0018] As a preferred solution of the distributed architecture-based power SCADA message bus service method described in the present invention, the SCADA application subscribes to message data according to preset topics, including topics representing telemetry data and topics representing alarm events, wherein the telemetry data topic is used to refresh the power grid topology map in real time, and the alarm event topic is used to trigger fault alarm response, push operation and maintenance work orders or perform automatic load shedding operations. The SCADA application executes corresponding business processing procedures according to different topic types.
[0019] As a preferred solution of the distributed architecture-based power SCADA message bus service method described in the present invention, the parallel processing and load balancing achieved through the partitioning mechanism include Kafka dividing each message topic into multiple partitions, and allocating different partitions to multiple consumer instances in the SCADA application through the partitioning mechanism. Multiple consumer instances form the same consumer group and consume data from the corresponding partitions in parallel, thereby achieving high-concurrency processing and overall load balancing of telemetry and alarm data.
[0020] As a preferred solution of the distributed architecture-based power SCADA message bus service method described in the present invention, the status of each Broker node is monitored through a periodic heartbeat mechanism, including that each Broker node periodically sends a heartbeat packet. If no heartbeat response is received within a preset timeout period, the corresponding node is marked as a faulty node and removed from the message routing ring. At the same time, the message throughput, transmission delay and node health status information of each Broker node are displayed in real time in the monitoring system.
[0021] Another object of the present invention is to provide an electric power SCADA message bus service system based on a distributed architecture.
[0022] To solve the above technical problems, the present invention provides the following technical solutions: an electric power SCADA message bus service system based on a distributed architecture, comprising: a protocol adaptation layer, a distributed message bus layer, a data consistency layer, and a fault tolerance and monitoring layer; the protocol adaptation layer is used to convert heterogeneous power equipment data into a standard format; the distributed message bus layer is composed of multiple stateless Broker nodes, dynamically allocates message routing paths based on an improved consistent hashing algorithm, and distributes and forwards messages in the bus through a Kafka distributed publish-subscribe message system; the data consistency layer realizes cross-node log synchronization through the Raft protocol, and adopts a hybrid persistence strategy to store message data; the fault tolerance and monitoring layer ensures system reliability through heartbeat detection and message retransmission mechanisms.
[0023] The present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the power SCADA message bus service method based on a distributed architecture are implemented.
[0024] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a power SCADA message bus service method based on a distributed architecture.
[0025] Beneficial effects of the present invention: Through C++ memory pool optimization and zero-copy technology, the present invention has a measured single-node message throughput of 800,000 messages per second (1KB / message), and an end-to-end delay of ≤1.5ms (99.9% percentile).
[0026] It supports online addition / removal of Broker nodes, and the cluster size can be dynamically expanded to 100+ nodes to meet the access needs of millions of terminals in provincial power grids.
[0027] Cross-node data synchronization is achieved based on the Raft protocol, and the message delivery success rate during master-slave switching is ≥99.999%. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0029] Figure 1 The present invention provides an overall flow chart of a distributed architecture-based power SCADA message bus service method according to an embodiment of the present invention.
[0030] Figure 2 A flow chart of a dynamic routing algorithm for a power SCADA message bus service method based on a distributed architecture is provided in accordance with an embodiment of the present invention.
[0031] Figure 3 A system layered architecture diagram of a distributed-architecture-based power SCADA message bus service system provided by one embodiment of the present invention. DETAILED DESCRIPTION
[0032] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.
[0033] Example 1, with reference to Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a power SCADA message bus service method based on a distributed architecture, including: collecting real-time operation data from power equipment; generating corresponding message topics according to data type and area to which it belongs, and Broker nodes dynamically allocate message routing paths through hash rings to avoid single-point bottlenecks; distributing and synchronizing the collected message data; SCADA applications subscribe to message data according to preset topics, and perform monitoring, alarm or analysis operations according to topic types, and achieve parallel processing and load balancing through a partitioning mechanism; monitoring the status of each Broker node through a periodic heartbeat mechanism, and displaying message throughput, transmission delay and node status information in real time in the monitoring system.
[0034] S1. Collect real-time operating data from power equipment.
[0035] Power equipment includes remote terminal units, intelligent electronic devices or smart meters, and data is transmitted through various communication protocols.
[0036] Collect data in real time from power equipment (such as RTUs, IEDs, smart meters) through different communication protocols (IEC 61850, Modbus, DNP3, etc.).
[0037] S2. Generate corresponding message topics based on data type and region. Broker nodes dynamically allocate message routing paths through hash rings to avoid single-point bottlenecks.
[0038] The improved consistent hashing algorithm dynamically calculates the hash weight according to the real-time load of the node CPU, memory and network bandwidth, which is expressed as,
[0039]
[0040] Among them, W i is the hash weight, α, β, γ are adjustable coefficients, CPU usage is the node CPU usage, Memory usage is the node memory usage, Bandwidth free Bandwidth is the available bandwidth value of the current node. totalis the total bandwidth value of the current node.
[0041] like Figure 2 As shown in the figure, KafkaTopic is defined according to the data type (telemetry, telesignaling, alarm) and area (substation, distribution line). Broker nodes dynamically allocate message routing paths through hash rings to avoid single point bottlenecks.
[0042] S3. Distribute and synchronize the collected message data.
[0043] High-frequency data is written to the in-memory database, and low-frequency data is compressed and written to the distributed file system.
[0044] The message logs are synchronized between multiple Broker nodes through the consistency protocol to ensure the consistency of message data between nodes.
[0045] In the event of a failure of the master Broker node, a new master node is elected through the consensus protocol and the continuity of the message queue is restored.
[0046] Specifically, real-time, high-frequency data (such as second-level telemetry) is stored in an in-memory database (Redis), while low-frequency alarm and event data is stored in a distributed file system (such as HDFS). The Raft protocol is then used to synchronize cross-node logs to ensure data consistency between the primary and backup brokers. When the primary node fails, Raft elects a new primary node and restores the message queue.
[0047] S4. SCADA applications subscribe to message data based on preset topics and perform monitoring, alarm, or analysis operations based on topic types, achieving parallel processing and load balancing through a partitioning mechanism.
[0048] Topics representing telemetry data and alarm events. The telemetry data topic is used to refresh the power grid topology in real time, and the alarm event topic is used to trigger fault alarm responses, push operation and maintenance work orders, or perform automatic load shedding operations. The SCADA application executes the corresponding business processing flow based on different topic types.
[0049] Kafka divides each message topic into multiple partitions and uses the partitioning mechanism to assign different partitions to multiple consumer instances in the SCADA application. Multiple consumer instances form the same consumer group and consume data from the corresponding partitions in parallel, achieving high-concurrency processing and overall load balancing of telemetry and alarm data.
[0050] Specifically, SCADA applications (such as real-time monitoring and fault analysis) subscribe to related topics as consumers. For example, the monitoring system subscribes to / substation* / telemetry to refresh the power grid topology in real time, and the alarm system subscribes to / grid-alarm to trigger work order push or automatic load shedding. For load balancing, Kafka automatically assigns partitions to consumer groups to achieve parallel processing.
[0051] S5. Monitor the status of each Broker node through a periodic heartbeat mechanism, and display message throughput, transmission delay and node status information in real time in the monitoring system.
[0052] Each Broker node periodically sends a heartbeat packet. If no heartbeat response is received within the preset timeout period, the corresponding node will be marked as a faulty node and removed from the message routing ring. At the same time, the message throughput, transmission delay and node health status information of each Broker node will be displayed in real time in the monitoring system.
[0053] Specifically, Broker nodes send heartbeat packets every 5 seconds. The timed-out node is marked as offline. When a message fails to be sent, retransmission is triggered through the ACK mechanism (maximum retries 3 times). The monitoring layer displays the message throughput, latency, and node health status in real time.
[0054] Furthermore, the protocol adaptation layer dynamically loads protocol parsing through a plug-in mechanism, converting the chaotic data (in different formats and protocols) sent by different devices (such as electricity meters and sensors) into a unified "standard language" (such as JSON format). The converted data takes up memory and needs to be cleaned up after use. The release method is like a cleaner, responsible for clearing away unnecessary data packets to prevent memory accumulation (preventing program jams).
[0055] Embodiment 2, the second embodiment of the present invention, provides a power SCADA message bus service method based on a distributed architecture.
[0056] Message routing process:
[0057] Device A sends DNP3 protocol data to the nearest broker node.
[0058] The protocol adaptation layer calls the DNP3 plug-in to parse the data and generate a standard format message body (including fields such as device ID, timestamp, and telemetry value).
[0059] The dynamic routing service calculates the hash value based on the device ID and selects the target Broker node with the lowest load.
[0060] The target node writes the message to the local memory queue and synchronizes it to other nodes through the Raft protocol.
[0061] The SCADA client that subscribes to the device data pulls the message from the optimal node.
[0062] Fault-tolerant recovery process:
[0063] The monitoring service detected that the heartbeat of Broker node N1 timed out.
[0064] Mark N1 as failed and remove it from the hash ring.
[0065] The message originally processed by N1 is rerouted to N2 and N3 nodes.
[0066] After node N1 recovers, the data synchronization engine resends the messages during the offline period.
[0067] Kafka C++ client integration
[0068] To implement producer / consumer using the librdkafka library, you need to install the development package.
[0069] The producer sample code uses the librdkafka library to implement Kafka producer functionality:
[0070] Create a producer: initialize the configuration and set the Kafka cluster address (kafka1:9091, kafka2:9091).
[0071] Send data: The payload data is copied and asynchronously pushed to the scada_data topic, without specifying partitions (default strategy) and callbacks.
[0072] Consumer implementations need to configure enable.auto.commit = false to ensure accurate control of offsets.
[0073] Message protocol design
[0074] Data serialization: Use Protobuf binary format to define the SCADA data model;
[0075] Defines an industrial equipment monitoring data protocol (Protobuf format), including timestamp, voltage, current, device ID and operating status (normal / warning / fault), for standardized transmission and status monitoring of equipment data, and for serialization and cross-platform communication of equipment data in systems such as SCADA / Kafka
[0076] Topic planning: scada_realtime: high-frequency telemetry data (QoS 0), scada_events: device status change events (QoS 1), scada_commands: control command issuance.
[0077] Reliability guarantee mechanism: on the producer side: set acks=all to ensure that all replicas confirm the write, enable the retry mechanism: retries=5, retry.backoff.ms=1000; on the consumer side: manually submit offsets to avoid data loss, and use consumer groups to achieve load balancing
[0078] Performance optimization strategy, throughput improvement, batch sending configuration: batch.size = 16384, linger.ms = 2037, partition strategy by substation ID hash partitioning to achieve data parallel processing.
[0079] To ensure real-time performance, zero-copy transmission is enabled: compression.type = zstd to reduce network load. Memory pool optimization: pre-allocate message buffers to avoid frequent memory requests. For high availability design, the Kafka cluster deploys more than 3 Brokers, sets the replication factor to 36, and integrates ZooKeeper to achieve automatic failover.
[0080] This solution decouples SCADA systems using Kafka. In real-world environments, it supports data throughput of over 100,000 points per second, with end-to-end latency below 200ms, meeting the real-time requirements of power monitoring. In actual deployment, the number of partitions and thread pool parameters should be adjusted based on the specific hardware configuration.
[0081] Example 3, the second embodiment of the present invention, provides a power SCADA message bus service method based on a distributed architecture. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.
[0082] In a 500kV substation monitoring scenario, the comparison of key indicators after deploying this system is shown in Table 1.
[0083] Table 1 Comparison between traditional method and the method of the present invention
[0084] index Traditional solution The present invention Maximum number of connected devices 5,000 100,000+ Average message delay 85ms 1.2ms Active / standby switchover time 30s 200ms
[0085] In actual deployment tests at 500kV substations, this system achieved significant improvements in key performance indicators compared to traditional centralized monitoring solutions:
[0086] The device access capacity has seen a breakthrough growth, with the maximum number of connected devices increased from 5,000 in traditional solutions to more than 100,000. It can support real-time access to a full range of IoT devices such as smart meters, relay protection devices, and environmental sensors in power stations, meeting the needs of large-scale deployment of power grids.
[0087] The real-time response reaches millisecond level, and the average message transmission delay is reduced from 85ms of the traditional solution to 1.2ms, making the end-to-end transmission of substation fault signals from collection to the monitoring master station faster.
[0088] High availability has been improved, with the switching time between the primary and standby nodes optimized from 30 seconds in traditional solutions (relying on manual intervention) to 200ms automatically completed. The switching process is unaware of the business, ensuring the power monitoring system's continuous operation with zero interruption in hardware failure scenarios.
[0089] Example 4 is the third embodiment of the present invention, which differs from the first two embodiments in that:
[0090] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0091] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.
[0092] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.
[0093] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0094] Example 5, reference Figure 3 , which is the fourth embodiment of the present invention, provides an electric power SCADA message bus service system based on a distributed architecture, including a protocol adaptation layer, a distributed message bus layer, a data consistency layer, and a fault tolerance and monitoring layer.
[0095] Protocol adaptation layer, used to convert heterogeneous power equipment data into a standard format.
[0096] The distributed message bus layer consists of multiple stateless broker nodes, which dynamically allocates message routing paths based on an improved consistent hashing algorithm and distributes messages in the forwarding bus through the Kafka distributed publish-subscribe messaging system.
[0097] The data consistency layer implements cross-node log synchronization through the Raft protocol and adopts a hybrid persistence strategy to store message data.
[0098] The fault tolerance and monitoring layer ensures system reliability through heartbeat detection and message retransmission mechanisms.
[0099] The protocol adaptation layer implements protocol parsing plug-ins based on the polymorphic factory model and supports dynamic loading and hot replacement of protocols such as DNP3, IEC 61850, and ModbusTCP.
[0100] Provides a unified data interface (IDataPacket) to convert heterogeneous device data into an internal standard format (based on Protobuf definition).
[0101] Real-time power grid data is collected through RTU / PLC devices, using IEC 60870-5-104 or DNP3 protocol for communication.
[0102] Distributed message bus layer, message broker cluster: Multiple Broker nodes form a decentralized cluster, and a single node uses a C++ high-performance network library (such as Boost.Asio) to implement non-blocking I / O, supporting more than 100,000 concurrent connections.
[0103] Dynamic routing service: Based on the improved consistent hashing algorithm (WeightedConsistentHash), the message path is dynamically allocated according to the node load weight to avoid hot spots.
[0104] Build a distributed message queue based on Kafka to achieve data buffering and asynchronous transmission.
[0105] Data consistency layer, distributed transaction engine: uses the Raft algorithm to achieve cross-node log synchronization, ensuring zero data loss on the backup node when the primary node fails.
[0106] Hybrid persistence strategy: High-priority messages (such as fault recordings) are written to the Redis cluster in real time, and historical data is batch-compressed and stored in a time series database (such as InfluxDB).
[0107] The data processing layer deploys consumer services for data analysis, alarm calculation and storage
[0108] Fault tolerance and monitoring layer, heartbeat detection mechanism: broadcast node status through UDP multicast periodically, define heartbeat timeout threshold, and trigger automatic isolation of faulty nodes.
[0109] Message retransmission queue: Enable local SQLite cache for unconfirmed messages (identified by the ACK mechanism) and retransmit them according to priority after the network is restored.
[0110] The status and historical trends of power equipment are displayed in real time through the UI real-time interface.
[0111] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A power SCADA message bus service method based on a distributed architecture, characterized by: include, Collect real-time operating data from power equipment; Generate corresponding message topics based on data type and region. Broker nodes dynamically allocate message routing paths through hash rings to avoid single point bottlenecks. Distribute and synchronize the collected message data; SCADA applications subscribe to message data based on preset topics and perform monitoring, alarm, or analysis operations based on topic types, achieving parallel processing and load balancing through a partitioning mechanism. The status of each Broker node is monitored through a periodic heartbeat mechanism, and the message throughput, transmission delay and node status information are displayed in real time in the monitoring system.
2. The power SCADA message bus service method based on a distributed architecture as claimed in claim 1, characterized in that: The power equipment includes a remote terminal unit, an intelligent electronic device or a smart meter, and data is transmitted through a variety of communication protocols.
3. The power SCADA message bus service method based on a distributed architecture as claimed in claim 2, characterized in that: The Broker node dynamically allocates message routing paths through the hash ring. include, The improved consistent hashing algorithm dynamically calculates the hash weight according to the real-time load of the node CPU, memory and network bandwidth, which is expressed as, Among them, W i is the hash weight, α, β, γ are adjustable coefficients, CPU usage is the node CPU usage, Memory usage is the node memory usage, Bandwidth free Bandwidth is the available bandwidth value of the current node. total is the total bandwidth value of the current node.
4. The power SCADA message bus service method based on a distributed architecture as claimed in claim 3, characterized in that: The distributed storage and synchronous processing of the collected message data includes: Write high-frequency data into the in-memory database and compress low-frequency data into the distributed file system; Synchronize message logs between multiple Broker nodes through the consistency protocol to ensure consistency of message data between nodes; In the event of a failure of the master Broker node, a new master node is elected through the consensus protocol and the continuity of the message queue is restored.
5. The power SCADA message bus service method based on a distributed architecture as claimed in claim 4, characterized in that: The SCADA application subscribes to message data according to preset topics, including topics representing telemetry data and topics representing alarm events. The telemetry data topic is used to refresh the power grid topology map in real time, and the alarm event topic is used to trigger fault alarm response, push operation and maintenance work orders or perform automatic load shedding operations. The SCADA application executes corresponding business processing procedures according to different topic types.
6. The power SCADA message bus service method based on a distributed architecture as claimed in claim 4, characterized in that: The parallel processing and load balancing achieved through the partitioning mechanism include Kafka dividing each message topic into multiple partitions, and allocating different partitions to multiple consumer instances in the SCADA application through the partitioning mechanism. Multiple consumer instances form the same consumer group and consume data from the corresponding partitions in parallel, thereby achieving high-concurrency processing and overall load balancing of telemetry and alarm data.
7. The power SCADA message bus service method based on a distributed architecture as claimed in claim 4, characterized in that: The periodic heartbeat mechanism is used to monitor the status of each Broker node, including: Each Broker node sends a heartbeat packet periodically. If no heartbeat response is received within the preset timeout period, the corresponding node will be marked as a faulty node and removed from the message routing ring. At the same time, the message throughput, transmission delay and node health status information of each Broker node will be displayed in real time in the monitoring system.
8. A power SCADA message bus service system based on a distributed architecture, applying the power SCADA message bus service method based on a distributed architecture as claimed in any one of claims 1 to 7, characterized in that: include: Protocol adaptation layer, distributed message bus layer, data consistency layer, and fault tolerance and monitoring layer; The protocol adaptation layer is used to convert heterogeneous power equipment data into a standard format; The distributed message bus layer consists of multiple stateless broker nodes, dynamically allocates message routing paths based on an improved consistent hashing algorithm, and distributes and forwards messages in the bus through the Kafka distributed publish-subscribe messaging system; The data consistency layer implements cross-node log synchronization through the Raft protocol and adopts a hybrid persistence strategy to store message data; The fault tolerance and monitoring layer ensures system reliability through heartbeat detection and message retransmission mechanism.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the power SCADA message bus service method based on a distributed architecture according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the power SCADA message bus service method based on a distributed architecture according to any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Engineering structure monitoring-oriented time series data edge caching and reliable transmission method and system
CN121397041A
IEC104 multi-channel parallel acquisition method and system, terminal and medium
CN121442021A