Power transmission line monitoring master station system for massive terminal access

Abstract

The invention belongs to the technical field of power grid monitoring systems, and particularly relates to a power transmission line monitoring master station system oriented to mass terminal access, in particular to a scene which needs to support high-frequency concurrent access of tens of thousands of terminals such as image and video monitoring devices and process mass heterogeneous data. Comprising a terminal access layer, an application service layer, a data storage layer and a system integration layer which are in communication connection in sequence, and the terminal access layer, the application service layer, the data storage layer and the system integration layer cooperatively achieve concurrent access, heterogeneous data processing and full-link stable operation of massive power transmission line monitoring terminals. According to the invention, the requirements of a power transmission line monitoring master station system on high concurrency, high availability and high safety under the background of an intelligent power grid can be comprehensively met, and a stable technical support is provided for real-time monitoring and fault early warning of the power transmission line.

Description

Technical Field

[0001] This invention belongs to the field of power grid monitoring system technology, specifically to a transmission line monitoring master station system for massive terminal access, especially for scenarios that need to support tens of thousands of terminals (such as image and video monitoring devices) to access and process massive heterogeneous data at high frequency. Background Technology

[0002] With the development of smart grid and Internet of Things technologies, the number of transmission line monitoring terminals has increased from thousands to tens of thousands, and the data acquisition frequency has increased to once every 10 minutes. The data flow that the main station system needs to process is growing exponentially, which puts extremely high demands on concurrent processing, storage stability and service continuity.

[0003] However, the existing main station system has significant technical bottlenecks:

[0004] 1. Database bottleneck: Most databases use a single central database or a simple master-slave architecture, which is difficult to withstand high-frequency data writing pressure, resulting in a sharp drop in query performance for massive historical data, and there is a risk of single point of failure.

[0005] 2. Application performance bottleneck: Traditional monolithic architectures or basic microservices (such as the Spring Cloud framework) have poor scalability. Microservices do not achieve effective clustering and load balancing. The failure of a single service can easily lead to system paralysis and cannot meet the 24 / 7 operation requirements.

[0006] 3. Access and processing bottlenecks: The lack of efficient concurrency mechanisms and distributed storage solutions can easily lead to request blocking, data loss, and service outages when the number of terminals surges.

[0007] 4. Integration security bottleneck: When integrating with third-party systems such as IoT platforms, simple interface calls are used, making it difficult to balance the security requirements of the power industry with the efficiency and stability of data interaction.

[0008] The aforementioned shortcomings make the existing system unable to adapt to the trend of expanding monitoring scale and increasing data demand, and there is an urgent need for a main station system solution to break through the bottleneck. Summary of the Invention

[0009] To overcome the shortcomings of existing technologies and solve the aforementioned technical problems, this invention proposes a transmission line monitoring master station system for massive terminal access.

[0010] The technical solution adopted by this invention to solve its technical problem is to propose a transmission line monitoring master station system for massive terminal access, including a terminal access layer, an application service layer, a data storage layer and a system integration layer connected in sequence. The terminal access layer, application service layer, data storage layer and system integration layer work together to realize the concurrent access of massive transmission line monitoring terminals, heterogeneous data processing and stable operation of the entire link.

[0011] The terminal access layer is used to receive monitoring data access requests from tens of thousands of transmission line monitoring terminals (including image monitoring devices and video monitoring devices) and distribute the access requests to the application service layer.

[0012] The application service layer is used to receive access requests distributed by the terminal access layer, verify and standardize the monitoring data, and execute business logic.

[0013] The data storage layer adopts an architecture that combines read-write separation and clustering. It is used to receive monitoring data transmitted from the application service layer and perform write, storage and query operations to avoid data processing interruption caused by the failure of a single node.

[0014] The system integration layer is a controllable data integration gateway, used to realize data interaction between the main station system and the external IoT platform, and to ensure the security and stability of data transmission.

[0015] Furthermore, the terminal access layer is equipped with a load balancer that supports dynamic expansion. When the number of terminal access requests exceeds a preset processing threshold, the access request distribution capability is improved by adding load balancer nodes, adapting to the expansion needs of the number of terminals from thousands to tens of thousands.

[0016] Furthermore, the application service layer is built on a microservice framework, and each microservice under the microservice framework is deployed in a cluster. The application service layer also includes a service registration and discovery center and a client load balancer. The service registration and discovery center is used to manage the registration information and availability status of microservices, and the client load balancer is used to implement call routing and fault isolation between microservices.

[0017] Furthermore, the microservice framework is the Spring Cloud framework; the service registration and discovery center is the Eureka component, which maintains a list of available microservices by receiving heartbeat information from each instance in the microservice cluster in real time; and the client-side load balancer is the Ribbon component, which uses round-robin or weighted algorithms to distribute the load of microservice calls based on the list of available microservices.

[0018] Furthermore, the microservices of the application service layer include device management service, data receiving service, video processing service, and data query service; wherein, the device management service is used to maintain the device information, online status, and configuration parameters of the transmission line monitoring terminal; the data receiving service is used to connect to the terminal access layer and perform preliminary verification and format unification of monitoring data; the video processing service is used to decode, extract frames, and identify abnormal features of video data uploaded by the terminal; and the data query service is used to respond to historical data and real-time data query requests from users or external systems.

[0019] Furthermore, the data storage layer includes a write cluster and a read cluster; the write cluster adopts a master-slave replication structure and is configured with a health monitoring tool and an automatic failover tool. The health monitoring tool collects the CPU utilization, memory usage and data write latency of the master database in real time. When a master database failure is detected, the automatic failover tool promotes a healthy slave database to the new master database within 10 seconds, ensuring that real-time monitoring data writing is uninterrupted.

[0020] Furthermore, the read cluster of the data storage layer includes multiple read-only slave databases and at least one dedicated data warehouse; the read-only slave databases are used to process regular query requests for single or small amounts of data, and the dedicated data warehouse is used to process complex query requests such as statistical analysis and trend prediction of massive historical data. The dedicated data warehouse is a ClickHouse database or a time-series database.

[0021] Furthermore, the data storage layer also includes a data synchronization module, which is used to achieve near real-time data synchronization between the write cluster and the read cluster. When the data synchronization module is a native database replication tool, data is synchronized through binlog logs between master and slave nodes. When the data synchronization module is a Kafka message queue, the write cluster pushes monitoring data to the Kafka topic in real time, and the read cluster subscribes to the corresponding topic and pulls data to complete the synchronization, with a synchronization delay of no more than 30 seconds.

[0022] Furthermore, the controllable data integration gateway has a built-in identity authentication module, protocol conversion module, traffic control module, and circuit breaker module. The identity authentication module verifies the access rights of external IoT platforms through API keys or tokens. The protocol conversion module is used to adapt to different data interaction protocols. The traffic control module limits the maximum concurrent request volume of a single platform through a token bucket algorithm. The circuit breaker module temporarily blocks request transmission to protect the core services of the main station system when the external request error rate exceeds a preset threshold (such as 50%).

[0023] Furthermore, the protocol conversion module specifically converts the MQTT protocol format monitoring data sent by the external IoT platform into HTTPRESTful protocol format data adapted to the data receiving service of the application service layer. During the conversion process, the fields of the monitoring data are simultaneously standardized (such as unifying the timestamp format to UTC time) and redundant fields (such as duplicate device identification fields) are removed to ensure that the converted data can directly enter the verification process of the data receiving service without secondary format processing.

[0024] The beneficial effects of this invention are as follows:

[0025] 1. The transmission line monitoring master station system for massive terminal access described in this invention deploys a dynamically expandable load balancer at the terminal access layer and combines it with a microservice cluster based on the Spring Cloud framework (including Eureka service registration and Ribbon load balancing) at the application service layer to achieve uniform distribution of access requests from tens of thousands of terminals and automatic isolation of microservice faults. This solves the technical problems of access blockage and overall paralysis caused by a single service failure when the number of terminals in the existing system surges.

[0026] 2. The transmission line monitoring master station system for massive terminal access described in this invention adopts a "read-write separation + one master and multiple slave MGR automatic fault switching architecture" in the data storage layer, and configures ClickHouse / time series database as a dedicated data warehouse and Kafka data synchronization module to achieve high-frequency real-time stable data writing (5000+ TPS), database fault switching within 10 seconds, and rapid query of massive historical data (≤3 seconds), solving the technical problems of database write overload, single point of failure, and high latency of historical data query in existing systems;

[0027] 2. The transmission line monitoring master station system for massive terminal access described in this invention, through the built-in OAuth2.0+JWT authentication, MQTT-HTTPRESTful protocol conversion, token bucket traffic control and circuit breaker modules in the controllable data integration gateway of the system integration layer, realizes secure access to external IoT platforms, seamless protocol adaptation and protection of core services from abnormal traffic surges, and solves the technical problems of existing systems such as lack of security control for cross-platform integration, protocol incompatibility and susceptibility to traffic surges. Attached Figure Description

[0028] The invention will now be further described with reference to the accompanying drawings.

[0029] Figure 1 This is a system block diagram of the present invention. Detailed Implementation

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] This invention provides a transmission line monitoring master station system for massive terminal access, adopting a four-layer collaborative architecture of "terminal access layer - application service layer - data storage layer - system integration layer". Each layer achieves stable operation across the entire link through specific technical designs. The specific scheme is as follows:

[0032] Terminal access layer design:

[0033] Deploy a load balancer that supports dynamic expansion (such as F5BIG-IP) and use the "least number of connections" scheduling algorithm to evenly distribute access requests from tens of thousands of monitoring terminals (image, video surveillance devices, etc.) to the application service layer; preset request processing thresholds, and automatically add load balancer nodes when the number of requests exceeds the threshold, adapting to the expansion needs of the number of terminals from thousands to tens of thousands, and avoiding concentrated blocking of access requests.

[0034] Application service layer design:

[0035] A microservice cluster is built based on the Spring Cloud framework, and automatic scaling is achieved through containerized deployment:

[0036] Basic support components: Configure Eureka service registration and discovery center (maintaining a list of available microservices) and Ribbon client load balancer (using a "round-robin + weight" algorithm to implement microservice call routing) to ensure reliable calls and fault isolation between services;

[0037] The core microservices are broken down into: device management service (terminal ledger maintenance and status monitoring), data receiving service (data verification and standardization), video processing service (video decoding and anomaly detection, GPU acceleration configuration), and data query service (real-time / historical data query response), enabling specialized processing of heterogeneous data.

[0038] Data storage layer design:

[0039] By adopting a "read-write separation + high-availability cluster" architecture, storage bottlenecks can be overcome.

[0040] Write cluster: Build a master-slave architecture with MySQL, configure the MGR (MySQLGroupReplication) automatic failover tool, and promote a healthy slave to master within 10 seconds when the master fails, ensuring high-frequency real-time data writing (supports 5000+ TPS).

[0041] Read cluster: includes MySQL read-only slave database (handles regular queries, supports 10,000+ QPS) and ClickHouse / time series database (handles complex statistical queries);

[0042] Data synchronization: Near real-time synchronization (latency ≤30 seconds) between the write cluster and the read cluster is achieved through Kafka message queues or database native replication tools, ensuring consistency between read and write data.

[0043] System integration layer design:

[0044] The controllable data integration gateway serves as the sole channel for interaction with external IoT platforms, and it incorporates four main functional modules:

[0045] Identity authentication module (OAuth2.0 + JWT verification of access permissions);

[0046] Protocol conversion module (converts external MQTT protocol to internal HTTPRESTful protocol, and synchronizes and standardizes data fields);

[0047] Traffic control module (token bucket algorithm to limit concurrent requests on a single platform, such as 500 QPS);

[0048] The circuit breaker module (temporarily blocks requests when the error rate exceeds 50% to protect core services) achieves a balance between security and efficiency.

[0049] The present invention is further illustrated below through examples.

[0050] Example: Combining Figure 1 As shown, the main station system in this embodiment is planned to connect 30,000 monitoring terminals (covering 2,000 110kV and above transmission lines). The terminal data acquisition frequency is 10 minutes / time, and the data types include images (JPEG format), videos (H.264 format), and environmental parameters (numerical data such as temperature, humidity, and icing thickness). The system achieves stable operation across the entire link through a four-layer architecture of "terminal access layer - application service layer - data storage layer - system integration layer". The hardware selection, software configuration, and function implementation of each layer strictly follow the core design of the technical disclosure document, as detailed below:

[0051] Terminal access layer implementation:

[0052] The core function of the terminal access layer is to receive access requests from 30,000 terminals and distribute them evenly. The specific configuration is as follows:

[0053] Hardware selection: Deploy 2 F5BIG-IP5250 load balancers (master-slave mode), supporting dynamic expansion (maximum expandable to a 4-node cluster); the load balancers are connected to the terminal access gateway (Huawei USG6000E firewall) via 10Gbps fiber optic cable to ensure sufficient access link bandwidth.

[0054] Software configuration: The load balancer uses the "least connections" scheduling algorithm to distribute the TCP connection requests of the terminal to 10 data receiving service instances in the application service layer; a health check mechanism is configured (sending a heartbeat check every 5 seconds), and when a data receiving service instance fails, the request is automatically routed to other healthy instances to avoid access interruption.

[0055] Dynamic scaling trigger: A preset access request threshold (1000 connections per second per node) is set. When the load balancer detects that the request volume exceeds the threshold for 30 consecutive seconds, the scaling process is automatically triggered. A new load balancer node is added through the cloud platform (Huawei FusionSphere) until the request processing capacity matches the terminal access requirements.

[0056] Application service layer implementation:

[0057] The application service layer is built on the Spring Cloud framework to create a microservice cluster. The specific configuration and functions are as follows:

[0058] Basic environment: 15 application servers (Huawei RH2288HV5, configured with 2 Intel Xeon Gold 6248 CPUs, 128GB DDR4 memory, 2TB SSD) are deployed, with CentOS 7.9 operating system and containerized deployment (Docker 20.10.12 + Kubernetes 1.23.5).

[0059] Microservice components and instance deployment:

[0060] Service registration and discovery: Deploy 3 Eureka 1.9.20 instances (master-slave synchronous mode) to manage the registration and status monitoring of all microservices. The Eureka client heartbeat interval is 30 seconds, and the service removal threshold is 90 seconds.

[0061] Client-side load balancing: Each microservice instance integrates Ribbon 2.3.0 and adopts a hybrid algorithm of "round-robin + weight" (core services such as data receiving service are weighted at 2, and non-core services such as device management service are weighted at 1).

[0062] Microservice instance allocation: 10 instances of data receiving service, 3 instances of device management service, 6 instances of video processing service (each instance is configured with an NVIDIA Tesla T4 GPU accelerator card for video decoding), and 4 instances of data query service. Each instance is automatically scaled up and down through Kubernetes.

[0063] Microservice functionality implementation:

[0064] Data receiving service: Receives terminal data distributed by the load balancer, validates the data format (such as device ID format and timestamp range) through JSONSchema, standardizes non-standard fields (such as "device number" to "device_id"), and routes the data to the data storage layer according to the data type (real-time data / historical data);

[0065] Video processing service: Decodes H.264 format video streams uploaded by terminals (using FFmpeg 5.1), extracts keyframes (1 frame every 5 seconds), identifies abnormal features such as tower tilt and conductor icing using OpenCV 4.6.0, and synchronizes the processing results to the data storage layer;

[0066] Equipment management service: Maintains terminal equipment ledger (including equipment model, installation location, IP address), sends configuration commands (such as adjusting the collection frequency) to terminals via MQTT protocol (version 3.1.1), and monitors the online status of terminals in real time (triggers an alarm if offline for more than 5 minutes);

[0067] Data Query Service: Receives query requests from users (such as dispatchers) or external systems, routes them to the read cluster of the data storage layer according to the request type (real-time data / historical statistics), and returns query results in JSON format with a response time of less than 1 second.

[0068] Data storage layer implementation:

[0069] The data storage layer adopts a "read-write separation + high-availability cluster" architecture, with the following specific configuration:

[0070] Write to the cluster (master cluster):

[0071] Hardware: Deploy 3 database servers (Huawei RH5885HV5, configured with 4 Intel Xeon Platinum 8369B CPUs, 256GB DDR4 memory, and 10TB SAS hard drive);

[0072] Software: MySQL 8.0.32 is used, configured with a master-slave replication architecture (the master database is responsible for writing, and the two slave databases synchronize data with the master database in real time, serving as backup master databases);

[0073] High availability tools: Deploy an MGR (MySQL Group Replication) cluster and configure automatic failover (failure detection interval of 5 seconds, failover time ≤ 10 seconds). When the master database CPU utilization exceeds 90% or the network is interrupted, the healthy slave database is automatically promoted to master database to ensure uninterrupted data writing.

[0074] Data writing strategy: Real-time data (such as real-time monitoring values ​​uploaded by the terminal) is directly written to the main database, and the write concurrency supports 5000 TPS (TransactionsPerSecond).

[0075] Reading the cluster (slave cluster + data warehouse):

[0076] Read-only slave database: Deploy 6 MySQL 8.0.32 slave databases to synchronize data with the cluster master database in real time. They are responsible for handling regular queries (such as real-time data queries on a single device) and support a query concurrency of 10,000 QPS (QueriesPerSecond).

[0077] Data warehouse: Deploy 4 ClickHouse 23.3.1.2823 servers (using column storage engine), receive and write historical data (such as monitoring data of the past year) to the cluster synchronization via Kafka 3.4.0, and be responsible for complex queries (such as monthly icing trend analysis of a certain line), with a query response time of ≤3 seconds;

[0078] Data synchronization module: Kafka is used as middleware. Data is written to the cluster master database and binlog is captured by Debezium 1.9.7. Data is pushed to Kafka topics in real time (separate topics "real_time_data" and "history_data" according to data type). Data is read from the cluster slave database and synchronized by subscribing to Kafka topics through Canal 1.1.6. The data synchronization delay is ≤30 seconds.

[0079] System integration layer implementation:

[0080] The system integration layer is a controllable data integration gateway, with the following specific configuration and functions:

[0081] Hardware and basic software: Deploy 2 gateway servers (Huawei RH2288HV5, configured with 2 Intel Xeon Gold 6248 CPUs and 64GB DDR4 memory), use Nginx 1.21.6 as the gateway base, and integrate Spring Cloud Gateway 2.2.9 to implement dynamic routing.

[0082] Core functional module implementation:

[0083] Identity authentication module: It adopts the OAuth2.0+JWT (JSON Web Token) authentication mechanism. External IoT platforms (such as power IoT management platforms) need to apply for an API key before accessing the network. The gateway verifies the validity of the JWT token (valid for 2 hours). Invalid tokens will be rejected directly.

[0084] Protocol Conversion Module: Supports MQTT (external platform) and HTTPRESTful (internal microservice) protocol conversion. It parses MQTT messages sent by the IoT platform (topic " / power / transmission / monitor") into JSON format and converts them into HTTPPOST requests (interface address " / api / v1 / data / receive"). At the same time, it standardizes fields (e.g., unifies the "time" field to UTC time format "yyyy-MM-dd'T'HH:mm:ss'Z') and removes redundant "device_type" fields).

[0085] Traffic control module: Employs the token bucket algorithm, configuring a maximum concurrent request volume of 500 QPS for a single IoT platform. When the threshold is exceeded, a "429TooManyRequests" response is returned to avoid impacting internal microservices.

[0086] Circuit Breaker Module: The circuit breaker mechanism is implemented through Resilience4j1.7.1. When the error rate of external requests (such as timeouts and format errors) exceeds 50% and lasts for 10 seconds, the circuit breaker is triggered (circuit breaker duration is 30 seconds). During this period, the platform's requests are temporarily blocked. After the circuit breaker ends, the requests are probed in a "half-open" state. If the connection is normal, the connection is restored.

[0087] Key business process implementation: The massive data concurrent processing flow is as follows:

[0088] Tens of thousands of monitoring terminals (such as image monitoring devices) send monitoring data (such as JPEG images and numerical parameters) to the load balancer of the terminal access layer via 4G / 5G network at a frequency of 10 minutes / time.

[0089] The load balancer uses the "least connections" algorithm to distribute data requests to 10 data receiving service instances;

[0090] The data receiving service instance performs format verification (such as verifying whether the image resolution is 1920×1080) and standardization processing (such as unifying the device ID to the format "SL-XX-XXXX");

[0091] Standardized data is routed by type: real-time numerical data is directly written to the cluster master database in the data storage layer, while image / video data is first temporarily stored in the distributed file system (MinIO 2023.07.07) and then the file path is written to the master database.

[0092] Data changes are captured by Debezium and pushed to the corresponding Kafka topic when written to the cluster master database. Data is read from the cluster slave database and ClickHouse data warehouse subscribed to the Kafka topic to complete data synchronization.

[0093] After the data receiving service instance completes the data writing / temporary storage, it immediately returns a "200 OK" response to the terminal. The entire processing time is ≤500 milliseconds, avoiding data accumulation on the terminal.

[0094] High availability failover process:

[0095] The monitoring system (Prometheus 2.45.0 + Grafana 10.1.0) collects the running status of each layer of components in real time: every 10 seconds, it collects the CPU utilization, memory usage, and data write latency of writing to the cluster master database, and every 5 seconds, it collects the heartbeat status of microservice instances.

[0096] When the monitoring system detects that the CPU utilization for writing to the cluster master database exceeds 95% for 30 consecutive seconds (determined as a fault), it immediately sends an alarm to the high availability management component (KubernetesOperator);

[0097] The high availability management component triggers automatic failover of the MGR cluster: the master database is marked as "unavailable", and the slave database with the smallest synchronization latency (≤1 second) is selected from the two slave databases and promoted to the new master database;

[0098] After the new master database starts up, the high availability management component automatically updates the data source configuration of the data receiving service instance in the application service layer (switching the database connection address to the new master database IP), with an update time of ≤5 seconds;

[0099] Meanwhile, the monitoring system detected that the heartbeat of a data receiving service instance was interrupted (determined to be an instance crash). Kubernetes automatically triggered the instance to restart, with a restart time of ≤30 seconds. During this period, the load balancer distributed requests to other healthy instances, and no data was lost.

[0100] After the fault is recovered, the original primary database is repaired and automatically joins the write cluster as a secondary database, synchronizing data with the new primary database and restoring it to standby status.

[0101] Secure integration process with external IoT platforms:

[0102] When an external power IoT platform needs to access the main station system, it first applies for an API key from the main station administrator. The administrator then creates a platform account through the device management service (with associated permissions: only able to upload / query terminal data of specified lines) and generates a JWT token.

[0103] When an IoT platform sends a data request, it carries a JWT token in the HTTP request header and requests to pass through the controllable data integration gateway of the system integration layer.

[0104] The gateway's authentication module verifies the validity of the JWT token (such as whether the signature is correct and whether it has expired). If the verification is successful, it proceeds to the protocol conversion stage.

[0105] The protocol conversion module converts MQTT protocol data (such as “{"device_id":"SL-01-0001","time":"2024-10-0112:00:00","value":5.2}”) sent by the IoT platform into HTTPRESTful protocol data, normalizes the timestamp to “2024-10-01T12:00:00Z”, and removes redundant fields (if there is no redundancy, it is directly retained).

[0106] The traffic control module checks the current concurrent request volume of the platform (e.g., if the current volume is 300 QPS, which does not exceed the 500 QPS threshold), allows the request to pass, and routes it to the data receiving service instance;

[0107] If the IoT platform experiences a request timeout rate of 60% (exceeding the 50% threshold) due to network fluctuations, the circuit breaker module will trigger a circuit breaker, rejecting the platform's requests within 30 seconds. After the circuit breaker ends, a test request (single data entry) will be sent. If the test passes, normal access will be restored; if the test fails, the circuit breaker duration will be extended.

[0108] Through the above embodiments, the present invention effectively breaks through the technical bottlenecks of existing systems and meets the high concurrency, high availability, and high security requirements of smart grid transmission line monitoring.

[0109] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A transmission line monitoring master station system for massive terminal access, characterized in that, It includes a terminal access layer, an application service layer, a data storage layer, and a system integration layer that are connected in sequence. The terminal access layer, application service layer, data storage layer, and system integration layer work together to realize concurrent access, heterogeneous data processing, and stable operation of the transmission line monitoring terminal. The terminal access layer is used to receive monitoring data access requests from transmission line monitoring terminals and distribute the access requests to the application service layer. The application service layer is used to receive access requests distributed by the terminal access layer, verify and standardize the monitoring data, and execute business logic. The data storage layer adopts an architecture that combines read-write separation and clustering, and is used to receive monitoring data transmitted from the application service layer and perform write, storage and query operations. The system integration layer is a controllable data integration gateway, used to realize data interaction between the main station system and the external IoT platform, and to ensure the security and stability of data transmission.

2. The transmission line monitoring master station system for massive terminal access as described in claim 1, characterized in that, The terminal access layer is equipped with a load balancer, which supports dynamic expansion. When the number of terminal access requests exceeds a preset processing threshold, the access request distribution capability is improved by adding load balancer nodes.

3. The transmission line monitoring master station system for massive terminal access as described in claim 1, characterized in that, The application service layer is built on a microservice framework, and each microservice under the microservice framework is deployed in a cluster. The application service layer also includes a service registration and discovery center and a client load balancer. The service registration and discovery center is used to manage the registration information and availability status of microservices, and the client load balancer is used to implement call routing and fault isolation between microservices.

4. The transmission line monitoring master station system for massive terminal access according to claim 3, characterized in that, The microservice framework is the Spring Cloud framework; the service registration and discovery center is the Eureka component, which maintains a list of available microservices by receiving heartbeat information from each instance in the microservice cluster in real time; the client-side load balancer is the Ribbon component, which uses round-robin or weighted algorithms to distribute the load of microservice calls based on the list of available microservices.

5. The transmission line monitoring master station system for massive terminal access according to claim 3, characterized in that, The microservices in the application service layer include device management service, data receiving service, video processing service, and data query service. The device management service maintains the device information, online status, and configuration parameters of the transmission line monitoring terminal. The data receiving service interfaces with the terminal access layer and performs preliminary verification and format unification of the monitoring data. The video processing service decodes, extracts frames, and identifies abnormal features from the video data uploaded by the terminal. The data query service responds to historical and real-time data query requests from users or external systems.

6. The transmission line monitoring master station system for massive terminal access according to claim 1, characterized in that, The data storage layer includes a write cluster and a read cluster. The write cluster adopts a master-slave replication structure and is equipped with a health monitoring tool and an automatic failover tool. The health monitoring tool collects the CPU utilization, memory usage and data write latency of the master database in real time. When a master database failure is detected, the automatic failover tool promotes a healthy slave database to the new master database.

7. The transmission line monitoring master station system for massive terminal access according to claim 1, characterized in that, The read cluster of the data storage layer includes multiple read-only slave databases and at least one dedicated data warehouse; the read-only slave databases are used to process regular query requests for single or small amounts of data, and the dedicated data warehouse is used to process complex query requests such as statistical analysis and trend prediction of massive historical data. The dedicated data warehouse is a ClickHouse database or a time-series database.

8. The transmission line monitoring master station system for massive terminal access according to claim 1, characterized in that, The data storage layer also includes a data synchronization module, which is used to achieve near real-time data synchronization between the write cluster and the read cluster. When the data synchronization module is a native database replication tool, data is synchronized through binlog logs between master and slave nodes. When the data synchronization module is a Kafka message queue, the write cluster pushes monitoring data to the Kafka topic in real time, and the read cluster subscribes to the corresponding topic and pulls data to complete the synchronization.

9. The transmission line monitoring master station system for massive terminal access according to claim 1, characterized in that, The controllable data integration gateway has a built-in identity authentication module, protocol conversion module, traffic control module, and circuit breaker module. The identity authentication module verifies the access rights of external IoT platforms through API keys or tokens. The protocol conversion module is used to adapt to different data interaction protocols. The traffic control module limits the maximum concurrent request volume of a single platform through a token bucket algorithm. The circuit breaker module temporarily blocks request transmission to protect the core services of the main station system when the external request error rate exceeds a preset threshold.

10. The transmission line monitoring master station system for massive terminal access according to claim 9, characterized in that, The protocol conversion module specifically converts the MQTT protocol format monitoring data sent by the external IoT platform into HTTPRESTful protocol format data adapted to the data receiving service of the application service layer; during the conversion process, the fields of the monitoring data are standardized and redundant fields are removed simultaneously.

Images