A water conservancy construction management system based on data link system

Abstract

The present invention relates to the technical field of water conservancy system construction, and in particular to a water conservancy construction management system based on a data link system. The system includes a collection system for obtaining original message data from a water conservancy and hydrological platform; a distribution system for distributing the data obtained by the collection system to other modules according to different needs to form an original message data link; a virtual RTU for receiving the original message data of the distribution system, and for transmitting the original message data to the water conservancy and hydrological platform in real time by simulating the TCP transmission mode of the real RTU; a monitoring system for real-time monitoring of the original message data link and the forwarding link channel; the present invention adopts high concurrency, high reliability and multi-platform technology to allow real-time data from different sources to be exchanged at high speed between shared units or different networks.

Description

Technical Field

[0001] The present invention relates to the technical field of water conservancy system construction, and in particular to a water conservancy construction management system based on a data link system. Background Art

[0002] In the construction of water conservancy systems, a large amount of system data is stored and forwarded between different network areas, and the demand for data and application sharing is also increasing, resulting in disordered opening of ports in the DMZ area and other areas, and a large increase in the behavior of isolated applications directly interoperating with each other's data. These behaviors cause great uncertainty to network security and system stability. At the same time, when sharing data between units, the channel construction time is long and the cost is high, which is not conducive to the rapid flow and sharing of real-time data in the water conservancy industry. At the same time, the data provision service flexibility is small and the development volume is large, which makes it difficult to meet the timeliness of various data services. Therefore, it is necessary to strengthen the real-time sharing capability of flash flood disaster real-time data in various units and systems, comprehensively converge the exposure of various receiving platform ports, greatly reduce the probability of attack, and provide convenient deployment and high-speed, secure and reliable real-time data transmission and sharing capabilities for sharing units. Summary of the invention

[0003] In order to solve the above-mentioned problems, the present invention provides a water conservancy construction management system based on a data link system.

[0004] In the first aspect, the present invention provides a water conservancy construction management system based on a data link system, which adopts the following technical solutions:

[0005] A water conservancy construction management system based on a data link system, comprising:

[0006] Collection system, used to obtain original message data from the water conservancy and hydrology platform;

[0007] The distribution system distributes the data obtained by the collection system to other modules according to different needs to form an original message data chain;

[0008] The virtual RTU receives the original message data of the distribution system and transmits the original message data to the water conservancy and hydrology platform in real time by simulating the TCP transmission method of the real RTU;

[0009] Monitoring system, real-time monitoring of the original message data link and forwarding link channel;

[0010] The analysis system analyzes, processes, stores and applies the data of each channel to form an analysis data chain.

[0011] Furthermore, the collection system includes water conservancy data monitoring points. The collection system exchanges real-time monitoring data with all established water conservancy data monitoring points and various units to receive, safely analyze and filter messages of the main and bypass channels and then send them to the distributor.

[0012] Furthermore, the collection system also includes a memory database, and the collection system stores the original message data in the memory database, wherein the data transmission adopts the TCP protocol.

[0013] Furthermore, the data transmission adopts the TCP protocol, including the use of Go language to implement TCTIP based on the advanced cross-chain communication protocol, by identifying and connecting different data receiving ports, establishing a communication channel between the ports, so that different types of blockchains can share data and perform data collection and transfer.

[0014] Furthermore, the distribution system realizes real-time calculation and distribution of data based on the distributed message queue NSQ and the memory database, wherein the message queue is maintained based on the Nsqd node and the message sending and receiving logic is processed, and the information registration and address list query of the Nsqd node are performed through nsqlookupd; the real-time status of the NSQ cluster is displayed and monitored through Nsqadmin.

[0015] Furthermore, the virtual RTU uses a virtual device that simulates a real RTU to exchange data with various standard hydrological and water resources receiving platforms, and virtually forwards message data by virtualizing any number of RTU devices and establishing a data sharing channel with the distribution system.

[0016] Furthermore, the monitoring system uses a Web framework based on golang technology to achieve visual display, and parses the binary message into a standard JSON format and sends it to a parsing module.

[0017] Furthermore, the analysis system includes implementing data analysis on each channel data of the forwarding system in a microservice architecture according to different water conservancy businesses, including deserializing various types of existing data into original messages, sending them to the original message data chain, providing original data sharing services for various units, and providing customized interfaces for data services with special requirements, providing service interfaces for internal networks and external networks.

[0018] Furthermore, the microservice architecture includes Spring Cloud, Nacos for the registration center and the configuration center, REST for the service communication protocol, API gateway for the service entry, Skywalking for the link tracking, and Sentinel for the fuse.

[0019] Furthermore, the original message data link adopts binary messages, the parsing data link synchronously parses the original message data link, stores the parsed message data in standard JSON format, and provides parsed data sharing externally, and the application data link synchronously processes the original message data link to form secondary data results shared externally, including rainfall, soil moisture, water level process lines, real-time segmentation and sharing of short-term and actual GRIB of meteorology, soil conditions, and dynamic real-time segmentation and sharing of land use GIS files.

[0020] In summary, the present invention has the following beneficial technical effects:

[0021] The present invention adopts high concurrency, high reliability and multi-platform technology, allowing real-time data from different sources to be exchanged at high speed between shared units or different networks; constructs an Internet of Things security monitoring interface, provides functions such as message monitoring, analysis, filtering and traffic control, and reduces the real-time data ports of the DMZ area and the private network and government external network to one respectively, comprehensively converging the exposure of various receiving platform ports, and greatly reducing the probability of attack.

[0022] Comprehensively improve data transmission security. The data sharing transmission link uses national secret algorithms for security encryption, and full-link security protection is achieved from monitoring data reception to data transmission and sharing.

[0023] Comprehensively improve database security. All real-time and near real-time data involving water conservancy units are shared through data links, changing the original database sharing mechanism, effectively realizing database isolation, and ensuring database security.

[0024] Open up data channels between networks, units and departments, establish a real-time, comprehensive and accurate data sharing mechanism with the Ministry of Water Resources, river basin organizations and water resources departments of various units in the region, and improve the level of data application and collaborative management.

[0025] When an emergency water conservancy incident occurs, relevant data can be quickly integrated to provide comprehensive data support for emergency command and improve the speed and accuracy of emergency response. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of a water conservancy construction management system based on a data link system according to Example 1 of the present invention.

[0027] Figure 2 It is a flow chart of the distribution system of Example 1 of the present invention. DETAILED DESCRIPTION

[0028] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0029] Example 1

[0030] Reference Figure 1 , a water conservancy construction management system based on a data link system of this embodiment includes:

[0031] Collection system, used to obtain original message data from the water conservancy and hydrology platform;

[0032] The distribution system distributes the data obtained by the collection system to other modules according to different needs to form an original message data chain;

[0033] The virtual RTU receives the original message data of the distribution system and transmits the original message data to the water conservancy and hydrology platform in real time by simulating the TCP transmission method of the real RTU;

[0034] Monitoring system, real-time monitoring of the original message data link and forwarding link channel;

[0035] The analysis system analyzes, processes, stores and applies the data of each channel to form an analysis data chain.

[0036] Specifically,

[0037] This embodiment builds a water conservancy data link system based on five major subsystems, including the collection system, distribution system, virtual RTU, monitoring system, and analysis system (including three dimensions: original message data link, analysis data link, and application data link). The system adopts a full-memory distributed architecture and microservice technology. Without affecting various existing receiving platforms, it performs functions such as trunk and bypass message reception, security analysis and filtering, virtual forwarding, change forwarding, analysis forwarding, data process line forwarding, automatic correction of specific error messages, real-time data monitoring, and real-time data national secret encryption. Except for the analysis system, the remaining subsystems do not use databases, and functional actions and data temporary storage must be completed in dynamic memory. According to the requirements of system compatibility and localization, the collection system, distribution system, virtual RTU, and monitoring system must use the Golang technology route, and the analysis system uses the JAVA microservice technology route.

[0038] The original message data link stores binary messages forwarded by other components and has message verification and error correction functions. Its message format must comply with the national standard SL651 hydrological protocol and the national standard SYZ206 water resources regulations. The message data on the original message data link can be forwarded to the standard receiving platform of the water conservancy industry.

[0039] The parsing data link parses the original message data link synchronously, and stores the parsed message data in standard JSON format to provide parsing data sharing to the outside world. The storage and transmission of the parsing data link are encrypted using the national secret algorithm. The parsing data link can run in all network areas.

[0040] The application data link synchronously processes the original message data link to form secondary data results for external sharing, such as rainfall, soil moisture, water level process lines, real-time segmentation and sharing of short-term and live GRIB of meteorology, dynamic real-time segmentation and sharing of GIS files such as soil conditions, land use, DEM, etc., and provides point-surface interpolation calculation results such as Thiessen polygons, inverse distance weights, and Kriging. The application data link runs in the DMZ area.

[0041] in,

[0042] (I) Regarding the collection system, in terms of flood control, ice prevention and drought relief information collection, 3,795 automatic rainfall monitoring stations, 637 automatic water level monitoring stations, and 395 automatic soil moisture monitoring points have been built; 31 real-time video monitoring points for river sections. In terms of water resources information collection, 1,770 automatic water intake and use monitoring points, 2,521 automatic groundwater level monitoring points, 5 online water quality monitoring points for drinking water sources, 1,232 dam safety monitoring points, and 26 ecological flow monitoring points have been built. In terms of agricultural and animal husbandry water conservancy information collection, the basic data entry of nearly 320,000 agricultural irrigation electromechanical wells has been completed, integrating 11,035 electromechanical well water volume monitoring stations in two leagues and cities, 871 water level monitoring stations, more than 2,000 meteorological stations, 2,350 groundwater level monitoring points, and 81 soil moisture stations. Basic data and online monitoring data.

[0043] The collection system is compatible with the existing standard communication protocols such as "Technical Specifications for Hydrological Automatic Measurement and Reporting System SL61-2003", "Hydrological Monitoring Data Communication Protocol SL651-2014", "Water Resources Monitoring Capability Information Process SZY503-2014", "Water Resources Monitoring Data Transmission Protocol SZY206-2016" and local related transmission extension protocols, and is backward compatible with the existing RTU transmission protocol, and can provide access to public interfaces for future RTUs. Based on all the built water conservancy data monitoring points and the real-time monitoring data exchanged by various units, the main and bypass messages are received, safely analyzed and filtered, and then sent to the distributor.

[0044] The collection system is closely related to the original message data chain, mainly running in the DMZ area, and interacting with the Ministry of Water Resources, leagues, cities, hydrology and other unit platforms for original message data. The data is stored in the memory database, and the data transmission adopts the TCP protocol. The main functions are as follows:

[0045] 1. It can replace various existing water conservancy data receiving platforms to receive real-time data from the Internet and meet the high concurrency requirements of instantaneous reporting of massive monitoring points.

[0046] 2. It has the ability of side reception, and can obtain data messages from the side of the network without affecting the data reception of the original water conservancy data receiving platform.

[0047] 3. It has the ability of two-way forced sharing. When both sides have standard receiving platforms, it can force the side to obtain the original messages of both parties and forward them to each other.

[0048] 4. It has the ability to correct message errors, which can automatically correct some specific errors in the messages from the front-end monitoring points, reducing the urgency of field operation and maintenance.

[0049] 5. Basic data serialization capability, which can serialize data from other sources according to hydrological standard messages, so that it has the ability to flow in the original data chain.

[0050] TCTIP is implemented using the Go language based on an advanced cross-chain communication protocol, which can identify and connect different data receiving ports and establish communication channels between the ports, allowing different types of blockchains to share data, collect data, and transfer data.

[0051] 2. Data Distribution System

[0052] The real-time data of the collection system is corrected and distributed according to different needs. The system not only meets the needs of the current system but also meets the needs of data exchange of each unit and data transmission under special circumstances. It has link data transmission encryption function, supports high concurrency, massive message cache, channel customization function, and forms the original message data link.

[0053] The data distribution system is the data exchange center of the entire system. The data distribution system can be deployed in multiple network areas to achieve forwarding sharing in different areas. The main functions are as follows:

[0054] 1. It has the ability to share multi-user, multi-channel, binary and text data.

[0055] 2. Support asynchronous communication. The sender can continue to perform other operations after sending data, and the receiver receives the message and processes it at the appropriate time.

[0056] 3. The system supports horizontal expansion to handle the transmission and processing of large amounts of data.

[0057] 4. The system can provide security protection for data transmission, such as identity authentication, encryption and secure transmission.

[0058] 5. Other modules can communicate by transmitting data through this system, reducing the coupling between systems and improving the flexibility and maintainability of the system.

[0059] The data distribution system of this embodiment realizes real-time calculation and distribution of data based on distributed message queue NSQ and memory database. Figure 2As shown in the figure, Nsqd, the daemon process, is responsible for maintaining the message queue and handling all logic related to message sending and receiving. Each nsqd node is independent and does not communicate directly with other nsqd instances, avoiding the single point of failure risk brought by centralized management. Its internal message queue adopts the FIFO (first in, first out) strategy and supports storing messages in memory and disk to cope with system pressure.

[0060] Nsqlookupd, the service discovery daemon, acts as a naming service. When the nsqd node starts, it registers its information with nsqlookupd. Consumers can query nsqlookupd to obtain a list of nsqd addresses for a specific topic. The polling mechanism allows new nsqd instances to be quickly identified and integrated into the system, increasing the system's scalability.

[0061] Nsqadmin can display and monitor the real-time status of the NSQ cluster and provide a unified operation interface for message queues. Administrators can use nsqadmin to monitor the status of topics and channels, create or delete topics and channels, and adjust cluster configurations in a timely manner.

[0062] Producer Topic, the data produced by the collection system, analysis system and application computing is sent to the corresponding Topic for message classification and subscription.

[0063] Consumer Channel distributes and subscribes to the data produced on the Topic.

[0064] 3. Virtual RTU

[0065] In order to ensure the stable operation of the original receiving platform (water and rainfall conditions, water resources, dam safety supervision, ecological flow, electromechanical wells), virtual forwarding is realized to meet the current system operation requirements. Any number of RTUs can be virtualized to connect with various standard hydrological and water resources receiving platforms, simulating real RTUs to exchange data with them. It has whitelist and station code mapping functions, and can cooperate with the distributor to form a data sharing channel, so that various types of receiving platforms that have been built have data sharing capabilities.

[0066] RTU (Remote Terminal Unit) is a hardware device deployed on various sensors for transmitting data. It is used to monitor and measure sensors and equipment installed in remote sites. RTU converts the measured state or signal into a data format that can be sent on the communication media. It also converts the data sent from the central computer into commands to achieve functional control of the equipment. Usually RTU can only send four channels at most at the same time. For multi-party data sharing requirements, hardware RTU is unable to meet the requirements. Therefore, this system implements virtual RTU to enable it to forward data to any number of targets.

[0067] The virtual RTU receives the original message data from the data distribution system, simulates the TCP transmission mode of the real RTU, and transmits the original message data to each standard water conservancy receiving platform in real time. The main functions are as follows:

[0068] 1. Provide the ability to forward to an unlimited number of targets.

[0069] 2. With whitelist function, only the data packets in the whitelist can be forwarded.

[0070] 3. It has the function of station code conversion and mapping, which can change the station code in the original source binary message and then re-encode and forward it, reducing the urgency of field operation and maintenance.

[0071] 4. Ability to encode national standard protocols for hydrology and water resources.

[0072] 5. Support high concurrent processing.

[0073] 6. Transparent access to all existing water conservancy receiving platforms.

[0074] This embodiment simulates a virtual device that exchanges data between a real RTU and various standard hydrological and water resources receiving platforms. Through the virtual RTU, the operation stability of existing receiving platforms (such as water and rainfall conditions, water resources, dam safety monitoring, ecological flow, electromechanical wells, etc.) can be guaranteed and the current system operation requirements can be met.

[0075] The functions of virtual RTU include simulating the actual data transmission process by virtualizing any number of RTU devices, connecting with various receiving platforms, and realizing data sharing and exchange. At the same time, virtual RTU also has whitelist and station code mapping functions, and can work with the data distributor to form a data sharing channel, so that data can be interoperable and shared between various receiving platforms.

[0076] The application of virtual RTU can not only simplify the data transmission process between systems and improve the efficiency of data exchange, but also help various built receiving platforms to have the ability to share data, further strengthening the integration and coordination of the system. The introduction of virtual RTU technology provides a more flexible and efficient data processing and transmission method for system operation in the fields of hydrology, water resources, etc., which helps to improve the overall operation efficiency and management level of the system.

[0077] This embodiment implements communication with devices and simulated devices based on golang technology. After acquiring data from the distribution system, the protocol is converted according to the protocols of different devices in the previous stage, and the device communication mode is simulated for data distribution.

[0078] 4. Monitoring system

[0079] Use Golang technology to achieve real-time monitoring of messages and forwarding link channels, and monitor the data resources, analysis, and data forwarding volume of the entire data link system in real time. You can view the original message data and the data of each channel of the forwarding link, including message query, message data viewing, channel link data, etc.

[0080] The monitoring system obtains the original message data through the data distribution system, processes the data in real time through the analysis module and the secondary result calculation module, and provides real-time uplink data for the analysis data link and the application data link. The main functions are as follows:

[0081] 1. Provide standard hydrological protocol and water resources protocol parsing functions.

[0082] 2. Provide field RTU commissioning message test function and Web interface.

[0083] 3. Provide field RTU commissioning query function and Web interface.

[0084] 4. Provide data volume counting function and web interface for all data sources.

[0085] 5. Provide daily activity counting function and web interface for monitoring points.

[0086] 6. Support parsing binary messages into standard JSON format and sending them to the parsing data link capability.

[0087] 7. Support real-time calculation of secondary results with real-time data in memory and provide application data link capability.

[0088] 8. Support real-time cutting and sharing of various static GIS files, such as GRB2, ASC, SHP, TIF, etc.

[0089] This embodiment uses the Web framework based on golang technology to achieve visual display and monitor the data distribution system in real time.

[0090] 5. Analysis system

[0091] Use the microservice technology route to perform unified service registration, unified service configuration, unified service monitoring, microservice monitoring, and mutual calls between microservices. Implement data analysis, processing, storage and application of each channel data of the forwarding system in a microservice manner according to different water conservancy businesses. Provide different customized services, shared services, and exchange services to form analysis data chains and application data chains.

[0092] The main functions are as follows:

[0093] 1. Data parsing service, receiving data distribution system data service, compatible with the parsing of existing protocols and the expansion of future protocol parsing.

[0094] 2. Deserialization data service: deserialize various types of existing data into original messages, send them to the original message data chain, and provide original data sharing services for various units and systems.

[0095] 3. Data interface service: provide customized interfaces for data services with special requirements and provide service interfaces for internal and external networks.

[0096] 4. Application data computing services, which provide application data services through real-time calculations using a large amount of analytical data and algorithm models, such as rainfall process lines, water level process lines, flow process lines and other water conservancy applications.

[0097] 5. Task scheduling service: dispatching scheduled tasks for parsing data link and applying data link data as needed.

[0098] 6. Authentication interface service: perform permission authentication when providing services for sensitive data.

[0099] The parsing system of this embodiment adopts the microservice architecture model for overall design, and uses advanced technical architecture to reduce the complexity of coupling between business components and functional modules, so as to fully dispatch and utilize cloud computing resource capabilities, flexibly use data resource pool data content, and dynamically use support services on demand to ensure the safe, stable, efficient and smooth operation of the platform. The microservice framework adopts Spring Cloud, the registration center and configuration center adopt Nacos, the service communication protocol adopts REST, the service entry adopts API gateway, the link tracking adopts Skywalking, and the fuse adopts Sentinel.

[0100] in,

[0101] 1) The service registration center uses NACOS to manage microservices, and the service configuration center is uniformly registered to Nacos for unified management.

[0102] 2) Service monitoring: monitor service indicators and track links for all microservices.

[0103] 3) Microservice calls: Feign is used to call services from each other through the service bus.

[0104] 4) Microservice cluster: When the data processing business reaches the peak of service resources, microservice clusters can be deployed to collaborate to complete the business.

[0105] 5) Microservices integrate public services such as relational databases, non-relational databases, buffered databases, message queues NSQ, and task scheduling.

[0106] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A water conservancy construction management system based on a data link system, characterized in that: include: Collection system, used to obtain original message data from the water conservancy and hydrology platform; The distribution system distributes the data obtained by the collection system to other systems according to different needs, and adopts the national secret encryption system to form the original message data chain; The virtual remote terminal unit RTU receives the original message data of the distribution system and transmits the original message data to the water conservancy and hydrology platform in real time by simulating the TCP / IP transmission mode of the real RTU; Monitoring system, real-time monitoring of the original message data link and forwarding link channel; The parsing system performs data parsing, data processing, warehousing and application on the data of each channel to form a parsing data chain. The parsing system uses the microservice technology route to perform unified service registration, unified service configuration, unified service monitoring, microservice monitoring and mutual calls between microservices. It implements data parsing, processing, warehousing and application of each channel data of the distribution system in a microservice manner according to different water conservancy businesses, provides different customized services, shared services, and exchange services, and forms a parsing data chain and an application data chain. The parsing system adopts the microservice architecture model for overall design. The microservice framework adopts Spring Cloud, the registration center and configuration center adopt Nacos, the service communication protocol adopts REST, the service entry adopts API Gateway, the link tracking adopts Skywalking, and the fuse adopts Sentinel. The collection system includes water conservancy data monitoring points. The collection system exchanges and monitors real-time data through all the built water conservancy data monitoring points, receives messages from trunk and bypass channels, performs security analysis and filtering, and then sends the messages to the distribution system. The collection system also includes a memory database, and the collection system stores the original message data in the memory database, wherein the data transmission adopts the TCP protocol; The data transmission adopts the TCP protocol, including using the Go language to implement TCT / IP based on an advanced cross-chain communication protocol, by identifying and connecting different data receiving ports, establishing a communication channel between the ports, so that different types of blockchains can share data, collect and transfer data; wherein, using the Go language to implement TCT / IP based on an advanced cross-chain communication protocol, it can identify and connect different data receiving ports, and establish a communication channel between the ports; The distribution system realizes real-time calculation and distribution of data based on the distributed message queue NSQ and the memory database of the collection system. The message queue is maintained based on the Nsqd node and the message sending and receiving logic is processed. The information registration and address list query of the Nsqd node are performed through nsqlookupd; the real-time status of the NSQ cluster is displayed and monitored through Nsqadmin; The virtual RTU uses a virtual device that simulates a real RTU to exchange data with the water conservancy and hydrology platform, and virtualizes any number of RTU devices and establishes a data sharing channel with the distribution system to perform virtual forwarding of message data; The monitoring system uses a Web framework based on golang technology to achieve visual display, and parses binary messages into standard JSON format and sends them to the parsing system; The analysis system includes implementing data analysis of each channel data of the distribution system in a microservice architecture according to different water conservancy businesses, including deserializing various types of existing data into original messages, sending them to the original message data chain, providing original data sharing services, and providing customized interfaces for data services with special requirements and providing service interfaces for internal networks and external networks; The original message data link adopts binary message, the parsing data link synchronously parses the original message data link, stores the parsed message data in standard JSON format, and provides parsed data sharing externally, and the application data link synchronously processes the original message data link to form secondary data results shared externally, including rainfall, soil moisture, water level process lines, short-term meteorological and real-time segmentation and sharing of GRIB, and dynamic real-time segmentation and sharing of soil condition GIS files.

Images