Dam safety monitoring data management system

Abstract

The utility model provides a dam safety monitoring data management system, and relates to the field of data management. The system comprises a reservoir dam on-site terminal, a big data storage device and a big data application server which are connected in sequence, wherein the reservoir dam field terminal is deployed in a reservoir dam field and is used for collecting field safety monitoring data and uploading the field safety monitoring data to the big data storage device; the big data storage device is used for receiving and storing safety monitoring data; and the big data application server is in butt joint with the big data storage device through a data interface, and is used for reading the safety monitoring data stored in the big data storage device for a data mining program to perform retrieval query and analysis mining. According to the scheme, the management and utilization efficiency of dam safety monitoring data can be improved by optimizing the topological structure of the safety monitoring data management system.

Description

Technical Field

[0001] The present application relates to the field of data management, and in particular, to a dam safety monitoring data management system. Background Art

[0002] The country attaches great importance to the management of reservoirs and dams. The degree of automation of dam safety monitoring is increasing year by year, and the amount of safety monitoring data collected is growing exponentially. At the same time, with the increase in the number of years of project operation, document data such as safety monitoring observations and data compilation reports, inspection reports, etc. are also accumulating, forming a huge data resource. Especially for river basin development companies and water conservancy and hydropower group companies, which have many reservoirs and dams under their jurisdiction, the accumulated safety monitoring data and information are massive, with big data characteristics such as large data capacity, many types of data, fast data increase speed, and low data value density.

[0003] At present, the safety monitoring data of various reservoirs and dams are usually managed by local storage or stored separately as structured data. The data access and utilization efficiency is low, and it is difficult to achieve centralized management of the safety monitoring data of multiple reservoirs and dams. This also hinders the development of data retrieval, query, analysis and mining applications. Utility Model Content

[0004] In order to solve the above problems, the present application provides a dam safety monitoring data management system to improve the management and utilization efficiency of dam safety monitoring data.

[0005] This application is implemented as follows:

[0006] In a first aspect, the present application provides a dam safety monitoring data management system, which includes a reservoir dam field terminal, a big data storage device and a big data application server connected in sequence;

[0007] The reservoir dam on-site terminal is deployed at the reservoir dam site to collect on-site safety monitoring data and upload it to a big data storage device; the big data storage device is used to receive and store safety monitoring data; the big data application server is connected to the big data storage device through a data interface to read the safety monitoring data stored in the big data storage device for retrieval, query and analysis and mining by a data mining program.

[0008] Further, based on the above scheme, the big data storage device includes a relational database server and an HDFS distributed file system interconnected through a switch, the input end of the switch is connected to the reservoir dam field terminal through a first big data gateway, and the output end of the switch is connected to the big data application server through a second big data gateway.

[0009] Furthermore, based on the above scheme, the security monitoring data includes structured data and unstructured data; the relational database server is used to store the structured data in the security monitoring data, and the HDFS distributed file system is used to store the unstructured data in the security monitoring data.

[0010] Furthermore, based on the above scheme, the structured data includes safety monitoring data, hydrological and meteorological monitoring data and monitoring compilation results data; the unstructured document materials include engineering design data, compilation analysis reports, inspection reports and special analysis materials.

[0011] Furthermore, based on the above solution, the HDFS distributed file system includes at least one management node server and multiple data node servers; the network topology structure between the management node server and the data node servers is a star network connection with a switch as the center.

[0012] Furthermore, based on the above scheme, the management node server is responsible for processing file access requests of the big data application server, and dynamically allocating data storage and access tasks according to the load conditions of the data node, thereby achieving load balancing; the data node is responsible for the storage and reading of unstructured data in the security monitoring data.

[0013] Furthermore, based on the above solution, the reservoir dam site terminal is deployed in a distributed manner at different reservoir dam sites, and is connected to the big data storage device through the first big data gateway.

[0014] Compared with the prior art, the present invention has at least the following advantages or beneficial effects:

[0015] The safety monitoring data collected by the on-site terminal of the reservoir dam is received and stored through the big data storage device, and then the big data application server is connected to the big data storage device through the data interface to read the safety monitoring data stored in the big data storage device for retrieval, query and analysis and mining by the data mining program. Thus, it will be possible to centrally manage multiple reservoir dams or multiple safety monitoring data of the same reservoir dam, facilitate unified data retrieval, query and analysis and mining, and thus improve the management and utilization efficiency of dam safety monitoring data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is a structural diagram of an embodiment of a dam safety monitoring data management system of the present application;

[0018] Figure 2 A schematic diagram of the structure of the HDFS distributed file system according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, rather than all of the embodiments. The components of the embodiment of the present application described and shown in the accompanying drawings here can be arranged and designed in various configurations. In this article, relational terms such as first and second, etc. are only used to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations.

[0020] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0021] Example:

[0022] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0023] The embodiment of the present application provides a dam safety monitoring data management system, which can solve the problems of low efficiency, difficulty in centralized management and disadvantageous analysis and mining in the traditional reservoir dam data management mode, and realize the efficient management and utilization of the dam safety monitoring data.

[0024] Please refer to Figure 1 , a dam safety monitoring data management system includes a reservoir dam field terminal, a big data storage device and a big data application server connected in sequence;

[0025] The reservoir dam on-site terminal is deployed at the reservoir dam site to collect on-site safety monitoring data and upload it to a big data storage device; the big data storage device is used to receive and store safety monitoring data; the big data application server is connected to the big data storage device through a data interface to read the safety monitoring data stored in the big data storage device for retrieval, query and analysis and mining by a data mining program (mainly big data application software).

[0026] In the above embodiment, multiple reservoir dam site terminals are used to be deployed at different locations of the reservoir dam site, or deployed at different reservoir dam sites, to collect corresponding safety monitoring data, and then upload the collected safety monitoring data to the big data storage device for storage. Finally, the safety monitoring data stored in the big data storage device is retrieved and analyzed and mined by the big data application server, so that the safety monitoring data can be centrally managed and analyzed and mined. Among them, in one implementation of the present application, the reservoir dam site terminals are deployed in a distributed manner at different reservoir dam sites, and can simultaneously access the big data storage device through the first big data gateway and connect to the big data storage device, and input safety monitoring data concurrently.

[0027] It should be noted that the reservoir dam site terminal mainly includes the following types of equipment:

[0028] Data collection terminal MCU: This type of terminal has real-time data collection function and can be unattended 24 hours a day. They can not only transmit data through 4G, but also be installed and deployed in mountain reservoirs with weak mobile network coverage using Lora+ solar power supply, thereby achieving flexible and convenient data transmission. In addition, these terminals can be updated and upgraded remotely, and can be connected to multi-channel safety monitoring sensors and remote relay control, so that one terminal can collect a large amount of safety monitoring data. These functions make the perception of reservoir safety status more thorough and operation and maintenance more intelligent.

[0029] Water and rainfall monitoring equipment: such as rainfall monitoring stations, reservoir water level monitoring stations, flow monitoring stations, etc. These devices are used to dynamically monitor the reservoir operation process and the flood control situation it faces, and also provide environmental information for monitoring data analysis.

[0030] Safety monitoring instruments and equipment: mainly include monitoring instruments such as deformation monitoring, seepage monitoring, stress strain and temperature monitoring, including displacement meters, crack meters, piezometers, water weir meters, strain gauges, steel bar meters, anchor dynamometers, etc., which are used to comprehensively monitor the operating performance of hydraulic structures.

[0031] That is to say, the above-mentioned reservoir dam site terminal is mainly a device deployed according to the actual situation of the reservoir dam site. Different devices will be used to collect different types of safety monitoring data, but these safety monitoring data will be uploaded to the big data storage device for unified storage, and then the big data application server can query, analyze and mine different safety monitoring data. Among them, big data storage devices can be divided into direct regional storage and network-based storage. Direct regional storage (such as hard disk drive HDD, solid state drive SSD, etc.) is usually located in the local area and directly connected to the computer that accesses it. Network-based storage (such as network-attached storage NAS and storage area network SAN) supports multiple computers to access it through the network, which is more conducive to data sharing and collaboration. The specific selection of big data storage devices needs to select different hard disk devices and storage methods according to different actual application requirements to achieve efficient, safe and reliable data storage. In addition, big data storage devices need to support the processing and query of massive data while ensuring the security and reliability of data. Therefore, big data storage devices are usually equipped with high-performance processors and large memories to quickly respond to complex computing and analysis tasks. Big data application architectures such as MapReduce, Hadoop and Spark can be used to process and analyze data. For example, MapReduce is used for parallel computing of large-scale data sets, Hadoop is used to develop data-intensive distributed applications, and Spark improves the execution and scheduling rate of distributed tasks through an in-memory computing model.

[0032] Please refer to Figure 1 In some embodiments of the present application, the big data storage device includes a relational database server and an HDFS distributed file system interconnected by a switch, the input end of the switch is connected to the reservoir dam field terminal through a first big data gateway, and the output end of the switch is connected to the big data application server through a second big data gateway.

[0033] Among them, relational database servers are suitable for storing structured data, such as dam basic parameters, equipment status, monitoring data, etc. It provides powerful data query and management functions, allowing managers to quickly retrieve and update data. The HDFS distributed file system is better at processing unstructured and semi-structured data, such as on-site inspection data. HDFS uses its unique distributed storage mechanism to store data on multiple nodes, which not only improves data reliability, but also greatly improves data processing speed through parallel processing.

[0034] As a key device connecting these components, the switch is responsible for data forwarding and routing. It transmits the data collected by the reservoir dam site terminal to the big data storage device through a high-speed network connection, and also forwards the processed data to the big data application server. The role of the first big data gateway is to achieve secure data transmission and protocol conversion. It receives data from the reservoir dam site terminal, performs necessary security checks and protocol conversions, and then sends it to the switch. In this way, data interaction between different devices and systems can be interconnected. Finally, the big data application server obtains data from the switch and conducts further analysis and processing to provide decision support for the safe operation of the dam.

[0035] In other words, the collaborative work of the entire system realizes the complete process from data collection, storage to application, providing strong technical support for the dam's safety status monitoring and abnormal warning. At the same time, this architecture also ensures the high availability, high scalability and high security of the system, which can meet the long-term operation requirements of dam safety monitoring.

[0036] In other words, in the above embodiment, the relational database server is a database based on the relational database model, which uses a table structure to organize and store data. It usually has a fixed data structure and adopts a centralized data storage method, and the data is stored on the disk of a single computer. The HDFS (Hadoop Distributed File System) distributed file system is a distributed storage system for storing and processing large-scale data sets. It divides the data into multiple data blocks and stores and replicates them on multiple nodes in the cluster to ensure the reliability and fault tolerance of the data. In general, the relational database server focuses on the structured storage and query of data, and is suitable for scenarios that require fixed data structures and efficient queries; while the HDFS distributed file system focuses on the storage and processing of large-scale data, and is suitable for processing ultra-large-scale data sets and large-scale computing scenarios. The safety monitoring data collected through the reservoir dam site terminal includes structured data and unstructured data. Therefore, according to the different safety monitoring data collected, the relational database server is used to store the structured data in the safety monitoring data, and the HDFS distributed file system is used to store the unstructured data in the safety monitoring data, so as to improve the storage and processing efficiency of the safety monitoring data.

[0037] Among them, structured data refers to data with a fixed format and predefined attributes. These data generally have clear meanings and units and can be directly used for analysis and comparison to assess the safety status of the dam. Unstructured data refers to data without a fixed format or predefined attributes, usually in the form of text, images, videos or audio.

[0038] Specifically, the structured data include safety monitoring data, hydrological and meteorological monitoring data, and monitoring compilation results data; the unstructured document data include engineering design data, compilation analysis reports, inspection reports, and special analysis data. Furthermore, in dam safety monitoring, structured data may include dam geometric parameters (such as dam height, dam width), material properties, equipment information, safety monitoring data, rain monitoring data, etc. These data generally have clear meanings and units, can be directly used for analysis and comparison, and can be used to assess the safety status of the dam. Unstructured data may include on-site video records, text descriptions or voice reports of inspectors, etc. Although these data are difficult to directly quantify and analyze, they provide intuitive information on the operating status of the dam and are also crucial for identifying potential risks and problems.

[0039] That is to say, by combining structured data and unstructured data, the dam safety monitoring data management system can assess the safety status of the dam more comprehensively and accurately. Structured data provides basic parameters and indicators to quantify the performance and status of the dam; unstructured data provides more contextual information and details to help discover potential problems and hidden dangers.

[0040] Please refer to Figure 2 In some embodiments of the present application, the HDFS distributed file system includes at least one management node server ( Figure 2 The exemplary embodiment includes 2 management node servers), and multiple data node servers ( Figure 2 The exemplary embodiment includes 3 data node servers); the network topology between the management node server and the data node server is a star network connection with a switch as the center.

[0041] In the above embodiment, the management node server is mainly responsible for processing the file access request of the big data application server, and dynamically allocates data storage and access tasks according to the load of the data node, so as to achieve load balancing; the data node is mainly responsible for the storage and reading of unstructured data in the security monitoring data. Specifically, the HDFS distributed file system includes a management node server and a data node server. After the security monitoring data is input into the HDFS distributed file system, it will be divided into multiple data blocks. After being copied to multiple copies, these data blocks will be dispersed and stored on multiple data node servers to achieve distributed storage of security monitoring data. If a data node server fails, the HDFS distributed file system can recover the data from the copies on other nodes, thereby improving the security of security monitoring data storage.

[0042] That is to say, in the architecture of the above embodiment, the network topology between the management node server and the data node server is a star network connection with a switch as the center. This structure enables each node server to communicate directly with the management node server, improving the efficiency and reliability of data transmission. At the same time, the switch, as a central device, is responsible for data forwarding and routing to ensure smooth transmission of data between nodes. This HDFS distributed file system architecture not only provides high fault tolerance and high reliability, but also can handle scenarios with large data volumes and high throughput. It is suitable for application scenarios such as dam safety monitoring that require real-time collection, storage and analysis of large amounts of data, and can provide strong technical support for dam safety management.

[0043] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A dam safety monitoring data management system, characterized in that: It includes a reservoir dam field terminal, a big data storage device and a big data application server connected in sequence; The reservoir dam on-site terminal is deployed at the reservoir dam site to collect on-site safety monitoring data and upload it to a big data storage device; the big data storage device is used to receive and store safety monitoring data; the big data application server is connected to the big data storage device through a data interface to read the safety monitoring data stored in the big data storage device for retrieval, query and analysis and mining by a data mining program.

2. A dam safety monitoring data management system according to claim 1, characterized in that: The big data storage device includes a relational database server and an HDFS distributed file system interconnected by a switch, the input end of the switch is connected to the reservoir dam field terminal through a first big data gateway, and the output end of the switch is connected to the big data application server through a second big data gateway.

3. A dam safety monitoring data management system according to claim 2, characterized in that: The security monitoring data includes structured data and unstructured data; the relational database server is used to store the structured data in the security monitoring data, and the HDFS distributed file system is used to store the unstructured data in the security monitoring data.

4. A dam safety monitoring data management system according to claim 3, characterized in that: The structured data include safety monitoring data, hydrological and meteorological monitoring data and monitoring compilation results data; The unstructured document materials include engineering design materials, compilation analysis reports, inspection reports and special analysis materials.

5. A dam safety monitoring data management system according to claim 2, characterized in that: The HDFS distributed file system includes at least one management node server and multiple data node servers; the network topology structure between the management node server and the data node servers is a star network connection with a switch as the center.

6. A dam safety monitoring data management system according to claim 5, characterized in that: The management node server is responsible for processing file access requests of the big data application server and dynamically allocating data storage and access tasks according to the load of the data node, thereby achieving load balancing; the data node is responsible for storing and reading unstructured data in the security monitoring data.

7. A dam safety monitoring data management system according to claim 1, characterized in that: The reservoir dam site terminals are deployed in a distributed manner at different reservoir dam sites, and are connected to the big data storage device through the first big data gateway.

Images