Image recognition-based reservoir dam safety intelligent monitoring system

Abstract

This invention relates to the field of image data processing, specifically to an intelligent monitoring system for reservoir dam safety based on image recognition. The system includes: a data acquisition module for continuously acquiring video images and sensor time-series data at fixed intervals; a model training module for training a specialized AI algorithm model; a real-time inference module for automatic target detection; a data structuring module for outputting standardized recognition results; a data fusion and verification module for performing source fusion and cross-verification on data; a risk assessment module for analysis and outputting accurate risk assessment results; an early warning triggering module for triggering corresponding level early warnings based on the assessment results; a message push module for pushing messages to management personnel through multiple channels; and an event handling module for synchronously recording the entire process of early warning events, tracking the handling progress, and forming a closed-loop business process.

Description

Technical Field

[0001] This invention belongs to the field of image data processing, specifically relating to an intelligent monitoring system for reservoir dam safety based on image recognition. Background Technology

[0002] As a core water conservancy hub in the region, Yugang Reservoir in Kedong County undertakes key functions related to people's livelihood, such as flood control and water storage, agricultural irrigation, and urban and rural water supply. The safe and stable operation of its dam is directly related to the safety of people's lives and property downstream and the economic and social development of the region.

[0003] Against the backdrop of the national policy of vigorously promoting the construction of smart water conservancy and digital twin river basins, the Ministry of Water Resources' "Technical Guidelines for the Construction of Digital Twin Water Conservancy Projects (Trial)" clearly stipulates that water conservancy projects should take "data, algorithms, and computing power" as the core to build a digital and intelligent safety monitoring and decision support system. The "Regulations on the Safety Management of Reservoirs and Dams" and the "Technical Specifications for Safety Monitoring of Water Conservancy Projects" also put forward mandatory requirements for the safety monitoring, risk warning and closed-loop management of reservoirs and dams throughout their entire life cycle.

[0004] The traditional safety management model for the Yugang Reservoir dam faces multiple industry pain points and technical bottlenecks: First, monitoring methods rely on manual inspections and single-point sensor threshold alarms, resulting in limited coverage, discontinuous data collection, delayed detection of abnormal events, and high rates of missed and false detections, making it impossible to achieve 24 / 7 intelligent control; second, the front-end sensing data is multi-source and heterogeneous, with unstructured video surveillance data, time-series structured sensor data, and BIM / GIS spatial geographic data stored separately, forming data silos that prevent fusion and linkage analysis, thus hindering the release of the value of massive monitoring data; third, data processing and analysis capabilities are weak. Traditional methods can only display data and issue over-limit alarms, but cannot accurately predict water and rainfall trends or dam structural behavior, resulting in a severe lack of early warning capabilities. Fourth, algorithm models are managed and deployed independently, and models such as AI recognition, numerical simulation, and big data analysis lack a unified registration, scheduling, evaluation, and iteration system, making it impossible to achieve multi-model collaborative computing and resulting in extremely poor scalability and reusability. Fifth, business processes have not formed a digital closed loop, with breakpoints in the entire process from data collection, intelligent analysis, early warning push, and disposal tracking, making it impossible to meet the actual needs of reservoir safe operation in terms of emergency response efficiency and decision-making scientificity. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent monitoring system for reservoir dam safety based on image recognition, so as to solve the problems mentioned in the background art.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: The image recognition-based intelligent monitoring system for reservoir dam safety includes: The data acquisition module is used to continuously acquire video images of the dam body, water surface, gatehouse, and perimeter area at fixed intervals, and simultaneously acquire time-series data from water level, seepage, and displacement sensors. The model training module is used to train specialized AI algorithm models; The real-time inference module is used to perform real-time inference on the acquired video stream based on a pre-trained specialized AI algorithm model, and to automatically detect open smoke and flames, water level readings, gate status, violations, and floating objects on the water surface. The data structuring module is used to simultaneously complete the structuring transformation of unstructured image data and output standardized recognition results; The data fusion and verification module is used to perform source fusion and cross-verification of image recognition results with sensor monitoring data and engineering space data; The risk assessment module is used to perform quantitative analysis of water and rainfall trends and dam safety status through preset algorithms, and output accurate risk assessment results. The early warning triggering module is used to trigger corresponding level early warnings based on the analysis results; the message push module is used to push messages to management personnel through multiple channels. The event handling module is used to synchronously record the entire process of early warning events, track the handling progress, and form a business closed loop; the model management module is used to perform unified registration, scheduling, effect evaluation, and iterative optimization of all algorithm models.

[0007] Furthermore, the method of continuously collecting video images of the dam, water surface, gatehouse, and perimeter area using high-definition cameras throughout the reservoir at fixed intervals, and simultaneously collecting time-series data from water level, seepage, and displacement sensors, includes: determining the deployment rules of high-definition network cameras at key control points throughout the reservoir based on the reservoir safety monitoring business scenario and algorithm model input requirements; continuously collecting video images of the control area and capturing key frames; simultaneously accessing drone patrol images; completing standardized processing of video images and recording of traceability information; and simultaneously collecting time-series data from various safety monitoring sensors of the reservoir according to a time benchmark aligned with video acquisition, completing initial data screening, standardized processing, and invalid data removal.

[0008] Furthermore, the training of the specialized AI algorithm model includes: determining the construction rules for the specialized algorithm dataset based on the business scenario of reservoir dam safety monitoring and the AI ​​algorithm recognition target; completing the collection, classification, labeling, and balanced partitioning of multi-scenario sample data; performing standardized preprocessing and data augmentation on labeled samples, removing invalid samples, and ensuring that the samples are compatible with the model input standard; matching and optimizing the appropriate deep learning network architecture for each recognition task, and completing iterative training and incremental optimization of the model; conducting multi-dimensional performance testing and on-site testing on the trained model, and after passing the tests, completing the model format solidification, specification formulation, and registration.

[0009] Furthermore, based on pre-trained specialized AI algorithm models, real-time inference is performed on the collected video streams to automatically detect open smoke and flames, water level readings, gate status, violations, and floating objects on the water surface. This includes: matching specialized algorithm models for corresponding scenarios to the video streams of each monitoring point in the reservoir, achieving full coverage of business scenarios in key control areas; performing frame-by-frame standardized preprocessing on the real-time collected video streams to ensure that the input conforms to model specifications; calling corresponding specialized models for different scenarios to complete real-time inference; identifying and marking various abnormal events; simultaneously completing the structured transformation of inference results; recording traceable information across the entire chain; allocating computing resources according to business priorities; and pushing abnormal events to the platform's early warning panel.

[0010] Furthermore, the simultaneous completion of the structured transformation of unstructured image data and the output of standardized recognition results includes: formulating standardized transformation rules for various unstructured image reasoning results based on the structured data standard for water conservancy project safety monitoring, determining unified structured data fields, performing format regularization and numerical standardization processing on the original reasoning results output by the special AI algorithm model, eliminating redundant and invalid data, conducting compliance verification on the transformed structured data to ensure that the data traceability relationship is uniquely bound, classifying and collecting the verified data into the database and establishing an associated index, generating a unique traceability code, and specifically marking abnormal event data in the standardized recognition results and simultaneously pushing them to the platform's early warning panel.

[0011] Furthermore, the process of fusing and cross-validating image recognition results with sensor monitoring data and engineering spatial data includes: relying on the platform's unified spatiotemporal reference system to achieve precise alignment of the time dimension of image recognition results, sensor monitoring data, and engineering spatial data; based on the full-domain spatial coordinate system constructed by the engineering BIM model and the WebGIS platform, to achieve one-to-one correspondence of the spatial dimensions of the three types of data; to complete the fusing and cross-validation of the three types of data according to the business scenario of reservoir dam safety monitoring; to formulate cross-validation rules for different business scenarios; to conduct cross-validation on the fused dataset; to trigger source tracing and verification and classify and handle abnormal data; and to update the compliant fused dataset that has passed the verification into the database and associate it with the engineering spatial model.

[0012] Furthermore, the quantitative analysis of water and rainfall trends and dam safety status through preset algorithms, and the output of accurate risk assessment results, include: based on a unified spatiotemporal benchmark, using compliant multi-source datasets that have undergone homogeneous fusion and cross-validation as the core data, constructing an adaptive algorithm system to complete multi-dimensional trend prediction of water and rainfall and flood discharge capacity matching analysis, building a multi-dimensional quantitative analysis index system for dam safety status, completing the calculation of multi-dimensional safety characteristics of the dam body and the overall safety level assessment, linking the two types of analysis results to carry out comprehensive risk assessment and source tracing analysis, with full-process traceability, outputting standardized risk assessment results and pushing them to the corresponding business modules.

[0013] Furthermore, the triggering of corresponding level early warnings based on the assessment results includes: establishing a four-level early warning classification system that matches the comprehensive risk level of reservoir operation; determining the triggering conditions, control requirements, push scope, and handling procedures corresponding to each level; ensuring that the triggering rules are deeply matched with business scenarios; performing pre-compliance verification on the standardized results output by the risk assessment module; eliminating invalid assessment results to avoid false early warning triggering; executing differentiated early warning triggering logic according to the risk type of different business scenarios; generating a globally unique traceability code for the event after the early warning is triggered; completing archiving and visual positioning by associating information across the entire chain; deduplicating and merging duplicate early warnings; dynamically adjusting the early warning level based on real-time data and assessment results; ensuring traceability throughout the entire early warning lifecycle; and simultaneously pushing standardized early warning information to the corresponding business modules.

[0014] Furthermore, the multi-channel push notifications to management personnel include: determining the scope of personnel, channels, frequency, and content specifications for different warning levels based on the platform's four-level warning grading system; ensuring that all push recipients are accounts that have completed real-name authentication and authorization within the platform; matching differentiated push trigger modes and execution rules according to the warning level; covering multiple compliant push channels; setting up retry for failed pushes and switching to backup channels; tracking the status of warning reception and confirmation; automatically triggering a second push and simultaneously notifying the superior if no confirmation is received within the specified period; using encrypted transmission throughout the entire chain; and implementing hierarchical control and display of confidential data.

[0015] Furthermore, the simultaneous recording and tracking of the entire process of early warning events to form a closed-loop business mechanism includes: establishing standardized ledger management rules for the entire lifecycle of early warning events; assigning globally unique traceability codes to early warning events; collecting information across the entire chain to ensure traceability; matching corresponding standardized handling process templates based on early warning levels; determining the responsibilities and rights of each node in the entire process; automatically dispatching early warning events to the corresponding responsible personnel; tracking and recording the handling progress and status throughout the entire process; standardizing the closed-loop process for handling review and acceptance; conducting post-mortem analysis on closed-loop events and pushing optimization suggestions; and setting up full-process access control and tamper-proof operation logs to meet the traceability verification, audit supervision, and compliance management requirements for water conservancy project safety monitoring event management.

[0016] Furthermore, all algorithm models undergo unified registration, scheduling, performance evaluation, and iterative optimization. This includes: establishing a unified model classification system and registration management specifications; setting access controls; completing standardized verification of submitted models throughout the entire process; registering models after passing verification and incorporating them into a unified model library for management, with full traceability; establishing a unified model scheduling management system that adapts to multiple scheduling modes; completing model binding and matching according to business scenarios; and tracking the execution status of scheduling tasks throughout the entire process; constructing a full lifecycle performance evaluation system for models; conducting multi-dimensional evaluations and handling them in a tiered manner; and determining model iterative optimization rules, completing model optimization, verification, version updates, and archiving according to trigger conditions.

[0017] This application also discloses an electronic device, including: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to execute the image recognition-based intelligent monitoring system for reservoir dam safety of the present invention.

[0018] Beneficial Effects: This application breaks through the limitations of traditional manual inspections and single-point sensor monitoring, achieving intelligent monitoring coverage of the entire reservoir area and all time periods. This significantly reduces the rate of missed and false detections of abnormal events, effectively solving the problems of delayed anomaly detection and discontinuous control. Simultaneously, it breaks down the barriers between unstructured video image data, sensor time-series data, and engineering spatial data, achieving the fusion and cross-verification of multi-source data, completely solving the data silo problem and fully releasing the application value of massive monitoring data. The solution constructs a multi-dimensional quantitative analysis and risk assessment system, breaking through the limitations of traditional simple data display and over-limit alarms. It achieves accurate prediction of water and rainfall trends and dam structural behavior, significantly improving the ability to provide early warning of risks. Furthermore, it establishes a unified management system for the entire lifecycle of algorithm models, realizing the collaborative scheduling and continuous iterative optimization of multiple models, greatly improving model reusability and system scalability. Finally, it constructs a full-process digital closed loop from data collection, intelligent analysis, early warning push, and response tracking, comprehensively improving emergency response efficiency and decision-making scientificity, providing full-chain intelligent support for the safe and stable operation of reservoirs and dams. Attached Figure Description

[0019] Figure 1 This is a block diagram of the intelligent monitoring system for reservoir dam safety based on image recognition, as described in this invention. Figure 2 This is a flowchart illustrating the training process of a specific AI algorithm model in an embodiment of the present invention. Figure 3 This is a flowchart of the real-time inference and automatic target detection process for video streams in an embodiment of the present invention; Figure 4 This is a flowchart illustrating the process of converting unstructured image data into structured data in an embodiment of the present invention. Figure 5 This is a flowchart of the multi-source data homogeneous fusion and cross-validation process in an embodiment of the present invention; Figure 6 This is a flowchart of the quantitative analysis and risk assessment of water and rainfall conditions and dam safety status in an embodiment of the present invention; Figure 7This is a flowchart illustrating the tiered early warning triggering and multi-channel message push process in an embodiment of the present invention. Figure 8 This is a flowchart illustrating the unified management process for the entire lifecycle of the algorithm model in this embodiment of the invention. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0021] This invention provides an intelligent monitoring system for reservoir dam safety based on image recognition, such as... Figure 1 As shown, it includes: The data acquisition module is used to continuously acquire video images of the dam body, water surface, gatehouse, and perimeter area at fixed intervals, and simultaneously acquire time-series data from water level, seepage, and displacement sensors. The model training module is used to train specialized AI algorithm models; The real-time inference module is used to perform real-time inference on the acquired video stream based on a pre-trained specialized AI algorithm model, and to automatically detect open smoke and flames, water level readings, gate status, violations, and floating objects on the water surface. The data structuring module is used to simultaneously complete the structuring transformation of unstructured image data and output standardized recognition results; The data fusion and verification module is used to perform source fusion and cross-verification of image recognition results with sensor monitoring data and engineering space data; The risk assessment module is used to perform quantitative analysis of water and rainfall trends and dam safety status through preset algorithms, and output accurate risk assessment results. The early warning triggering module is used to trigger corresponding level early warnings based on the analysis results; the message push module is used to push messages to management personnel through multiple channels. The event handling module is used to synchronously record the entire process of early warning events, track the handling progress, and form a business closed loop; the model management module is used to perform unified registration, scheduling, effect evaluation, and iterative optimization of all algorithm models.

[0022] The process involves continuously collecting video images of the dam, water surface, gate control room, and perimeter area using high-definition cameras throughout the reservoir at fixed intervals, while simultaneously collecting time-series data from water level, seepage, and displacement sensors. Specifically, this includes: determining the deployment rules for high-definition network cameras at key control points within the reservoir's upstream and downstream slopes, spillway, discharge tunnel, gate control room, power distribution room, the entire reservoir surface, perimeter protection zone, no-fishing zone, and management protection area, based on the reservoir safety monitoring business scenario and algorithm model input requirements; continuously collecting real-time video images of the corresponding areas at a frequency of no less than 25 frames per second; simultaneously capturing keyframe images at 10-second intervals for offline algorithm training and backtracking analysis; and simultaneously accessing real-time video and aerial image data transmitted from drone patrols, including areas difficult to reach by manual inspections such as the reservoir edge and shoreline. All collected video images are uniformly converted to a standardized format, and the collection point number, collection time, and spatial coordinate information are recorded synchronously to ensure data traceability and location.

[0023] Simultaneously, various safety monitoring sensors, including water level gauges, piezometers, displacement gauges, and stress gauges, are used to continuously collect time-series data for the corresponding monitoring items at a fixed interval of 1 minute, based on a time reference aligned with video acquisition. These data items include real-time monitoring values, acquisition time, sensor device number, spatial location of the measuring point, and equipment operating status information. Initial data screening and format standardization are performed concurrently to remove invalid or garbled data generated during acquisition, ensuring the continuity and integrity of the collected data. All acquired video image data and sensor time-series data are encrypted and transmitted through the reservoir's dedicated intranet and simultaneously stored in the platform's dedicated database. Strict access control rules are followed, with only authorized accounts able to access and retrieve the raw and pre-processed data.

[0024] The training-specific AI algorithm model, such as Figure 2 As shown, the implementation specifically includes: based on the business scenario of safety monitoring of the Yugang Reservoir dam in Kedong County and the AI ​​algorithm identification targets, determining the dataset construction rules corresponding to each special algorithm, collecting historical video frames, on-site real-shot images, and drone patrol aerial images from all monitoring points in the entire reservoir area as basic sample data, and simultaneously supplementing publicly compliant sample data under the same type of water conservancy project scenario, including scenarios with different lighting, weather, seasons, and shooting angles, and completing the classification and labeling of sample data for each special identification target, including open smoke and fire, drowning, floating objects on the water surface, gate opening and closing status, water gauge scale, personnel intrusion, vehicles, illegal dumping, illegal mining, illegal construction, fishing, and equipment operation status. The labeling content includes target category, bounding box coordinates, and status attribute information. After labeling, the training set, validation set, and test set are divided in a 7:2:1 ratio to ensure that the sample distribution within each dataset is balanced and there are no scene biases or category imbalances.

[0025] Based on the input requirements of each specialized AI algorithm model, standardized preprocessing operations are performed on the labeled sample data. Image data is normalized in size and pixel value. For low-quality images in backlight, low light, rain, or foggy weather, denoising, contrast enhancement, and distortion correction are performed. Data augmentation operations such as random flipping, cropping, scaling, and color dithering are performed on the sample data to expand the sample richness and improve the model's generalization ability. At the same time, invalid samples that are blurry, occluded, or incorrectly labeled are removed to ensure that the preprocessed sample data meets the input standards for model training.

[0026] Based on the real-time and accuracy requirements of the Yugang Reservoir dam safety monitoring, suitable deep learning network architectures were determined for each specialized identification task. For tasks with high real-time requirements, such as perimeter intrusion and open smoke / fire identification, a lightweight edge-side inference network architecture was selected. For tasks with high accuracy requirements, such as water level readings, gate opening calculations, and identification of disorderly phenomena, a high-precision detection network architecture was selected. According to the hardware computing power environment deployed in the reservoir, the network architecture was trimmed and optimized, adjusting the number of convolutional kernels and network depth to balance the inference speed and recognition accuracy of the model, ensuring that the trained model can be adapted to the operating environment of the reservoir's existing GPU inference server.

[0027] Based on the determined network architecture and preprocessed sample dataset, a model training environment is built, and hyperparameters for model training are configured, including learning rate, batch size, iteration rounds, loss function, and optimizer type. The training set data is input into the network architecture to perform iterative training. After each round of training, the generalization ability of the model is verified using validation set data. Based on the validation results, the hyperparameters are dynamically adjusted, and the weights of the loss function are optimized. For target categories with high false positive and false negative rates, sample data corresponding to the scenario is supplemented to perform incremental training, continuously optimizing the model's feature extraction capability until the model's recognition accuracy and recall on the validation set reach the preset business standards.

[0028] In accordance with the operational specifications for reservoir dam safety monitoring, multi-dimensional performance tests were conducted on the trained models. Independent test sets were used to test the models' core performance indicators, including recognition accuracy, recall, precision, and single-inference response time. This ensured that the target detection confidence of each specialized model reached the preset threshold and that the single-inference response time met the operational requirements for real-time reservoir monitoring. Simultaneously, online testing was conducted in the actual reservoir environment, connecting real-time video streams from various monitoring points to test the model's stable operation under different on-site scenarios. For misidentification and missed identification issues that occurred during on-site testing, corresponding scenario samples were added to further optimize the model until it passed the full-scenario operational testing.

[0029] In accordance with the platform deployment requirements, the tested specialized AI algorithm models are converted and solidified into a standardized model format supported by the platform. Simultaneously, the model interface is defined, input and output specifications are formulated, and a model description document is written for each specialized model, including model functions, recognition targets, performance indicators, input and output requirements, and operating environment parameter information. After completion, the model is submitted to the model management module to complete the model registration process, providing standardized model support for subsequent real-time inference calls.

[0030] Based on a pre-trained specialized AI algorithm model, real-time inference is performed on the acquired video stream to automatically detect open smoke and flames, water level readings, gate status, violations, and floating objects on the water surface. Figure 3 As shown, the implementation specifically includes: based on the pre-trained AI algorithm models that have been registered on the platform and passed full-scenario business testing, determining one-to-one or many-to-one binding rules with cameras at each monitoring point of the reservoir, matching each real-time video stream with a corresponding business scenario-specific algorithm model, ensuring that video streams from all key control points, including the upstream and downstream slopes of the dam, spillway, water release tunnel, gate opening and closing machine room, power distribution room, the entire water surface of the reservoir, the perimeter protection zone of the reservoir area, the no-fishing zone, and the management and protection area, are matched with the detection algorithm corresponding to the scenario, achieving full coverage of business scenarios without blind spots. Based on the input standards of each pre-trained model, the real-time acquired video streams undergo frame-by-frame standardized preprocessing, completing image size normalization and pixel value standardization. For low-quality images in scenarios such as backlight, low light, rain and fog, strong light reflection, and water surface ripples, noise reduction, contrast enhancement, and distortion correction operations are performed, eliminating invalid occluded areas in the image, ensuring that each frame of the image input to the algorithm model conforms to the input specifications of the pre-trained model, guaranteeing the stability and accuracy of the inference results.

[0031] For open smoke and open flame detection scenarios, a pre-trained specialized model for open smoke and open flame recognition is invoked to perform frame-by-frame inference on real-time video streams from key fire prevention areas such as switch rooms, power distribution rooms, and material warehouses. The model automatically identifies open flame and smoke targets in the scene and simultaneously outputs target category, confidence level, and spatial location coordinates. When the confidence level of a detected target reaches a preset business threshold, it is immediately marked as an abnormal event. The entire inference process strictly controls the response time of a single frame, meeting the performance requirements of the platform's real-time monitoring.

[0032] For water level reading detection scenarios, a pre-trained water level recognition model is invoked to continuously infer the real-time video stream from cameras bound to water level monitoring points. First, the overall area of ​​the water level monitoring point in the image is located, and then the sequence of water level monitoring scale numbers is accurately identified. Based on the previously calibrated zero-point coordinates and scale parameters of the water level monitoring point, the current water level reading is automatically calculated and stably output. Simultaneously, the identified water level data is bound and stored with the corresponding timestamp and point number to ensure the continuity and traceability of the water level reading. During the inference process, interference caused by water surface fluctuations, debris obstruction, and water flow impact is automatically filtered to ensure that the reading accuracy meets the business standards for reservoir water level monitoring.

[0033] For gate status detection scenarios, a pre-trained gate opening and closing status recognition model is invoked to perform real-time inference on the real-time video streams of each gate control point. This accurately locates the gate body and the opening scale area in the image. Based on the calibration sample data of the fully open and fully closed gate positions, the real-time gate opening value is automatically calculated to determine the real-time operating status of the gate (open, closed, or half-open). The gate status results, opening data, and trends over a period of time are output simultaneously. When it is detected that the gate is in a state below the limit for a long time or that the opening changes abnormally, it is immediately marked as an abnormal event, providing real-time data support for daily gate inspection and zero-position calibration.

[0034] For violation detection scenarios, based on the business management requirements of different control areas, corresponding pre-trained specialized AI algorithm models are invoked to perform synchronous parallel inference on real-time video streams from points such as the reservoir perimeter, no-fishing areas, the entire water surface, and the management and protection area. Specifically, for the perimeter protection area, a personnel intrusion detection model is invoked, which automatically identifies targets of illegal intrusion, loitering, and crossing the boundary based on pre-defined virtual fences and trigger rules; for the open water area of ​​the reservoir, a drowning recognition model is invoked to automatically distinguish between normal water surface activities and abnormal drowning states; for the no-fishing control area, a fishing recognition model is invoked to automatically identify the violation of personnel fishing with rods; for the reservoir management and protection area, a specialized model for identifying four types of illegal activities is invoked to automatically detect illegal dumping, illegal mining, illegal occupation, and illegal construction; and for the reservoir entrances and internal road areas, a vehicle detection model is invoked to automatically identify illegal intrusion and abnormal parking behavior of heavy vehicles. The inference results of all violations are output simultaneously with complete information such as violation type, target location, confidence level, and occurrence time. When the confidence level of a detected violation reaches a preset threshold, it is immediately marked as an abnormal event.

[0035] For the detection of floating objects on the water surface, a pre-trained specialized model for recognizing floating objects is invoked to continuously infer the real-time video streams from monitoring points across the entire reservoir area. This model automatically identifies individual floating debris and large-area debris accumulations in the footage, simultaneously outputting the type, coverage area, and spatial location information of the floating objects. It continuously tracks the movement trajectory and range change trends of the floating objects, and immediately marks them as abnormal events when the detected floating object coverage area reaches a preset alarm threshold. The inference process adapts to complex on-site environments such as water surface fluctuations, light reflection, and water flow changes, ensuring the accuracy of the recognition results.

[0036] All real-time inference results of specialized models synchronously complete the structured transformation of unstructured image data, outputting standardized data that conforms to the platform's unified specifications. Simultaneously, the entire traceable information of inference time, model version, location number, and collection timestamp is recorded. During the inference process, serial, parallel, and hybrid model scheduling are supported. Computing resources are allocated according to the priority of business scenarios to ensure that the inference response of high-priority alarm scenarios is given priority. All abnormal events in the inference output are synchronously pushed to the platform's early warning information panel, providing real-time and accurate data support for subsequent data fusion verification, risk assessment, and graded early warning triggering.

[0037] The process of synchronously transforming unstructured image data into structured data and outputting standardized recognition results is as follows: Figure 4 As shown, the specific implementation includes: based on the unified structured data standard for water conservancy project safety monitoring established by the platform, determining the standardized conversion rules corresponding to each type of unstructured image reasoning result; for different identification targets such as open smoke and fire, water level readings, gate opening and closing status, personnel intrusion, drowning, fishing, floating objects on the water surface, vehicles, illegal dumping, illegal mining, illegal construction, and equipment operation status, determining fixed structured data fields respectively. The field content includes the target's unique identifier, target category code, target category name, confidence score, target spatial boundary coordinates, the monitoring point number to which the target belongs, the target's spatial geographic coordinates, image acquisition timestamp, algorithm reasoning completion timestamp, algorithm model version number, abnormal event judgment result, and event level code fixed items, ensuring that all identification results have uniform field formats and consistent data types, with no custom non-standard fields.

[0038] Based on the original inference results output by each specialized AI algorithm model, the unstructured image features, detection box information, and classification labels are formatted and numerically standardized one by one. The category labels of the original output are mapped to the platform's unified water conservancy safety monitoring event classification code. The bounding box coordinates of the original output are converted into a unified pixel coordinate system and spatial geographic coordinate dual-dimensional value. The confidence value of the original output is retained to four significant digits. The time information of the original output is uniformly converted to year-month-day hour:minute:second.

[0039] The standard time format is milliseconds. Redundant feature data and invalid intermediate variables in the original inference results are removed, retaining only valid data content that meets the standardized field requirements. Based on the platform's data quality control rules, the structured data that has undergone initial transformation undergoes compliance verification. The data type, value range, and format specifications of each field are checked one by one to ensure they conform to preset standards. Abnormal data with missing fields, excessive values, or incorrect formats are marked and intercepted. Simultaneously, the consistency between the structured data and the original image acquisition points and timestamps is verified to ensure that each piece of structured data is uniquely bound to its corresponding original image frame or video stream segment, without data mismatches or broken traceability issues.

[0040] Based on the business scenario type of the identified target, the standardized structured data that has passed the verification is classified and collected. The data is grouped into four major business categories: safety production monitoring, water and engineering monitoring, violation control, and equipment status monitoring. The standardized identification results are simultaneously written into the corresponding structured database of the platform and indexed and associated with the time-series data of sensors at the same time dimension and location. At the same time, a unique traceability code is generated for each standardized identification result. Through the traceability code, the corresponding original image frames, video stream segments, and algorithm inference full-link logs can be retrieved to ensure that the data is traceable and verifiable throughout the entire process. According to the preset business alarm threshold, abnormal event data in the standardized identification results are specially marked. Fixed fields such as abnormal event level, alarm triggering conditions, and handling priority are added to the structured data. Abnormal data that reaches the alarm threshold is pushed to the platform's early warning information panel to ensure that the standardized identification results can directly support the subsequent data fusion verification, risk assessment, and early warning triggering of the entire business process without secondary format conversion and data processing.

[0041] The process involves fusing and cross-validating image recognition results with sensor monitoring data and engineering spatial data, such as... Figure 5 As shown, the specific implementation includes: based on the platform's unified spatiotemporal reference system, determining the spatiotemporal alignment rules for image recognition results, sensor monitoring data, and engineering spatial data; using Beijing time as the unified time reference, standardizing the timestamps of all data to millisecond-level precision; for image recognition results, using the acquisition timestamp of the corresponding video frame as the core reference; for sensor monitoring data, using the timestamp of data acquisition completion as the reference; and for engineering spatial data, using the time node of data acquisition or model update as the reference, ensuring that the three types of data achieve accurate alignment in the same time dimension and eliminating fusion errors caused by time deviations.

[0042] Based on the BIM model of the reservoir project and the global spatial coordinate system constructed on the WebGIS platform, spatial association mapping rules for three types of data are determined. A unique spatial geographic coordinate and engineering structure code are assigned to each monitoring camera, each sensor monitoring point, and each engineering structure. The target location coordinates in the image recognition results are converted into a global spatial coordinate system consistent with the engineering spatial data. The sensor monitoring data is bound to the corresponding engineering structure code and spatial coordinates of the measuring points, realizing a one-to-one correspondence between the image recognition results, sensor monitoring data, and engineering spatial data in the spatial dimension. This ensures that each piece of data can be accurately located to the specific structural part and spatial position of the reservoir project.

[0043] Based on the classification of business scenarios for reservoir dam safety monitoring, the data from the three categories were aligned and fused together. For the hydrological monitoring scenario, the water level identification results output by the water gauge recognition model were fused with the sensor time-series data collected by the corresponding water level gauges and integrated into the spatial location of the corresponding water level monitoring section in the reservoir engineering spatial data, forming a water level time-series dataset with spatial attributes and dual data source verification. For the dam structural safety scenario, the deformation and crack identification results output by the dam slope video AI recognition were fused with the sensor time-series data collected by the dam displacement gauges, piezometers, and stress gauges and integrated into the structural location of the corresponding dam monitoring section and measuring points in the engineering spatial data, forming a multi-dimensional structural safety fusion dataset including the dam surface and interior. For the equipment operation scenario, the AI ​​recognition results of the hoisting and gate equipment operation status were fused together with the data from the dam slope video AI recognition model. The results of the gate opening and closing status recognition are combined with the time-series data collected by the opening degree sensor and the operation status sensor of the corresponding equipment and integrated into the BIM model location of the corresponding equipment in the engineering space data to form a full-dimensional fusion dataset of equipment operation status. For the reservoir area safety production scenario, the results of open smoke and open flame recognition are combined with the time-series data collected by the smoke and heat sensors of the corresponding area and integrated into the spatial layer of the corresponding fire prevention and control area in the engineering space data to form a safety production monitoring fusion dataset. For the reservoir area management scenario, the AI ​​recognition results of personnel intrusion detection, drowning recognition, fishing recognition, disorderly conduct recognition, floating object recognition, and vehicle detection are combined with the environmental monitoring sensor data of the corresponding area and integrated into the spatial layer of the corresponding control area in the engineering space data to form a fusion dataset of full-area reservoir area management.

[0044] Based on the technical specifications for safety monitoring of water conservancy projects and the requirements for reservoir business management, cross-validation rules and allowable deviation thresholds for three types of data are determined for different business scenarios. For the water level monitoring scenario, an allowable deviation threshold is determined between the AI ​​recognition results of the water gauge and the data from the water level sensor. When the deviation between the two types of data exceeds the preset threshold, the data is automatically marked as abnormal. For the gate operation scenario, an allowable deviation threshold is determined between the AI ​​recognition results of the gate opening status and the data from the gate opening sensor. When the deviation between the two types of data exceeds the preset threshold, the data is automatically marked as abnormal. For the dam structure safety scenario, an allowable deviation threshold is determined between the AI ​​deformation recognition results of the dam surface and the data from the displacement sensor inside the dam. When the deviation between the two types of data exceeds the preset threshold, the data is automatically marked as abnormal. For the safe production scenario, verification and matching rules are determined between the AI ​​recognition results of open smoke and open flame and the data from smoke and heat sensors. When the state descriptions of the two types of data are inconsistent, the data is automatically marked as abnormal. Based on the established verification rules, a full cross-validation is performed on the fused dataset. For each set of three types of spatiotemporally aligned data, the logical consistency and numerical deviation between the data are compared one by one. First, the two-way verification between the AI ​​recognition results and the sensor monitoring data is completed to verify the consistency of the two types of data in describing the state of the same monitoring object. Then, based on the design parameters and historical operating thresholds of the corresponding monitoring object in the engineering spatial data, the compliance verification between the first two types of data and the inherent attributes of the engineering is completed to confirm whether the data conforms to the physical operating laws and design safety range of the engineering structure.

[0045] For abnormal data marked during cross-validation, a data tracing and verification process is automatically triggered. Based on the unique data tracing code, the corresponding original video frames, original sensor acquisition messages, and original parameters of the engineering spatial model are retrieved in reverse order to investigate the cause of the anomaly one by one. Three different situations are distinguished: algorithm error, sensor acquisition failure, and actual engineering status anomaly. For anomalies caused by algorithm error, the data is automatically pushed to the model management module to trigger the model optimization and incremental training process. For anomalies caused by sensor acquisition failure, the data is automatically pushed to the equipment operation and maintenance module to trigger the equipment inspection and calibration process. For verification deviations caused by actual engineering status anomalies, the data is automatically marked as a high-priority risk event and simultaneously pushed to the risk assessment module and the early warning triggering module to provide accurate basis for subsequent risk handling.

[0046] For compliant fusion datasets that have passed cross-validation, the data is automatically updated to the platform's unified fusion database and synchronously linked to the BIM / GIS model layers of the corresponding engineering spatial locations. This enables the visualization and overlay of monitoring data on the 3D engineering model. At the same time, the compliant fusion datasets are used as input data into the big data analysis model to provide high-quality basic data with multi-source verification for water and rainfall trend prediction, dam structural behavior analysis, and flood risk assessment, ensuring the accuracy and reliability of the analysis and prediction results.

[0047] The aforementioned algorithm performs quantitative analysis of water and rainfall trends and dam safety status, outputting accurate risk assessment results, such as... Figure 6 As shown, the specific implementation includes: determining a unified spatiotemporal benchmark for the quantitative analysis of water and rainfall trends and dam safety status; determining a compliant dataset after source fusion and cross-validation as the core data input; the data content includes structured data such as standardized water level, gate opening, and dam surface anomalies output by AI image recognition; time-series monitoring data collected by sensors such as water level gauges, piezometers, displacement gauges, and stress gauges; BIM and GIS spatial data of the reservoir project; historical hydrological and meteorological data of the basin; and reservoir design parameters and historical operating condition data. It is ensured that all input data passes data quality verification, and that the time and spatial dimensions are fully aligned, with no missing values, outliers, or data mismatch issues.

[0048] The algorithm system and calculation rules for quantitative analysis of water and rainfall trends were determined. For reservoir water level prediction scenarios, an appropriate time series analysis algorithm was determined. Based on historical water levels, rainfall, inflow, meteorological forecast data, and real-time water level data output by AI water gauge recognition, the model input features were selected and weighted. According to the preset prediction time dimension, the water level change trend was calculated for the next 6 hours, 12 hours, 24 hours, and 72 hours respectively. The predicted water level value, rate of change, confidence interval, and corresponding probability distribution were output simultaneously. During the calculation process, the four types of thresholds of dead water level, normal storage water level, warning water level, and guaranteed water level were automatically compared, and the safe range of the water level prediction value was marked.

[0049] For the inflow forecasting scenario, a suitable machine learning algorithm is determined. Based on rainfall intensity, watershed underlying surface characteristics, historical inflow, previous soil moisture, and upstream hydrological station monitoring data, feature engineering processing and model calibration are completed. The predicted inflow, peak arrival time, peak flow, and cumulative inflow are calculated for future periods under different rainfall scenarios. The uncertainty analysis of the forecast results is completed simultaneously, and the flow forecast intervals at different confidence levels are output.

[0050] For flood discharge capacity matching analysis, based on predicted inflow, real-time water level data, and design parameters of the reservoir's flood discharge structures, the reservoir water level change process and discharge flow process under different flood discharge scheduling schemes are calculated. The matching degree between the flood discharge scheme and the downstream river's safe flood passage capacity is quantitatively assessed, and scheduling risk points exceeding the downstream safe carrying capacity are marked. A multi-dimensional indicator system for the quantitative analysis of dam safety status is established. The indicator system includes four core dimensions: dam deformation, seepage characteristics, structural stress, and equipment operating status. Each dimension has corresponding quantitative calculation indicators, safety thresholds, and weight coefficients. The safety thresholds are determined based on reservoir dam design specifications, safety assessment conclusions, and historical operating extreme values. The weight coefficients are quantitatively allocated using the analytic hierarchy process (AHP) to ensure that the indicator system comprehensively includes the core control elements for dam safe operation.

[0051] For dam deformation safety analysis, based on the dam surface deformation and crack identification results output by AI image recognition, and the horizontal displacement, vertical displacement, and settlement time series data collected by displacement sensors on the dam top, dam foundation, and upstream and downstream slopes, the correlation between the deformation sequence and the water level and temperature time series is quantitatively calculated. The water pressure component, temperature component, time component, and noise component in the deformation data are separated, and the development rate, cumulative deformation amount, and deformation uniformity of the dam deformation are quantitatively calculated. The results are compared with the preset deformation control threshold, and abnormal deformation points and development trends that exceed the safe range are marked.

[0052] For the safety analysis of dam seepage, based on the time series data of dam seepage line, seepage flow, and pore water pressure collected by piezometers and seepage flow sensors, and the abnormal identification results of seepage and water accumulation behind the dam output by AI image recognition, the seepage gradient, seepage flow rate, piezometer depth and trend of the dam are quantitatively calculated. By comparing with the seepage control threshold allowed by the design, the seepage resistance stability of the dam is quantitatively evaluated, and the probability of seepage anomaly areas and piping and soil erosion risks is marked.

[0053] For stress safety analysis of dam structure, based on the stress-strain time series data of dam structure collected by stress gauges and strain gauges, and the operation status data of gates, spillways and other structures identified by AI, the stress distribution, stress change rate and load matching degree of dam structure are quantitatively calculated, compared with the stress limit allowed by structural design, and the stress over-limit area and structural damage risk point are marked.

[0054] For the overall structural performance assessment of the dam, based on the quantitative calculation results of multi-dimensional indicators, the fuzzy comprehensive evaluation method is used to complete the quantitative scoring of the overall safety status of the dam, determine the dam safety status level, and classify the safety level into four levels: normal, basically normal, abnormal, and dangerous. At the same time, the contribution of each indicator to the safety level is quantitatively calculated to identify the core risk factors affecting the dam safety status.

[0055] Based on the quantitative analysis results of water and rainfall trends and the quantitative analysis results of dam safety status, a joint assessment of the comprehensive risk of reservoir operation is completed, and the linkage rules for risk assessment are determined. When three scenarios occur—predicted water level exceeding warning level, abnormal dam safety status indicators, and inflow exceeding reservoir regulation capacity—a multi-dimensional joint assessment is automatically triggered. The comprehensive risk value of reservoir operation under different scenario combinations is quantitatively calculated, and the risk level is determined. The risk level is divided into four levels: general, relatively severe, serious, and extremely serious. The quantitative assessment of the probability of risk occurrence, scope of impact, and degree of harm is completed simultaneously.

[0056] In a further embodiment, the assessment based solely on three discrete scenarios suffers from drawbacks such as threshold triggering lag, insufficient quantification of multi-scenario coupled risks, and failure to consider the temporal transmission effect of risk evolution. This makes it unsuitable for the nonlinear coupled risk prediction requirements of multi-source heterogeneous data from reservoirs and dams. Therefore, this application proposes a dynamic linkage assessment algorithm for comprehensive dam operation risks, specifically including: The marginal distribution fitting of risk indicators uses nonparametric kernel density estimation to fit the marginal cumulative distribution function F of each risk indicator. i (x i (i=1,2,...,n, where n is the total number of risk indicators), complete the standardized mapping of indicators. The Gumbel-Hougaard Copula function is used to characterize the extreme tail nonlinear dependence of water and rainfall conditions on dam safety status, and a multi-index joint distribution function is constructed, as shown in the following formula: C(u1,u2,...,u n )=exp{-[(-lnu1) θ +(-lnu2) θ +...+(-lnu n ) θ ] 1 / θ}; In the formula: u i Let u be the marginal distribution value of the i-th risk indicator. i =F i (x i The value range is [0,1]; θ is the Copula coupling parameter, θ∈[1,+∞). The larger θ is, the stronger the correlation between extreme value risks between indicators. The parameter is estimated by the maximum likelihood method.

[0057] The comprehensive risk probability is calculated based on the joint distribution function, using the formula: P = 1 - C(u1, u2, ..., u n ), P∈[0,1], the larger the P value, the higher the overall risk level.

[0058] A normal cloud model is introduced to address the fusion of fuzziness and randomness in risk level determination. Each risk level is determined by the expected value E.x Entropy E n Hyperentropy H e Three numerical characteristics are used to pre-construct a four-level risk standard cloud model: general risk C1(E) x1 E n1 H e1 C2 (E) is a relatively heavy risk x2 E n2 H e2 ), serious risk C3 (E) x3 E n3 H e3 C4 (E) is a particularly serious risk x4 E n4 H e4 ).

[0059] The membership degree of the comprehensive risk probability P to each level of the cloud model is calculated using the following formula: ; In the formula: k=1,2,3,4 correspond to four risk levels; E nk′ =E nk +H ek ×randn(); randn() is a standard normally distributed random number, used to characterize the random discreteness of risk assessment; μ k (P) represents the membership degree of the risk value P to the k-th level risk cloud.

[0060] The risk level determination and evolution prediction take the level corresponding to max(μ1,μ2,μ3,μ4) as the final comprehensive risk level. At the same time, the temporal evolution transition probability of the risk level is calculated based on the first-order Markov chain to complete the advanced prediction of the risk development trend.

[0061] The system outputs the comprehensive risk level of reservoir operation, comprehensive risk probability, core risk influencing factors, probability of risk evolution trend transfer, and quantitative value of risk impact scope, and simultaneously triggers the corresponding early warning process.

[0062] For identified risk events, risk tracing rules are established. Based on the unique tracing code of the fused dataset, the monitoring points, engineering structures, data sources, and triggering conditions corresponding to the risks are located in reverse. The causes, development paths, and evolution trends of the risks are quantitatively analyzed, and targeted risk handling priorities and control recommendations are output. All quantitative analysis processes and risk assessment results are synchronously recorded with complete calculation parameters, algorithm versions, input data sources, and timestamps to ensure that the entire analysis and assessment process is traceable and verifiable. The final risk assessment results include standardized content of all elements, including hydrological and rainfall trend prediction data, dam safety status quantitative scores, comprehensive risk levels, risk points, impact range, evolution trends, and handling recommendations. These are simultaneously pushed to the early warning triggering module and the event handling module, providing accurate and reliable decision-making basis for subsequent graded early warning and closed-loop handling.

[0063] The corresponding level of early warning is triggered based on the assessment results, such as... Figure 7 As shown, the implementation specifically includes: establishing a four-level early warning classification system that corresponds one-to-one with the comprehensive risk level of reservoir operation. The early warning levels are divided into four levels: general, relatively severe, serious, and extremely serious. Simultaneously, the triggering conditions, control requirements, notification scope, and handling procedures corresponding to each early warning level are determined. The triggering conditions are strictly based on the results of water and rainfall trend prediction, dam safety status quantitative scoring, comprehensive risk assessment conclusions, reservoir design safety thresholds, and industry standard control requirements to ensure that the early warning triggering rules of each level are deeply matched with the business scenarios.

[0064] Based on the standardized assessment results output by the risk assessment module, the pre-warning compliance verification is completed. The comprehensive risk level, spatial coordinates of risk points, trigger timestamp, data traceability code, probability of occurrence and scope of impact of the risk are checked one by one in the assessment results. The input data corresponding to the assessment results are verified to see if they have passed the same source fusion and cross-verification. Invalid assessment results that have not passed the data quality verification are eliminated to avoid false warnings. Based on the risk type of different business scenarios, differentiated early warning triggering logic is executed. For water situation scenarios, when the predicted water level exceeds the corresponding threshold of normal storage level, warning level, or guaranteed water level, or when the predicted inflow exceeds the reservoir's regulation capacity or the outflow exceeds the downstream river's safe flood discharge capacity, the corresponding level of early warning is automatically triggered. For dam structural safety scenarios, when the dam deformation, seepage, or stress indicators exceed the safety control threshold, or when the overall safety status of the dam is assessed as abnormal or dangerous, the corresponding level of early warning is automatically triggered. For safe production scenarios, when the open smoke and flame detection results reach the alarm threshold, or when equipment operation status is identified as abnormal, the corresponding level of early warning is automatically triggered. For reservoir area management scenarios, when illegal intrusion, drowning, illegal fishing, disorderly conduct, illegal intrusion of heavy vehicles, or excessive accumulation of floating debris on the water surface are detected, the corresponding level of early warning is automatically triggered. For equipment operation scenarios, when the gate opening and closing status is abnormal or the deviation between the opening data and sensor data exceeds the limit, the corresponding level of early warning is automatically triggered.

[0065] Once an early warning is triggered, a globally unique traceability code is automatically generated for each individual early warning event. Simultaneously, the original video frames, sensor-collected data, fusion verification records, risk assessment logs, and engineering spatial location information corresponding to the early warning event are linked. The complete information of the early warning event is written into the platform's early warning information panel and early warning event ledger. Based on the BIM and WebGIS spatial coordinate system of the reservoir project, early warning markers are overlaid on the corresponding points in the 3D engineering model to achieve precise visualization and positioning of the early warning location.

[0066] For multiple early warning messages triggered repeatedly within a short period of time for the same risk source, automatic deduplication and merging are performed to retain the latest assessment results and the highest early warning level, avoiding duplicate early warnings from interfering with business operations. Based on real-time updated monitoring data and risk assessment results, a dynamic early warning level adjustment mechanism is implemented. When the risk situation continues to escalate, an automatic upgrade of the early warning level is triggered, simultaneously updating control requirements and the scope of notification. When the risk situation eases and the potential risks are eliminated, based on the latest compliance assessment results, an early warning level downgrade or de-escalation process is triggered, simultaneously recording the entire chain of level adjustment and de-escalation evidence to ensure traceability and verifiability throughout the early warning lifecycle. After an early warning is triggered, standardized early warning event information is simultaneously pushed to the message push module and event handling module, providing complete event evidence for subsequent multi-channel alarm notifications and closed-loop handling. The multi-channel push notifications to management personnel specifically include: Based on the platform's four-level early warning classification system, determining the scope of personnel, channels, frequency, and content specifications for different early warning levels. General early warnings are pushed to daily reservoir inspection personnel and on-site maintenance personnel in the corresponding area; more severe early warnings are pushed to the duty officer of the reservoir management station and the team leader of the maintenance team; serious early warnings are pushed to the main person in charge of the reservoir management unit and the relevant leader of the county-level water resources department; and particularly serious early warnings are pushed to the main person in charge of the county-level water resources department, the emergency management department, and the relevant leader of the local government. Simultaneously, notifications are sent to all relevant maintenance and management personnel. All recipients must be accounts that have completed real-name authentication and authorization within the platform; unauthorized personnel cannot receive early warning information. Based on the unique traceability code of the early warning event, standardized push content is determined. This content includes the unique traceability code of the early warning event, the early warning level, the early warning type, the trigger time, the precise spatial location, a description of the risk source, the basis for judgment, the priority of handling, guidelines for standardized handling procedures, and links to retrieve on-site video footage, ensuring the completeness and traceability of the push content.

[0067] The push triggering mode and execution rules are determined based on the warning level. The push process is started immediately after the warning is generated. For severe and extremely severe high-priority warnings, a multi-channel synchronous parallel push mode is adopted to ensure that the push is completed within 1 minute. For relatively severe and general warnings, a sequential push mode is adopted, which first pushes to the on-site personnel. If the receipt is not confirmed within the specified time limit, the push is automatically upgraded to the superior person in charge.

[0068] Multiple compliant push channels have been identified. The first category is the platform's internal network-exclusive channel, including pop-up push notifications on the system's web-based warning information panel and desktop pop-up notifications on the platform's client, accompanied by sound alarms within the platform. High-level warnings trigger continuous audio and visual alerts. The second category is the mobile-exclusive channel, including push notifications from the reservoir's dedicated operation and maintenance app and encrypted SMS push notifications. The third category is the emergency voice channel, which simultaneously pushes automatic voice call notifications for particularly severe warnings, ensuring that multiple channels are included without blind spots.

[0069] In case of push notification failure, the system automatically executes a retry mechanism with a 30-second retry interval. If the push fails after three consecutive retries, it automatically switches to a backup push channel, records the push failure log, and simultaneously pushes it to the system administrator account to ensure a closed-loop push process without interruptions. After the push is completed, the system tracks the reception status in real time. After the recipient opens the push message and completes the confirmation operation, the system automatically records the confirmer, confirmation time, and confirmation terminal information, and simultaneously updates the handling status in the warning event log. For severe and extremely severe warnings, the recipient must complete the confirmation receipt within the specified time limit. If confirmation is not completed within the time limit, the system automatically triggers a second push and simultaneously notifies the recipient's superior management, ensuring that the warning information reaches 100% of the responsible personnel.

[0070] All push notifications employ encrypted transmission. Internal network channels use a dedicated reservoir intranet for encrypted transmission, while public network channels use national cryptographic algorithms for end-to-end encryption of the content. Confidential engineering structural parameters and sensitive monitoring data are only fully displayed on the internal network channel; public network channels only display core warning information and handling guidelines, without disclosing confidential engineering data. The system comprehensively records the entire process of each warning push, including the warning event source code, push time, push channel, push recipient, reception status, confirmation receipt information, retry records, and failure reasons. All log data is immutable and permanently stored in the platform's dedicated database. Based on the warning event source code, the entire push process can be fully traced, meeting the traceability and verification requirements for water conservancy project safety management.

[0071] The synchronous recording and tracking of the entire process of early warning events to form a closed-loop business process includes: establishing standardized ledger management rules for the entire lifecycle of early warning events; assigning a globally unique traceability code to each triggered early warning event; and synchronously collecting the full-link basic information of early warning events within the ledger, including early warning level, early warning type, trigger time, precise spatial location, risk source description, risk assessment basis, original monitoring data source, algorithm model version, and corresponding engineering structure code, ensuring that all basic information of a single early warning event is complete and traceable, with no missing information.

[0072] Based on the four-level early warning classification system, standardized handling procedures, responsible entities, handling time limits, key node requirements, and closed-loop acceptance standards are determined for each level of early warning event. A unique handling procedure template is matched for different types and levels of early warning events. The template includes fixed nodes for event confirmation, on-site verification, risk management, rectification review, and closed-loop acceptance. Each node specifies the corresponding responsible personnel, operating permissions, completion time limits, and mandatory archiving information, ensuring that the entire handling process has standards to follow. After an early warning event is generated and pushed, the system automatically synchronizes the early warning event to the event handling module. Based on the early warning level and event type, it automatically distributes the event to the platform account and mobile maintenance app of the corresponding responsible personnel, and simultaneously updates the early warning event status to "pending confirmation." After the responsible personnel complete the receipt and confirmation of the early warning information, the system automatically records the confirmer, confirmation time, and confirmation terminal information, simultaneously updates the event status to "pending handling," and simultaneously sends confirmation receipt information to the superior management personnel who pushed the early warning.

[0073] When responsible personnel conduct on-site verification and risk management, they can retrieve real-time video streams, historical monitoring data, fusion verification records, engineering design parameters, BIM model spatial information, and handling process guidelines for the corresponding locations based on the early warning event source code within the platform. During the handling process, on-site verification photos, videos, handling measures, on-site situation descriptions, and other archived materials must be uploaded according to the nodes. The system records every step of the handling process in real time, including the operator, operation time, and uploaded archived materials, and updates the event handling progress and current status synchronously. The status includes four categories: handling, pending review, rectification, and pending acceptance. The handling progress is synchronized in real time to the platform's early warning information panel and the corresponding management personnel's account, ensuring that the entire handling process is traceable and supervised.

[0074] In response to situations where the risk situation escalates or the difficulty of handling exceeds the authority of on-site personnel, the system supports personnel in submitting escalation requests. The requests include the current risk situation, the handling measures already taken, the reasons for the escalation request, and the support needs. After the request is submitted, the system automatically pushes it to the corresponding superior management. After the manager completes the approval, the responsible party for handling the incident can be adjusted, the authority of the handling personnel can be expanded, and the handling process and time limit can be updated. The system fully records the information of the entire application and approval process and updates it to the incident log simultaneously. After completing on-site risk mitigation, the personnel handling the situation must submit a completion application and a complete set of archived documents within the system. Upon submission, the system automatically assigns the application to the corresponding reviewer's account. The reviewer, based on the original triggering conditions of the warning event, the handling process standards, and risk control requirements, conducts on-site review and document verification. Once the risk has been eliminated and the mitigation measures meet the required standards, the reviewer submits a review approval opinion within the system, and the system simultaneously updates the event status to "Pending Acceptance." If the review finds that the risk has not been completely eliminated or the mitigation measures do not meet the requirements, the system automatically rejects the application, marks the rectification requirements and deadline, reverts the event status to "In Rectification," and simultaneously notifies the original personnel handling the situation to continue the rectification work. The system fully records the reasons for rejection, rectification requirements, and rectification process information.

[0075] For early warning events that pass the review, the corresponding person in charge of the reservoir management unit will conduct the final closed-loop acceptance work. Based on the water conservancy project safety management specifications, reservoir operation management system, and the archived materials of the entire event handling process, the compliance of the entire event handling process, the thoroughness of risk and hidden danger elimination, and the completeness of the archived materials will be verified. After the acceptance is passed, the acceptance opinion will be submitted in the system, and the system will automatically update the event status to closed-loop and simultaneously complete the final archiving of the early warning event ledger. The archived ledger information cannot be tampered with and will be permanently stored in the platform's dedicated database. If the acceptance fails, the system will automatically roll back the event to the corresponding handling node and re-execute the handling process until the acceptance is passed and the closed loop is completed.

[0076] For early warning events that have completed closed-loop archiving, the system automatically conducts event debriefing analysis. Based on the event's full-process log data, it analyzes the causes of early warning triggers, the accuracy of algorithm model recognition, the efficiency of handling procedures, and the effectiveness of handling measures. For early warning events caused by algorithm false alarms or missed alarms, the system automatically pushes sample data of the corresponding scenario to the model management module, triggering incremental model training and optimization iteration processes. For early warning events caused by equipment failures or data anomalies, the system automatically pushes them to the equipment operation and maintenance module, triggering equipment inspection, calibration, and maintenance processes. For links where the handling process is not executed smoothly or the handling efficiency is not up to standard, the system automatically optimizes the corresponding handling process templates and responsibility allocation rules. At the same time, the debriefing analysis results and optimization suggestions are pushed to the person in charge of the reservoir management unit, providing data support for the optimization of the reservoir safety management system.

[0077] The system has strict access control rules for the entire process of early warning events recording, handling progress tracking, and closed-loop management. Only authorized accounts can view early warning event information and execute handling operations at corresponding nodes within their authorized scope. All operations generate tamper-proof operation logs simultaneously. The log content includes the operator, operation time, operation terminal, operation content, and changes in event status before and after the operation. Based on the early warning event traceability code, all ledger information, handling archives, and operation logs throughout the entire lifecycle of the event can be retrieved completely, meeting the traceability verification, audit supervision, and compliance management requirements for water conservancy project safety monitoring event management.

[0078] Perform unified registration, scheduling, performance evaluation, and iterative optimization on all algorithm models, such as Figure 8 As shown, the specific implementation includes: determining a unified classification system and registration management specifications for all algorithm models on the platform. The classification system includes three core categories: artificial intelligence models, visualization models, and big data analysis models. Each category corresponds to sub-model classification rules for specific business scenarios. The registration management specifications determine the mandatory information items, verification standards, release process, and access control requirements for model registration. Mandatory information items include model name, unique identifier code, version number, model type, business adaptation scenario, functional description, performance indicators, input / output format specifications, operating environment requirements, supporting documentation, and model firmware files. All information items must comply with the platform's unified data standards and water conservancy project safety monitoring business specifications, with no custom non-standard fields or content.

[0079] The entire process of model registration and release is subject to strict access control. Only the platform administrator account can perform model registration, release, and deactivation operations. Unauthorized accounts do not have relevant operation permissions. Model files and supporting documents are transmitted and stored in an encrypted manner through the reservoir's dedicated intranet to ensure the security of model resources.

[0080] For the algorithm models submitted for registration, a standardized verification process is performed throughout the entire process. First, the compliance of the model interface, the matching of input and output formats, and the adaptability of the operating environment are verified. Then, the model's functional and performance are tested using the platform's testing tools. The test content includes core indicators such as recognition accuracy, recall rate, single inference response time, and computational stability. The test samples include real-world scene data from Yugang Reservoir in Kedong County under different lighting, weather, seasons, and shooting angles. Models that pass the test complete the registration process, are assigned a globally unique traceability code, and their status is updated to "released" and included in the platform's model library for unified management. Models that fail the test are returned for optimization and are not allowed to be included in the model library or business call processes. All registration process operations and verification results generate tamper-proof operation logs, which are bound to the model traceability code for storage, ensuring that the entire process is traceable and verifiable.

[0081] Based on the full set of algorithm models registered and released in the platform's model library, a unified model scheduling and management system is established. The scheduling system supports three scheduling modes: serial, parallel, and hybrid. It adapts to the computing power scheduling and allocation rules of the distributed computing environment, determines the model scheduling priority, triggering conditions, execution frequency, error retry mechanism, and task full-process tracking rules for different business scenarios, and allocates priority computing power resources to ensure the real-time performance of model scheduling and inference response for high-priority real-time monitoring and alarm business scenarios.

[0082] Based on the reservoir's on-site business scenarios, the system completes the binding and matching of models with corresponding monitoring points, sensor nodes, and business processes. It determines the calling model and scheduling rules corresponding to each video stream and each set of monitoring data. It supports three triggering methods: automatic scheduling triggered by fixed periods, real-time scheduling triggered by events, and manual scheduling triggered by manual instructions. Before the scheduling instructions are executed, deduplication verification is automatically performed to avoid repeated execution of invalid calculation tasks. During the scheduling process, the system automatically tracks the task execution status, which includes four categories: running, successful, failed, and timed out. For tasks that fail or time out, the system automatically performs retry operations according to preset rules. The number of retries and the interval are determined according to the model's business type. Tasks that still fail after retrying are automatically marked as abnormal, pushed to the platform administrator account, and recorded in the abnormal log. The entire execution process of all scheduling tasks is bound and stored with the model's traceability code, business scenario, input data, and output results to ensure that the entire scheduling link is traceable.

[0083] Based on the platform's unified evaluation standards for water conservancy project safety monitoring algorithm models, a full lifecycle performance evaluation system for all published models is established. The evaluation system defines fixed evaluation dimensions, quantitative indicators, evaluation cycles, data sources, and result application rules. The evaluation dimensions include five core dimensions: model functional adaptability, business accuracy, operational performance, stability, and security. The quantitative indicators include identification accuracy, recall rate, precision rate, false alarm rate, false negative rate, average response time per inference, maximum response time, task execution success rate, data compliance, and effectiveness of access control. All indicators have qualified thresholds, which are determined based on the business needs and industry standards for safety monitoring of Yugang Reservoir. The assessment is divided into two categories: regular routine assessment and event-triggered special assessment. Regular routine assessment is performed on a monthly, quarterly, or annual basis and includes all published models in the model library. Event-triggered special assessment is triggered immediately when a model experiences false alarms or missed alarms, excessive verification deviations, operational anomalies, or changes in business scenarios. The assessment data sources include historical model operation records, actual business scenario test data, early warning event ledgers, cross-validation anomaly records, and on-site handling feedback information to ensure that the assessment data is authentic and effective and comprehensively reflects the actual business operation effect of the model.

[0084] For each model's evaluation results, a quantitative score and grading system is implemented, categorizing models into four levels: Excellent, Qualified, Needs Optimization, and Unqualified. Evaluation results are simultaneously written into the model ledger and stored in conjunction with the model's traceability code. For models needing optimization or unqualified models, an automatic optimization and rectification process is triggered, simultaneously suspending the model's business scheduling permissions until it passes re-evaluation after optimization. The evaluation results also serve as the core basis for model iteration optimization and version updates, ensuring the model continuously adapts to the needs of reservoir safety monitoring. Based on model performance evaluation results, changes in business scenarios, cross-validation of abnormal data, and post-event analysis conclusions, rules for the full lifecycle iterative optimization management of the algorithm model are determined. These rules define the triggering conditions, execution process, testing standards, version management, and redistribution requirements for model iteration optimization. Triggering conditions include a model evaluation result of Needs Optimization or Unqualified, new control requirements arising in on-site business scenarios, persistent false alarms and missed alarms in the model, persistent algorithmic deviations discovered during cross-validation, and adaptation adjustments required by updated water conservancy industry standards.

[0085] For models that trigger iterative optimization, the optimization requirements and objectives are first identified. These objectives include specific, quantifiable metrics such as improved accuracy, reduced false positive rate, improved response speed, addition of new recognition targets, and adaptation to new business scenarios. Then, based on the optimization objectives, corresponding on-site sample data for the relevant scenarios is supplemented, and sample annotation, preprocessing, and data augmentation are completed. Incremental model training, network architecture adjustments, hyperparameter optimization, and inference performance pruning are then performed. Throughout the optimization process, the parameters adjusted, training results, and performance changes are recorded simultaneously to ensure traceability. Once the optimized model is completed, a standardized full-process verification and on-site business testing, consistent with new model registration, are performed. After successful testing, a version update is completed, a new version number and updated traceability code are assigned, and the model is incorporated into the model library for unified management. Simultaneously, historical version models are retained, and a version rollback mechanism is set up. When a new version model experiences operational anomalies, it can be immediately switched back to a stable historical version to ensure continuous business operation.

[0086] For the models that have been iteratively optimized, the accompanying documentation, input / output specifications, and scheduling rules are updated simultaneously. The models are also re-bound and adapted to the corresponding business scenarios, monitoring points, and sensor nodes. All operation records, test reports, and version update notes throughout the entire process are bound to the model traceability code and archived permanently in the platform's dedicated database. At the same time, based on the results of the model iterative optimization, the platform's model evaluation standards and scheduling rules are updated simultaneously to continuously improve the adaptability and standardization of the platform's model management system, ensuring that all algorithm models continuously and stably meet the business needs of safety monitoring of the Yugang Reservoir Dam in Kedong County.

[0087] This application also provides an embodiment of an electronic device. The electronic device is manifested in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: one or more processors or processing units, memory, and buses connecting different components (including memory and processing units).

[0088] A bus refers to one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. Examples of these architectures include, but are not limited to, Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses.

[0089] Electronic devices typically include a variety of computer-readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, and removable and non-removable media.

[0090] The memory may include computer-readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. Electronic devices may further include other removable / non-removable, volatile / non-volatile computer device storage media. By way of example only, the storage system may be used to read and write non-removable, non-volatile magnetic media.

[0091] The electronic device can also communicate with one or more external devices (e.g., keyboard, pointing device, camera, etc.), may include a display, and may communicate with one or more devices that enable a user to interact with the electronic device, and / or with any device that enables the electronic device to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via an input / output (I / O) interface. Furthermore, the electronic device can also communicate with one or more networks (e.g., Local Area Network (LAN), Wide Area Network (WAN) and / or public networks, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. The processor executes various functional applications and data processing by running programs stored in memory, such as implementing the image recognition-based intelligent monitoring system for reservoir dam safety provided in the above embodiments of the present invention.

Claims

1. A reservoir dam safety intelligent monitoring system based on image recognition, characterized in that, include: The data acquisition module is used to continuously acquire video images of the dam body, water surface, gatehouse, and perimeter area at fixed intervals, and simultaneously acquire time-series data from water level, seepage, and displacement sensors. The model training module is used to train specialized AI algorithm models; The real-time inference module is used to perform real-time inference on the acquired video stream based on a pre-trained specialized AI algorithm model, and to automatically detect open smoke and flames, water level readings, gate status, violations, and floating objects on the water surface. The data structuring module is used to simultaneously complete the structuring transformation of unstructured image data and output standardized recognition results; The data fusion and verification module is used to perform source fusion and cross-verification of image recognition results with sensor monitoring data and engineering space data; The risk assessment module is used to perform quantitative analysis of water and rainfall trends and dam safety status through preset algorithms, and output accurate risk assessment results. The early warning triggering module is used to trigger early warnings of the corresponding level based on the analysis results; The message push module is used to push messages to administrators through multiple channels; The event handling module is used to synchronously record the entire process of early warning events, track the handling progress, and form a closed business loop; The model management module is used to perform unified registration, scheduling, performance evaluation, and iterative optimization of all algorithm models.

2. The intelligent monitoring system for reservoir dam safety based on image recognition according to claim 1, characterized in that, The process involves continuously collecting video images of the dam, water surface, gate control room, and perimeter area using high-definition cameras throughout the reservoir at fixed intervals, while simultaneously collecting time-series data from water level, seepage, and displacement sensors. This includes: determining the deployment rules for high-definition network cameras at key control points throughout the reservoir based on the reservoir safety monitoring business scenario and algorithm model input requirements; continuously collecting video images of the control area and capturing key frames; simultaneously accessing drone patrol images; completing standardized processing of video images and recording traceability information; and simultaneously collecting time-series data from various safety monitoring sensors of the reservoir according to a time benchmark aligned with video acquisition, completing initial data screening, standardized processing, and invalid data removal.

3. The intelligent monitoring system for reservoir dam safety based on image recognition according to claim 1, characterized in that, The training of the specialized AI algorithm model includes: determining the construction rules for the specialized algorithm dataset based on the business scenario of reservoir dam safety monitoring and the AI ​​algorithm recognition target; completing the collection, classification, labeling, and balanced partitioning of multi-scenario sample data; performing standardized preprocessing and data augmentation on labeled samples, removing invalid samples, and ensuring that the samples are compatible with the model input standard; matching and optimizing the appropriate deep learning network architecture for each recognition task, and completing iterative training and incremental optimization of the model; conducting multi-dimensional performance testing and on-site testing on the trained model, and after passing the tests, completing the model format solidification, specification formulation, and registration.

4. The intelligent monitoring system for reservoir dam safety based on image recognition according to claim 1, characterized in that, Based on pre-trained specialized AI algorithm models, the system performs real-time inference on the collected video streams, automatically detecting open smoke and flames, water level readings, gate status, violations, and floating objects on the water surface. This includes: matching specialized algorithm models for corresponding scenarios to the video streams of each monitoring point in the reservoir, achieving full coverage of business scenarios in key control areas; performing frame-by-frame standardized preprocessing on the real-time collected video streams to ensure that the input conforms to model specifications; calling corresponding specialized models for different scenarios to complete real-time inference; identifying and marking various abnormal events; simultaneously completing the structured transformation of inference results; recording traceable information across the entire chain; allocating computing resources according to business priorities; and pushing abnormal events to the platform's early warning panel.

5. The intelligent monitoring system for reservoir dam safety based on image recognition according to claim 1, characterized in that, The process of simultaneously converting unstructured image data into structured data and outputting standardized recognition results includes: formulating standardized conversion rules for various unstructured image reasoning results based on the structured data standard for water conservancy project safety monitoring; determining unified structured data fields; performing format regularization and numerical standardization processing on the original reasoning results output by the special AI algorithm model; removing redundant and invalid data; conducting compliance verification on the converted structured data to ensure that the data traceability relationship is uniquely bound; classifying and collecting the verified data into the database and establishing an associated index; generating a unique traceability code; and specifically marking abnormal event data in the standardized recognition results and simultaneously pushing them to the platform's early warning panel.

6. The intelligent monitoring system for reservoir dam safety based on image recognition according to claim 1, characterized in that, The process of fusing and cross-validating image recognition results with sensor monitoring data and engineering spatial data includes: relying on the platform's unified spatiotemporal reference system to achieve precise alignment of the time dimension of image recognition results, sensor monitoring data, and engineering spatial data; based on the full-domain spatial coordinate system constructed by the engineering BIM model and WebGIS platform to achieve one-to-one correspondence of the spatial dimensions of the three types of data; completing the fusing and cross-validation of the three types of data according to the business scenario of reservoir dam safety monitoring; formulating cross-validation rules for different business scenarios; conducting cross-validation on the fused dataset; triggering source tracing and verification for abnormal data and classifying and handling it; and updating the compliant fused dataset that has passed the verification into the database and associating it with the engineering spatial model.

7. The intelligent monitoring system for reservoir dam safety based on image recognition according to claim 1, characterized in that, The process involves using a pre-defined algorithm to perform quantitative analysis of water and rainfall trends and dam safety status, outputting accurate risk assessment results. This includes: based on a unified spatiotemporal benchmark, using compliant multi-source datasets that have undergone source fusion and cross-validation as the core data, constructing an adaptive algorithm system to complete multi-dimensional trend prediction of water and rainfall and flood discharge capacity matching analysis; building a multi-dimensional quantitative analysis index system for dam safety status; completing the calculation of multi-dimensional safety characteristics of the dam body and the overall safety level assessment; linking the two types of analysis results to carry out comprehensive risk assessment and source tracing analysis; ensuring full-process traceability; and outputting standardized risk assessment results and pushing them to the corresponding business modules.

8. The intelligent monitoring system for reservoir dam safety based on image recognition according to claim 1, characterized in that, The process of triggering corresponding warning levels based on the assessment results includes: establishing a four-level warning classification system that matches the comprehensive risk level of reservoir operation; determining the triggering conditions, control requirements, push scope, and handling procedures for each level; ensuring that the triggering rules are deeply matched with business scenarios; performing pre-compliance verification on the standardized results output by the risk assessment module; eliminating invalid assessment results to avoid false warning triggers; executing differentiated warning triggering logic based on the risk type of different business scenarios; generating a globally unique traceability code for the event after the warning is triggered; completing archiving and visual positioning by associating information across the entire chain; deduplicating and merging duplicate warnings; dynamically adjusting the warning level based on real-time data and assessment results; ensuring traceability throughout the entire lifecycle of the warning; and simultaneously pushing standardized warning information to the corresponding business modules.

9. The intelligent monitoring system for reservoir dam safety based on image recognition according to claim 1, characterized in that, The multi-channel push notifications to management personnel include: determining the scope of personnel, channels, frequency, and content specifications for different warning levels based on the platform's four-level warning classification system; all push recipients are accounts that have completed real-name authentication and authorization within the platform; matching differentiated push trigger modes and execution rules according to the warning level; covering multiple compliant push channels; setting up retry for failed pushes and switching to backup channels; tracking the status of warning reception and confirmation; automatically triggering a second push and simultaneously notifying the superior if no confirmation is received within the time limit; using encrypted transmission throughout the entire chain; and implementing hierarchical control and display of confidential data.

10. The intelligent monitoring system for reservoir dam safety based on image recognition according to claim 1, characterized in that, The synchronous recording and tracking of the entire process of early warning events to form a closed-loop business system includes: establishing standardized ledger management rules for the entire lifecycle of early warning events; assigning globally unique traceability codes to early warning events; collecting information across the entire chain to ensure traceability; matching corresponding standardized handling process templates based on early warning levels; determining the rights and responsibilities of each node in the entire process; automatically dispatching early warning events to the corresponding responsible personnel; tracking and recording the handling progress and status throughout the entire process; standardizing the closed-loop process for handling review and acceptance; conducting post-mortem analysis on closed-loop events and pushing optimization suggestions; and setting up full-process access control and tamper-proof operation logs to meet the traceability verification, audit supervision, and compliance management requirements for water conservancy project safety monitoring event management.

Images