Water conservancy project safety monitoring intelligent edge computing terminal, system and method integrated with AI recognition

Abstract

The application discloses a kind of integrated AI identification's water conservancy project safety monitoring intelligent edge computing terminal, system and method, terminal includes acquisition interface unit, network communication unit, data storage unit, IOT middleware and central processing unit;Central processing unit includes data analysis module, AI identification module and alarm generation module;Data analysis module is checked to structured data after pre-processing time sequence integrity, logic rationality and multi-source consistency;AI identification module dynamically schedules NPU, GPU resource and runs lightweight AI video identification model, realizes target all-weather detection and target identification;When data analysis module judges that sensor monitoring data is wrong or abnormal, then trigger AI identification module to review verification, and according to the check determination result, start alarm generation module.The application is verified by multi-source data association and hierarchical multi-modal early warning algorithm, to realize the more accurate abnormal identification and risk assessment of water conservancy project safety.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of water conservancy safety monitoring, and particularly relates to a water conservancy engineering safety monitoring intelligent edge computing terminal integrated with AI recognition, a system and a method. BACKGROUND

[0002] With the deepening of the digital transformation of the water conservancy industry, a dense safety monitoring and video monitoring network has been constructed in major water areas, and real-time collection and supervision of multi-source water conservancy elements have been realized. The current mainstream system is based on a cloud computing architecture: monitoring devices transmit raw data to the cloud for centralized processing through independent channels, and then instructions are issued. Although this mode is good at processing large-scale data, with the explosive growth of Internet of Things data, cloud computing faces problems such as high transmission delay and large bandwidth pressure, and it is difficult to meet the demand for high real-time monitoring and early warning. Edge computing technology provides an optimization path. By deploying edge computing terminals with network, computing and storage capabilities near the data source, local data collection, real-time processing and intelligent analysis can be realized. The terminal supports flexible configuration and dynamic update of processing strategies, rules and algorithms, adapts to various scenarios, effectively reduces the load of the cloud, improves the real-time response capability, and provides key technical support for the real-time and reliability of water conservancy safety monitoring.

[0003] However, the existing system still has many defects in the data collection, transmission and processing process. First of all, the poor compatibility of multi-source heterogeneous sensor devices is the primary constraint factor. During the integration process of the monitoring system in previous automation upgrades, different brands and specifications of sensor interface standards are not the same and the transmission protocols are not compatible with each other; this makes it necessary to deploy multiple independent data collection and transmission systems and corresponding platform servers for the same water conservancy project, which not only greatly increases the complexity of operation and maintenance, but also seriously hinders the standardization of multi-source monitoring data fusion and collaborative analysis. Secondly, the cloud centralized processing mode faces serious challenges such as bandwidth pressure, response delay and privacy security. Although cloud computing has the advantages of centralized storage and batch processing of massive data, the exponential growth of the number of Internet of Things monitoring terminals puts the cloud system under great bandwidth resource pressure and computing delay. During critical periods such as the flood season, data transmission congestion often leads to delays in real-time early warning response, and the large-scale uploading of raw data to the cloud also poses a significant information security risk. In addition, the lack of edge computing and intelligent analysis capabilities in local data collection devices constitutes another key constraint. Most existing terminals only have raw data collection and uploading functions, and lack local computing capabilities. The architecture design results in the complete dependence of cloud processing for key functions such as abnormal data identification and missing value measurement. When the communication conditions are limited, the system is difficult to complete data quality verification and emergency warning in a timely manner, which seriously damages the timeliness and reliability of monitoring.

[0004] Therefore, how to provide an edge computing terminal capable of on-site and timely processing, intelligent identification and abnormal alarm of multi-source heterogeneous data is one of the technical problems to be solved by those skilled in the art. SUMMARY

[0005] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide an integrated AI identification water conservancy engineering safety monitoring intelligent edge computing terminal, system and method. Through multi-source data correlation verification and hierarchical multi-modal early warning algorithm, more accurate abnormal identification and risk assessment of water conservancy engineering safety are realized.

[0006] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0007] In the first aspect, the present application provides an integrated AI identification water conservancy engineering safety monitoring intelligent edge computing terminal, which adopts a multi-core heterogeneous architecture design and includes a data acquisition interface unit, a network communication unit, a data storage unit, an IOT middleware and a central processing unit.

[0008] The data acquisition interface unit integrates multiple data acquisition interfaces and supports multi-protocol adaptive matching, and is used to acquire monitoring data of external monitoring sensor devices;

[0009] The network communication unit supports multi-mode redundant transmission and is used to transmit the collected monitoring data to the data storage unit;

[0010] The data storage unit is used for encrypted storage of raw monitoring data and result data and temporary data caching;

[0011] The IOT middleware is used for protocol conversion, data encryption and data routing of raw data;

[0012] The central processing unit is used for dynamically scheduling data analysis, AI identification and real-time alarm generation tasks; the central processing unit comprises a data analysis module, an AI identification module and an alarm generation module; the data analysis module adopts a streaming computing engine to perform time sequence integrity verification, logical reasonableness verification and multi-source consistency verification on the preprocessed structured data; the AI identification module comprises a semantic segmentation model, an AI video identification model and a fusion decision module, the AI identification module receives an input image and inputs the input image into the semantic segmentation model and the AI video identification model; the semantic segmentation model performs water area segmentation on the input image and outputs a water area mask; the AI video identification model detects a target in the input image and outputs a target bounding box list; the fusion decision module combines the water area mask and the target bounding box list, and outputs an early warning result through a spatial relationship judgment process of water area existence judgment, center point inclusion judgment, intersection ratio calculation and early warning judgment; the alarm generation module starts a hierarchical early warning according to a data verification judgment result: data error triggers a device-level early warning, data anomaly but not exceeding a threshold value triggers a system-level early warning, and data exceeding a safety threshold value triggers an emergency-level early warning; a monitoring data and video monitoring linkage verification mechanism is adopted for early warning, when the data analysis module judges that the sensor monitoring data is incorrect or abnormal, the AI identification module is triggered for rechecking and verification, forming a double verification mechanism, and the alarm generation module is started according to the verification judgment result.

[0013] As a preferred technical solution, the edge computing terminal adopts a hierarchical pipeline type service logic processing mechanism, specifically as follows:

[0014] In the data uplink channel, the data acquisition interface unit obtains the original data of various monitoring sensors in real time through the multi-modal interface, performs preliminary filtering and time scale alignment through the hardware accelerated preprocessing unit, and then performs multi-threaded protocol analysis and data standardization processing by the IOT middleware, the processed structured data is written into the encrypted storage area in parallel through the DMA channel, and is pushed to the streaming computing engine of the central processing unit for real-time analysis at the same time;

[0015] In the data downlink channel, the network communication unit receives cloud instructions, verifies the digital signature by the hardware security area, and after the data storage unit, the standard protocol instructions are directly executed by the central processing unit, and the non-standard protocol instructions need to be processed by the protocol conversion engine of the IOT middleware; the whole link tracking is implemented in the instruction execution process, and the execution result is fed back to the cloud operation and maintenance platform together with the system health status through the heartbeat mechanism;

[0016] As a preferred technical solution, the data analysis module adopts a data quality control algorithm including time sequence integrity verification, logical reasonableness verification and multi-source consistency verification, to perform multi-level quality verification on the real-time collected monitoring data.

[0017] As a preferred technical solution, the semantic segmentation model adopts a DeepLabv3+ model, which is used for water area region segmentation of an input image and outputs a water area mask.

[0018] The AI video recognition model adopts an improved YOLOv5s model, which is used for target detection of an input image and outputs bounding box information of a task target; the AI video recognition model embeds an ECA+CBAM combined attention module in the last C3 layer of a backbone network and the first C3 layer of a neck network, respectively; the ECA+CBAM combined attention module in the backbone network strengthens key channel information of a target from original features and establishes a cross-scale correlation relationship; the ECA+CBAM combined attention module in the neck network further optimizes multi-scale feature fusion effect to enhance the representation ability of small target features; the CARAFE content-aware resampling technology is used in the first upsampling layer of the neck network to enhance the receptive field of a feature map.

[0019] The fusion decision module is used for spatial relationship judgment of the target bounding box and the water area mask, and outputs an early warning result; the spatial relationship judgment includes water area existence judgment, center point inclusion judgment, intersection ratio calculation and early warning judgment.

[0020] As a preferred technical solution, the ECA+CBAM combined attention module includes a channel attention layer and a spatial attention layer.

[0021] The channel attention layer performs average pooling and maximum pooling operations on the input features to aggregate feature information; after extracting the features through a fully connected layer, a Sigmoid activation function is used to generate a first weight value W1 of a convolution layer, and a first feature X o1 is outputted; the calculation formula of the first feature X o1 is as follows:

[0022] ;

[0023] Wherein, represents the input feature, represents the first weight of the convolution layer, and B1 represents the first bias of the convolution layer.

[0024] The spatial attention layer performs average pooling and maximum pooling operations on the first feature X o1 outputted by the channel attention layer to aggregate feature information; after extracting the features through multiple convolution layers, a Sigmoid activation function is used to generate a second weight value W2 of a convolution layer, and a second feature X o2 is outputted after weighted fusion with the first feature X o1 ; the calculation formula of the second feature X o2 is as follows:

[0025] ;

[0026] wherein, represents the second weight of the convolution layer, and B2 represents the second bias of the convolution layer;

[0027] The CARAFE content-aware resampling technology comprises an up-sampling prediction module and a content-aware reorganization module.

[0028] As a preferred technical solution, the water area existence determination is to calculate the total number of pixels with a pixel value of 1 in the water area mask, and if the total number is greater than 0, it is determined that there is a water area;

[0029] The center point containment determination is to calculate the geometric center of the target bounding box and map it to the discrete pixel index of the water area mask, and if the pixel value corresponding to the index is 1, it is determined that the target center point is within the water area range;

[0030] The intersection ratio calculation is the ratio of the intersection area of the target bounding box and the water area mask to the area of the target bounding box;

[0031] The early warning determination is to output an early warning signal when the target category belongs to the dangerous target category set, and the intersection ratio is greater than or equal to the preset spatial threshold or the target center point is within the water area range.

[0032] As a preferred technical solution, the improved YOLOv5s model is subjected to sparse training, channel pruning and model fine-tuning for lightweight processing, specifically:

[0033] The original weights of the convolution layer and the BN layer in the improved YOLOv5s model are subjected to sparse training using L1 regularization training;

[0034] The parameters on the BN layer are As a channel pruning determination factor of the model structure, part of the scale factors is forced to approach 0 through L1 regularization, so as to identify and reduce the corresponding channels;

[0035] The model fine-tuning restores the model accuracy by iteratively training the pruned model.

[0036] As a preferred technical solution, the AI recognition module adopts a multi-model fixed polling mechanism, coordinates the running of multiple lightweight video recognition task models through a risk adaptability scheduling strategy, and performs AI image recognition on images or video frames collected by multiple video channels; the multi-model fixed polling mechanism sets different monitoring intervals according to target behavior characteristics and risk levels, uses a short interval for high-frequency dynamic change events, uses a medium interval for medium persistence events, and uses a long interval for low-frequency long-time events; through the polling mechanism, all-around and all-weather intelligent monitoring of complex water area scenes of water conservancy projects is realized.

[0037] As a preferred technical solution, the linkage verification mechanism of the monitoring data and the video monitoring is specifically:

[0038] Time synchronization mechanism: missing and deviation of time stamps are processed through interpolation, calibration and dynamic synchronization to realize time synchronization of monitoring data and video monitoring;

[0039] Space correlation modeling: a space mapping database of sensors and cameras is established, and a GIS coordinate system is used to record device location information, so that sensor data can be associated with corresponding video monitoring areas;

[0040] Intelligent review process: after confirming that the data is wrong or abnormal, double verification is performed, T-second video clips before and after the abnormal period of sensor data are automatically intercepted, and target detection, recognition and comparison are performed through the AI recognition module;

[0041] Cloud collaborative analysis: according to the verification and judgment result, the alarm generation module is started, and the key data is encrypted and uploaded to the cloud.

[0042] As a preferred technical solution, in the central processing unit, normal data is compressed and then enters the waiting upload queue; data errors will immediately trigger automatic re-measurement, interpolation compensation and original data encryption backup operations; data abnormalities will automatically adjust the sampling frequency and perform encrypted data priority upload; all abnormal processing operations will be recorded completely, including the time, type, processing measures taken and detailed information of the final judgment result, forming a complete quality control log.

[0043] In a second aspect, the present application provides a water conservancy project safety monitoring on-site early warning system, comprising:

[0044] The edge computing terminal;

[0045] The multi-source heterogeneous monitoring equipment connected to the edge computing terminal, wherein the multi-source heterogeneous monitoring equipment comprises one or more of a osmometer, a weir meter, a rain gauge, a water level meter, a flow meter, a displacement monitoring device, a pressure gauge and a video camera;

[0046] Linkage verification unit: when the sensor data is wrong or abnormal, the associated camera video segment is automatically intercepted for AI video review;

[0047] Hierarchical early warning feedback unit: trigger different levels of three-level early warning according to monitoring data and video recognition verification results, three-level early warning includes: device level early warning, system level early warning and emergency level early warning.

[0048] As a preferred technical solution, the trigger conditions of the three-level early warning include:

[0049] Device level early warning: data is wrong;

[0050] System level early warning: data deviates from historical rules but does not exceed the set threshold;

[0051] Emergency level early warning: data continues to exceed the threshold and affects the engineering function.

[0052] In the third aspect, the application provides a safety monitoring method of an intelligent edge computing terminal for water conservancy project safety monitoring integrated with AI recognition, comprising the following steps:

[0053] Adapt the protocol of multi-source heterogeneous sensors dynamically through the edge computing terminal, and collect monitoring data and video streams in real time;

[0054] Perform data quality control on the edge side: execute in the order of time sequence integrity verification, logical reasonableness verification and multi-source consistency verification;

[0055] Call the AI recognition module to detect and identify floating objects, weeds, water level readings and fishing, swimming human behavior targets;

[0056] Verify the sensor data and video recognition results, and start hierarchical early warning according to the determination results.

[0057] Compared with the prior art, the application has the following advantages and beneficial effects:

[0058] 1. The application replaces the traditional distributed architecture with an intensive edge computing terminal, effectively solving the problems of multi-source heterogeneous sensor device redundancy and protocol island. The terminal supports plug and play of multiple sensors and dynamic switching of protocols, significantly reducing hardware deployment complexity, improving system compatibility and scalability, and laying a foundation for monitoring data standardization and fusion.

[0059] 2. The application realizes localized data quality control and video intelligent analysis based on edge computing, greatly reducing the need for remote transmission of raw data. Through real-time data processing and target recognition on the edge side, the high dependence of traditional systems on network bandwidth is effectively overcome, the response delay is significantly reduced, and the timeliness and reliability of key monitoring data are ensured.

[0060] 3、The application breaks through the limitation of traditional single-point static threshold early warning, realizes more accurate abnormal identification and risk assessment through multi-source data correlation verification and hierarchical multi-modal early warning algorithm. The edge terminal completes the closed-loop control of data cleaning, intelligent diagnosis and early warning feedback, reduces the cloud load, improves the early warning accuracy and response speed, and forms a more intelligent and autonomous monitoring and early warning system. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0062] Figure 1 The data acquisition and processing flowchart of the water conservancy project safety monitoring intelligent edge computing terminal integrated with AI recognition of the embodiments of the present application is shown in the figure.

[0063] Figure 2 The data quality control and video linkage verification flowchart of the embodiments of the present application is shown in the figure.

[0064] Figure 3 The improved yolov5s network architecture diagram of the embodiments of the present application is shown in the figure.

[0065] Figure 4 The hierarchical multi-modal early warning flowchart of the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0066] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0067] In the present application, the phrase "embodiment" means that the specific features, structures or characteristics described in conjunction with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described in the present application can be combined with other embodiments.

[0068] The embodiment provides a water conservancy project safety monitoring intelligent edge computing terminal integrated with AI recognition, which is suitable for water conservancy project safety monitoring multi-source data collection, data on-site analysis and early warning. The edge computing terminal integrates a multi-source monitoring data collection and transmission module, an AI-driven data processing and analysis module, an adaptive unit supporting multiple communication interfaces and networks and a dynamic protocol library, realizes edge data inspection and instant repair through a built-in data quality control algorithm, realizes target detection and recognition by introducing a lightweight AI recognition model, and realizes on-site abnormal alarm and early warning in combination with an audible and visual early warning device. The system realizes water conservancy project safety monitoring multi-source heterogeneous data collection, on-site instant verification, intelligent recognition and abnormal alarm based on edge computing, and realizes dynamic updating of a protocol library and optimization and upgrading of an algorithm model in cloud-edge cooperation, thereby significantly improving the efficiency of water conservancy project operation management and the intelligent level of safety monitoring.

[0069] The water conservancy project safety monitoring intelligent edge computing terminal integrated with AI recognition in the embodiment is constructed with the Rockchip RK3588 as a core. The chip adopts a multi-core heterogeneous architecture (4xARM Cortex-A76 and 4xARM Cortex-A55) 8nm advanced process, and is built-in with an NPU processing unit with a computing power of 6TOPS, which can efficiently support AI model deployment at the edge, significantly reduce the dependence on cloud computing power, and effectively solve the problem that real-time detection cannot be performed due to unstable network in a water conservancy scene.

[0070] The embedded software of the intelligent edge computing terminal intelligently calls a processor core according to a task type, realizes dynamic load scheduling, and the like. For example, a small core is called for a low-power-consumption-demand task such as sensor data collection, device state monitoring and communication protocol stack processing; a large core, a GPU and an NPU are cooperatively accelerated for a high-computing-density task such as AI intelligent recognition and complex algorithm operation. Through this mechanism, the terminal can simultaneously process multiple different types of workloads, realizes parallel operation of the CPU, the GPU and the NPU, respectively processes control logic and AI intelligent recognition tasks, and truly realizes multi-task parallel processing.

[0071] In addition, the terminal is equipped with LPDDR4X memory, supports a data transmission rate of up to 4266Mbps, meets the multi-source heterogeneous data throughput demand under multi-task parallelism, and has lightning protection circuit protection level of up to 6KV and wide-range DCDC power supply circuit design, thereby guaranteeing stable operation in a complex power supply environment of water conservancy.

[0072] Meanwhile, the edge computing terminal supports expansion of a high-computing-power card with a computing power of up to 26 TOPS, so that the maximum computing power of the terminal can reach 32 TOPS, and sufficient computing power support is provided for front-end data collection, data processing and edge intelligent analysis.

[0073] For example, Figure 1As shown, the edge computing terminal includes a data acquisition interface unit, a network communication unit, a data storage unit, an IOT middleware, a central processing unit, and various functional units interconnected through a high-speed internal bus.

[0074] The data acquisition interface unit integrates RS485, RS232, USB, switching value control, analog quantity acquisition, and other industrial standard interfaces, supports multi-protocol adaptation such as Modbus / Profinet, and is responsible for data interaction communication.

[0075] The network communication unit supports multi-mode redundant transmission such as Ethernet, WIFI, 4G / 5G, and is responsible for data interaction transmission.

[0076] The data storage unit is responsible for encrypted storage of raw data and result data and temporary data caching.

[0077] The IOT middleware implements functions such as protocol conversion, data encryption, and data routing.

[0078] The central processing unit adopts a micro-service architecture and dynamically schedules tasks such as data analysis, AI recognition, and real-time alarm generation; the central processing unit includes a data analysis module, an AI recognition module, and an alarm generation module; the data analysis module uses a streaming computing engine to perform time series integrity verification, logical reasonableness verification, and multi-source consistency verification on preprocessed structured data; the AI recognition module dynamically schedules NPU and GPU resources to run lightweight AI video recognition models, enabling all-weather detection and recognition of floating objects, weeds, water gauge readings, and fishing and swimming behaviors; the alarm generation module initiates a hierarchical warning based on data verification determination results: data errors trigger device-level warnings, data anomalies but not exceeding thresholds trigger system-level warnings, and data exceeding safety thresholds trigger emergency-level warnings; a monitoring data and video monitoring linkage verification mechanism is used for alarm warning, and when the data analysis module determines that sensor monitoring data is incorrect or abnormal, the AI recognition module is triggered for rechecking and verification, forming a double verification mechanism, and the alarm generation module is started according to the verification determination result.

[0079] In addition, the edge computing terminal also includes a system interface module and a power supply module.

[0080] The system interface module provides standardized APIs for external systems to access data or control devices.

[0081] The power supply module is connected to an external power source to supply power to the edge computing terminal or monitoring devices connected to the edge computing terminal by wire.

[0082] Further, the edge computing terminal adopts a layered pipeline business logic processing mechanism, including processing of uplink and downlink channels:

[0083] In the data uplink channel, the data acquisition interface unit obtains various types of sensor raw data in real time through the multi-modal interface, and after preliminary filtering and time scale alignment by the hardware accelerated preprocessing unit, the IOT middleware implements multi-threaded protocol analysis and data standardization processing. The processed structured data is written into the encrypted storage area through the DMA channel in parallel, and is pushed to the streaming computing engine of the central processing module for real-time analysis.

[0084] In the data downlink channel, the cloud instructions are received through the dual-channel redundant network interface, verified by the hardware security module (HSM), and after the data storage unit, the standard protocol instructions are directly executed by the central processor, and the non-standard protocol instructions need to be processed through the protocol conversion engine of the middleware. The execution process implements full-link tracking, and the execution results are fed back to the cloud operation and maintenance platform through the heartbeat mechanism together with the system health status.

[0085] Further, the central processing unit relies on the micro-service architecture to dynamically schedule computing resources such as CPU / GPU / NPU, and to perform monitoring data quality inspection, AI video recognition, and linkage verification of monitoring data and video monitoring. Normal data is compressed and entered into the upload queue, data errors trigger data repair and original data encryption backup operations, and data abnormalities automatically adjust the sampling frequency and perform emergency upload of encrypted data.

[0086] Further, the edge computing terminal integrates multiple communication interfaces and multiple network access methods, adopts a hybrid protocol architecture, and through hardware acceleration modules and intelligent network control technology, builds a multi-level protocol compatible system that can cover Modbus RTU / TCP, MQTT, LoRaWAN, SL 651, RTSP, ONVIF, GB / T 28181, and other commonly used protocols in the water conservancy industry, standard video transmission protocols, and custom communication protocols. The terminal can flexibly access heterogeneous devices according to the water conservancy safety monitoring requirements, adapt to different scales and complexity of application scenarios, improve the compatibility and expandability of data acquisition, while maintaining the simplicity and maintainability of business logic.

[0087] The protocol processing engine of the terminal adopts a microkernel plug-in design, i.e., a modular architecture design, which realizes flexible expansion through dynamic loading of protocol analysis components, can stably support multiple devices to achieve plug-and-play, and ensures high compatibility for multiple protocols. In terms of data processing flow, hardware acceleration capabilities are fully utilized to improve processing efficiency and reduce latency.

[0088] The uplink data processing is firstly transmitted by the data acquisition module to the preprocessing unit through the high-speed DMA channel, and the key operations such as protocol feature matching, data frame check sum and timestamp alignment are quickly completed by the special NPU chip. After entering the middleware layer, the protocol conversion and data standardization processing are performed, the data encryption and decryption operations are realized by using the built-in security chip to ensure data security. When processing the downlink instructions, the central processing unit will strictly check the operation permission in the trusted execution environment, and complete the protocol reverse packaging and transmission encryption and other key steps.

[0089] In order to maintain the timeliness and flexibility of the protocol, the system adopts a cloud-edge collaborative management mode, which can update the protocol library remotely through the cloud, and can cache commonly used protocols locally to deal with network interruption. In terms of security protection, the system constructs a comprehensive defense system through protocol plug-in running isolation, data encryption key management and transmission link encryption protection to ensure system security and reliability. This design enables the terminal to efficiently process various protocol data and flexibly adapt to changes in demand in different application scenarios.

[0090] Further, the edge computing system builds a real-time data quality control system, forming a complete closed-loop management process from rule making to verification processing.

[0091] As shown in Figure 2 The data quality control algorithm deployed on the edge computing terminal side is responsible for multi-level quality verification of real-time monitoring data, mainly including three stages of time sequence integrity verification, logical reasonableness verification and multi-source consistency verification:

[0092] Time sequence integrity verification: The integrity of the monitoring data is checked according to the preset acquisition frequency, and the ratio of the actual acquired monitoring data quantity to the theoretical acquisition quantity is compared to determine whether the data is complete, and whether the data has abnormal interruption is also checked.

[0093] Logical reasonableness verification: The value range and change trend of the monitoring data are verified. The value range is judged based on the engineering characteristic parameters and historical monitoring data threshold (for example, the water level of the reservoir should be between the dead water level and the check flood level); the change trend is identified by analyzing the data change rate.

[0094] Multi-source consistency verification: The multi-source monitoring data with correlation are verified cooperatively, focusing on the space-time synchronization (such as the water level should not rise suddenly without rainfall). Considering the reasonable error in sensor measurement, the system sets a dynamic tolerance range (usually ±5% of the threshold), when the data is in a critical state, the AI video recognition is not triggered immediately, but a multi-sensor joint analysis mechanism is started, and the monitoring data of multiple sensors are cross-verified to comprehensively judge the data reliability.

[0095] When data errors or abnormalities are detected, corresponding hierarchical processing strategies will be taken: for data errors, automatic retesting, interpolation compensation, and video linkage verification will be triggered immediately; for data abnormalities, multi-sensor joint analysis mechanism and video linkage verification will be triggered. All abnormal processing operations will be recorded completely, including the time, type, processing measures taken, and final judgment results, etc. detailed information, forming a complete quality control log, facilitating subsequent system maintenance and problem tracing.

[0096] In a specific example: when the system detects that the water level rises rapidly in a short time (change rate exceeds 1 m / h) but has not reached the warning threshold, it will immediately start the multi-sensor joint analysis mechanism. First, it analyzes the field rain gauge data to confirm whether there is heavy rainfall; at the same time, it retrieves the osmometer monitoring data to check whether the dam body seepage pressure is abnormal synchronously; if necessary, it will also call video monitoring and use AI image recognition technology to visually review the water level scale. Only when the judgment conclusion after multiple verifications points to an abnormality, the system will finally determine it as a valid abnormal event, triggering the corresponding warning mechanism. This rigorous verification process maximizes the prevention of false positives and false negatives, ensuring the authenticity and reliability of the monitoring data.

[0097] Further, the AI recognition module supports multi-channel video and multi-scene intelligent recognition through the built-in AI video recognition model in the edge computing terminal, which can realize intelligent detection, recognition and warning of floating objects, weeds, water gauge readings, fishing and swimming behaviors, etc.

[0098] The AI video recognition model is based on the YOLO lightweight model, which directly predicts target bounding boxes and classes through a single-stage end-to-end network architecture, uses multi-scale feature fusion and dense prediction mechanism to achieve high-speed and high-precision detection, and its lightweight design is suitable for real-time video analysis. Through dynamic label assignment and loss function optimization, the small target detection performance is improved, with high recognition accuracy and inference speed, which can be deployed on edge computing terminals to immediately warn of abnormal information identified.

[0099] It can be understood that for different types of target object training task models, they are mainly divided into floating object, weed detection task model, water gauge detection task model, swimming and fishing detection task model.

[0100] For the recognition scene of such detection targets in water conservancy, which has a small proportion in the shooting scene and unclear edge features, ECA+CBAM combined attention modules and CARAFE up-sampling modules are introduced into the video recognition model structure to strengthen the feature extraction and detail restoration capabilities and improve the recognition accuracy. A multi-model fusion inference strategy is adopted to effectively solve the high false positive rate problem of traditional single modal methods by weighting fusion or NMS integration of model outputs in different training stages, and to improve the detection robustness of the model in complex scenes. At the same time, the model is optimized by sparse training and channel pruning to adapt to the power limit of the edge computing terminal. The model is deployed to the edge computing terminal through RKNN quantization acceleration to achieve the optimal balance among computing resources, running efficiency and recognition accuracy, and to meet the real-time detection demand of ≥25FPS under 1080P video stream of reservoir safety supervision.

[0101] Further, the AI recognition module includes a semantic segmentation model, an AI video recognition model, and a fusion decision module. The AI recognition module receives an input image and inputs it to the semantic segmentation model and the AI video recognition model. The semantic segmentation model performs water area segmentation on the input image and outputs a water mask. The AI video recognition model detects targets in the input image and outputs a target bounding box list. The fusion decision module combines the water mask and the target bounding box list, and outputs an early warning result through a spatial relationship judgment process including water existence judgment, center point inclusion judgment, intersection ratio calculation, and early warning judgment.

[0102] Further, the semantic segmentation model uses a DeepLabv3+ model, constructs a semantic segmentation dataset containing multiple forms of water areas such as ponds, lakes, rivers, and oceans, and accurately annotates the dataset. The dataset is also distinguished by water quality, lighting, and wave conditions. The annotation uses a binary mask, with water pixels being 1 and non-water pixels being 0, generating a high-precision water mask (Water Mask W) with dimensions of where represents a pixel belonging to a water area.

[0103] As shown in Figure 3 , the AI video recognition model uses an improved YOLOv5s model, which is trained and fine-tuned using an image dataset containing water area targets and non-water area targets as follows:

[0104] The AI video recognition model is based on a YOLOv5v7.0 network, and an ECA+CBAM combined attention module is embedded in the last C3 layer of the backbone network and the first C3 layer of the neck network, respectively. The ECA+CBAM module in the backbone network strengthens the key channel information of the target from the original features and establishes a cross-scale correlation relationship, and the ECA+CBAM module in the neck network further optimizes the multi-scale feature fusion effect, so that the representation ability of small target features is more prominent. In the first upsampling layer of the neck network, the CARAFE content-aware resampling technology is used, so that the model can adaptively generate an up-sampling kernel according to the local feature content, and the edge details and texture information can be better preserved while the feature map is enlarged.

[0105] Further, the ECA+CBAM combined attention module includes a channel attention layer and a spatial attention layer, and the output of the C3 in the model is taken as the input feature X i In the channel attention layer, the average pooling operation and the maximum pooling operation are used to aggregate feature information, and then a plurality of fully connected layers are used for feature extraction, a Sigmoid activation function is used to generate the first weight W1 of the convolution layer, and the first feature X o1 is output. o1 In the spatial attention layer, the average pooling operation and the maximum pooling operation are used to aggregate the first feature X o2 , and then a multi-layer convolution method is used for feature extraction, a Sigmoid activation function is used to generate a weighted spatial attention second weight W2, and finally the second feature X i is output. The calculation formula is as follows:

[0106] ;

[0107] ;

[0108] The channel attention layer is composed of a convolution layer, W1 represents the first weight of the convolution layer, B1 represents the second bias of the convolution layer, X i is the input feature, W2 represents the second weight of the convolution layer, and B2 represents the second bias of the convolution layer.

[0109] CARAFE includes an up-sampling prediction module and a content-aware reorganization module; wherein the up-sampling prediction module maps the features of the target image to CxHxW, after channel compression using a 1x1 convolution kernel with C2, the number of channels is redistributed for convolution encoding, then the pixel random shuffling is performed to expand the up-sampling receiving domain of the feature map, then the feature mapping is normalized to reduce the number of parameters in the operation, and finally the kernel normalization is performed. In the content-aware reorganization module, the feature maps obtained by the up-sampling prediction module and the nearest neighbor up-sampling are dot product processed, and the features are reorganized using the prediction kernel.

[0110] ;

[0111] ;

[0112] ;

[0113] where K up is the size of the recombination kernel; K encoder is the convolution kernel for encoding convolution; represents the kernel prediction module, is the k x k region centered at l of X; W o is the output of the prediction module; is the recombination module for perceiving content, Z o is the total output of the up-sampling model.

[0114] Further, the collected image is input in parallel to a semantic segmentation branch and a target detection branch, the semantic segmentation branch outputs a water area mask graph, the target detection branch outputs a target bounding box list, and then the spatial relationship between the target frame and the water area mask is judged in a fusion decision module, and the final early warning result is output.

[0115] Still further, the core decision algorithm of the fusion decision module includes the following processes:

[0116] ① Water area existence judgment; the water area existence flag function is as follows:

[0117] ;

[0118] If , it means that there is no water area in the image, and the water area target early warning is directly excluded. When , the next step is to determine the spatial relationship quantization algorithm.

[0119] Spatial relationship quantization algorithm determination. In the spatial relationship quantization algorithm, two mechanisms of center point inclusion judgment and intersection ratio calculation are used for fusion determination.

[0120] ② Center point inclusion judgment; the geometric center of the detection target bounding box bboxk is defined as follows:

[0121] ;

[0122] The coordinate is a continuous floating point number, and the water area mask W is a discrete matrix with a size of H x W, so it is necessary to map the continuous coordinate to the discrete pixel index. The specific mapping rule is:

[0123] ;

[0124] where denotes a floor operation to ensure that the index of the center point of the target detection falls within the valid range of the mask matrix, i.e. , indicates that the center point of the target detection is within the water area range.

[0125] ③ Intersection ratio calculation; area of target detection :

[0126] ;

[0127] The area of the intersection region of the target detection region and the water area segmentation is :

[0128] ;

[0129] The intersection ratio of the target frame and the water area :

[0130] ;

[0131] ④ Finally, through the logical complement of the center point judgment and the intersection ratio calculation, the final warning condition is determined through the cooperative warning decision logic mechanism:

[0132] ;

[0133] wherein is a set of dangerous target categories, and α is a spatial threshold value, taking a value of 0.4.

[0134] Further, in order to ensure the accuracy and speed of the video recognition model, the video recognition model deployed in the edge computing terminal is lightened, including sparse training, channel pruning and model fine-tuning, and the specific contents are as follows:

[0135] The YOLOv5s model is followed by a BN layer, and the BN layer is composed as follows:

[0136] ;

[0137] ;

[0138] wherein is the input channel number, is the output channel number, are the mean value and the standard deviation value of the BN activation, respectively, and and are the scales on the BN linear transformation, which are the weight parameter and the displacement parameter, respectively. The initial parameters of are 1 and 0, respectively, and then is adjusted, which is conducive to the distribution and mutual coordination of data.

[0139] The original weights of the convolutional layer and the BN layer in the YOLOv5s model are sparsely trained by using L1 regularization. The sparse mode generated by L1 regularization has channel consistency, that is, multiple filters in the same channel are sparsely synchronized, and all weights tend to 0. The all-zero or near-zero channel can be safely pruned, while the complete network structure is maintained, realizing channel-level sparsity and pruning.

[0140] L1 regularization is the sum of the absolute values of each weight, and the loss function of L1 regularization includes the loss function of the weight during training of the convolutional neural network and the loss function of the BN layer scale factor .

[0141] ;

[0142] ;

[0143] wherein is a regularization term used to control the complexity of the model and prevent overfitting; represents the summation of all parameters that need to be regularized, is the set of these parameters; is the regularization coefficient used to balance the weight between the data fitting term and the regularization term. A larger value will place more emphasis on the regularization term, thereby reducing the complexity of the model; a smaller value will focus more on data fitting.

[0144] The model is subjected to channel pruning processing, and the parameters on the BN layer are used as the channel pruning judgment factor of the model structure. By L1 regularization, some scale factors are forced to approach 0, so as to identify the prunable channels. For example, if the number of pruned channels is set to 50%, then the channels with a size smaller than the average will be pruned, thereby obtaining a more compact model with fewer parameters, faster running speed, and less memory occupation.

[0145] After channel pruning, the detection accuracy of the model is slightly lower than that of the original model, so the model is then fine-tuned by performing a small amount of iterative training on the pruned model to restore the model accuracy.

[0146] Further, the lightweight AI video recognition model is deployed to an edge computing terminal through domestic adaptation:

[0147] ①Convert the trained YOLOv5s model (.pt file) to ONNX intermediate format using the conversion interface recommended by Swiftest. Considering the wide field of view and relatively small target objects in water conservancy scenarios, a high-resolution input setting of 1920x1080 is used instead of the traditional 640x640 to improve small target recognition capability.

[0148] ②Use Swiftest RKNN-Toolkit2 tool to convert the optimized ONNX model to RKNN format specific to RK3588. Use INT8 asymmetric quantization method (asymmetric_quantized-8) to significantly reduce model size while ensuring accuracy. During the quantization process, representative water conservancy scenario datasets are used for calibration, dynamic range analysis is performed on the input and output of each layer, and a per-channel quantization strategy is used to improve the recognition accuracy of small targets (pixel area < 32x32).

[0149] ③In code implementation, use more efficient Numpy library for data processing. Use Numpy to implement image normalization, size adjustment and channel conversion to avoid the overhead of PyTorch tensor operations; for high-resolution input of 1920x1080, optimize memory access mode to reduce data copying; rewrite the original PyTorch-based post-processing of YOLOv5s (including non-maximum suppression NMS) to pure Numpy implementation, use vectorized operations to speed up the bounding box decoding and screening process; optimize the NMS threshold setting for the characteristics of sparse targets in water conservancy scenarios to reduce false positives.

[0150] Further, use the target tracking method based on video stream to send warning information, assign a unique ID value to the detected target, if the target moves less than 10 pixels in the previous and subsequent frames, it is considered that the target has not moved, and the target recognition will not be performed again. This method does not reduce resource consumption, but the recognition ability and repeated recognition ability are greatly improved.

[0151] Further, use the multi-model fixed polling mechanism to coordinate the operation of multiple lightweight video recognition task models through risk adaptability scheduling strategy, perform AI image recognition on the images or video frames collected by multiple video channels, and realize all-round and all-weather intelligent monitoring of complex water area scenarios in water conservancy projects.

[0152] The mechanism implements a fine time interval setting strategy according to the behavior characteristics and risk levels of different monitoring targets. Personnel behavior is instantaneous and unpredictable, so a short interval of 5 seconds is used to poll high-frequency dynamic changes in personnel activities, ensuring real-time capture of sudden behavior and key events. Fishing and swimming behaviors have a long duration and pose significant safety risks, so a medium interval of 5 minutes is set for such medium-high risk persistent behaviors to meet the need for timely intervention while avoiding resource waste caused by high-frequency detection. Natural growth of weeds or water level readings use a long interval of 2 hours, which meets the "minimum necessary frequency" principle and can significantly reduce system computing load. This hierarchical and time-based scheduling strategy precisely matches the monitoring frequency to the characteristics of the target, achieving an optimal balance between risk monitoring and resource efficiency.

[0153] Further, by establishing a model scheduling management module, the task queue is dynamically scheduled, and multiple task models are orderly organized for collaborative work, ensuring comprehensive coverage of various monitoring targets while avoiding resource competition problems between models.

[0154] In the AI recognition module, a lightweight scheduling hub is deployed as the decision core to achieve multi-model collaborative management and synchronize strategies with the cloud. The scheduling hub has a built-in "risk adaptability strategy" engine that combines video stream sources, water risk levels, lighting conditions, and other contexts to collect real-time data such as GPU / NPU occupancy and inference delay, dynamically adjust scheduling weights, mark task model priorities, generate scheduling instructions, and push them into the dynamic task queue (such as Kafka / RabbitMQ) to avoid running resource overload.

[0155] The multi-model fixed polling mechanism ensures the system's second-level response to high-risk events through differentiated interval settings, eliminating the coexistence of "over-monitoring" and "monitoring blind spots" in traditional monitoring. At the same time, the scheduling hub implements orderly model scheduling management to avoid memory overflow risks and ensure the stable operation of edge computing terminals. The combination of the two fully considers the actual processing capacity of edge computing terminals, extends the service life of the equipment through reasonable resource allocation, and provides an efficient and reliable intelligent solution for water conservancy project safety monitoring.

[0156] Further, the linkage verification mechanism of the monitoring data and video monitoring is a closed-loop management process of "data error or abnormal trigger-video detection and identification review-linkage verification judgment", which realizes the intelligent linkage of sensor monitoring and AI video identification. When the data analysis module detects that the water level meter, osmotic pressure meter and other sensor monitoring data are incorrect or abnormal, the system will trigger AI video identification for review and verification, forming a double verification mechanism. For example, when the water level meter data is abnormal, the AI video identification will trigger the water level identification function, accurately measure the water gauge scale through image analysis technology, and compare the results with the sensor data. If the osmotic pressure meter is offline, the system will intelligently analyze the monitoring screen to determine whether there are abnormal conditions such as equipment damage or extreme weather.

[0157] In terms of technical implementation, the system adopts the following key measures to ensure the reliability of linkage:

[0158] (1) Time synchronization mechanism: Through interpolation, calibration and dynamic synchronization, etc. Handle the missing and deviation of time stamp, to ensure the time synchronization of monitoring data and video monitoring, provide accurate basis for data linkage judgment.

[0159] (2) Space correlation modeling: Establish a spatial mapping database of sensors and cameras, use GIS coordinate system to record device location information, so that sensor data can be associated with the corresponding video monitoring area.

[0160] (3) Intelligent review process: After confirming that the data is incorrect or abnormal, the system will perform double verification, automatically intercept 10-second video clips before and after the abnormal period of sensor data, and perform target detection, identification and comparison through lightweight AI video identification model.

[0161] (4) Cloud collaborative analysis: According to the verification judgment result, start the alarm generation module, and at the same time, the key data will be encrypted and uploaded to the cloud, providing judgment basis for management personnel, and reducing the false alarm rate.

[0162] Further, the early warning of the water conservancy project safety monitoring system refers to the early identification and warning of potential safety hazards or abnormal states of water conservancy projects through real-time monitoring, data analysis and risk assessment.

[0163] Use monitoring equipment to obtain monitoring data, analyze and process monitoring data, identify abnormal patterns such as data missing, value mutation, trend deviation, and automatically trigger early warning when monitoring value exceeds preset safety threshold or multi-source data change trend is inconsistent.

[0164] Device-level early warning: data is incorrect, there may be problems in some links of the monitoring system, send early warning request for manual review;

[0165] System-level warning: data deviates from historical rules but does not exceed the design threshold, which may affect the normal function of water conservancy projects, and a warning is sent to take measures;

[0166] Emergency-level warning: data continues to exceed the threshold and affects project function, such as major risks of dam collapse or levee breach, which requires emergency disposal or personnel relocation.

[0167] Data error refers to data that does not conform to objective reality, caused by sensor, transmission system or human error, which can feedback monitoring system problems and need to repair data or repair equipment.

[0168] Data anomaly refers to data that is true but its value exceeds the normal range or safety threshold of project safe operation, which can feedback the physical state change of the project and need to take necessary measures to intervene the project operation.

[0169] The warning means include sound and light warning devices, backend platform message push warning, mobile phone short message warning, telephone language warning and warning broadcast, etc. The configuration of warning means should match the warning level, response timeliness and notification range.

[0170] As shown in Figure 3 , the hierarchical multi-modal warning is specifically:

[0171] When device-level warning is triggered, low-frequency yellow light flashes and intermittent beeps are triggered in the management room and key areas of the project to quickly remind on-site personnel to review; the monitoring system interface of the backend platform generates a pending review and processing prompt, but does not trigger a push alarm to avoid unnecessary panic caused by false alarms.

[0172] When system-level warning is triggered, medium-frequency orange light flashes and intermittent beeps are triggered in the management room and key areas of the project; the monitoring system of the backend platform triggers an orange warning message push; a mobile phone short message containing the monitoring elements and abnormal information triggering the warning is sent to relevant personnel to ensure that technical personnel intervene in time and judge the abnormal phenomenon.

[0173] When emergency-level warning is triggered, high-frequency red light flashes and continuous beeping sound and light prompts are triggered in the management room and key areas of the project; the monitoring system of the backend platform triggers a red warning message pop-up window; voice telephone automatic calls related persons are triggered; downstream warning broadcast sends warning information, and multiple channels of warning related parties ensure full coverage of emergency response.

[0174] Through hierarchical and differentiated warning means configuration, response efficiency and social cost can be considered, and excessive warning or warning failure can be avoided.

[0175] Further, in order to ensure the long-term reliability of the edge computing terminal, a model optimization and dynamic updating strategy is adopted, and the system adopts a cloud-edge collaborative computing architecture to realize model optimization and updating through the cooperation of the cloud and the edge computing terminal. The cloud relies on strong computing power to be responsible for model training and algorithm development, and the edge computing terminal connects the monitoring equipment to complete data collection and perform edge computing tasks. The two achieve efficient collaboration based on a self-developed cloud-edge framework, which develops an OTA engine and uses differential update technology to issue to the edge computing terminal, supporting dynamic deployment and version management. At the same time, the framework realizes remote application deployment and cross-layer metadata synchronization through the built-in TSL (Thing Specification Language), and builds a complete and efficient and reliable cloud-edge collaborative solution. The differential update technology ensures transmission safety through digital signature and verification mechanism to prevent rules from being tampered with; and only the changed part is transmitted, greatly reducing the occupation of network bandwidth, especially suitable for unstable network conditions in the field monitoring environment.

[0176] The cloud optimizes the algorithm and trains the model according to the data uploaded by the edge, and updates the algorithm and model deployed in the edge computing terminal at irregular intervals to adapt to the changes of the water conservancy system, so that the system has stronger dynamic adaptability. The edge computing terminal collects monitoring sensor and video monitoring data, and uses the latest algorithm model to perform tasks, ensuring the long-term reliability and stability of safety monitoring.

[0177] The edge computing terminal of the application is compatible with multiple commonly used and customized communication protocols in the water conservancy industry, automatically identifies the type of connected devices and switches protocols to ensure standardized transmission of multi-source heterogeneous monitoring data. It integrates multiple communication interfaces, supports gigabit Ethernet, 4G / 5G, LoRa, Beidou and other multi-mode communication, and can uniformly collect data from heterogeneous devices such as flow meters, osmotic pressure meters, GNSS receivers and video cameras. It has built-in time sequence integrity analysis, logical rationality detection, multi-source consistency verification algorithms, etc., which can judge the data integrity and reliability in real time, and automatically trigger data repair, multi-source data verification or video linkage verification when errors or abnormalities are found. The terminal deploys a lightweight AI video recognition model, supports hydrological element identification, engineering safety monitoring, water behavior identification and environmental monitoring, etc. Intelligent analysis of scenes, and realizes functions such as target detection and identification of floating objects, weeds, water level reading and fishing, swimming, etc. The terminal establishes a hierarchical early warning system based on the dual verification mechanism of "monitoring data-video monitoring", automatically triggers different levels of early warning response according to the data verification result, cooperates with the Internet of Things platform to realize remote functions such as protocol library update and algorithm model optimization, and maintains the continuous optimization and stable operation of the system through OTA technology.

[0178] In another embodiment of the application, a water conservancy engineering safety monitoring on-site early warning system is provided, comprising:

[0179] The water conservancy project safety monitoring intelligent edge computing terminal integrated with AI recognition in the above embodiment;

[0180] Multi-source heterogeneous monitoring equipment: including osmometer, weir meter, rain gauge, water level meter, flow meter, displacement monitoring equipment, pressure gauge, video camera, connected to edge computing terminal;

[0181] Linkage verification mechanism: when the sensor data is wrong or abnormal, the associated camera video clip is automatically intercepted for AI video review;

[0182] Hierarchical early warning feedback mechanism: according to the monitoring data and video recognition verification judgment result, trigger different levels of three-level early warning:

[0183] Device-level early warning: trigger local low-frequency yellow light flashing and intermittent buzzing, and back-end platform review prompt;

[0184] System-level early warning: trigger local medium-frequency orange light flashing and intermittent buzzing, back-end platform message push and SMS alarm;

[0185] Emergency-level early warning: trigger local high-frequency red light flashing and continuous buzzing, telephone call and early warning broadcast;

[0186] The trigger conditions of the three-level early warning include:

[0187] Device-level early warning: data is wrong;

[0188] System-level early warning: data deviates from historical rules but does not exceed the design threshold;

[0189] Emergency-level early warning: data continuously exceeds the threshold and affects the function of the project.

[0190] The edge computing terminal adopts a multi-core heterogeneous architecture design, including a collection interface unit, a network communication unit, a data storage unit, an IOT middleware, and a central processing unit;

[0191] Data collection interface unit, integrated with multiple data collection interfaces, supporting multi-protocol adaptive matching, used to obtain monitoring data from external monitoring sensor equipment;

[0192] Network communication unit, supporting multi-mode redundant transmission, used to transmit the collected monitoring data to the data storage unit;

[0193] Data storage unit, used for encrypted storage of raw monitoring data, result data and temporary data caching;

[0194] IOT middleware, used for protocol conversion, data encryption and data routing of raw data;

[0195] The central processing unit is used for dynamically scheduling data analysis, AI identification and real-time alarm generation tasks; the central processing unit comprises a data analysis module, an AI identification module and an alarm generation module; the data analysis module adopts a streaming computing engine to perform time sequence integrity verification, logical rationality verification and multi-source consistency verification on the preprocessed structured data; the AI identification module comprises a semantic segmentation model, an AI video identification model and a fusion decision module, the AI identification module receives an input image and inputs the input image into the semantic segmentation model and the AI video identification model; the semantic segmentation model performs water area segmentation on the input image and outputs a water area mask; the AI video identification model detects a target in the input image and outputs a target bounding box list; the fusion decision module combines the water area mask and the target bounding box list, and outputs a warning result through a water area existence judgment and a target box and water area mask spatial relationship judgment process; the alarm generation module starts a hierarchical warning according to a data verification judgment result: data error triggers a device-level warning, data anomaly but not exceeding a threshold value triggers a system-level warning, and data exceeding a safety threshold value triggers an emergency-level warning; a monitoring data and video monitoring linkage verification mechanism is adopted to perform alarm warning, when the data analysis module judges that sensor monitoring data is incorrect or abnormal, the AI identification module is triggered to perform rechecking verification, forming a double verification mechanism, and the alarm generation module is started according to the verification judgment result.

[0196] In another embodiment of the application, a multi-source heterogeneous sensor protocol is dynamically adapted by an edge computing terminal, and monitoring data and video streams are collected in real time.

[0197] Data quality control is performed at the edge side: time sequence integrity verification, logical rationality verification and multi-source consistency verification are performed in sequence.

[0198] An AI video identification model is called to detect and identify floating objects, weeds, water gauge readings and fishing, swimming and other human behaviors;

[0199] Sensor data and video identification results are verified and judged, and hierarchical warning is performed according to the judgment result:

[0200] If it is a device-level warning: trigger local low-frequency yellow light flashing and intermittent buzzing, and back-end platform rechecking prompt;

[0201] If it is a system-level warning: trigger local medium-frequency orange light flashing and intermittent buzzing, back-end platform message push and SMS alarm;

[0202] If it is an emergency-level warning: trigger local high-frequency red light flashing and continuous buzzing, telephone call and warning broadcast.

[0203] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiment methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0204] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0205] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present application should be considered as equivalent replacement methods, and are included in the protection scope of the present application.

Claims

1. An integrated AI-identified water conservancy safety monitoring intelligent edge computing terminal, characterized in that, The edge computing terminal adopts a multi-core heterogeneous architecture design, including a collection interface unit, a network communication unit, a data storage unit, an IOT middleware and a central processing unit; The data collection interface unit integrates multiple data collection interfaces and supports multi-protocol adaptive matching, and is used to obtain monitoring data of external monitoring sensor devices; The network communication unit supports multi-mode redundant transmission and is used to transmit the collected monitoring data to the data storage unit; The data storage unit is used for encrypted storage of raw monitoring data and result data and temporary data caching; The IOT middleware is used for protocol conversion, data encryption and data routing of raw data; The central processing unit is used for dynamically scheduling data analysis, AI identification and real-time alarm generation tasks; the central processing unit includes a data analysis module, an AI identification module and an alarm generation module; the data analysis module uses a streaming computing engine to perform time sequence integrity verification, logical reasonableness verification and multi-source consistency verification on preprocessed structured data; the AI identification module includes a semantic segmentation model, an AI video identification model and a fusion decision module, and the AI identification module receives input images and inputs them to the semantic segmentation model and the AI video identification model; the semantic segmentation model performs water area segmentation on the input images and outputs a water mask; the AI video identification model detects targets in the input images and outputs a target bounding box list; The fusion decision module combines the water mask and the target bounding box list, and outputs an early warning result through a spatial relationship judgment process of water existence judgment, center point inclusion judgment, intersection ratio calculation and early warning judgment; The alarm generation module starts a hierarchical early warning according to the data verification judgment result: data error triggers a device-level early warning, data anomaly but not exceeding the threshold triggers a system-level early warning, and data exceeding the safety threshold triggers an emergency-level early warning; A monitoring data and video monitoring linkage verification mechanism is adopted for alarm and early warning, when the data analysis module judges that the sensor monitoring data is incorrect or abnormal, the AI identification module is triggered for review and verification, forming a double verification mechanism, and the alarm generation module is started according to the verification judgment result; The edge computing terminal adopts a hierarchical pipeline type business logic processing mechanism, specifically as follows: In the data uplink channel, the data collection interface unit obtains raw data of various monitoring sensors in real time through multi-modal interfaces, performs preliminary filtering and time alignment through a hardware accelerated preprocessing unit, and then multi-threaded protocol analysis and data standardization processing are performed by the IOT middleware, the processed structured data is written into the encrypted storage area through the DMA channel in parallel, and at the same time, the data is pushed to the streaming computing engine of the central processing unit for real-time analysis; In the data downlink channel, the network communication unit receives cloud instructions, verifies the digital signature by the hardware security area, and after the data storage unit, the standard protocol instructions are directly executed by the central processing unit, and the non-standard protocol instructions need to be processed by the protocol conversion engine of the IOT middleware; the execution process implements full-link tracking, and the execution result together with the system health status is fed back to the cloud operation and maintenance platform through the heartbeat mechanism. 2.The integrated AI-identified water conservancy safety monitoring intelligent edge computing terminal of claim 1, wherein The data analysis module adopts a data quality control algorithm including time sequence integrity verification, logical reasonableness verification and multi-source consistency verification to perform multi-level quality verification on the real-time collected monitoring data. 3.The integrated AI-identified water conservancy safety monitoring intelligent edge computing terminal of claim 1, wherein The semantic segmentation model adopts a DeepLabv3+ model for water area region segmentation of an input image, and outputs a water area mask; The AI video recognition model adopts an improved YOLOv5s model for target detection of an input image, and outputs the bounding box information of the task target; the AI video recognition model embeds an ECA+CBAM combined attention module in the last C3 layer of the backbone network and the first C3 layer of the neck network, respectively; the ECA+CBAM combined attention module in the backbone network strengthens the key channel information of the target from the original features and establishes a cross-scale correlation; the ECA+CBAM combined attention module in the neck network further optimizes the multi-scale feature fusion effect to enhance the representation ability of small target features; the CARAFE content-aware resampling technology is used in the first upsampling layer of the neck network to enhance the receptive field of the feature map; The fusion decision module is used for spatial relationship judgment of the target bounding box and the water area mask, and outputs an early warning result; The spatial relationship judgment includes water area existence judgment, center point inclusion judgment, intersection ratio calculation and early warning judgment.

4. The integrated AI-identified water conservancy safety monitoring intelligent edge computing terminal of claim 3, characterized in that, The ECA+CBAM combined attention module includes a channel attention layer and a spatial attention layer; The channel attention layer performs average pooling and maximum pooling operations on the input features to aggregate feature information. After extracting the features through a fully connected layer, a sigmoid activation function is used to generate the first weight value W1 of the convolution layer, and the first feature X o1 is output o1 The calculation formula is as follows: ； wherein, represents an input feature, represents a first weight of the convolution layer, and B1 represents a first bias of the convolution layer; The spatial attention layer outputs a first feature X o1 The average pooling and maximum pooling operations are performed to aggregate feature information. After the multi-layer convolution extracts the features, a sigmoid activation function is used to generate a second weight value W2 of the convolution layer, and the first feature X o1 The second feature X o2 is output after the weighted fusion. o2 The calculation formula is as follows: ； wherein, B2represents a second bias of the convolution layer. The CARAFE content-aware resampling technology includes an upsampling prediction module and a content-aware reorganization module; the upsampling prediction module performs channel compression, encoding convolution, pixel random shuffling and normalization processing on the target image feature mapping, and outputs a prediction kernel; the content-aware reorganization module performs dot product processing on the prediction kernel output by the upsampling prediction module and the feature map obtained by nearest neighbor upsampling to complete feature reorganization.

5. The integrated AI-identified water conservancy safety monitoring intelligent edge computing terminal of claim 3, wherein The water area existence judgment is to calculate the total number of pixels with a pixel value of 1 in the water area mask, and if the total number is greater than 0, it is determined that there is water area; The center point inclusion judgment is to calculate the geometric center of the target bounding box and map it to the discrete pixel index of the water area mask, and if the pixel value corresponding to the index is 1, it is determined that the target center point is within the water area range; The intersection ratio calculation is the ratio of the intersection area of the target bounding box and the water area mask to the area of the target bounding box; The early warning judgment is to output an early warning signal when the target category belongs to the dangerous target category set and the intersection ratio is greater than or equal to the preset spatial threshold or the target center point is within the water area range.

6. The integrated AI-identified water conservancy safety monitoring intelligent edge computing terminal of claim 3, wherein, The improved YOLOv5s model is subjected to sparse training, channel pruning and model fine-tuning for lightweight processing, specifically: L1 regularization training is used to perform sparse training on the original weights of the convolutional layer and the BN layer in the improved YOLOv5s model; Parameters on the BN layer As a channel pruning decision factor for the model structure, the partial scale factor is forced to approach 0 by L1 regularization to identify and reduce the corresponding channel; tends to 0. The model fine-tuning restores the model accuracy by iteratively training the pruned model.

7. The integrated AI-identified water conservancy safety monitoring intelligent edge computing terminal of claim 1, wherein The AI recognition module adopts a multi-model fixed polling mechanism, coordinates the running of multiple lightweight video recognition task models through a risk adaptability scheduling strategy, and performs AI image recognition on images or video frames collected by multiple video channels; the multi-model fixed polling mechanism sets different monitoring intervals according to target behavior characteristics and risk levels, uses a short interval for high-frequency dynamic change events, uses a medium interval for medium persistence events, and uses a long interval for low-frequency long-time events; through the polling mechanism, all-around and all-weather intelligent monitoring of complex water area scenes of water conservancy projects is realized. 8.The integrated AI-identified water conservancy safety monitoring intelligent edge computing terminal of claim 1, wherein, The linkage verification mechanism of the monitoring data and the video monitoring specifically comprises: A time synchronization mechanism: missing and deviation of time stamps are processed through interpolation, calibration and dynamic synchronization to realize time synchronization of the monitoring data and the video monitoring; Space correlation modeling: a space mapping database of sensors and cameras is established, and device position information is recorded by using a GIS coordinate system, so that sensor data can be associated with corresponding video monitoring areas; An intelligent review process: after confirming that data is wrong or abnormal, double verification is performed, video clips T seconds before and after the abnormal period of sensor data are automatically intercepted, and target detection, recognition and comparison are performed through the AI recognition module; Cloud collaborative analysis: according to the verification judgment result, an alarm generation module is started, and key data is encrypted and uploaded to the cloud. 9.The integrated AI-identified water conservancy safety monitoring intelligent edge computing terminal of claim 1, wherein, In the central processing unit, normal data is compressed and then enters a queue for uploading; wrong data will immediately trigger automatic re-measurement, interpolation compensation and original data encryption backup operations; data abnormality will automatically adjust the sampling frequency and perform priority uploading of encrypted data; all abnormal processing operations will be recorded completely, including the time, type, processing measures taken and detailed information of the final judgment result, to form a complete quality control log.

10. A hydraulic engineering safety monitoring on-site early warning system characterized in that, It comprises: The edge computing terminal according to any one of claims 1-9; The multi-source heterogeneous monitoring equipment connected to the edge computing terminal, wherein the multi-source heterogeneous monitoring equipment comprises one or more of a tonometer, a weir meter, a rain gauge, a water level gauge, a flow meter, a displacement monitoring device, a pressure gauge and a video camera; A linkage verification unit: when sensor data is wrong or abnormal, the video clips of the associated camera are automatically intercepted for AI video review; A hierarchical early warning feedback unit: different levels of three-level early warning are triggered according to the monitoring data and the video recognition verification judgment result, and the three-level early warning comprises device-level early warning, system-level early warning and emergency-level early warning.

11. The hydraulic engineering safety monitoring and early warning system in situ according to claim 10, characterized in that, The trigger conditions of the three-level early warning comprise: Device-level early warning: data is wrong; System-level early warning: data deviates from historical rules but does not exceed the set threshold; Emergency-level early warning: data continuously exceeds the threshold and affects the function of the project.

12. A safety monitoring method of an integrated AI-identified water conservancy safety monitoring intelligent edge computing terminal based on any one of claims 1-9, characterized in that, It comprises the following steps: Adapt the multi-source heterogeneous sensor protocol through the edge computing terminal to collect monitoring data and video streams in real time; Perform data quality control on the edge side: execute time sequence integrity verification, logical reasonableness verification and multi-source consistency verification in sequence; Call the AI recognition module to detect and recognize floating objects, weeds, water gauge readings and fishing and swimming behavior targets; Verify the sensor data and the video recognition result, and start hierarchical early warning according to the judgment result.

Images