A high earth-rock dam embankment animal motion track intelligent prediction and real-time early warning system and method

Abstract

The application provides a high earth-rock dam embankment damage animal motion track intelligent prediction and real-time early warning system and method, relates to the field of hydraulic structure safety monitoring and risk prevention and control, and realizes all-weather data acquisition through a multi-modal sensing unit, completes precise detection and instance segmentation of embankment damage animals by using an improved single-stage target detection network, realizes three-dimensional space positioning in combination with sensing device parameters and dam area geographic information, fuses a physical kinematics model and a long short-term memory network to predict a future motion track, establishes a graded early warning mechanism based on dangerous area division and performs differentiated linkage driving away, and the application realizes a change from passive detection to active prediction, significantly improves the intelligentization and active level of high earth-rock dam embankment damage animal prevention and control, and effectively guarantees safe and stable operation of a dam body.

Description

Technical Field

[0001] This invention relates to the field of safety monitoring and risk prevention technology for hydraulic structures, and particularly to the field of intelligent management of the safe operation of high earth-rock dams. Specifically, it relates to an intelligent prediction and real-time early warning system and method for the movement trajectory of animals that may cause damage to high earth-rock dams. Background Technology

[0002] Earth-rock dams are an important component of large and medium-sized water conservancy projects in my country, playing an irreplaceable role in flood control, irrigation, water supply, and power generation. However, because their dam bodies are mainly composed of earth and rock, they are highly susceptible to structural hazards due to the physical and mechanical properties of natural materials and their susceptibility to environmental factors and external disturbances. Among these hazards, seepage, piping, and local instability caused by wildlife activity have become significant risks to the safe operation of high earth-rock dams.

[0003] Animals that damage dams mainly include rodents, badgers, snakes, and some burrowing animals that inhabit the area around the dam. These animals often burrow and dig tunnels near the dam slope, crest, or toe, creating pathways that disrupt the continuity and density of the dam structure. When there is rainfall, flooding, or fluctuation in reservoir water levels, water can easily seep into the dam structure through these animal tunnels, causing the seepage channels to expand or even piping to occur. In severe cases, this can lead to partial collapse and overall instability of the dam, threatening the lives and property of people downstream.

[0004] Existing methods for controlling animals that infest dikes mainly fall into two categories: manual patrols and physical control, and video surveillance and infrared triggering devices. Manual patrols are labor-intensive, inefficient, and susceptible to blind spots caused by weather, lighting, and terrain, making it difficult to promptly grasp the true activity patterns of animals. While traditional video surveillance systems have improved the level of automation to some extent, they suffer from limitations such as limited monitoring range, insufficient recognition accuracy, and difficulty in long-term trajectory prediction. They often only allow for passive responses to intrusions that have already occurred, failing to provide early warnings or quantify potential future movement paths and hazards.

[0005] In recent years, with the development of artificial intelligence, sensor technology, and big data analysis, animal intrusion detection methods based on video image recognition and thermal imaging monitoring have gradually emerged. However, current research mostly remains at the "detection" level, lacking the ability to analyze patterns in animal movement behavior and predict trajectories. Animals that threaten dams exhibit diurnal activity differences, seasonal migration patterns, and group distribution characteristics; relying solely on instantaneous detection results is insufficient to support comprehensive safety early warning decisions. Furthermore, traditional monitoring systems often operate in isolation, failing to form a data closed loop and intelligent early warning mechanism, and cannot combine dam structure, animal habits, and historical activity data to classify and warn of risk levels. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide an intelligent prediction and real-time early warning system and method for the movement trajectory of animals that cause damage to high earth-rock dams. This invention solves the problems of limited monitoring range, insufficient identification accuracy, lack of trajectory prediction capability, and untimely early warning response in the prior art. It can manage the invasion behavior of animals that cause damage to dams throughout the entire process, realize the transformation from passive detection to active prediction, and from single alarm to hierarchical response, and significantly improve the intelligence, initiative, and precision of animal control for high earth-rock dams.

[0007] To achieve the above-mentioned objectives, the first aspect of this invention provides an intelligent prediction and real-time early warning system for animal movement trajectories in high earth-rock dams, comprising: Multimodal sensing unit, used for all-weather collection of multi-source sensing data on animals that harm the dam in the high earth-rock dam area; The intelligent recognition unit is used to perform target detection and instance segmentation on the perceived data, and output the category, image location and pixel-level outline of the animal that harms the dike; The spatial positioning unit is used to invert the image coordinates of animals that damage the dam into three-dimensional spatial coordinates in the world coordinate system based on the spatial parameters of the sensing equipment and the geographical information of the dam area. The trajectory prediction unit is used to predict the future trajectory of the animal based on its historical time series coordinates and by integrating a physical kinematics model with a deep learning time series model. The graded early warning unit is used to calculate the risk level and trigger the corresponding early warning response based on the spatial relationship between the predicted trajectory and the preset danger zone; The linkage and removal unit is used to perform differentiated on-site warnings, remote notifications, and automatic removal operations based on the warning level.

[0008] Preferably, the multimodal sensing unit includes a visible light / infrared coaxial imaging device, a motorized pan-tilt unit, a hybrid power supply module, and a data transmission module; The visible light / infrared coaxial imaging device is fixed on an electric pan-tilt head and simultaneously acquires visible light and infrared thermal imaging data. The electric gimbal supports horizontal omnidirectional and vertical pitch adjustment, and integrates an attitude stabilization system to suppress environmental vibration interference. The hybrid power supply module consists of an energy storage unit and a renewable energy auxiliary power supply unit, providing continuous power to the system in the field. The data transmission module supports both wireless and wired dual-link communication, enabling real-time data uploading and remote command distribution.

[0009] Preferably, the intelligent recognition unit adopts an improved single-stage target detection network, whose backbone network uses a dynamic convolution module to adapt to complex pose targets, the neck network uses a bidirectional feature pyramid network to achieve multi-scale feature weighted fusion, and the detection head integrates a channel-space attention module to enhance the response of key features.

[0010] Preferably, the spatial positioning unit is specifically used to: establish the mapping relationship between the world coordinate system, the imaging device coordinate system and the image coordinate system; obtain the intrinsic parameter matrix according to the calibration parameters of the imaging device; obtain the extrinsic parameter matrix according to the device installation position and the real-time attitude of the gimbal; solve the depth information based on the prior assumption of the movement of animals on the dam surface and the digital elevation model of the dam area; and finally complete the conversion from image coordinates to three-dimensional world coordinates.

[0011] Preferably, the trajectory prediction unit is specifically used to: interpolate and complete missing frames in the historical coordinate sequence, calculate the future position of the dam-damaging animal using a kinematic model and a long short-term memory network (LSTM) respectively, adaptively calculate weight coefficients based on the historical prediction errors of the two models, and obtain the final predicted trajectory through weighted fusion.

[0012] Preferably, the graded early warning unit is specifically used to: divide the dam area into a safe zone, an early warning zone, and a warning zone according to the importance of the dam structure; calculate the expected arrival time and invasion probability of animals that harm the dam into each dangerous zone based on the predicted trajectory; comprehensively determine four risk levels: low, medium, high, and extremely high; and trigger corresponding early warnings.

[0013] Preferably, the linkage drive-away unit includes: The on-site warning module includes a brightness-adaptive display device and an intelligent audio playback device, used to display warning information in real time and play warning voice and eviction audio. The remote notification module is used to send tiered early warning information and handling instructions to the mobile terminals of management personnel; The automatic deportation module is used to automatically activate non-contact deportation equipment under high-risk conditions to accurately drive away animals that may harm the dike.

[0014] A second aspect of this invention provides a method for intelligent prediction and real-time early warning of animal movement trajectories in high earth-rock dams, the method being based on an early warning system and comprising the following steps: S1. Multimodal data acquisition: Collect multi-source sensing data on animals that damage the dam around the clock using multimodal sensing devices deployed in the dam area; S2. Intelligent identification of animals that damage dikes: An improved single-stage target detection network is used to process the perceived data and identify the category, image location and pixel-level outline of animals that damage dikes. S3. Three-dimensional spatial coordinate inversion: Based on the spatial parameters of the sensing equipment and the geographical information of the dam area, the image coordinates of the animals that damage the dam are converted into three-dimensional spatial coordinates in the world coordinate system. S4. Intelligent prediction of movement trajectory: Based on the historical time series coordinates of the animals that cause damage to the dike, the physical kinematics model and the deep learning time series model are integrated to predict their future movement trajectory. S5. Risk Level Assessment and Early Warning: Calculate the risk level and trigger the corresponding early warning response based on the spatial relationship between the predicted trajectory and the preset danger zone; S6. Joint Removal and Handling: Execute differentiated on-site warnings, remote notifications, and automatic removal operations based on the warning level, and record the entire event process.

[0015] Preferably, in step S1, image data under different lighting conditions are collected simultaneously using a visible light / infrared coaxial imaging device, and the automatic cruise function of the electric pan-tilt unit is used to achieve full coverage monitoring of key areas in the dam area.

[0016] Preferably, in step S2, the training method of the improved single-stage target detection network includes: constructing a dataset of animals that cause damage to dikes under different lighting conditions, weather conditions, and postures; expanding the sample diversity through data augmentation; performing domain-adaptive fine-tuning using transfer learning; and simultaneously optimizing detection and segmentation accuracy using a multi-task loss function.

[0017] Preferably, in step S3, the intrinsic parameter matrix is ​​obtained through camera calibration, the extrinsic parameter matrix is ​​obtained through equipment installation parameters and gimbal real-time attitude, and the depth information is solved by combining the digital elevation model of the dam area to complete the mapping from pixel coordinates to three-dimensional world coordinates.

[0018] Preferably, in step S4, the missing coordinate frames are filled in using a quadratic interpolation method, and the future positions are predicted using a uniform-acceleration kinematic model and an LSTM network, respectively. The weights are dynamically adjusted based on the prediction errors of the two models at the previous time step, and the weighted fusion is used to obtain the final prediction result.

[0019] Preferably, in step S5, the dam area is pre-divided into a safe zone, a warning zone, and a restricted zone. The estimated arrival time of animals that may damage the dam is calculated for each zone. When the estimated arrival time is greater than a first threshold, it is considered low risk; when it is between the first and second thresholds, it is considered medium risk; when it is between 0 and the second threshold, it is considered high risk; and when the animals have entered the restricted zone, it is considered extremely high risk.

[0020] Preferably, in step S6, the differentiated expulsion operation specifically includes: continuous monitoring for low risk; activation of on-site display and remote notification for medium risk; activation of audible and visual warning and emergency notification for high risk; and activation of automatic expulsion equipment and notification to management personnel for immediate on-site handling for extremely high risk.

[0021] The present invention has the following beneficial effects: 1. This invention employs an improved single-stage target detection network, which significantly improves the detection accuracy and robustness of dam-damaging animals in complex environments by combining dynamic convolution, bidirectional feature pyramids and channel-space attention modules, and greatly enhances the ability to identify small dam-damaging animals and animals with complex postures.

[0022] 2. This invention proposes a trajectory prediction method that integrates physical kinematics models and deep learning temporal models. Through an adaptive weight adjustment mechanism, it balances the real-time performance of the kinematics model with the fitting ability of the LSTM network for nonlinear motion, and the trajectory prediction accuracy is significantly improved compared with the single method.

[0023] 3. This invention establishes a graded early warning mechanism based on the expected arrival time, combined with a multi-channel linkage drive-away system, realizing the transformation from passive response to active defense. It can provide early warning before harmful animals enter critical areas, effectively reducing the safety risks of the dam.

[0024] 4. This invention adopts multimodal sensing and hybrid power supply technology to achieve all-weather, full-coverage monitoring of the dam area. The equipment can operate stably for a long time without mains power and adapt to the complex field dam environment. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 The flowchart illustrates an intelligent prediction and real-time early warning system for the trajectory of animals invading a high earth-rock dam, as provided in an embodiment of the present invention.

[0027] Figure 2 This is a component diagram of the intelligent detection, data acquisition, and processing equipment for animal invasion in high earth-rock dams provided in an embodiment of the present invention.

[0028] Figure 3 This invention relates to the training and application process of an image recognition network in the intelligent prediction process for the trajectory of animals invading high earth-rock dams, as provided in this embodiment of the invention.

[0029] Figure 4 The trajectory prediction process that integrates LSTM and kinematic methods is provided in the high earth-rock dam intrusion animal prediction process of the present invention embodiment. Detailed Implementation

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0031] Example 1: This embodiment provides a specific implementation method for an intelligent prediction and real-time early warning system for animal movement trajectories in high earth-rock dams. The overall system flow is as follows: Figure 1 As shown, the data acquisition and processing equipment consists of the following components: Figure 2 As shown.

[0032] 1. Specific implementation of the multimodal sensing unit The multimodal sensing unit is used to collect multi-source sensing data on harmful animals in the high earth-rock dam area around the clock. It includes a visible light / infrared coaxial imaging device, an electric pan-tilt unit, a hybrid power supply module, and a data transmission module.

[0033] The visible light / infrared coaxial imaging device is fixed to an electric pan-tilt head via connectors, enabling simultaneous acquisition of visible light images and infrared thermal imaging data of the dam area. The visible light imaging mode is used for high-definition detection of animals damaging the dam in daytime environments, while the infrared thermal imaging mode is used for target detection at night or in adverse weather conditions. Through automatic switching and fusion of the two modes, uninterrupted 24 / 7 monitoring is achieved.

[0034] The motorized pan-tilt unit supports omnidirectional horizontal rotation and vertical pitch adjustment, allowing for flexible adjustment of the imaging device's viewing angle and achieving full coverage monitoring of key areas in the dam area. The pan-tilt unit integrates an attitude stabilization system, effectively suppressing image jitter caused by environmental interference such as wind and dam vibration, ensuring that the imaging device can stably capture clear target images even under complex operating conditions. The pan-tilt unit supports preset position settings and automatic cruise functions, enabling periodic scanning of key areas in the dam area according to a preset path.

[0035] The hybrid power supply module consists of an energy storage unit and a renewable energy auxiliary power supply unit, providing continuous power supply for the system in outdoor environments. The energy storage unit uses a high-capacity lithium battery pack and is equipped with a smart power management system (BMS) that can monitor the battery's voltage, current, temperature, and other statuses in real time, and has overcharge, over-discharge, overcurrent, and short-circuit protection functions. The renewable energy auxiliary power supply unit uses solar panels and improves energy conversion efficiency through an MPPT power optimization controller. Under sufficient sunlight conditions, it can meet the daily power supply needs of the equipment and extend the system's outdoor endurance.

[0036] The data transmission module supports both wireless and wired dual-link communication, enabling real-time uploading of collected sensing data to edge computing devices and receiving remote control commands, thus achieving bidirectional data transmission and remote system management. Wireless communication utilizes 5G / 4G cellular networks, while wired communication employs fiber optic Ethernet, ensuring real-time performance and reliability of data transmission.

[0037] 2. Specific Implementation of the Intelligent Recognition Unit The intelligent recognition unit is deployed on an edge computing device to perform target detection and instance segmentation on the acquired image data, outputting the category, image location, and pixel-level contour of the harmful animal. The intelligent recognition unit employs an improved single-stage target detection network; the network training and application process is as follows: Figure 3 As shown.

[0038] In response to the following challenges in detecting animals that intrude into high earth-rock dams: (1) Complex target morphology: Animals that intrude into dams are diverse and exhibit complex postures under different behaviors and angles, making it difficult for traditional convolutional kernels to adapt to complex target boundaries. (2) Multi-scale detection requirements: Animals that intrude into dams cover multiple families and genera, with significant scale variations, making it difficult for ordinary FPN structures to fully integrate multi-scale information. (3) Complex ambient lighting: Lighting conditions such as daytime, nighttime, rain, and fog significantly affect detection accuracy, and existing models are unable to adapt to the challenges of varying lighting conditions. In response to these challenges, the currently advanced YOLOv11 method is modified to adapt to the task of efficiently detecting animals that intrude into high earth-rock dams. YOLOv11 adopts an end-to-end target detection approach. After the image is input into the network, it is first transformed to a standard size (640, 640), and then normalized to the [0, 1] interval. Subsequently, the image is processed by the backbone network to extract multi-scale features, resulting in multi-channel feature maps of different sizes. Subsequently, the network uses a pyramid-structured neck layer to fuse the obtained multi-scale feature maps, ensuring that large-scale feature maps contain overall target information while small-scale feature maps contain local target information. Next, a target detection head is used to predict target information, including target category, location, and confidence level. Furthermore, a target segmentation head performs pixel-level segmentation of the target's outline and provides a refined mask. Finally, a non-maximum suppression method is used to remove duplicate detection results, and the target detection result with the highest confidence level is given.

[0039] Backbone Network Improvement: The original YOLOv11 backbone network module is replaced with dynamic snake convolution to improve the network's ability to detect and segment various dam-damping animal targets and complex-pose animal targets. The calculation process of the snake convolution module includes the following steps: (1) Convolution kernel generation: Based on the local structure of the input image, a convolution kernel suitable for the local area is dynamically generated. The shape of the convolution kernel is determined by the changes in the edges and local features in the image, and can be a curve or a polygon. The convolution kernel is not limited to rectangles or squares, but can be bent or stretched according to the image content, so as to better fit the boundaries and details in the image; (2) Feature extraction: Based on the generated snake convolution kernel, convolution operation is performed with image pixels; (3) Dynamic adjustment: During the forward propagation of the entire network, the snake convolution kernel will be dynamically adjusted according to different regions of the image. This adjustment not only depends on local image information, but is also affected by the global context to ensure that the convolution kernel can better adapt.

[0040] Neck Network Improvements: A BiFPN module is introduced into the neck network of YOLOv11 to better fuse multi-scale target features extracted by the backbone network. In BiFPN, learnable weights are added to features at each scale. During training, the network can learn the contribution of feature maps in different layers to subsequent segmentation networks and automatically adjust the weights. In this process, weighted feature fusion can better reflect the characteristics of invasive animal targets, highlighting information-rich feature layers and suppressing less information-rich feature layers. A fast normalization fusion mechanism is adopted in BiFPN. The feature process in the BiFPN layer after the target features are extracted by the backbone network is as follows: ; In the formula For the top-down path i Intermediate features of the layer For the bottom-up path i The output features of the layer For learnable weights, Set a small value to avoid numerical instability. Resize( P i ) indicates the network P i Features can be upsampled or downsampled. Conv Depthwise separable convolution can be used here to improve convolution efficiency.

[0041] Furthermore, a channel-spatial attention module is introduced into the network detection head to further improve the head's ability to utilize fused features. The channel-spatial attention module is an attention mechanism module used to enhance the performance of convolutional neural networks, consisting of two key parts: a channel attention module and a spatial attention module. These two modules can be embedded into different layers of the CNN to enhance feature representation. The channel attention module aims to enhance the feature representation of each channel. For the input feature map, global max pooling and global average pooling operations are first performed on each channel to calculate the maximum and average eigenvalues ​​for each channel, generating two vectors containing the number of channels to represent the global maximum and average features of the phenotypic defects extracted from each channel. Then, the module inputs the feature vectors obtained from global max pooling and average pooling into a shared fully connected layer to learn the attention weights for each channel. After the network learns the feature importance, the module adaptively determines which channels are more important for the current defect detection task. Finally, the global maximum eigenvector and the average eigenvector are added and normalized using the sigmoid function to obtain the final attention weight vector. The spatial attention module aims to emphasize the importance of different locations in the image. For the feature map enhanced by channel attention, the spatial attention module performs max pooling and average pooling operations along the channel dimension to generate features at different contextual scales. Then, the module concatenates the max-pooled and average-pooled features along the channel dimension to obtain a feature map with contextual information at different scales. This feature map is then processed by convolutional layers to generate spatial attention weights. Similar to the channel attention operation, the network finally uses the sigmoid function to normalize the weights to between 0 and 1.

[0042] 3. Specific Implementation of Spatial Positioning Unit The spatial positioning unit is used to convert the image coordinates of animals damaging the dam into three-dimensional spatial coordinates in the world coordinate system based on the spatial parameters of the sensing equipment and the geographical information of the dam area. Specifically, it includes: Given the camera's installation location (its position in the world coordinate system), the spatial coordinates of the dam surface, and the current rotation angle obtained via the pan-tilt unit, and assuming that the animals damaging the dam are all moving on the dam surface, the position of a pixel in the camera's image within the world coordinate system can be determined, thus providing the spatial coordinates of the animals damaging the dam within that system. Specifically, the center position of the animals is determined by the points within the target bounding box provided by the aforementioned network in the animal detection results.

[0043] In the solution process, the world coordinate system is first set. W and with ( X w , Y w , Zw () represents the coordinates of a ground point, defining the camera coordinate system. C and with ( X c , Y c , Z c () represents the coordinates from the camera's perspective, defining the pixel coordinate system. I and with ( u , v () represents pixel coordinates. During camera installation, the camera's position in the world coordinate system can be recorded as () X c0 , Y c0 , Z c0 When the camera is shooting at the corresponding position, the current yaw, pitch, and roll angles of the camera can be determined by the position of the gimbal, which is ( θ y , θ p , θ r Next, spatial position inversion from the camera coordinate system to the world coordinate system is performed. First, the camera's intrinsic parameter matrix is ​​obtained based on the camera's factory calibration information. K , is represented as: ; in, f x as well as f y For the camera x as well as y Focal length in direction, c x as well as c y These are the principal point coordinates of the camera's imaging element. Then, through the intrinsic parameter matrix parameters, it is possible to obtain the coordinates from the pixel coordinates (...). u , v Transformation to the normalized camera coordinate system: ; here Z c =1 because only directional information can be obtained from a two-dimensional image, but depth information is lacking. Subsequent depth information is calculated based on the previous assumption of the movement of animals damaging the dam surface.

[0044] In the camera coordinate system, a point P c The coordinates can be represented as: ; Next, the coordinates of this point in the camera coordinate system are transformed to the world coordinate system using the camera's spatial position and rotation direction. (Camera rotation matrix) R and translation matrix T It can be defined as follows: ; ; The rotation matrix can be determined by the rotation angle obtained from the gimbal, while the translation matrix is ​​the translation amount of the camera relative to the origin of the world coordinate system, i.e., the spatial coordinate position of the camera. Based on R and T The transformation from the camera coordinate system to the world coordinate system can be represented as: ; Expanding the above equation, we get: ; Since all points are on the surface of the dam, the depth information of all dam-damping animals in the world coordinate system can be known, i.e. Z c Ultimately, Z c Substituting the above transformation formula from camera coordinate system to world coordinate system, we can calculate the pixel value at a point in the camera's field of view. u , v ) to world coordinates ( X w , Y w , Z w The calculation process of ).

[0045] 4. Specific implementation of the trajectory prediction unit The trajectory prediction unit is used to predict the future movement trajectory of animals based on their historical time-series coordinates, integrating physical kinematics models and deep learning time-series models. The trajectory prediction process is as follows: Figure 4 As shown.

[0046] The specific implementation process includes: based on the movement of the spatial coordinates of the dike-damping animals in the frame sequence captured by the camera, a method for estimating the position of the dike-damping animals based on LSTM and kinematics is proposed. In each frame image, the position of the object in the world coordinate system is calculated using camera parameters and an object detection algorithm. ; Based on the assumption of uniform motion, using the nearest... n Frame data is used to calculate the velocity of an object: ; Therefore, acceleration can be estimated as follows: ; Assuming constant acceleration, the predicted position of the dam-damming animal at the next moment based on the kinematic equations is: ; The LSTM module predicts the subsequent positions of animals by inputting a one-dimensional temporal position vector of the animal that caused the flood. It mitigates the vanishing and exploding gradient problems during long sequence training through a gating mechanism. In LSTM, the gating mechanism is implemented by training fully connected layers. The calculations for the forget gate, input gate, unit state update, and output gate are shown below: ; ; ; ; ; In the formula, W and b These are the weights and bias matrices of different fully connected layers. C t This is the current cell state. h t This is the current hidden state. x t It is the input one-dimensional matrix. tanh and σ These are the tanh activation function and the sigmoid activation function, and their expressions are as follows: ; ; Set time window T That is, input the past T The spatial coordinates of the frame can be used to obtain the input sequence of the LSTM network, represented as: ; Through the computation process of the LSTM network described above, the next position of the animal that caused the damage to the dike can be calculated: ; Then, weighting factors are used. α The locations of dike-damping animals predicted by different methods are adjusted using an adaptive strategy: ; Among them, weighting factors α The prediction error from the previous step is calculated using two methods: ; In the above formula, ELTSM and E motion The calculation is as follows: ; ; That is, if the LSTM prediction error is small, its weight is increased, and vice versa.

[0047] 5. Specific Implementation of the Hierarchical Early Warning Unit and the Linked Expulsion Unit The graded early warning unit is used to calculate the risk level and trigger the corresponding early warning response based on the spatial relationship between the predicted trajectory and the preset danger zone.

[0048] First, based on the structural characteristics, importance, and historical risk data of high earth-rock dams, the dam area is divided into three levels of danger zones: Safe zone: The area beyond a certain distance around the dam, where the activity of animals that harm the dam has a relatively small impact on the safety of the dam. Warning zone: The transitional area around the dam body, where there is a potential risk of harmful animals entering the dam. Warning zone: The dam body itself and the core protection area. Animals that harm the dam will directly threaten the safety of the dam if they enter this area.

[0049] Then, based on the future movement trajectories output by the trajectory prediction unit, the estimated arrival time and invasion probability of the dam-damming animals entering each danger zone are calculated. Combining these two indicators—estimated arrival time and invasion probability—the risk level is divided into four levels: low, medium, high, and extremely high. Low risk: Expected arrival time is greater than the first threshold; continuous monitoring is required. Medium risk: The expected arrival time is between the first and second thresholds, triggering a Level 1 warning; High risk: The expected arrival time is between 0 and the second threshold, triggering a level 2 warning; Extremely high risk: Animals that threaten the dike have entered the warning zone, triggering a Level 3 alert.

[0050] The linkage-based eviction unit is used to perform differentiated on-site warnings, remote notifications, and automatic eviction operations based on the warning level, including: On-site warning module: Includes a brightness-adaptive LED display and an intelligent audio playback device. The LED display can automatically adjust its brightness according to ambient light, displaying the location, predicted trajectory, and danger level of dam-damping animals in real time. The intelligent audio playback device can dynamically adjust the playback content according to the type of dam-damping animal and the warning level. Indoor speakers play warning information such as the type, location, and movement of dam-damping animals, while outdoor speakers play the calls of the dam-damping animals' natural predators to achieve initial deterrence.

[0051] Remote notification module: This module transmits tiered early warning information and response instructions to managers' mobile terminals in real time via the 5G network. Level 1 warnings send SMS notifications, Level 2 warnings send both SMS and telephone notifications, and Level 3 warnings send emergency response instructions.

[0052] Automatic Removal Module: In extremely high-risk situations, it automatically activates non-contact removal equipment, such as ultrasonic repellers and directional spray devices, to accurately remove animals that threaten the dike, thus avoiding unnecessary impacts on the environment and other organisms.

[0053] Example 2: See Figure 1 This embodiment discloses the overall workflow of the system of the present invention, including a method for intelligent prediction and real-time early warning of animal movement trajectories in high earth-rock dams. The complete workflow of the system is as follows: Step 11, Invasion of dam-damping animals: When dam-damping animals enter the monitoring range of the dam area, the system detection process is triggered.

[0054] Step 12, Visible / Infrared Camera: The visible / infrared camera of the multimodal sensing unit automatically switches the imaging mode according to the ambient light intensity and collects real-time image data of the dam area; the motorized pan-tilt unit automatically rotates according to the preset cruise path and periodically scans the key areas of the dam area.

[0055] Step 13, Single-stage target detection network detection: The intelligent recognition unit uses an improved single-stage target detection network to process the collected image data in real time, detect and identify harmful animals, and output their category, image location and pixel-level contour.

[0056] Dual-path trajectory prediction: After the spatial localization unit inverts the pixel coordinates of the dam animal into three-dimensional spatial coordinates in the world coordinate system, it independently predicts its future position through the long short-term memory network in step 141 and the kinematic method in step 142.

[0057] Step 15, Future Movement Trajectory: The trajectory prediction unit adaptively calculates the fusion weights based on the historical prediction errors of the two models, and obtains the future continuous movement trajectory of the dam-damped animal through weighted fusion.

[0058] Step 16, Invasion Area Judgment: The graded early warning unit spatially matches the predicted trajectory with the pre-divided safe zone, early warning zone, and alert zone, and calculates the expected arrival time and invasion probability of the dam-damming animals entering each dangerous area.

[0059] Step 17, Warning Trigger: Based on the intrusion area assessment results and risk level determination rules, determine the current risk level and trigger the corresponding warning response. The linkage expulsion unit executes differentiated on-site warnings, remote notifications, and automatic expulsion operations according to the warning level.

[0060] Step 18, Termination of Intrusion Behavior Judgment: The system continuously monitors the movement status of animals that harm the dam and determines in real time whether the intrusion behavior has terminated. If the intrusion behavior has not terminated (the animals that harm the dam are still active within the monitoring range), the system returns to Step 13 to continue tracking, detection, and early warning; if the intrusion behavior has terminated (the animals that harm the dam have left the dam area or have been successfully driven away), the system proceeds to Step 19.

[0061] Step 19, the camera enters patrol mode: the warning is lifted, the system returns to normal operation, and the camera continues to automatically patrol along the preset path, waiting for the next intrusion event.

[0062] Example 3: See Figure 2 This embodiment discloses the hardware composition and deployment method of the multimodal sensing unit of the present invention. The multimodal sensing device in this embodiment is an integrated field monitoring device, which is installed on a galvanized steel mounting pole 22. Multiple devices are deployed along the key sections of the dam to form a full-coverage monitoring network. All device data are finally aggregated to the power plant central server 21.

[0063] The visible light / infrared camera 23 is rigidly fixed to the rotating gimbal 24 via a standard connector, and coaxially integrates a visible light imaging module and an infrared thermal imaging module. During the day, it automatically activates the visible light channel to achieve high-definition target recognition, and at night, in rain, fog, sandstorms, or other adverse weather conditions, it automatically switches to the infrared thermal imaging channel, clearly capturing the target's thermal radiation signal without the need for active supplemental lighting, achieving uninterrupted monitoring around the clock.

[0064] The 24-inch rotating gimbal employs a high-precision motorized rotating structure, supporting continuous horizontal rotation and large-angle pitch adjustment, and possesses high-precision positioning capabilities. The gimbal incorporates a three-axis gyroscope stabilization system, which can compensate for image jitter caused by environmental factors such as dam vibration and wind disturbance in real time, ensuring image clarity. The gimbal supports multiple preset position settings and automatic cruise function, and can periodically scan key areas such as the dam slope, dam toe, and dam shoulder according to preset paths.

[0065] The power supply assembly 25 consists of a hybrid power supply system comprised of solar panels 26 and batteries 27. Solar panels 26 utilize an MPPT (Maximum Power Point Tracking) controller to optimize charging efficiency, providing high-efficiency charging capabilities. Batteries 27 employ high-capacity lithium-ion battery packs and are equipped with a smart power management system (BMS) that monitors battery status in real time and provides overcharge, over-discharge, overcurrent, and short-circuit protection. The system supports intelligent switching between mains power and battery power, quickly switching to battery power during mains power outages and enabling fully off-grid operation under sufficient sunlight conditions.

[0066] The multi-source data transmission device 28 integrates a cellular communication module and an Ethernet interface, supporting dual-link hot backup communication via wireless and wired connections. The wireless link is used for data transmission in remote areas without fiber optic connections, while the wired link is used for high-speed and stable transmission in core areas. It enables real-time uploading of video streams, detection results, trajectory data, and early warning information, as well as remote distribution of control commands.

[0067] The data synchronization server 29 is deployed in the on-site monitoring station and uses an industrial-grade embedded computer. It is responsible for local caching, preprocessing, and time synchronization of the collected data. The processed data is uploaded to the power plant's central server 21 in real time, and the server also receives control commands from the central server, such as equipment configuration, patrol path, and early warning thresholds.

[0068] The power plant's central server 21 is deployed in the power plant's monitoring center, which houses the dam. Utilizing a high-performance server cluster, it is responsible for the unified management, data aggregation and storage, global analysis and calculation, and remote dispatch of all monitoring equipment throughout the dam area. It provides management personnel with a visual monitoring interface, displaying real-time information on animal activity affecting the dam, early warning messages, and equipment operating status.

[0069] This embodiment, through the organic combination of the aforementioned hardware devices, achieves all-weather, full-coverage, and highly reliable multi-source sensing data acquisition in the high earth-rock dam area, providing a data foundation for subsequent intelligent identification, spatial positioning, trajectory prediction, and early warning and relocation. Example 4: See Figure 3 This embodiment discloses the training and application method of the improved single-stage target detection network used in the intelligent recognition unit of the present invention. The training and application process of the improved single-stage target detection network is as follows: Step 31, Data Collection: Construct a multi-scenario, multi-dimensional dataset specifically for animals that damage dikes. The collected content includes: Step 311, Visible light animal images: Collect images of typical dam-damming animals under different light intensities, shooting angles, and distances during the day; Step 312, Infrared animal images: Acquire infrared thermal imaging images under different ambient brightness conditions at night; Step 313, Complex Environment Images: Images are specifically collected under common harsh conditions in the dam area, such as low light, direct strong light, low visibility due to rain and fog, and low contrast due to shadows, to simulate actual application scenarios.

[0070] All images were annotated using professional annotation tools in step 313. The annotations included the target category, minimum bounding rectangle, and pixel-level contour, forming a standardized annotation dataset.

[0071] Step 32, Dataset partitioning: The labeled dataset is randomly divided into training set images and validation set images according to a preset ratio. The training set is used for network parameter learning, and the validation set is used for accuracy evaluation and model selection during the training process.

[0072] Step 37, Improved Single-Stage Object Detection Network Training: The improved single-stage object detection network architecture proposed in this invention is used for training. The core improvements of the network include: Step 331 Dynamic Convolution Module: Replace the standard convolution module in the original backbone network, dynamically generate an adapted convolution kernel based on the local edge features of the input image, better fit the complex contours and posture changes of the dammed animals, and significantly improve the effectiveness of feature extraction. Step 341 Bidirectional Feature Pyramid Module: As the neck network, it adds bidirectional connections on the basis of the traditional feature pyramid and assigns learnable weights to feature maps at each scale, automatically learning the contribution of features at different levels to the detection task, and achieving efficient fusion of multi-scale features. Step 351 Channel-Spatial Attention Module: Integrated into the detection and segmentation head, it adaptively enhances the feature channels and spatial regions important to the target animal in the embankment through channel attention and spatial attention mechanisms in turn, and suppresses background noise interference.

[0073] The training process is executed sequentially: Step 33, Feature Extraction: Extract multi-scale depth features of the input image through an improved backbone network; Step 34, Feature Fusion: Multi-scale features are weighted and fused using the bidirectional feature pyramid module; Step 35, Mask Prediction: The detection segmentation head simultaneously outputs the target's category, bounding box coordinates, confidence score, and pixel-level segmentation mask; Step 36, Accuracy Validation: Perform accuracy validation on the validation set for the trained network and calculate the core evaluation metrics.

[0074] Step 38, Accuracy Threshold Judgment: Compare the accuracy index obtained from the verification with the preset threshold. If the threshold requirement is not met, return to the network training step to continue iterative optimization; if the threshold requirement is met, save the network model trained in Step 391.

[0075] Step 39, Practical Application: Deploy the trained network model on an edge computing device to perform end-to-end inference on the video stream collected in real time in step 392, and output the category, location and pixel-level mask of the animals that damage the dike in step 393, which are then sent to the subsequent spatial localization unit for processing.

[0076] This embodiment solves the problem of low recognition accuracy of traditional detection networks for small dike-damping animals and animals with complex postures in complex dam area environments through the above-described network training and application process. Example 5: See Figure 4 This embodiment provides a method for implementing the trajectory prediction unit of the present invention. The trajectory prediction process, which integrates long short-term memory networks and kinematic methods, is as follows: Step 41, Simulated Trajectory Data Generation: Since it is difficult to obtain a large amount of on-site levee damage animal trajectory data, a simulated trajectory dataset containing multiple movement modes is first generated. Taking into account the movement characteristics of the levee damage animals in Step 411 (velocity), Step 412 (acceleration), and Step 413 (route form), random route data in Step 414 is generated, covering multiple typical movement modes. Gaussian noise consistent with actual positioning errors is added to the trajectory data to simulate real-world application scenarios.

[0077] Step 42, Dataset Partitioning: The generated simulated trajectory data is divided into the training set (Step 421) and the validation set (Step 422) according to a preset ratio, which are used for training and performance evaluation of the Long Short-Term Memory Network trajectory prediction model.

[0078] Step 43, Long Short-Term Memory Network Training: Construct and train a trajectory prediction network containing multiple layers of long short-term memory and fully connected layers. The process includes: Step 431, Parameter optimization: An adaptive optimizer is used to iteratively optimize the network parameters using a preset loss function; Step 432, Effect Verification: Perform effect verification on the validation set once every preset training round and calculate the core error index; Step 433, accuracy requirement judgment: If the verification error does not meet the preset accuracy requirement, return to the parameter optimization step to continue training; if the requirement is met, save the trained Long Short-Term Memory network model in Step 434.

[0079] Step 44, Practical Application: The trained Long Short-Term Memory network is combined with a physical kinematic model to predict the movement trajectories of animals damaging the dike in real time. The process includes: Input the actual field data in step 441: Receive the three-dimensional spatial coordinates of the historical time series of animals that damage the dikes output by the spatial positioning unit, and use interpolation to complete the missing coordinate frames to generate a continuous and complete coordinate sequence. Dual-model parallel prediction: The position of the dam-damping animal at the next moment is independently calculated using the uniform-uniform acceleration kinematic model in step 442 and the long short-term memory network in step 434. Step 443, Adaptive weight calculation: Based on the prediction errors of the two models at the previous time step, dynamically calculate the fusion weights. The smaller the prediction error, the greater the weight of the model. Step 444, Weighted prediction result: The final predicted position is obtained through weighted fusion, and the continuous motion trajectory within a preset time period is obtained through iterative calculation and sent to the hierarchical early warning unit.

[0080] This embodiment employs a trajectory prediction method that integrates physical kinematic models and deep learning temporal models, thus balancing the real-time performance of kinematic models with the fitting ability of long short-term memory networks for nonlinear motion. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart prediction and real-time early warning system for animal movement trajectories in high earth-rock dams, characterized in that, include: Multimodal sensing unit, used for all-weather collection of multi-source sensing data on animals that harm the dam in the high earth-rock dam area; The intelligent recognition unit is used to perform target detection and instance segmentation on the perceived data, and output the category, image location and pixel-level outline of the animal that harms the dike; The spatial positioning unit is used to invert the image coordinates of animals that damage the dam into three-dimensional spatial coordinates in the world coordinate system based on the spatial parameters of the sensing equipment and the geographical information of the dam area. The trajectory prediction unit is used to predict the future trajectory of the animal based on its historical time series coordinates and by integrating a physical kinematics model with a deep learning time series model. The graded early warning unit is used to calculate the risk level and trigger the corresponding early warning response based on the spatial relationship between the predicted trajectory and the preset danger zone; The linkage and removal unit is used to perform differentiated on-site warnings, remote notifications, and automatic removal operations based on the warning level.

2. The intelligent prediction and real-time early warning system for animal movement trajectories in high earth-rock dams according to claim 1, characterized in that, The multimodal sensing unit includes a visible light / infrared coaxial imaging device, a motorized gimbal, a hybrid power supply module, and a data transmission module; The visible light / infrared coaxial imaging device is fixed on an electric pan-tilt head and simultaneously acquires visible light and infrared thermal imaging data. The electric gimbal supports horizontal omnidirectional and vertical pitch adjustment, and integrates an attitude stabilization system to suppress environmental vibration interference. The hybrid power supply module consists of an energy storage unit and a renewable energy auxiliary power supply unit, providing continuous power to the system in the field. The data transmission module supports both wireless and wired dual-link communication, enabling real-time data uploading and remote command distribution.

3. The intelligent prediction and real-time early warning system for animal movement trajectories in high earth-rock dams according to claim 1, characterized in that, The intelligent recognition unit adopts an improved single-stage target detection network. Its backbone network uses a dynamic convolution module to adapt to complex pose targets, and the neck network uses a bidirectional feature pyramid network to achieve multi-scale feature weighted fusion. The detection head integrates a channel-space attention module to enhance the response of key features.

4. The intelligent prediction and real-time early warning system for animal movement trajectories in high earth-rock dams according to claim 1, characterized in that, The spatial positioning unit is specifically used to: establish the mapping relationship between the world coordinate system, the imaging device coordinate system and the image coordinate system; obtain the intrinsic parameter matrix according to the calibration parameters of the imaging device; obtain the extrinsic parameter matrix according to the device installation position and the real-time attitude of the gimbal; solve the depth information based on the prior assumption of the movement of animals on the dam surface and the digital elevation model of the dam area; and finally complete the conversion from image coordinates to three-dimensional world coordinates.

5. The intelligent prediction and real-time early warning system for animal movement trajectories in high earth-rock dams according to claim 1, characterized in that, The trajectory prediction unit is specifically used to: interpolate and complete missing frames in the historical coordinate sequence; calculate the future position of the dam-damaging animal using both a kinematic model and a Long Short-Term Memory (LSTM) network; adaptively calculate weight coefficients based on the historical prediction errors of the two models; and obtain the final predicted trajectory through weighted fusion.

6. The intelligent prediction and real-time early warning system for animal movement trajectories in high earth-rock dams according to claim 1, characterized in that, The graded early warning unit is specifically used to: divide the dam area into a safe zone, an early warning zone, and a warning zone according to the importance of the dam structure; calculate the expected arrival time and invasion probability of animals that harm the dam into each dangerous zone based on the predicted trajectory; comprehensively determine four risk levels—low, medium, high, and extremely high—and trigger corresponding early warnings.

7. The intelligent prediction and real-time early warning system for animal movement trajectories in high earth-rock dams according to claim 1, characterized in that, The linkage expulsion unit includes: The on-site warning module includes a brightness-adaptive display device and an intelligent audio playback device, used to display warning information in real time and play warning voice and eviction audio. The remote notification module is used to send tiered early warning information and handling instructions to the mobile terminals of management personnel; The automatic deportation module is used to automatically activate non-contact deportation equipment under high-risk conditions to accurately drive away animals that may harm the dike.

8. A method for intelligent prediction and real-time early warning of animal movement trajectories in high earth-rock dams, characterized in that, The method, based on the early warning system according to any one of claims 1-7, includes the following steps: S1. Multimodal data acquisition: Collect multi-source sensing data on animals that damage the dam around the clock using multimodal sensing devices deployed in the dam area; S2. Intelligent identification of animals that damage dikes: An improved single-stage target detection network is used to process the perceived data and identify the category, image location and pixel-level outline of animals that damage dikes. S3. Three-dimensional spatial coordinate inversion: Based on the spatial parameters of the sensing equipment and the geographical information of the dam area, the image coordinates of the animals that damage the dam are converted into three-dimensional spatial coordinates in the world coordinate system. S4. Intelligent prediction of movement trajectory: Based on the historical time series coordinates of the animals that cause damage to the dike, the physical kinematics model and the deep learning time series model are integrated to predict their future movement trajectory. S5. Risk Level Assessment and Early Warning: Calculate the risk level and trigger the corresponding early warning response based on the spatial relationship between the predicted trajectory and the preset danger zone; S6. Joint Removal and Handling: Execute differentiated on-site warnings, remote notifications, and automatic removal operations based on the warning level, and record the entire event process.

9. The early warning method according to claim 8, characterized in that, In step S1, image data under different lighting conditions are collected simultaneously using a visible light / infrared coaxial imaging device, and the automatic cruise function of the electric pan-tilt unit is used to achieve full coverage monitoring of key areas in the dam area.

10. The early warning method according to claim 8, characterized in that, In step S2, the training method of the improved single-stage target detection network includes: constructing a dataset of animals that cause damage to dikes under different lighting conditions, weather conditions, and postures; expanding the sample diversity through data augmentation; using transfer learning for domain-adaptive fine-tuning; and using a multi-task loss function to simultaneously optimize detection and segmentation accuracy.

11. The early warning method according to claim 8, characterized in that, In step S3, the intrinsic parameter matrix is ​​obtained through camera calibration, the extrinsic parameter matrix is ​​obtained through equipment installation parameters and gimbal real-time attitude, and the depth information is solved by combining the digital elevation model of the dam area to complete the mapping from pixel coordinates to three-dimensional world coordinates.

12. The early warning method of claim 8, wherein, In step S4, the missing coordinate frames are filled using a quadratic interpolation method. The future positions are predicted using a uniform-acceleration kinematic model and an LSTM network, respectively. The weights are dynamically adjusted based on the prediction errors of the two models at the previous time step, and the weighted fusion is used to obtain the final prediction result.

13. The early warning method of claim 8, wherein, In step S5, the dam area is pre-divided into a safe zone, a warning zone, and a restricted zone. The estimated arrival time of animals that may damage the dam is calculated for each zone. When the estimated arrival time is greater than the first threshold, it is considered low risk; when it is between the first and second thresholds, it is considered medium risk; when it is between 0 and the second threshold, it is considered high risk; and when the animals have entered the restricted zone, it is considered extremely high risk.

14. The early warning method of claim 8, wherein, In step S6, the differentiated expulsion operation specifically involves: continuous monitoring of low-risk cases; For medium-risk situations, an on-site warning will be displayed and a remote notification will be sent; for high-risk situations, an audible and visual warning will be activated and an emergency notification will be sent; for extremely high-risk situations, automatic decoy equipment will be activated and management personnel will be notified to take immediate on-site action.

Images