Hydro-junction fish passage fish behavior dynamic monitoring and data analysis method

Abstract

The invention discloses a hydro-junction fish passage fish behavior dynamic monitoring and data analysis method, and belongs to the field of hydro-junction fish school ecological monitoring. According to the method, multiple types of sensors are arranged at key nodes of a fish passing channel, multi-modal synchronous fusion of video, sonar and environmental data is realized based on a global clock, and time alignment and spatial relocation processing are performed on the data; continuously identifying fish targets and behavior changes thereof in real time by utilizing feature extraction and motion trail splicing; through behavior clustering and event interpretation, microscopic behaviors of fish individuals and groups and large-scale behavior events in channels are distinguished, and modeling analysis is carried out on the causal relationship between fish behaviors and environmental parameters in combination with integrated nerve causal tensor mapping. The system can achieve dynamic early warning and feedback of abnormal behaviors, and the intelligent level of ecological protection of the hydro-junction and smooth management of fish swimming is improved.

Description

Technical Field

[0001] This invention belongs to the field of fish ecological monitoring in water conservancy projects, and more specifically relates to a method for dynamic monitoring and data analysis of fish behavior in fish passages of water conservancy projects. Background Technology

[0002] With the increasing demand for ecological environmental protection and sustainable development of water conservancy projects, the construction and scientific management of fish passageways have become an important component of modern water conservancy projects. As core members of aquatic ecosystems, the behavior patterns of fish in artificial passageways directly affect the protection of aquatic biodiversity and the maintenance of aquatic ecosystem health. Traditional methods for monitoring fish behavior mostly rely on manual observation or single-type sensors, such as ordinary surveillance cameras and sonar probes. These methods are often limited by space, lighting, and water visibility, making it difficult to continuously and accurately capture the true movement paths and complex interactions of fish.

[0003] Meanwhile, current research on fish group behavior identification, anomaly detection, and ecological response mostly remains at the stage of offline statistical analysis or simple behavioral parameter judgment, lacking the ability to fuse multiple information, continuously track in time and space, and accurately model ecological causal relationships. The multiple disturbances in the underwater environment, the high complexity of fish behavior, and the dynamic characteristics of group behavior create significant bottlenecks for traditional methods in real-time monitoring, anomaly early warning, and data self-learning evolution.

[0004] Therefore, there is an urgent need for an intelligent monitoring and closed-loop feedback technology that can integrate multimodal sensor data, possess spatiotemporal consistency and autonomous learning capabilities, in order to improve the precision of fish behavior recognition and the depth of ecological event analysis, and effectively support scientific decision-making in aquatic ecosystem management and hub operation. Summary of the Invention

[0005] This invention aims to address the common problems in current fish behavior monitoring at water conservancy projects, such as insufficient multimodal data fusion, coarse identification of individual and group behaviors, difficulty in achieving spatiotemporal continuity and efficient anomaly detection, and lack of ecological causal mechanism modeling and intelligent feedback loop. By constructing an integrated monitoring system that combines multi-source feature collaborative extraction, autonomous trajectory shaping, swarm intelligence aggregation, real-time early warning of behavioral anomalies, and in-depth ecological causal analysis, this invention achieves high-precision, full-process, and self-evolving intelligent monitoring and management of fish passage behavior, providing strong support for water ecological security and engineering operation optimization.

[0006] A method for dynamic monitoring and data analysis of fish behavior in fish passages of water conservancy projects, characterized by the following steps:

[0007] Step S1: Multimodal synchronous acquisition and preprocessing of video, sonar and environmental data sensors are deployed at key nodes of the fish passage. Through timestamp alignment and spatial relocation, the video, sonar and environmental data are synchronously fused to reduce the signal-to-noise ratio.

[0008] Step S2: Integrating feature extraction and autonomous return trajectory shaping

[0009] Extract the spatial morphology, motion vectors, and surrounding environmental change features of fish targets to generate multidimensional feature tensors; stitch together the fish body motion trajectories to achieve continuous recognition of fish targets in dynamic environments and generate cross-modal continuous motion trajectory tensors;

[0010] Step S3, Target Aggregation and Local Behavior Recognition

[0011] The fish populations are divided into subgroups and individual behavioral groups. The aggregation and subtle changes of each subgroup or individual are analyzed in real time, and the changes in behavioral status are dynamically tracked.

[0012] Step S4: Large-scale behavioral event identification and ecological causal modeling

[0013] Key behavioral events during the overall passage of fish schools through the channel, such as collective acceleration, congestion, and opposite flow, are interpreted; fish school behavioral events are correlated with environmental variables, such as flow velocity, light intensity, and sonar disturbances.

[0014] Step S5: Feedback and Dynamic Early Warning

[0015] By comparing historical normal behavior patterns with current real-time monitoring results, the critical threshold is dynamically refreshed. Once a large-scale abnormal behavior event is detected, such as collective stagnation or extreme reverse swimming, feedback is given to the water conservancy hub management. The timestamp alignment and spatial relocation in step S1 involve introducing a global master clock, uniformly allocating initial timestamps, and fine-tuning all data with the corrected time series. The geometric relationship of the underwater camera and sonar sensor placement is modeled as a rigid body transformation projection matrix to achieve mutual mapping of spatial coordinates. After unifying the data of different modalities in the time and spatial dimensions, a joint observation tensor is constructed.

[0016] If some sensor data is missing frames or out of sync, the nearest neighbor spatiotemporal interpolation resampling method is used to automatically fill in the missing data segments based on the shortest time distance and the nearest neighbor mapping in Euclidean space, so as to ensure data integrity and alignment accuracy.

[0017] The fusion feature extraction in step S2 is to extract the spatial morphological features, instantaneous motion vectors and local environmental change features of the fish target from different modal channels, taking each moment as the analysis base point. Through modal weighted stitching, a multidimensional fusion feature tensor for each target moment is generated.

[0018] The spatial morphological features are represented as vectors by the principal axis shape features of the pixel region, including the fish body principal axis length, width, aspect ratio, and sixth-order Hu moment features;

[0019] The instantaneous motion vector is obtained by differentiating the centroids of the principal axes in consecutive frames;

[0020] The environmental change characteristics consist of local water flow, sonar scattering energy, temperature, or turbidity normalized parameters.

[0021] The process of splicing the fish's movement trajectory in step S2 involves combining information from the same fish at consecutive moments to ultimately infer the complete trajectory. Specifically:

[0022] The set of detected targets is defined in each frame, and the pairing relationship between any target between t and t+1 is established by a feature similarity measurement function; this distance function integrates Euclidean / Mahathano distance weighting of spatial location, motion trend and local environmental changes;

[0023] Based on the distance matrix, the core idea of ​​the Hungarian algorithm, which minimizes the global optimal pairing, is transformed into a self-iterative matching mechanism: each time, only feature pairs based on the nearest neighbor below the threshold are allowed to be spliced ​​into the same fish trajectory; otherwise, a new trajectory branch is created.

[0024] The continuity of each trajectory is predicted by the survival probability—based on the probability of fish appearance, to prevent misjudgments caused by short-term occlusion or data loss.

[0025] The target aggregation in step S3 is based on the obtained multidimensional fusion feature tensor sequence to achieve aggregation discrimination and micro-variation recognition of individual fish and subgroup behaviors, specifically:

[0026] The aggregation discrimination is based on simulating the local potential field and collective behavioral dynamics among fish groups. Specifically, the distance between fish bodies and the velocity vector field at each moment are abstracted into dynamic interaction potential energy. The instantaneous potential field formed by all fish bodies is smoothed by Gaussian to obtain the local density function. Based on the local density extrema and the same velocity direction, an adaptive neighborhood aggregation discrimination is adopted. For neighborhood overlap and member continuity at multiple spatial scales, a multi-level fish subgroup label is formed through intra-scale aggregation and inter-scale subdivision strategy, taking into account both instantaneous patterns and persistent trends.

[0027] The micro-change identification involves extracting behavioral sequences in real time for each subgroup or individual and monitoring minute mutation events; autonomously identifying abnormal change points based on temporal behavioral state vectors; calculating a behavior indicator function for each trajectory over consecutive time periods; performing real-time weighted scoring; and determining that a turning, clustering, or sluggish micro-change event has occurred if the score exceeds the adaptive behavior threshold, and recording the start and end times of the event.

[0028] The large-scale behavioral event identification in step S4 specifically involves:

[0029] The continuous behavioral states of all individuals and subgroups and the overall aggregated labels are spliced ​​together in time to construct a multi-scale weighted temporal spectrum in the global feature space of the fish swarm, thereby realizing the identification of large-scale behavioral events.

[0030] The progressive window cascade algorithm is adopted. First, local extreme values ​​and behavioral change zones are extracted in smaller time windows. Then, the maximum, minimum and mean features are statistically analyzed in higher time windows to gradually lock in the range of significant changes in collective properties.

[0031] Collective acceleration is defined by an indicator; if it exceeds the historical average threshold, it is determined to be a collective acceleration event.

[0032] The blockage phenomenon is identified by the criterion of a sudden decrease in global spatial distribution variance and a bipolar increase in local density;

[0033] The opposite flow forms a principal component feature chain of the behavior spectrum based on the significant increase in the global velocity vector variance, and finally outputs key event segments and their corresponding behavior pattern labels.

[0034] The ecological causal modeling in step S4 involves dynamically associating each identified behavioral event with the real-time environmental parameter data stream; using an integrated neural causal tensor mapping, the intensity sequence of behavioral events and the environmental driving input are jointly embedded into a multi-layer nonlinear mapping network to output behavioral response components.

[0035] This invention significantly improves the accuracy and timeliness of fish behavior event identification in water conservancy projects by constructing a multimodal fusion and fully intelligent fish behavior monitoring and feedback system. It overcomes the limitations of traditional systems relying on single sensors and manual interpretation, such as information fragmentation and recognition lag. The system can adaptively integrate multi-source data, accurately track the spatiotemporal dynamics of individual fish and fish groups, and identify complex ecological events such as collective anomalies, abnormal lingering, and reverse swimming in real time. It also combines environmental parameters to deeply analyze the ecological driving forces behind these behaviors, enabling dynamic assessment and optimization of the ecological state of fish passageways. Compared with existing technologies, this invention not only possesses high versatility, strong adaptability, and proactive early warning capabilities, but also feeds all detection and response data back into the algorithm model, achieving a real-time self-learning closed loop. This significantly reduces the risks of manual intervention and false or missed reports, providing reliable and efficient intelligent technical support for water ecological protection and refined management of water projects. Attached Figure Description

[0036] Figure 1 This is a flowchart of the method of the present invention.

[0037] Figure 2 This is a flowchart of the multimodal synchronous acquisition and preprocessing process of the present invention.

[0038] Figure 3 This is a flowchart illustrating the integration of feature extraction and autonomous return trajectory shaping in this invention.

[0039] Figure 4 This is a flowchart of the target aggregation and local behavior recognition process of the present invention.

[0040] Figure 5 This is a flowchart of the large-scale behavioral event recognition and ecological causal modeling of the present invention. Detailed Implementation

[0041] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Typical embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0042] Example 1: A method for dynamic monitoring and data analysis of fish behavior in fish passages of water conservancy projects, specifically including the following steps:

[0043] Step S1: Multimodal synchronous acquisition and preprocessing

[0044] S101 deploys multiple types of sensor systems at key nodes of the fish passage, including high-resolution underwater cameras, sonar arrays, and environmental parameter monitors.

[0045] First, high-resolution underwater cameras (in modules) are deployed at key nodes of the fish passage. (representation), sonar array (in modules) (representation) and environmental parameter monitoring instruments (in modules) (represented by...) Each sensor independently collects its raw data stream. The data stream produced by each type of sensor is denoted as... , and , where t is the acquisition timestamp, (x, y) are the two-dimensional positioning coordinates, and d is the sonar range layer parameter.

[0046] S102 is designed with a "Multimodal Dynamic Synchronous Integration (MDSI)" mechanism, which synchronizes and fuses video, sonar, and environmental data through timestamp alignment and spatial relocation.

[0047] The Multimodal Dynamic Synchronization Integration (MDSI) mechanism first introduces a global master clock to uniformly allocate initial timestamps. All data is based on The corrected time series is fine-tuned; for multi-source spatial positioning differences, the geometric relationship of the underwater sensor placement of cameras and sonar is modeled as a rigid body transformation projection matrix. To achieve mutual mapping of spatial coordinates: After unifying the data from different modalities in both the temporal and spatial dimensions, a joint observation tensor is constructed. ,in This represents the channel index for different modalities. For situations where some sensor data suffers from frame loss or asynchrony, the MDSI mechanism employs the nearest neighbor spatiotemporal interpolation resampling method, based on... (Shortest time distance) and Euclidean space nearest neighbor mapping automatically fill in missing data segments to ensure data integrity and alignment accuracy.

[0048] S103 employs adaptive signal denoising and enhancement algorithms to reduce the signal-to-noise ratio and establish a high-quality data foundation for subsequent analysis.

[0049] The specific process of the adaptive signal denoising and enhancement algorithm is as follows:

[0050] First of all Adaptive frequency domain filtering is performed on each modal data to parameterize the sensor modal noise model into noise spectral density. Based on real-time observed signal power spectrum The filter bandwidth is dynamically adjusted, and the filter output is:

[0051] ;

[0052] in For Fourier transform operators;

[0053] Based on this, a local adaptive contrast and dynamic range enhancement algorithm is applied, with the following block contrast gain coefficients:

[0054] ;

[0055] and These represent the standard deviations of the local signal and noise statistical distributions, respectively. To stabilize small constants, the signal amplitude in each region is adjusted to suppress non-biological signals such as underwater floating objects and reflective interference, thereby improving the effectiveness of subsequent target identification. This series of processing steps outputs high-quality, multimodal, and spatially and temporally aligned data tensors, providing solid underlying data support for precise fish behavior analysis and dynamic tracking.

[0056] Step S2: Integrating feature extraction and autonomous return trajectory shaping

[0057] S201, for the multimodal data at each time step, uses "Cross-Domain Feature Stitching (COFS)" to jointly extract the spatial morphology, motion vectors and surrounding environmental change features of the fish target, and generate a multidimensional feature tensor;

[0058] First, regarding the preprocessed multimodal synchronization data tensor... This paper proposes a Cross-Domain Feature Stitching (COFS) algorithm. This algorithm uses each time step as the analysis base point to extract the spatial morphological features, instantaneous motion vectors, and local environmental change features of fish targets from different modal channels (such as video, sonar, environmental parameters, etc.). Specifically, the spatial morphology is represented as a vector by the principal axis shape features of the pixel region. Including the length of the main axis of the fish body ,width Aspect Ratio And the sixth-order Hu moment characteristics; the instantaneous motion vector is obtained by the principal axis centroid difference of consecutive frames. ,in This represents the difference between the current position and the centroid of the previous frame. For the modulus speed, The direction of motion is indicated. Environmental change characteristics are composed of normalized parameters such as local water flow, sonar scattering energy, temperature, or turbidity. Finally, through modal weighted stitching, a multidimensional fused feature tensor for each target time step is generated. ,in , , These are the weights for spatial, motion, and environmental features, respectively, and the weights can be adaptively adjusted based on data correlation.

[0059] S202. Based on the above feature tensors, the "Self-Iterative Trajectory Shaping (SITS)" process is adopted to automatically associate the multimodal performance of the same fish body in continuous time series, and to initially splice the fish body's motion trajectory, so as to realize the continuous recognition of fish targets in dynamic environments.

[0060] Feature tensor sequences based on temporal arrangement The Self-Iterative Trajectory Forming (SITS) process is initiated. Based on auto-incrementing similarity matching and a trajectory growth model in the feature space, the SITS process automatically combines information from the same fish at consecutive time points to ultimately infer the complete trajectory. First, the set of detected targets within each frame is defined. The pairing relationship of any target between t and t+1 is determined by the feature similarity measurement function. Established. This distance function integrates Euclidean / Mahanobis distance weights based on spatial location, movement trends, and local environmental changes. The expression is:

[0061] ;

[0062] in This is the adaptive balance factor for each feature subdomain;

[0063] Based on the distance matrix, the core idea of ​​the Hungarian algorithm, which minimizes the globally optimal pairing, is modified into a self-iterative matching mechanism: each time, only feature pairs based on nearest neighbors below a threshold are allowed to be concatenated into the same fish trajectory; otherwise, a new trajectory branch is created. The continuity of each trajectory is predicted by the survival probability—through the probability of fish appearance. ( To prevent misinterpretations caused by short-term occlusion or data loss, a sigmoid function and negative weights are used to accelerate and suppress abnormal stitching. This ultimately leads to the adaptive stitching of cross-modal, multi-source data into a time-series trajectory set for each fish. , where each trajectory This provides a continuous data foundation for subsequent behavior analysis and judgment.

[0064] Step S3, Target Aggregation and Local Behavior Recognition

[0065] S301, using the fusion features and spatiotemporal trajectories from step 2, achieve "multi-scale spatiotemporal aggregation discrimination": by simulating local potential field changes and collective behavior patterns, adaptively divide fish subgroups and individual behavior groups;

[0066] Based on the multimodal fusion feature trajectory set obtained in step S2 and the corresponding feature tensor sequence This enables the aggregation and discrimination of individual and group behaviors of fish, as well as the identification of subtle changes.

[0067] First, the local potential field and collective behavioral dynamics among the fish swarm are simulated using the Multi-Scale Spatiotemporal Convergence Discriminant (MSTAC) algorithm. Specifically, the distance between fish and the velocity vector field at each moment are abstracted as dynamic interaction potential energy, and the interaction potential between the i-th fish and the j-th fish is defined as:

[0068] ;

[0069] in Let i be the position of the center of gravity of the i-th fish. Its velocity vector, , , These are weighted parameters such as distance and cooperative coupling. The instantaneous potential field formed by all fish bodies. After Gaussian smoothing, the local density function is obtained. Based on local density extrema and velocity homogeneity, an adaptive neighborhood aggregation discrimination method is adopted to classify fish bodies that satisfy the following formula into the same behavioral main group; for neighborhood overlap and member continuity at multiple spatial scales, a multi-level fish subgroup label is formed through intra-scale aggregation and inter-scale subdivision strategy, taking into account both instantaneous patterns and persistent trends.

[0070] ;

[0071] in, Density threshold The angle between the velocity vectors, As a criterion for same directionality;

[0072] S302, under the aggregation discrimination label, analyzes the micro-mutations of behavior of each subgroup or individual in real time, including turning, gathering and dispersing, sluggishness, etc., and dynamically tracks changes in behavioral status.

[0073] Building upon the above, the system then employs Micro-Modulation Adaptive Detection (MBAD) to extract behavioral sequences in real time for each subgroup or individual, monitoring for minute mutation events. The core mechanism is based on temporal behavioral state vectors, autonomously identifying anomalous change points. For each trajectory, the following formula is used to calculate the behavior indicator function for consecutive time steps, representing the turning amplitude, clustering degree, and activity level, respectively.

[0074] ;

[0075] The steering angle is determined by the following formula, which changes the direction of velocity;

[0076] ;

[0077] Clustering and dispersion are measured by the average distance mutation rate of subgroups. Given: the variance of the rate of activity utilization. Characterization; the following behavioral micro-variation recognition operators are used for real-time weighted scoring:

[0078] ;

[0079] like Exceeding the adaptive behavior threshold (Based on historical behavioral fluctuation statistics that are self-updated), the system determines the occurrence of minor changes such as turning, gathering, dispersing, or slowing down, and records the start and end times of these events. Through real-time tracking of these fine-grained behavioral dynamics, the system can detect potential ecological anomalies in the evolution of group behavior and individual divergence trends at the first opportunity, enabling refined monitoring and analysis of fish behavior in water conservancy projects.

[0080] Step S4: Large-scale behavioral event identification and ecological causal modeling

[0081] S401, based on the behavioral sequences and aggregation patterns identified in step 3, integrates the "Multi-level Progressive Behavior Spectrum (MLPBS)" algorithm to perform high-level interpretation of key behavioral events such as collective acceleration, blockage, and opposite flow during the overall fish swarm crossing the channel.

[0082] Using the individual / group behavior sequences and their aggregation patterns output in step 3 as core inputs, the algorithm performs high-level identification of key behavioral events on a large scale in fish schools via the Multi-Level Progressive Behavior Spectrum (MLPBS) algorithm. MLPBS first analyzes the continuous behavioral states of all individuals and subgroups. and overall aggregation tags Temporal splicing is performed to construct the following multi-scale weighted temporal spectrum on the behavioral indicator space (such as turning rate, group velocity change, density aggregation, etc.);

[0083] ;

[0084] in For individual instantaneous speed, For individual shifts and changes, Let K be the local density of the k-th aggregate subgroup. The variance of the spatial distribution of the fish population;

[0085] MLPBS employs a progressive window cascade algorithm, first extracting local extrema and behavioral change zones in smaller time windows, and then recursively statistically analyzing the maximum / minimum / mean features in higher time windows to gradually lock in the range of significant changes in collective properties.

[0086] For collective acceleration, the following interpretation criteria can be defined:

[0087] ;

[0088] like Exceeding the historical average If the threshold is reached, it is determined to be a large-scale acceleration event;

[0089] The congestion phenomenon is caused by a sudden decrease in the variance of the global spatial distribution. Furthermore, the local density bipolar increase criterion is used for identification;

[0090] The opposite flow forms a principal component feature chain of the behavior spectrum based on the significant increase in the variance of the global velocity vector, and finally outputs the key event segments and their corresponding behavior pattern labels.

[0091] ;

[0092] S402, combined with real-time environmental parameters, synchronously establishes an "ecological causal deep mapping (ECDM)" model, automatically correlates fish behavior events with environmental variables (flow velocity, light intensity, sonar disturbance, etc.), and reveals the mechanism of action of different influencing factors;

[0093] While identifying large-scale behavioral events, the system simultaneously launches an Ecological Causal Deep Mapping (ECDM) model, linking each identified behavioral event with real-time environmental parameter data streams. (e.g., flow velocity, light intensity, sonar disturbance power, etc.) are dynamically correlated;

[0094] ECDM employs integrated neural causal tensor mapping to integrate behavioral event intensity sequences. The environment-driven input is embedded together with a multi-layer nonlinear mapping network to output behavioral response components: ;

[0095] in For parameter-learnable causal response mapping, For the corresponding network weights;

[0096] During model training, the loss for minimizing causal explanations is:

[0097] ;

[0098] To achieve in-depth understanding of the mechanisms by which ecological and environmental variables affect specific behavioral events;

[0099] Ultimately, ECDM can output the ranking of the main environmental driving factors, contribution weights, and response thresholds for each key behavioral event, providing quantifiable evidence to assist in the operation of the hub and the optimization of ecological hydraulics.

[0100] Step S5: Feedback and Dynamic Early Warning

[0101] By comparing historical normal behavior patterns with current real-time monitoring results, the critical threshold is dynamically updated. Once a large-scale abnormal behavior event is detected (such as mass stagnation, extreme reverse swimming, etc.), the system immediately feeds back to the water conservancy hub management end, generating detailed event assessments, impact analyses, and operational recommendations. All feedback and abnormal response data will be collected and traced back to the multimodal feature library and causal model constructed in steps S2-4 to continuously optimize the accuracy of algorithm discrimination and achieve intelligent closed-loop self-evolution.

[0102] First, leveraging the multimodal feature library and standardized behavioral event labels accumulated in previous steps, the system automatically establishes a baseline of historical behavioral patterns for fish groups and individuals. It can also continuously and adaptively fine-tune the "normal behavior range" based on different scenarios, seasons, and environmental variables. With the arrival of new real-time data, the system performs multi-dimensional comparisons between the observed fish behavior spectrum and historical data. Utilizing a statistical threshold adaptive update mechanism, it automatically refreshes the discrimination thresholds for various key behaviors, significantly reducing the probability of false alarms and false negatives.

[0103] Based on dynamic criteria, abnormal behavioral events such as large-scale fish stagnation, reverse congestion in passageways, and a sudden drop in fish passage efficiency are detected, immediately triggering an active anomaly response program. The system not only records in detail the type, location, time period, and individual fish involved in the abnormal behavior, but also provides potential environmental drivers, anomaly trend levels, and direct impacts on the fish passage ecosystem using a comprehensive causal model. For different levels of anomalies, the system automatically generates detailed event analysis reports, combining environmental parameters, behavioral change characteristics, and comparisons with the same period in previous years. These reports are then pushed to the water conservancy project management platform, providing on-site maintenance personnel with intuitive data visualizations and targeted operational suggestions, such as temporarily optimizing flow rates, adjusting lighting, or implementing other induction measures, effectively improving management efficiency and biodiversity conservation levels.

[0104] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0105] Example 2: A dam-type water conservancy project has an artificial fish passage (100 meters long, 3 meters wide, and 2 meters deep). The main economic fish species in the river where the dam is located are Chinese sturgeon, silver carp, and common carp.

[0106] The monitoring period is during the spring migration season, from March 10, 2024 to April 20, 2024.

[0107] The specific methods for dynamic monitoring and data analysis of fish behavior in the fish passage of this water conservancy project are as follows:

[0108] Step S1: Multimodal synchronous acquisition and preprocessing

[0109] Five high-definition video cameras and three high-precision sonars are deployed inside and outside the passage, and are stamped with a unified timestamp under the guidance of the master clock.

[0110] The monitoring system automatically captures video, sonar, and environmental variables such as water temperature, flow rate, and illumination in real time. It models a rigid body transformation matrix according to the spatial layout of the sensors and completes dynamic spatial correction. If a camera, numbered C2, loses its 500th frame, the system uses the two nearest frames (0.06 seconds away) and adjacent sonar data to perform nearest-neighbor interpolation and resampling to fill in the visual gaps.

[0111] The key parameters of the multimodal acquisition system are configured as shown in Table 1 below:

[0112] Table 1

[0113]

[0114] Step S2: Integrating feature extraction and autonomous return trajectory shaping

[0115] Fish bodies are detected in real time from the video using the YOLOv7 model, and their contour length / area is extracted. In this embodiment, the average length of Chinese sturgeon is 1.2m and that of carp is 0.5m. The synchronous calculation speed is as follows: the average speed of Chinese sturgeon can reach 50cm / s.

[0116] Sonar tracking revealed that the fish's movement trajectory exhibited obvious distortion or stagnation characteristics in high-velocity areas (above 1.3 m / s).

[0117] The system configured according to the method in the embodiment automatically stitches together the temporal positions of the fish body to generate a cross-modal continuous motion trajectory tensor.

[0118] Step S3, Target Aggregation and Local Behavior Recognition

[0119] By aggregating spatiotemporally neighboring fish using the DBSCAN algorithm, the number of subgroups was determined, and a total of 13 small groups of Chinese sturgeon were detected within the same time period.

[0120] Within an individual / subgroup, subtle behavioral changes are detected every 30 seconds, including sudden collective shifts or localized dense clustering and dispersal, enabling real-time labeling.

[0121] Step S4: Large-scale behavioral event identification and ecological causal modeling

[0122] Comparative analysis of data from March 12th and April 2nd:

[0123] On March 12, the channel flow rate was 0.6 m / s, and the average passage time for all fish was 38.4 seconds, with no blockages.

[0124] On April 2, the flow velocity in the channel suddenly increased to 1.7 m / s. The system identified 6 incidents of fish swimming backwards in real time and 2 large-scale congestion events. The average passage time was extended to 87.2 seconds, during which the average light intensity dropped to 3 lux.

[0125] Using the causal tensor network comprehensive analysis method in step 4 of Example 1, the corresponding behavioral response weight matrix is ​​output by mapping the event intensity with inputs such as flow velocity, illumination, and sonar anomalies.

[0126] Step S5: Feedback and Dynamic Early Warning

[0127] If congestion lasts for 5 minutes and more than 50% of the individuals are detected turning around and going against the flow, the system will trigger a red alert and automatically upload the information to the management terminal, suggesting that the gate flow rate be appropriately reduced.

[0128] Data from the historical model shows that after regulation, the flow velocity dropped back to 0.8 m / s, fish resumed normal one-way passage, and congestion and abnormal migration events disappeared.

[0129] The method of this invention, unlike traditional manual methods which cannot identify behavioral anomalies and congestion in real time and quantitatively, can automatically issue an alarm more than 5 minutes in advance. After system intervention and control, abnormal events disappear, throughput efficiency increases by more than 30%, and the normal passage rate of the channel increases from about 75% to more than 94%, greatly improving the ecological fish passage effect. It accurately captures causal relationships; for example, a flow velocity threshold exceeding 1.5 m / s can easily trigger collective reverse flow or stagnation, providing strong decision support for water conservancy scheduling. Specific data comparisons are shown in Table 2 below.

[0130] Table 2

[0131]

Claims

1. A method for dynamic monitoring and data analysis of fish behavior in fish passages of water conservancy projects, characterized in that... The method includes the following steps: Step S1: Multimodal synchronous acquisition and preprocessing; Video, sonar, and environmental data sensors are deployed at key nodes of the fish passage. By aligning timestamps and relocating spatially, the video, sonar, and environmental data are synchronously fused to reduce the signal-to-noise ratio. Step S2: Integrate feature extraction with autonomous return trajectory shaping; Extract the spatial morphology, motion vectors, and surrounding environmental change features of fish targets to generate multidimensional feature tensors; stitch together the fish body motion trajectories to achieve continuous recognition of fish targets in dynamic environments and generate cross-modal continuous motion trajectory tensors; Step S3, target aggregation and local behavior recognition; The fish populations are divided into subgroups and individual behavioral groups. The aggregation and subtle changes of each subgroup or individual are analyzed in real time, and the changes in behavioral status are dynamically tracked. Step S4: Large-scale behavioral event identification and ecological causal modeling; Key behavioral events during the overall passage of fish schools through the channel, such as collective acceleration, congestion, and opposite flow, are interpreted; fish school behavioral events are correlated with environmental variables, such as flow velocity, light intensity, and sonar disturbances. Step S5: Feedback and dynamic early warning; By comparing historical normal behavior patterns with current real-time monitoring results, the critical threshold is dynamically updated; once a large-scale abnormal behavior event is detected, such as collective stagnation or extreme reverse swimming, it is reported to the water conservancy hub management end.

2. The method for dynamic monitoring and data analysis of fish behavior in a fish passage of a water conservancy project as described in claim 1, characterized in that, Its features The timestamp alignment and spatial relocation in step S1 involve introducing a global master clock, uniformly allocating initial timestamps, and fine-tuning all data using the corrected time series; modeling the geometric relationship of the underwater camera and sonar sensor placement as a rigid body transformation projection matrix to achieve mutual mapping of spatial coordinates; and constructing a joint observation tensor after unifying the data of different modalities in the time and spatial dimensions. If some sensor data is missing frames or out of sync, the nearest spatiotemporal interpolation resampling method is used to automatically fill in the missing data segments based on the shortest time distance and Euclidean space nearest neighbor mapping, ensuring data integrity and alignment accuracy. The fusion feature extraction in step S2 is to extract the spatial morphological features, instantaneous motion vectors and local environmental change features of the fish target from different modal channels, taking each moment as the analysis base point. Through modal weighted stitching, a multidimensional fusion feature tensor for each target moment is generated. The spatial morphological features are represented as vectors by the principal axis shape features of the pixel region, including the fish body principal axis length, width, aspect ratio, and sixth-order Hu moment features; The instantaneous motion vector is obtained by differentiating the centroids of the principal axes in consecutive frames; The environmental change characteristics are composed of local water flow, sonar scattering energy, temperature or turbidity normalized parameters. The process of splicing the fish's movement trajectory in step S2 involves combining information from the same fish at consecutive moments to ultimately infer the complete trajectory. Specifically: The set of detected targets is defined in each frame, and the pairing relationship between any target between t and t+1 is established by a feature similarity measurement function; this distance function integrates Euclidean / Mahathano distance weighting of spatial location, motion trend and local environmental changes; Based on the distance matrix, the core idea of ​​the Hungarian algorithm, which minimizes the global optimal pairing, is transformed into a self-iterative matching mechanism: each time, only feature pairs based on the nearest neighbor below the threshold are allowed to be spliced ​​into the same fish trajectory; otherwise, a new trajectory branch is created. The continuity of each trajectory is predicted by the survival probability—by the probability of fish appearance, to prevent misjudgment caused by short-term occlusion or data loss; The target aggregation in step S3 is based on the obtained multidimensional fusion feature tensor sequence to achieve aggregation discrimination and micro-variation recognition of individual fish and subgroup behaviors, specifically: The aggregation discrimination is to simulate the local potential field and collective behavior dynamics among fish. Specifically, the distance between fish and the velocity vector field at each moment are abstracted into dynamic interaction potential energy. The instantaneous potential field formed by all fish is smoothed by Gaussian to obtain the local density function. Based on the local density extrema and the same velocity direction, an adaptive neighborhood aggregation discrimination is adopted. To address neighborhood overlap and member continuity across multiple spatial scales, a multi-level fish subgroup labeling strategy is employed, combining intra-scale clustering with inter-scale subdivision, to generate labels that take into account both transient patterns and persistent trends. The micro-change identification involves extracting behavioral sequences in real time for each subgroup or individual to monitor minute mutation events; and autonomously identifying abnormal change points based on time-series behavioral state vectors. For each trajectory, calculate the behavior indication function for consecutive time moments; perform real-time weighted scoring; if the score exceeds the adaptive behavior threshold, determine that a turning, convergence, or idling micro-change event has occurred, and record the start and end times of the event. The large-scale behavioral event identification in step S4 specifically involves: The continuous behavioral states of all individuals and subgroups and the overall aggregated labels are spliced ​​together in time to construct a multi-scale weighted temporal spectrum in the global feature space of the fish swarm, thereby realizing the identification of large-scale behavioral events. The progressive window cascade algorithm is adopted. First, local extreme values ​​and behavioral change zones are extracted in smaller time windows. Then, the maximum, minimum and mean features are statistically analyzed in higher time windows to gradually lock in the range of significant changes in collective properties. Collective acceleration is defined by an indicator; if it exceeds the historical average threshold, it is determined to be a collective acceleration event. The blockage phenomenon is identified by the criterion of a sudden decrease in global spatial distribution variance and a bipolar increase in local density; The opposite flow forms a principal component feature chain of the behavior spectrum based on the significant increase in the global velocity vector variance, and finally outputs key event segments and their corresponding behavior pattern labels. The ecological causal modeling in step S4 involves dynamically associating each identified behavioral event with a real-time environmental parameter data stream. An integrated neural causal tensor mapping is used to embed the sequence of behavioral event intensity and the environmental driving input into a multi-layer nonlinear mapping network, and output behavioral response components.

3. The method for dynamic monitoring and data analysis of fish behavior in a fish passage of a water conservancy project as described in claim 1, characterized in that: Step S1 employs an adaptive signal denoising and enhancement algorithm to reduce the signal-to-noise ratio and establish a high-quality data foundation for subsequent analysis. The specific process of the adaptive signal denoising and enhancement algorithm is as follows: First of all Adaptive frequency domain filtering is performed on each modal data to parameterize the sensor modal noise model into noise spectral density. Based on real-time observed signal power spectrum The filter bandwidth is dynamically adjusted, and the filter output is: ； in For Fourier transform operators; Based on this, a local adaptive contrast and dynamic range enhancement algorithm is applied, with the following block contrast gain coefficients: ； and These represent the standard deviations of the local signal and noise statistical distributions, respectively. To stabilize the small constant, the signal amplitude in each region is adjusted to suppress underwater floating objects and reflective interference with non-biological signals, thereby improving the effectiveness of subsequent target identification.

4. The method for dynamic monitoring and data analysis of fish behavior in a fish passage of a water conservancy project as described in claim 1, characterized in that: The target aggregation and local behavior recognition in step S3 utilize the fusion features and spatiotemporal trajectories from step 2 to achieve "multi-scale spatiotemporal aggregation and discrimination": by simulating local potential field changes and collective behavior patterns, it adaptively divides fish subgroups and individual behavior groups; Based on the multimodal fusion feature trajectory set obtained in step S2 and the corresponding feature tensor sequence This enables the aggregation and discrimination of individual and group behaviors of fish, as well as the identification of subtle changes. First, the local potential field and collective behavioral dynamics among fish schools are simulated using the Multi-Scale Spatiotemporal Convergence Discriminant (MSTAC) algorithm; Specifically, the distance between fish bodies and the velocity vector field at each moment are abstracted as dynamic interaction potential energy, and the interaction potential between the i-th fish and the j-th fish is defined as: ； in Let i be the position of the center of gravity of the i-th fish. Its velocity vector, , , For distance and cooperative coupling weight parameters; the instantaneous potential field formed by all fish bodies. After Gaussian smoothing, the local density function is obtained. ; Based on local density extrema and velocity homogeneity, an adaptive neighborhood aggregation discrimination method is used to classify fish bodies that satisfy the following formula into the same behavioral main group; To address neighborhood overlap and member continuity across multiple spatial scales, a multi-level fish subgroup labeling strategy is employed, combining intra-scale clustering with inter-scale subdivision, to generate labels that take into account both transient patterns and persistent trends. ； in, Density threshold The angle between the velocity vectors, As a criterion for same directionality; Under the aggregation and discrimination label, the behavior micro-mutations of each subgroup or individual are analyzed in real time, including turning, gathering and dispersing, and sluggishness, and the changes in behavior status are dynamically tracked. Building upon the above, we proceed to Micro-Modulation Adaptive Detection (MBAD), which extracts behavioral sequences in real time for each subgroup or individual to monitor minute mutation events. The core mechanism is based on temporal behavioral state vectors to autonomously discover abnormal change points. For each trajectory, the following formula is used to calculate the behavior indicator function at continuous time intervals, representing the turning amplitude, clustering degree, and activity level, respectively. ； The steering angle is determined by the following formula, which changes the direction of velocity; ； Clustering and dispersion are measured by the average distance mutation rate of subgroups. Given: the variance of the rate of activity utilization. Characterization; the following behavioral micro-variation recognition operators are used for real-time weighted scoring: ； like Exceeding the adaptive behavior threshold If a change in behavior is detected, it is determined that a turning, gathering, or slowing event has occurred, and the start and end times of the event are recorded. By tracking such fine-grained behavioral dynamics in real time, the system can detect potential ecological anomalies in the evolution of group behavior and individual divergence trends, and realize refined monitoring and analysis of fish behavior in water conservancy hubs.

5. The method for dynamic monitoring and data analysis of fish behavior in a fish passage of a water conservancy project as described in claim 1, characterized in that: The large-scale behavioral event identification and ecological causal modeling in step S4 is based on the behavioral sequences and aggregation patterns identified in step 3. It integrates the multi-level progressive behavior spectrum (MLPBS) algorithm to perform high-level interpretation of key behavioral events such as collective acceleration, blockage, and opposite flow during the overall fish swarm crossing the channel. Using the individual / group behavior sequences and their aggregation patterns output in step 3 as core inputs, the algorithm performs high-level identification of key behavioral events on a large scale in fish schools via the Multi-Level Progressive Behavior Spectrum (MLPBS) algorithm. MLPBS first analyzes the continuous behavioral states of all individuals and subgroups. and overall aggregation tags By performing temporal splicing, the following multi-scale weighted temporal spectrum is constructed in the behavioral indicator space; ； in For individual instantaneous speed, For individual shifts and changes, Let K be the local density of the k-th aggregate subgroup. The variance of the spatial distribution of the fish population; MLPBS employs a progressive window cascade algorithm, first extracting local extrema and behavioral change zones in smaller time windows, and then recursively statistically analyzing the maximum / minimum / mean features in higher time windows to gradually lock in the range of significant changes in collective properties. For collective acceleration, the interpretation metric is defined as follows: ； like Exceeding the historical average If the threshold is reached, it is determined to be a large-scale acceleration event; The congestion phenomenon is caused by a sudden decrease in the variance of the global spatial distribution. Furthermore, the local density bipolar increase criterion is used for identification; The opposite flow forms a principal component feature chain of the behavior spectrum based on the significant increase in the variance of the global velocity vector, and finally outputs the key event segments and their corresponding behavior pattern labels. ； By combining real-time environmental parameters, an ecological causal deep mapping (ECDM) model is simultaneously established to automatically correlate fish behavior events with environmental variables and reveal the mechanism of action of different influencing factors. While identifying large-scale behavioral events, the system simultaneously launches an Ecological Causal Deep Mapping (ECDM) model, linking each identified behavioral event with real-time environmental parameter data streams. Perform dynamic association; ECDM employs integrated neural causal tensor mapping to integrate behavioral event intensity sequences. The environment-driven input is embedded together with a multi-layer nonlinear mapping network to output behavioral response components: ; in For parameter-learnable causal response mapping, For the corresponding network weights; During model training, the loss for minimizing causal explanations is: ； To achieve in-depth understanding of the mechanisms by which ecological and environmental variables affect specific behavioral events; Ultimately, ECDM can output the ranking of the main environmental driving factors, contribution weights, and response thresholds for each key behavioral event, providing quantifiable evidence to assist in the operation of the hub and the optimization of ecological hydraulics.

Images