Fish monitoring management system and method based on multi-source data analysis

Abstract

The invention relates to the technical field of computer vision, in particular to a fish monitoring management system and method based on multi-source data analysis. The method comprises the following steps: performing video image acquisition and enhancement on a target fish school through multi-angle imaging equipment to obtain an enhanced video sequence; performing fish body swimming posture extraction on the enhanced video sequence to obtain a fish body feature original data set; performing body surface feature point classification on the fish body feature original data set to obtain a classification feature point set; constructing a fish body morphological feature vector according to the classification feature point set; performing fish body skeleton positioning and tracking according to the fish body morphological feature vector to obtain skeleton movement trajectory data; and performing body curvature quantification on the skeleton movement track data to obtain body curvature feature data. According to the method, the problem of unmarked individual identification of common cultured fishes is solved through dual biological characteristics of form fingerprints and behavior gaits, and technical support is provided for fine management of aquaculture.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a fish monitoring and management system and method based on multi-source data analysis. Background Art

[0002] Existing computer vision-based automated identification systems still face huge challenges in accurately identifying individuals of common farmed fish under unmarked conditions. Existing technologies mainly rely on obvious markings or color patterns on the fish's surface, making it difficult to effectively identify mainstream farmed species such as sea bass and tilapia that lack these features. At the same time, existing monitoring systems have limited ability to extract fine morphological and behavioral gait characteristics of fish, and are unable to simultaneously capture tiny surface features (such as tiny scars and missing scales) and complex movement patterns (such as tail fin swing frequency and body curvature), resulting in incomplete feature representation. In addition, traditional aquaculture management methods generally use group average indicators for health assessment, lack the ability to identify individual differences and long-term tracking capabilities, and cannot achieve precise and personalized health status monitoring and management.

[0003] In summary, existing technologies have problems such as the inability to perform unmarked individual identification of common farmed fish, the lack of comprehensive analysis capabilities of multi-dimensional features, and the inability to achieve long-term individualized health monitoring, which need to be urgently addressed. Summary of the Invention

[0004] Based on this, it is necessary to provide a fish monitoring and management system and method based on multi-source data analysis to solve at least one of the above technical problems.

[0005] To achieve the above objectives, a fish monitoring and management method based on multi-source data analysis includes the following steps:

[0006] Step S1: Capture and enhance video images of the target fish school using a multi-angle imaging device to obtain an enhanced video sequence; extract the fish swimming posture from the enhanced video sequence to obtain an original data set of fish features;

[0007] Step S2: classifying the surface feature points of the fish body feature original data set to obtain a classification feature point set; constructing a fish body morphological feature vector based on the classification feature point set;

[0008] Step S3: locating and tracking the fish skeleton according to the fish morphological feature vector to obtain skeleton motion trajectory data; quantifying the body curvature of the skeleton motion trajectory data to obtain body curvature feature data; analyzing the tail fin swing of the skeleton motion trajectory data to obtain the tail fin swing timing characteristics; extracting the turning behavior characteristics of the skeleton motion trajectory data according to the body curvature feature data to obtain the turning behavior feature set; analyzing the fish swimming gait according to the tail fin swing timing characteristics and the turning behavior feature set to obtain the fish gait feature spectrum;

[0009] Step S4: Perform multi-source data fusion and behavior change identification on the fish body morphological feature vector and the fish body gait feature spectrum to obtain a behavior change trend graph; perform health assessment on the behavior change trend graph to obtain health status assessment data.

[0010] This method effectively overcomes the challenges of complex underwater environments, rapid fish movement, and difficulty capturing detailed information by constructing a standardized underwater imaging environment, employing an infrared trigger mechanism to accurately capture fish passages, and applying multi-angle, high-resolution imaging, image enhancement, and fish segmentation and tracking. The resulting enhanced video sequences and raw fish feature datasets contain clear morphological information and continuous dynamic swimming posture data from multiple vantage points. This provides high-quality, multi-dimensional, and spatiotemporally correlated foundational data for subsequent fine-grained morphological feature extraction and behavioral gait analysis, significantly improving the accuracy and reliability of subsequent analysis. Fine-grained enhancement preprocessing of feature regions allows for the clear visualization of surface features that are difficult to track with the naked eye, such as minor scars, missing scales, and bifurcated fin rays. Combined with improved corner detection, spot detection, and sub-pixel precision positioning techniques, high-precision extraction of these fine feature points is achieved. Multi-view consistency verification and classification algorithms ensure the stability of feature points and the accuracy of fish type identification. Furthermore, a spatial relationship network of feature points is constructed and their temporal evolution is analyzed to form a unique, time-varying "morphological fingerprint" for each fish. This morphological feature vector, based on minute surface features, provides a critical and stable biological basis for the accurate identification of common farmed fish in unmarked conditions, overcoming the limitations of traditional methods that rely on distinct markings. Using this morphological feature vector as a reference, combined with deep learning pose estimation and a robust temporal tracking algorithm (incorporating motion prediction, structural constraints, multi-frame smoothing, and occlusion inference), the method achieves stable and high-precision tracking of key points in the fish skeleton during complex swimming movements. Furthermore, by quantifying the curvature of the body axis, analyzing bending wave propagation characteristics, assessing segment stiffness and muscle contraction strength, and analyzing tail fin oscillation (including spectrum, asymmetry, stiffness, and propulsion force estimation) and steering behavior (including trajectory, dynamics, energy efficiency, and pattern classification), the unique swimming gait of individual fish is comprehensively and meticulously characterized. This dynamic, quantitative behavioral gait signature complements static morphological information, providing a rich set of dynamic biological indicators for distinguishing individuals and assessing their physiological state and mobility. By performing weighted multimodal fusion on the standardized morphological feature vectors and behavioral gait feature spectra, a more comprehensive and discriminative individual representation is constructed, significantly improving the accuracy and robustness of individual identification under marker-free conditions. Based on the individual identification results, it is possible to track the long-term changes in the growth parameters (body length, height, and growth rate) of individual fish and detect the long-term trends and mutation points of their key behavioral patterns (such as swimming speed and turning frequency). By comparing these growth and behavioral change indicators with preset thresholds or their own historical baselines, early abnormality detection and quantitative scoring of individual health status are achieved.This individualized and refined health assessment capability breaks through the limitations of traditional group average monitoring, providing accurate individual health portraits and decision-making basis for aquaculture management, which is conducive to timely detection of problems, taking targeted measures, and improving aquaculture efficiency and fish health levels.

[0011] Therefore, the present invention provides a fish monitoring and management method based on multi-source data analysis. This method captures the fine morphological features of fish by constructing a multidimensional feature acquisition system. It also incorporates innovative skeletal tracking technology to analyze the unique swimming gait of fish. These two types of features are then multimodally fused to establish a comprehensive system for individual fish identification and health assessment. This comprehensive analysis method, based on the dual biological characteristics of "morphological fingerprints" and "behavioral gait," not only solves the problem of unmarked individual identification of common farmed fish, but also enables accurate health assessment based on individual differences, providing technical support for the refined management of aquaculture.

[0012] Preferably, the present invention further provides a fish monitoring and management system based on multi-source data analysis, which is used to execute the fish monitoring and management method based on multi-source data analysis as described above. The fish monitoring and management system based on multi-source data analysis includes:

[0013] The multi-angle data acquisition module is used to collect and enhance video images of the target fish school through multi-angle imaging equipment to obtain an enhanced video sequence; the enhanced video sequence is used to extract the swimming posture of the fish body to obtain the original data set of fish body features;

[0014] The morphological feature extraction module is used to classify the surface feature points of the fish body feature original data set to obtain a classification feature point set; and construct a fish body morphological feature vector based on the classification feature point set;

[0015] The swimming behavior analysis module is used to locate and track the fish skeleton based on the fish morphological feature vector to obtain skeleton motion trajectory data; quantify the body curvature of the skeleton motion trajectory data to obtain body curvature feature data; analyze the tail fin swing of the skeleton motion trajectory data to obtain the tail fin swing timing characteristics; extract the turning behavior characteristics of the skeleton motion trajectory data based on the body curvature feature data to obtain the turning behavior feature set; and analyze the fish swimming gait based on the tail fin swing timing characteristics and the turning behavior feature set to obtain the fish gait feature spectrum;

[0016] The health status assessment module is used to perform multi-source data fusion and behavioral change identification on the fish body morphological feature vector and the fish body gait feature spectrum to obtain a behavior change trend graph; perform health assessment on the behavior change trend graph to obtain health status assessment data.

[0017] This fish monitoring and management system, based on multi-source data analysis, achieves precise individual monitoring and health management of common farmed fish under marker-free conditions through the collaborative work of its various modules. The multi-angle data acquisition module efficiently acquires high-quality fish morphology and dynamic behavioral data, laying the foundation for comprehensive analysis. The morphological feature extraction module innovatively constructs a "morphological fingerprint" based on subtle surface features, addressing the difficulty of traditional methods in identifying common fish individuals. The swimming behavior analysis module quantifies the unique movement patterns of individual fish through detailed skeletal tracking and multi-dimensional gait analysis, providing a dynamic basis for individual identification and physiological status assessment. The health status assessment module effectively integrates morphological and behavioral characteristics, significantly improving the accuracy of individual identification. It can also track individual growth and behavioral changes over time, enabling early detection and quantitative assessment of health abnormalities based on individual differences. As a whole, the system breaks through the bottlenecks of existing technologies and provides a refined, personalized, and intelligent monitoring and management approach for aquaculture. It helps increase production, reduce disease risks, and optimize resource allocation, with significant practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the steps of a fish monitoring and management method based on multi-source data analysis.

[0019] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0020] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. It is obvious that the embodiments described are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts are within the scope of protection of the present invention.

[0021] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor and / or microcontroller approaches.

[0022] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. The term "and / or" as used herein includes any and all combinations of one or more of the listed associated items.

[0023] In the embodiment of the present invention, reference Figure 1 FIG. 1 is a flow chart showing the steps of a fish monitoring and management method based on multi-source data analysis according to the present invention. In this example, the fish monitoring and management method based on multi-source data analysis includes the following steps:

[0024] Step S1: Capture and enhance video images of the target fish school using a multi-angle imaging device to obtain an enhanced video sequence; extract the fish swimming posture from the enhanced video sequence to obtain an original data set of fish features;

[0025] In an embodiment of the present invention, a standardized underwater imaging environment is first constructed in a fish pond, including a uniform light source array, multiple high-resolution cameras with stereo coverage, and a water clarification device, and environmental parameters are recorded. When a fish enters the standardized area, an infrared sensor triggers the camera system to capture a raw multi-angle video sequence and simultaneously record environmental information. Next, the raw video is preprocessed and enhanced using algorithms such as dark channel priors, bilateral filtering, and CLAHE to correct for water effects and enhance detail. The fish outline is then segmented using background subtraction and morphological operations, and the spatiotemporal correlation of the fish in consecutive frames is established using multi-target tracking algorithms such as Kalman filtering. Based on the fish's visibility scores at different viewing angles, the image with the best viewing angle is selected from the enhanced video. Based on changes in the fish's posture (such as the curvature of the central axis and tail sway), representative key pose frames are extracted from the selected images. Fine feature regions such as the fish's head and fins are located in the key frames and locally enhanced. Finally, all collected key pose frames, feature region image blocks, fish segmentation trajectory data, and environmental parameters are organized and stored in a hierarchical structure to generate a raw fish feature dataset.

[0026] Step S2: classifying the surface feature points of the fish body feature original data set to obtain a classification feature point set; constructing a fish body morphological feature vector based on the classification feature point set;

[0027] In an embodiment of the present invention, the feature region image blocks in the original dataset of fish body features are enhanced, and methods such as CLAHE, bilateral filtering, and DoG are used to highlight tiny details. Then, algorithms such as Shi-Tomasi corner detection and Hessian blob detection are applied to roughly extract candidate feature points on the body surface from the enhanced feature image. By reprojecting the candidate points onto images from other viewpoints and calculating spatial consistency, high-quality feature points are screened out, and their coordinate accuracy is improved using a sub-pixel positioning algorithm. Using the SIFT descriptor and a pre-trained SVM classifier, precise feature points are classified into different types such as scale edge points, scar points, and pigment spots based on local texture and regional information to obtain a set of classified feature points. Based on the coordinates and types of the classified feature point set, the local spatial relationship (distance, angle) between the feature points is constructed. Stable feature points are selected as anchor points, and Delaunay triangulation is performed to construct a fish body morphological feature network structure, including geometric attributes and topological characteristics. The feature networks of the same fish body at different time points are aligned to extract the temporal variation patterns of feature point positions, network deformation, and the appearance / disappearance of feature points. Finally, the morphological network structure features and time-varying pattern features are encoded and weightedly concatenated, and then L2 normalized to construct a unique fish morphological feature vector for each fish.

[0028] Step S3: locating and tracking the fish skeleton according to the fish morphological feature vector to obtain skeleton motion trajectory data; quantifying the body curvature of the skeleton motion trajectory data to obtain body curvature feature data; analyzing the tail fin swing of the skeleton motion trajectory data to obtain the tail fin swing timing characteristics; extracting the turning behavior characteristics of the skeleton motion trajectory data according to the body curvature feature data to obtain the turning behavior feature set; analyzing the fish swimming gait according to the tail fin swing timing characteristics and the turning behavior feature set to obtain the fish gait feature spectrum;

[0029] In an embodiment of the present invention, a pre-trained deep learning posture estimation algorithm model (such as MaskR-CNN) is used in combination with a stable point reference in a morphological feature vector to locate anatomical key points (head, fin base, tail stalk, etc.) on a fish body key posture frame. The skeleton segment connection relationship and the fitting central axis are defined according to these key points to complete the skeleton initialization mark. Then, for the enhanced video sequence, starting from the initialization mark frame, a Kalman filter is used to track the key points, and observation data is provided by matching adjacent frames (such as using Lucas-Kanade optical flow), and motion prediction is combined. During the tracking process, skeletal structure constraints (such as skeletal segment length, joint angle range) are imposed, and the key point positions are corrected by iterative optimization. Multi-frame Savitzky-Golay smoothing is performed on the tracking results to improve temporal consistency. For key points that are blocked or unreliable for tracking, the position of the visible reliable point and the skeletal structure are used for position inference, and complete skeletal motion trajectory data is finally obtained. Then, the central axis discrete nodes are extracted from the skeletal motion trajectory, the node angles are calculated, the central axis curvature function is constructed, and the bending wave propagation speed, wavelength, and frequency are analyzed. The relative stiffness of the anterior, middle, and posterior segments of the fish body is estimated, and bending energy consumption is estimated based on stiffness, curvature, and angular change rate. The coordination between muscle contraction intensity distribution and bending wave propagation is analyzed to generate body bending feature data. Simultaneously, the local coordinates of key points of the caudal fin are extracted, and the lateral offset of the caudal fin is calculated. After Butterworth filtering, the swing period, amplitude, and phase are analyzed. Swing asymmetry (amplitude and time) is calculated to evaluate caudal fin stiffness. The propulsive force of the caudal fin is estimated based on an empirical model, generating a temporal signature of the caudal fin swing. Furthermore, head orientation is extracted from skeletal trajectories, and sudden changes in angular velocity are detected to identify turning events. Turning trajectories (radius), dynamic parameters (angular velocity), the relationship between body bending and turning, and velocity changes before and after turning are analyzed. Turning energy efficiency is evaluated, and turning events are classified into different modes through clustering to generate a turning behavior feature set. Finally, the body bending feature data, caudal fin swing timing features, and turning behavior feature set are integrated to construct a multidimensional temporal or statistical signature describing the fish's swimming pattern, forming a fish gait signature spectrum.

[0030] Step S4: performing multi-source data fusion and behavior change identification on the fish body morphology feature vector and the fish body gait feature spectrum to obtain a behavior change trend graph; performing health assessment on the behavior change trend graph to obtain health status assessment data;

[0031] In this embodiment of the present invention, fish morphological feature vectors and fish gait feature spectra are Z-score normalized to scale the features of each dimension to a uniform range. Next, based on the feature importance weights learned from historical recognition tasks, a weighted splicing method is used to fuse the standardized morphological and gait features into a unified fused feature representation. The fused feature representation of the current fish is then matched against the fused feature representations of known individuals in a historical fish feature library using cosine similarity. By setting a matching threshold and a score difference margin, it is determined whether the current fish is successfully identified as a historical individual, generating an individual recognition result set. For identified individuals, distance indicators related to body length and height are extracted from the historical morphological feature vectors. Body length and height at different time points are estimated using a pre-built regression model. Individual growth curves are plotted, and growth rates and deviations from the standard growth curve are calculated to form a growth parameter analysis table. Simultaneously, key behavioral indicators (such as swimming speed, swing frequency, and turning frequency) are extracted from the historical gait feature spectra, smoothed with a sliding window, and the CUSUM algorithm is applied to detect significant changes in behavioral patterns, generating a behavioral change trend chart. Based on the growth parameter analysis table and behavioral change trend chart, each indicator is compared with the preset normal range threshold or its own historical baseline. Indicators that exceed the range or have changed significantly are marked. Individuals with multiple abnormal indicators are marked as having health abnormalities, generating health abnormality marker data. Finally, based on the health abnormality marker data, growth parameters, and behavioral change trends, a weighted scoring function is designed to calculate the individual's comprehensive health score. The score is then divided into health status levels (such as Healthy, At Risk). The score change trends are analyzed to generate health status assessment data.

[0032] It is particularly important to analyze the fish swimming gait based on the tail fin swing timing characteristics and the turning behavior feature set:

[0033] Extract the swing spectrum characteristics of the tail fin swing time series characteristics to obtain the swing spectrum feature map;

[0034] The swimming efficiency index set is calculated based on the skeletal motion trajectory data, body bending feature data and tail fin swing timing characteristics;

[0035] According to the swimming efficiency index set, the turning behavior feature set and the swing spectrum feature map are used to summarize the individual gait characteristics and obtain the fish gait feature spectrum.

[0036] In an embodiment of the present invention, the tail fin swing filter sequence (the time series x_filtered[t] of the lateral offset of the tail fin) contained in the tail fin swing timing characteristics is used to apply the short-time Fourier transform (STFT) method for time-frequency analysis. The time series x_filtered[t] is divided into a series of overlapping time windows of length N_window (for example, the window length N_window corresponds to the number of video frames of 0.5 seconds, and the overlap ratio is 50%), and the Hanning window function is applied to each window to reduce spectral leakage. The discrete Fourier transform (DFT) of each windowed signal segment is calculated. The square of the amplitude of the DFT result is calculated to obtain the power spectrum density within the time window. The power spectrum density results of the continuous time windows are arranged in chronological order to form a two-dimensional matrix whose dimensions are time and frequency. The values ​​in the matrix represent the swing energy density at the corresponding time and frequency. This matrix is ​​the swing spectrum diagram. Quantitative features are extracted from the swing spectrum graph. For example, within each time window, the frequency with the highest energy is identified as the dominant frequency; the total energy within a specific frequency range (e.g., 0-1Hz, 1-2Hz, 2-3Hz) is calculated; and the rate of change of the dominant frequency over time (frequency modulation) is analyzed. These quantitative features extracted from the spectrum graph constitute the swing spectrum feature graph, which is actually stored as a time series of these statistical features.

[0037] From the skeletal motion trajectory data, calculate the instantaneous forward velocity V_forward(t) (the velocity component along the fish body's central axis) of the center point (e.g., the center of the head or the center of mass). From the body bending feature data, obtain the estimated instantaneous bending energy consumption rate P_bend(t). From the tail fin swing timing characteristics, obtain the estimated instantaneous tail fin propulsion force F_thrust(t) and tail fin tip velocity V_tip(t). Estimate the instantaneous total energy consumption rate P_total(t)≈P_bend(t)+P_fin(t), where P_fin(t) is the tail fin work power, which can be approximated as F_thrust(t)×V_tip(t). Calculate the instantaneous swimming efficiency η(t)=(F_thrust_forward(t)×V_forward(t)) / P_total(t), where F_thrust_forward(t) is the component of the propulsion force in the forward direction (this part of the estimation is more complicated, and a simpler efficiency indicator can also be used). Use a simpler efficiency index based on kinematic parameters, such as the Strouhal number (St). For each swing cycle or within a short time window, calculate the average forward speed V_avg, the average swing frequency f_avg, and the average swing amplitude A_avg. Calculate St = f_avg × A_avg / V_avg. For fish swimming, a Strouhal number between 0.2 and 0.4 usually corresponds to a higher propulsion efficiency. Calculate the average Strouhal number St_avg and its standard deviation σ_St within a time window (for example, 30 seconds). Calculate the average forward speed V_avg within this time window. These average Strouhal numbers, Strouhal number standard deviations, average forward speeds, etc. are combined into a swimming efficiency index set, which is actually stored as a time series of these statistical indicators.

[0038] The dynamic characteristic indicators calculated within a recent time window (e.g., the past minute) are statistically summarized to form a fixed-length vector describing the individual's typical swimming gait. From the time series of the swimming efficiency indicator set, the average Strouhal number, standard deviation of the Strouhal number, average forward speed, and standard deviation of forward speed are calculated. From the turning behavior feature set (which is itself a statistical summary or pattern classification result), the turning frequency (e.g., number of turns per minute), average turning radius, occurrence ratio of different turning modes (sharp turns, slow turns), and average turning energy efficiency are extracted. From the time series of the swing spectrum feature map, the average dominant frequency, standard deviation of the dominant frequency, average energy content of specific frequency bands (e.g., 0-1Hz, 1-2Hz), and frequency modulation degree indicators are calculated. These statistical summary values ​​are combined into a multidimensional vector. For example, a vector might include: [average forward speed, standard deviation of forward speed, average Strouhal number, standard deviation of Strouhal number, turning frequency, average turning radius, proportion of sharp turns, proportion of slow turns, average main swing frequency, standard deviation of main swing frequency, 0-1 Hz energy fraction, 1-2 Hz energy fraction, average turning energy efficiency]. This vector comprehensively describes the individual characteristics of the fish in terms of movement speed, bending and swinging patterns, turning strategy, and swimming efficiency, forming the fish gait characteristic spectrum.

[0039] Preferably, step S1 includes:

[0040] Step S11: constructing an underwater imaging environment for the fish in the breeding pond to obtain a standardized observation environment parameter table;

[0041] Step S12: collecting original multi-angle video sequences of the fish body through a camera according to the standardized observation environment parameter table;

[0042] Step S13: performing image preprocessing and enhancement on the original multi-angle video sequence to obtain an enhanced video sequence;

[0043] Step S14: performing fish segmentation and tracking on the enhanced video sequence to obtain fish segmentation trajectory data;

[0044] Step S15: selecting and synthesizing the optimal view angle of the enhanced video sequence according to the fish body segmentation trajectory data to obtain a multi-view optimal image set;

[0045] Step S16: extracting key frames from the multi-view optimal image set to obtain a fish body key posture frame set;

[0046] Step S17: performing fine feature region positioning on the fish body key posture frame set to obtain fine image blocks of the feature regions;

[0047] Step S18: Perform data structuring processing on the fine image blocks of the feature area and the fish body key posture frame set to obtain the original data set of fish body features.

[0048] In this embodiment of the present invention, a standardized observation area measuring 2 meters by 2 meters by 1.5 meters (length by width by depth) is defined within the aquaculture pond. A waterproof LED light array is installed on the top and sides of this area, with a power setting of 100 watts per square meter and angled to achieve illumination uniformity within the area of ​​less than 10%. At least four underwater high-resolution cameras, such as industrial-grade underwater cameras with a resolution of 3840 × 2160 pixels and a frame rate of 30 frames per second, are installed along the perimeter and bottom of the area. The camera mounting positions and angles are calibrated to ensure 360° coverage of the observation area. A water clarification device is installed at the entrance to the observation area. This device integrates physical filtration and ultraviolet disinfection functions to reduce the concentration of suspended solids in the water to less than 5 mg / L, thereby improving water transparency. The system records the camera serial number, spatial coordinates (xi, yi, zi), where i represents the camera number, and environmental parameters such as light intensity (lux) and water temperature (degrees Celsius), generating a standardized observation environmental parameter table.

[0049] An infrared sensor array is deployed at the entrance to an observation area defined by a standardized observation environment parameter table. This array consists of multiple pairs of transmitters and receivers, forming a network of intersecting sensor beams. When a fish swims through, interrupting any one or more pairs of beams, a trigger signal is sent to the control unit. Upon receiving the trigger signal, the control unit immediately activates all camera systems associated with that area and begins synchronously capturing a video sequence. Simultaneously, a guidance channel 0.5 meters wide and 1 meter long is designed to guide the fish into the standardized observation area. Once the camera system is triggered, it continuously captures a 2-second video sequence to ensure that the fish's complete passage through the observation area is captured. The system automatically records the video sequence's start timestamp (for example, 2023-10-27, 10:35:12.345) and associates and stores environmental parameters such as current water temperature and light intensity with the video sequence, generating an original multi-angle video sequence.

[0050] Underwater image enhancement algorithms are applied to the collected raw multi-angle video sequences. For example, a dark channel prior-based method is used to correct for the attenuation and scattering effects of water on light, restoring the image's contrast and color balance. Next, the enhanced images are subjected to noise suppression using a median filter or non-local mean filter to reduce interference introduced by water turbidity and equipment noise. A color-adaptive histogram equalization (CLAHE) algorithm is used to enhance contrast in localized areas of the fish body, making fine features such as scale texture and tiny scars more prominent. Inter-frame jitter correction is performed by calculating optical flow or feature point matching between consecutive frames to estimate and compensate for image translation and rotation caused by water flow, thereby improving the stability of the video sequence. Finally, all images are resized to a standard size (e.g., 1920 × 1080 pixels) and converted to a unified color space (e.g., RGB) to ensure consistency in subsequent processing and generate the enhanced video sequence.

[0051] For the enhanced video sequence, a Gaussian mixture model (GMM) or background modeling method is used to perform background subtraction, and a static background model of the observation area is learned and established. In each frame of the image, the current pixel value is compared with the background model to identify the moving foreground target that is significantly different from the background, namely the fish body. Morphological operations are applied to the foreground segmentation results, such as first performing an erosion operation to remove small isolated noise points and burrs, and then performing an expansion operation to fill the holes inside the fish body contour. Next, a multi-target tracking algorithm is used, such as trajectory association based on the Hungarian algorithm or a method combining Kalman filtering and data association, to establish the correspondence between the same fish body in consecutive frames. By calculating the changes in the coordinates of the center point of the fish body bounding box or contour over time, the motion trajectory of each tracked fish body is generated, and its contour pixel set in each frame is recorded to establish the fish body segmentation trajectory data.

[0052] Based on the contour information of each fish in each frame in the fish segmentation trajectory data, the visibility score of the fish body in each camera perspective is calculated. The visibility score comprehensively considers the area ratio of the fish body in the picture, the contour clarity (for example, calculating the contour gradient variance), and the integrity of the fish body (for example, whether it is blocked or partially out of the picture). For each time point (i.e., video frame), the system selects the two to three frames of images with the highest scores as preferred perspective images based on the calculated visibility scores. These preferred image sets retain the morphological and posture information of the fish body captured from different advantageous angles at the current moment. The output of this step is a set of the clearest and most complete images selected for each fish at each time point or key frame moment, forming a multi-perspective preferred image set.

[0053] Posture analysis is performed on the video sequences corresponding to the multi-view optimized image set to identify key posture points during the fish's swimming. By calculating metrics such as the curvature change rate of the fish's central axis and the swing angle of key tail points relative to the central axis, typical posture events are defined and detected, including body straightening (curvature change rate close to zero), maximum bending (curvature reaching a local extreme), acceleration, deceleration, turning (head orientation change rate exceeding a threshold), and caudal fin swinging to extreme positions on either side. Based on the time points of these detected key posture events, representative still image frames are extracted from the corresponding optimized image set. The extracted key frames ensure that they comprehensively cover various angles and key morphological features of the fish during different swimming states, such as close-up views of the head, full-body profile views, and tail swing moments. Each extracted key frame is assigned a precise timestamp and corresponding posture label (e.g., "straight," "left turn," "maximum right tail swing") to generate a set of fish key posture frames.

[0054] For each image in the key pose frame set, a pre-trained segmentation model or geometry-based method is used to locate high-information, fine-scale features on the fish body, such as the head, pectoral fin, dorsal fin, anal fin, caudal fin, lateral line, and specific surface spots (if any). Local contrast enhancement algorithms, such as contrast-limited adaptive histogram equalization (CLAHE), are applied to these located feature region image patches to further highlight subtle details within the region, such as small scars, missing scales, and bifurcated fin rays. A clarity metric is calculated for each feature region image patch, such as using the Tenengrad gradient function or the Laplacian operator variance, and regions with clarity above a preset threshold are selected for subsequent processing. A local two-dimensional coordinate system is created for each selected feature region, with its origin at the region's center and axes parallel to the image boundaries. This facilitates subsequent feature extraction and alignment, generating fine-scale image patches for the feature region.

[0055] A hierarchical data structure is designed to organize and store all collected raw data and intermediate processing results. The top-level structural unit is a collection batch, which is uniquely identified by the collection start timestamp. Each batch contains multiple fish instances, and each fish instance is assigned a unique identifier (for example, the batch timestamp plus the serial number of the fish in the batch). Under each fish instance, the corresponding fish key pose frame set (static features) and fish segmentation trajectory data (dynamic features) are stored. Each image in the key pose frame set is associated with its timestamp and posture label; the fish segmentation trajectory data stores the contour coordinates and motion parameters of each frame, and is associated with the key pose frame set through the timestamp. At the same time, the collection environment parameters (water temperature, lighting, etc.) and system configuration information (camera model, calibration parameters, etc.) recorded in steps S11 and S12 are used as metadata and stored in association with the corresponding collection batch or fish instance. All data are stored in a database or storage system in a structured file format (such as HDF5 or Protocol Buffers) to form a raw data set of fish features.

[0056] Preferably, classifying the fish body feature original data set in step S2 includes:

[0057] Perform feature region enhancement preprocessing on the original data set of fish features to obtain an enhanced feature image group;

[0058] Perform rough extraction of body surface key points on the enhanced feature image group to obtain a set of body surface candidate feature points;

[0059] Screen and accurately locate the candidate feature points on the body surface to obtain accurate feature point data;

[0060] The precise feature point data is classified into feature point types to obtain a classified feature point set.

[0061] In the embodiment of the present invention, the fine image blocks of the characteristic regions in the original data set of fish body features are enhanced and pre-processed. For example, the contrast-limited adaptive histogram equalization (CLAHE) algorithm is applied to each image block, and the image block is divided into 8×8 small regions. Each region is independently histogram-equalized, and the contrast limit is set to 0.02 to enhance the contrast of local details while avoiding excessive amplification of noise. Then, a bilateral filter is applied, and the filter kernel size is set to 5×5 pixels, and the spatial standard deviation is set to 0.02. Set to 3, the intensity standard deviation σ iSet to 50, while preserving the edge information. For the scale region, apply multi-scale Difference of Gaussians (DoG) decomposition to extract texture features at different scales, such as using different scale combinations of σ1=1.0 and σ2=1.6, σ3=2.5 and σ4=4.0, etc., to separate fine and coarse textures. For the fin region and other transparent or semi-transparent regions, apply color and texture based enhancement methods, such as adjusting the gamma value of specific color channels, and combining with local contrast stretching. These processed image patches constitute the enhanced feature image set.

[0062] Perform body surface key point rough extraction on the enhanced feature image set. On each enhanced feature region image patch, apply the Shi-Tomasi corner detection algorithm, set the maximum number of corner points to 500, the minimum feature value quality level k to 0.01, and the minimum Euclidean distance between two corners to 5 pixels, to detect points with high curvature in the image, which are often located at the intersection of scale edges, or the turning points of scar outlines. For fin image patches, apply a spot detection algorithm based on the Hessian matrix (e.g., using the determinant maximum value in the scale space) to identify small spots or damaged areas on the fin membrane. At the same time, by analyzing the image gradient direction, identify the direction of the fin strip, and detect the intersection points as candidate feature points where the fin strip diverges or converges. Record the pixel coordinates (u, v) and corresponding detection response intensity (e.g., the minimum feature value of the Shi-Tomasi algorithm) of all detected candidate feature points. This set of points constitutes the body surface candidate feature point set.

[0063] Perform feature point screening and accurate positioning on the body surface candidate feature point set. For each point in the body surface candidate feature point set, use the preferred image set of the same fish body at a similar time point and different viewing angles obtained in step S15, as well as the known camera calibration parameters and fish body pose estimation (trajectory data from S14), to project the point from the current viewing angle back to the three-dimensional space, and then re-project it into other preferred viewing angle images. Calculate the distance between the re-projected point and the candidate points detected in other viewing angles to evaluate the spatial consistency of the point under multiple viewing angles. Set a consistency threshold (e.g., re-projection error less than 3 pixels) to remove candidate points with poor spatial consistency. At the same time, set a detection response intensity threshold to remove candidate points with low response intensity. For the high-quality candidate points after screening, apply a sub-pixel accurate positioning algorithm, such as an iterative method based on image gradient, to calculate the accurate coordinates (u_sub, v_sub) at the sub-pixel level by analyzing the image gradient in a small neighborhood around the point. Assign a unique identifier to each accurate feature point to facilitate subsequent tracking and matching, and obtain the accurate feature point data.

[0064] The accurate feature points are classified into different types. For each accurate feature point, a local image patch (e.g., 32x32 pixels) centered at it is extracted. A local descriptor, e.g., a scale-invariant feature transform (SIFT) descriptor, is computed on this local image patch. The SIFT descriptor is a vector representing the gradient orientation histogram of the local region. A pre-trained classification model, e.g., a support vector machine (SVM) classifier, is used to classify the feature point. The input of the SVM model is the SIFT descriptor, and the output is the probability or score that the feature point belongs to different types of body surface features. The SVM model uses a radial basis function (RBF) as the kernel function, which is trained on a labeled dataset containing multiple fish bodies and multiple types of features. The classification model classifies the accurate feature points into predefined type categories, e.g., scale edge point (Type Code 1), pigment spot (Type Code 2), tiny scar point (Type Code 3), fin stripe bifurcation point (Type Code 4), etc. Each accurate feature point is assigned with the identified type code and the corresponding classification confidence score (e.g., the probability value output by the SVM). The set of accurate feature points with type and confidence information constitutes the classified feature point set.

[0065] Preferably, constructing the fish body morphological feature vector from the classified feature point set in step S2 comprises:

[0066] Constructing the feature point spatial relationship from the classified feature point set to obtain a feature point spatial relationship graph;

[0067] Generating a fish body morphological feature network from the feature point spatial relationship graph and the classified feature point set to obtain a fish body feature network structure;

[0068] Extracting the time variation pattern from the fish body feature network structure to obtain feature time variation data;

[0069] Constructing the morphological feature vector from the feature time variation data and the fish body feature network structure to obtain the fish body morphological feature vector.

[0070] In the embodiment of the present application, for each accurate feature point in the classified feature point set, it is mapped to the approximate local three-dimensional coordinate system on the fish body surface according to its two-dimensional image coordinates (u_sub, v_sub) and the known camera parameters and the estimated approximate posture of the fish body. A local neighborhood radius R (e.g., R=20 pixels is set on the two-dimensional image plane) is selected, and for each feature point p i , all other feature points p j (j≠i) in its neighborhood are searched. If point p j is in the neighborhood of p i , it is considered that there is a local spatial relationship between them. The distance between p i and p jThe two-dimensional Euclidean distance At the same time, calculate the vector p i p j With p i The angle θ between the main directions of the local area (for example, estimated by local principal component analysis) ij . Construct an adjacency list or sparse matrix to store these relationships, where each entry records a pair of feature points with a spatial relationship (p i ,p j ), and the distance d between them ij , relative angle θ ij and their respective type codes. This structured data representation constitutes the spatial relationship graph of feature points.

[0071] According to the spatial relationship diagram of feature points and the set of classified feature points, feature points with high stability (for example, continuous occurrence rate higher than 90%) and high classification confidence (for example, confidence higher than 0.8) at multiple time points are selected as anchor points. Based on these anchor points, the fish body feature network skeleton is constructed. For example, two-dimensional Delaunay triangulation is performed using anchor points to form a basic triangular mesh. For non-anchor points, they are added to the interior of the nearest triangle, or described by the distance and relative angle between them and the nearest anchor point. Calculate the area of ​​each triangle in the network Where k represents the triangle number and the length of each side is L e , where e represents the edge number. The network's topological properties are extracted, such as the average node degree, the spatial distribution density of different types of feature points, and the presence of significant connected regions or isolated points. This structured data, including anchor point locations, relative positions of non-anchor points, connectivity relationships (edges), local geometric properties (edge ​​lengths, angles, triangle areas), and global topological properties, constitutes the fish's characteristic network structure.

[0072] The fish body feature network structure of the same fish body at different time points t1 and t2 (for example, 1 second apart) is aligned. By matching feature points with the same unique identifier, the position change Δpi = pi(t2) - pi(t1) of each matching point pi from t1 to t2 is calculated, and its speed vi = Δpi / (t2-t1). The deformation of the network structure is analyzed, for example, the area change rate of the corresponding triangle or the length change rate of the side is calculated. Feature points that exist at t1 but disappear at t2 (indicating scar healing or scale regeneration) and feature points that appear at t2 (indicating new damage or pigment changes) are identified. The appearance / disappearance frequency of different types of feature points is counted. The average speed and variance of the feature point position change are calculated. These data such as position change, speed, deformation rate, appearance / disappearance events and their frequency constitute the feature time change data.

[0073] The fish body feature network structure (for example, the structure at the last collection time t0) and the feature time change data (for example, the change in the last period of time ΔT) are encoded into a numerical vector. The vector contains several dimensions: global features such as: total number of feature points, proportion of the number of different types of feature points (scale edge points, scar points, pigment spots, fin strip bifurcation points, etc.); local geometric features such as: feature point spacing histogram (for example, divided into 10 intervals), adjacent point relative angle histogram (for example, divided into 12 direction intervals); network topology features such as: average node degree, number of network connected components. Time series features are extracted from the feature time change data, such as: average point speed amplitude in the past ΔT, standard deviation of point speed amplitude, average occurrence rate of different types of feature points (per minute), average disappearance rate of different types of feature points (per minute), average area change rate of the network. Each dimension of the vector f j A weight wj is assigned, which is based on the evaluation of the feature between individuals and the stability within individuals. The final morphological feature vector V is composed of each dimension of the weighted feature: Where n is the vector dimension. The vector V is L2 normalized to obtain the final fish body morphological feature vector V_normalized = V / ||V||2, where

[0074] Preferably, the fish body skeleton positioning and tracking according to the fish body morphological feature vector in step S3 includes:

[0075] According to the fish body morphological feature vector, the fish body feature original data set is positioned to obtain the fish body skeleton key point set;

[0076] According to the fish body skeleton key point set, the skeleton initialization mark is performed to obtain the initialization skeleton mark;

[0077] According to the initialization skeleton mark, the skeleton structure time series tracking is performed on the enhanced video sequence to obtain the skeleton motion trajectory data.

[0078] In an embodiment of the present invention, a deep learning pose estimation algorithm model is used that has been pre-annotated on a large number of fish images with anatomical key points (e.g., the center of the head, the left / right base of the pectoral fin, the starting point of the dorsal fin, the starting point of the anal fin, the caudal peduncle, and the upper / lower tip of the caudal fin). For example, a model based on the MaskR-CNN or AlphaPose architecture is used. The model takes a single image in a set of key pose frames of the fish body as input, and outputs the pixel coordinates (u_kp, v_kp) and the detection confidence score of each key point of the fish body in the image, where kp represents the key point type. The output of the pose estimation algorithm model is post-processed and verified using the stable feature point information contained in the morphological feature vector of the fish body. For example, the relative positional relationship between the detected key point and the known stable feature point in the morphological feature vector is calculated. If the positional relationship between the detected key point and the morphological feature point deviates from the typical pattern learned through the morphological vector (e.g., the distance between the center of the head and a certain stable scale feature point is outside the statistical distribution), the confidence of the key point is reduced or the position is corrected. The final output is verified and corrected data containing the type, pixel coordinates and confidence of each key point, forming a set of key points of the fish skeleton.

[0079] In a selected key frame in the fish skeleton key point set (for example, the frame where the fish posture is closest to the straight state), a simplified skeletal model of the fish is constructed using the key point coordinates detected in the frame. Define the connection relationship of the bone segments: the center of the head is connected to the base of the left and right pectoral fins, the center of the head is connected to the starting point of the dorsal fin, the starting point of the dorsal fin is connected to the caudal peduncle, the center of the head is connected to the starting point of the anal fin, the starting point of the anal fin is connected to the caudal peduncle, and the caudal peduncle is connected to the upper / lower tip of the caudal fin. These connection relationships constitute the skeletal structure. At the same time, based on the key points such as the center of the head, the starting point of the dorsal fin, the starting point of the anal fin, and the caudal peduncle, a curve passing through the central axis of the fish body is fitted (for example, using cubic spline interpolation). This curve represents the trunk of the fish body and is used for subsequent body bending analysis. The initial coordinates of these key points, the connection relationship between them, and the central axis representation are stored as initialization bone markers.

[0080] Starting from the video frame where the skeleton marker is initialized, the skeleton key points are tracked between frames using a multi-target tracking algorithm. A multi-target tracking method based on Kalman Filter is used to establish an independent Kalman filter for each skeleton key point. The state vector of each filter contains the pixel coordinates (u, v) of the key point and its velocity (v) on the image plane. u ,v v ), that is, X=[u,v,v u ,v v ] T. The prediction step predicts the key point position of the current frame based on the state and motion model (for example, uniform linear motion model) of the previous frame. The update step uses the image information of the current frame to correct the predicted position. The image information obtains observation data by performing local feature matching (for example, using the optical flow method to calculate the motion vector of the area) or appearance model-based matching in a small area (for example, 15×15 pixels) around the predicted position. In the tracking process, skeletal structure constraints are introduced: the length changes of adjacent bone segments in a short period of time should be within a biologically reasonable range, and the relative position relationship between key points should maintain a certain topological structure. If a key point is occluded (judged by fish body segmentation data), its position depends entirely on the prediction of the Kalman filter and the motion inference of the adjacent visible key points. The coordinates and confidence data of all skeletal key points tracked in each frame are stored in chronological order to form skeletal motion trajectory data.

[0081] It is particularly important to track the bone structure of the enhanced video sequence in time according to the initialization bone markers:

[0082] Perform adjacent frame matching search on the enhanced video sequence according to the initialized skeleton markers to obtain inter-frame matching results;

[0083] Perform motion state prediction on the inter-frame matching results to obtain motion prediction data;

[0084] Perform skeletal structure constraints based on motion prediction data to obtain the constrained optimized position;

[0085] Perform multi-frame consistency check on the constraint optimization position to obtain a time-smooth trajectory;

[0086] The occlusion point positions of the enhanced video sequence are inferred based on the temporal smooth trajectory to obtain the skeleton motion trajectory data;

[0087] In an embodiment of the present invention, an adjacent frame matching search is performed on the enhanced video sequence based on the initialized skeleton markers (including keypoint type, initial coordinates, and skeleton connection relationships). For each frame t in the video sequence, the tracked skeleton keypoint positions in frame t-1 are used as the starting point. In frame t, a local search window (e.g., a window size of 21 pixels × 21 pixels) is defined with the position of each keypoint in frame t-1 as the center. Within this search window, the Lucas-Kanade sparse optical flow algorithm is applied to calculate the motion vector of the pixel. The motion vector of the pixel at the keypoint position in frame t-1 or its small neighborhood is used as the observation data for the predicted position of the keypoint in frame t. A minimum quality threshold for optical flow tracking (e.g., optical flow points less than 0.01 are not considered) and a maximum number of iterations (e.g., 30) are set. For each keypoint in frame t, its potential matching position and corresponding tracking confidence (e.g., based on the tracking quality of the feature point or the optical flow residual) are calculated. These potential matching positions and confidences constitute the inter-frame matching results.

[0088] Using the inter-frame matching results and the tracking history of the previous frames, the precise position of each skeleton key point in the current frame (frame t) is predicted. A simple linear motion model is used for prediction: assuming that the key point moves in a straight line at a constant speed in a short period of time. The predicted position of key point i in frame t Among them, P i (t-1) is the final tracking position of key point i in frame t-1, V i (t-1) is its velocity at frame t-1 (given by P i (t-1)-P i (t-2) estimate), where Δt is the inter-frame time interval (e.g., for 30 fps video, Δt = 1 / 30 second). If a Kalman filter is used, the Kalman filter's prediction step is performed here, predicting the current state (position and velocity) based on the previous state estimate and the system dynamics model. These predicted positions constitute the motion prediction data.

[0089] The keypoint positions in the motion prediction data are treated as initial estimates. Constraints are imposed using knowledge of the fish's anatomical skeletal structure. The main constraints include: 1. Segment length constraint: The distance between a pair of connected keypoints (e.g., the center of the head and the origin of the dorsal fin) should be close to their initial length determined in the initialization markers or their relative length estimated from the total length of the fish, allowing for a small fluctuation range (e.g., ±5%). 2. Joint angle constraint: The joint angle formed by three connected keypoints (e.g., the angle formed by the origin of the dorsal fin, the caudal peduncle, and the tip of the caudal fin at the caudal peduncle) should be within a biologically plausible angle range (e.g., 0 to 180 degrees). 3. Relative position constraint: Certain keypoints have stable relative positional relationships (e.g., the bases of the left and right pectoral fins should be roughly symmetrically distributed on both sides of the midline). An optimization problem is constructed with the goal of finding a set of keypoint positions that are both close to the motion prediction data and maximize the satisfaction of skeletal structural constraints. This optimization problem can be formulated as minimizing an energy function E = E_data + E_structure, where E_data measures the difference between the predicted and optimized positions, and E_structure measures the degree to which the optimized positions violate structural constraints. The problem is solved using an iterative optimization algorithm (eg, Levenberg-Marquardt algorithm or gradient-based optimization method) to obtain optimized key point positions that satisfy the constraints, constituting constrained optimized positions.

[0090] The constrained optimized position data of the most recent frames (for example, the most recent 5 frames) are collected to form a short-term trajectory for each key point. A Savitzky-Golay smoothing filter is applied to these short-term trajectories. A quadratic polynomial is fitted to each coordinate component (u and v) in the trajectory data using a smoothing window with a window size of 5 frames. The filter outputs the smoothed position of the center frame of the window to reduce random noise and local jitter. At the same time, the trajectory is detected for transient abnormal jumps in speed or acceleration. If the speed change rate (acceleration) of a key point in a certain frame exceeds a preset threshold (for example, more than 3 times the average acceleration), the position of the point is marked as a potential anomaly, and the smooth trajectories of the previous and next frames are used for interpolation correction. After multi-frame consistency verification and correction, the key point position sequence constitutes a temporal smooth trajectory.

[0091] In the current frame, if a key point is judged to be outside the fish body contour, blocked by other fish bodies, or its multi-frame consistency test results show an abnormality based on the fish body segmentation trajectory data (from S14), then the key point is considered to be blocked or unreliable to track. For these key points marked as unreliable, the performance of the point in the previous frames in the temporal smoothing trajectory is used, combined with the positions of other visible and reliable key points in the current frame and the skeletal structure constraints, to infer its precise position in the current frame. For example, if the tip of the tail fin is blocked, but the position of the tail stalk is reliable, the position of the tail fin tip can be inferred from the position of the tail stalk in the current frame based on the average distance and relative angle between the tail stalk and the tip of the tail fin in the previous frames. Another inference method is that if a Kalman filter is used, for the blocked key points, the observation data of the current frame is not used in the update step, and only the prediction step (based on the motion model and the state estimation of the previous moment) is relied upon to update the state. The tracked reliable key point positions are merged with the inferred unreliable key point positions to form a complete set of skeletal key point positions for the current frame. The skeleton key point position data of all video frames are stored in chronological order to form skeleton motion trajectory data.

[0092] Preferably, quantifying the body curvature of the skeletal motion trajectory data in step S3 includes:

[0093] Extract the coordinate point sequence of the central axis from the head to the tail of the fish body from the skeletal motion trajectory data, perform central axis discretization processing, and obtain central axis discrete node data;

[0094] Calculate the node angles of the discrete node data of the central axis to obtain the node angle sequence;

[0095] Construct the medial axis curvature function based on the node angle sequence and the discrete node data of the medial axis;

[0096] Analyze the propagation of bending waves according to the medial axis curvature function to obtain the propagation characteristic data of bending waves;

[0097] The fish body is divided into three main sections: front, middle and back. The section stiffness coefficient is estimated based on the medial axis curvature function and the node angle sequence to obtain the section stiffness distribution data.

[0098] The bending energy consumption data is estimated based on the segment stiffness distribution data and the node angle sequence;

[0099] The muscle contraction strength analysis was performed based on the segment stiffness distribution data and the medial axis curvature function to obtain the muscle contraction strength distribution data;

[0100] The bending coordination is evaluated based on the muscle contraction intensity distribution data and the bending wave propagation characteristic data to obtain the bending coordination index;

[0101] Body bending characteristic data is generated according to the bending coordination index.

[0102] In an embodiment of the present invention, the time series smooth trajectory data of key points such as the center of the head, the starting point of the dorsal fin, the starting point of the anal fin, and the caudal stalk obtained by locating the key points of the skeleton in step S3 are used to regard these key points as control points. The cubic spline interpolation method is used to fit a smooth curve through these control points to represent the central axis of the fish body. The parameterized form of this curve is C(s) = (x(s), y(s)), where s is a parameter that varies from 0 to 1. The arc length of the fitted central axis curve from the head (s = 0) to the tail (s = 1) is discretized into a fixed number of nodes, for example, 20 nodes. Each node p i The position of the curve is determined by its parameter value s i OK, i =(i-1) / (N-1), where i ranges from 1 to N, and N = 20. The two-dimensional coordinate sequence of these discrete nodes constitutes the central axis discrete node data, which records the central axis shape of the fish body in each frame.

[0103] For each frame of the central axis discrete node data, select three consecutive nodes p i-1 ,p i ,p i+1 (i ranges from 2 to N-1). Calculate vector v1 = p i -p i-1 and v2 = p i+1 -p i . Calculate the angle θ between these two vectors i Angle θ i It can be calculated by vector dot product and modulus length: cos(θ i )=(v1·v2) / (|v1|×|v2|). Angle θ i Reflects the degree of local bending at node pi. The angles calculated for all internal nodes (from node 2 to node N-1) are arranged in order to form a node angle sequence, which records the distribution of local bending angles of the fish along the central axis in each frame.

[0104] The curvature κ is a quantity that describes the degree of curvature of a curve. It is the rate of change of the tangent direction angle to the arc length. For discrete nodes, the node angle can be used to approximate the curvature. At the node pi, the curvature κ i It can be approximated as (π-θ i ) divided by the average length of two adjacent line segments: κ i ≈(π-θ i ) / ((|v1|+|v2|) / 2). Here π-θ i Indicates the deviation angle from the straight line state (angle is π). The calculated curvature value κi The arc length position along the central axis (for example, using the normalized arc length si) is interpolated to construct a continuous function κ(s), where s represents the normalized arc length along the central axis, ranging from 0 to 1. This function describes the curvature distribution of the fish body along the central axis and constitutes the central axis curvature function, which is actually stored as a sequence of curvature values ​​at discrete nodes.

[0105] Analyze the changes in the medial axis curvature function over time. Identify the positions of the local maximum and minimum values ​​(i.e., the crests and troughs of the bending waves) on the medial axis curvature function κ(s) of each frame. Track the trajectories of these crests and troughs as they move along the medial axis (s direction) over time. By calculating the position change Δs of the crests / troughs in consecutive frames and the time interval Δt between frames, estimate the propagation speed of the bending wave v_wave = Δs / Δt, in units of body length / second. Calculate the distance between the crests to obtain the wavelength λ of the bending wave, in units of body length. Calculate the time required for the bending wave to pass through the entire length of the fish body to obtain the frequency of the bending wave f_wave = 1 / T_pass, where T_pass is the period. Analyze the stability of the wave velocity, wavelength, frequency, and the number of bending waves present on the fish body at the same time. These calculated parameters constitute the bending wave propagation characteristic data.

[0106] The fish body is divided into three main sections: front, middle, and back. For example, the normalized arc length s of the central axis from the head to the tail is divided into: front section (0≤s<0.3), middle section (0.3≤s<0.7), and back section (0.7≤s≤1.0). According to the curvature function of the central axis κ(s) and the node angle sequence θ i , estimate the relative "stiffness coefficient" of each segment. Stiffness can be indirectly estimated by analyzing the degree of bending of the segment under muscle drive (reflected by the propagation of bending waves). For example, when the bending wave passes, if the average curvature change of a segment is smaller than that of other segments, the segment is considered to have a higher relative stiffness. Calculate the standard deviation σ_θ or σ_κ of the node angle or curvature value in each segment as a measure of the bending flexibility of the segment. The stiffness coefficient K_section can be negatively correlated with flexibility, for example, K_section∝1 / σ_θ. These estimated values ​​representing the relative bending difficulty of the front, middle and back segments constitute the segment stiffness distribution data.

[0107] Bending energy consumption is related to muscle work to overcome the internal stiffness and water resistance of the fish body. A simplified energy consumption model is used: instantaneous bending energy consumption rate P(t)∝∑ i K i (t)×(θ i (t)-π) 2 ×|dθ i (t) / dt|, where K i(t) is the local stiffness of node i at time t estimated based on the segment stiffness distribution, (θ i (t)-π) is the amount by which the bending angle of node i deviates from the straight line state, |dθ i (t) / dt| is the rate of change of the bending angle of node i. The time derivative of the node angle sequence is used to obtain the angle change rate. The stiffness coefficient of the segment to which each node belongs is assigned to the node as K i The total instantaneous energy consumption rate of the fish body is obtained by summing the instantaneous energy consumption rates of all nodes i. The total bending energy consumption during this time window (for example, a complete swing cycle) is obtained by integrating the instantaneous energy consumption rate. These estimated energy consumption values ​​constitute the bending energy consumption data.

[0108] The curvature of the fish body is mainly caused by the contraction of muscles along the sides of the body. Assuming that a certain local curvature κ is achieved i Requires corresponding muscle contraction intensity I i . The required strength is related to the magnitude of the local curvature and the stiffness of the tissue at that location. A simplified model: the instantaneous muscle contraction strength distribution I(s,t)∝K(s,t)×κ(s,t), where K(s,t) is the local stiffness at position s along the central axis at time t (obtained by interpolating the segment stiffness distribution), and κ(s,t) is the curvature at that location. Analyze the spatial distribution and temporal variation of the central axis curvature function, and combine the estimated local stiffness to estimate the relative muscle contraction strength distribution along the central axis of the fish body. For example, the estimated muscle strength will be higher on the side of the body in front of the bending crest. These estimated relative muscle contraction strength distribution curves along the central axis of the fish body that change with time constitute the muscle contraction strength distribution data.

[0109] Evaluate the spatiotemporal coordination between the muscle contraction wave (reflected by the temporal shift of the peak value of the muscle contraction intensity distribution) and the bending wave (reflected by the temporal shift of the peak value of the medial axis curvature function). Calculate the phase difference between the peak position of the muscle contraction intensity and the peak position of the curvature along the medial axis (for example, in units of normalized arc length s or in units of angle). In efficient propulsion, the muscle contraction wave should generally lead the bending wave by a small phase difference. Analyze the average value and stability of this phase difference. At the same time, evaluate the smoothness and continuity of the bending wave propagation, such as calculating the coefficient of variation of the wave velocity and wavelength between consecutive swing cycles. A smaller, more stable leading phase difference and a lower coefficient of variation of the wave parameters indicate better bending coordination. These quantitative indicators, such as the average phase difference, the standard deviation of the phase difference, the coefficient of variation of the wave velocity, etc., constitute the bending coordination index.

[0110] Based on the bending coordination index and the body bending related data obtained in the previous steps (such as the statistical characteristics of the central axis curvature function, such as maximum curvature, average curvature, and curvature variance; bending wave propagation characteristic data, such as average wave velocity, frequency, and wavelength; statistical characteristics of segment stiffness distribution data; statistical characteristics of bending energy consumption data, such as average energy consumption rate; and statistical characteristics of muscle contraction intensity distribution data), these quantitative indicators are integrated into a multidimensional vector or structured data set. For example, the vector may include: the average curvature of the central axis, the maximum curvature, the average stiffness coefficient of the front, middle, and back segments, the average bending wave velocity, the average bending wave frequency, the average bending coordination phase difference, the standard deviation of the bending coordination phase difference, and the estimated average bending energy consumption rate. These values ​​together describe the pattern, efficiency, and coordination of the fish body bending, and constitute the body bending characteristic data.

[0111] Preferably, performing tail fin swing analysis on the skeletal motion trajectory data in step S3 includes:

[0112] Extract the tail fin key points from the skeleton motion trajectory data to obtain the tail fin local coordinate data;

[0113] Calculate the tail fin offset based on the local coordinate data of the tail fin to obtain the original sequence of the tail fin offset;

[0114] Perform signal filtering optimization on the original sequence of tail fin offset to obtain the tail fin swing filter sequence;

[0115] Analyze the swing period characteristics of the tail fin swing filter sequence to obtain the swing period data;

[0116] The amplitude and phase characteristics of the swing period data are extracted according to the tail fin swing filter sequence to obtain the swing amplitude and phase characteristics;

[0117] The swing asymmetry is analyzed based on the swing amplitude and phase characteristics to obtain the swing asymmetry characteristics;

[0118] The tail fin stiffness is evaluated based on the swing amplitude phase characteristics and the tail fin local coordinate data to obtain the tail fin stiffness index;

[0119] The tail fin propulsion force data is estimated based on the tail fin stiffness index;

[0120] The tail fin swing timing characteristics are generated based on the tail fin propulsion force data and the swing asymmetry characteristics.

[0121] In an embodiment of the present invention, a time-series smooth trajectory of key points belonging to the caudal fin is extracted from the skeletal motion trajectory data. These key points include the caudal peduncle, the upper tail fin tip, and the lower tail fin tip. For each frame, the two-dimensional pixel coordinates of these key points are obtained. Based on the central axis of the fish body (provided by the body curvature quantification step) or the position of the caudal peduncle, a local coordinate system is established with the caudal peduncle as the origin, the X-axis roughly along the direction of the fish body's forward movement, and the Y-axis perpendicular to the X-axis. The pixel coordinates of the upper and lower points of the caudal fin tip are converted to this local coordinate system to obtain their local two-dimensional coordinates (x_tip_upper, y_tip_upper) and (x_tip_lower, y_tip_lower) relative to the caudal peduncle. These local coordinate data that change over time constitute the caudal fin local coordinate data.

[0122] In the local coordinate system of the tail fin, the direction of the fish's forward movement roughly corresponds to the local X-axis. The lateral swing of the tail fin is reflected in the offset on the local Y-axis. Calculate the average Y coordinate of the tip of the tail fin y_avg = (y_tip_upper + y_tip_lower) / 2. This average Y coordinate represents the lateral position of the tail fin body. This average Y coordinate is used as the lateral offset of the tail fin relative to the local X-axis (roughly representing the end of the fish's central axis). Record the sequence of this offset changing over time to form the original sequence of the tail fin offset. For example, if the positive direction of the local Y-axis points to the left side of the fish body, a positive offset indicates that the tail fin swings to the left, and a negative offset indicates that it swings to the right.

[0123] The raw sequence of tail fin offsets contains noise caused by tracking errors or water currents. A low-pass digital filter is applied to this sequence to remove high-frequency noise above the typical tail fin oscillation frequency of fish. For example, a fourth-order Butterworth low-pass filter is used with a cutoff frequency of 10 Hz. This frequency is above the typical tail fin oscillation frequency range of common farmed fish such as sea bass and tilapia (typically 0.5-5 Hz), but below the noise frequency. Applying the filter to the raw offset sequence x_raw[t] yields the smoothed filtered tail fin oscillation sequence x_filtered[t].

[0124] Perform periodic analysis on the filtered tail fin swing sequence x_filtered[t]. Determine the swing period by detecting the zero crossing points (points where the signal changes from negative to positive or from positive to negative) or local extreme points (peaks and valleys) in the sequence. For example, detect the time t0, t1, t2, ... when the signal changes from negative to positive through zero. The time interval T between two consecutive zero crossings in the same direction is i =t i+1 -ti That is a swing cycle. Calculate the duration T of each detected cycle i , and calculate the average period T_avg and its standard deviation σ_T within a time window (for example, 10 consecutive periods). Swing frequency f i =1 / T i Calculate the average frequency f_avg and its standard deviation σ_f. These period and frequency statistics constitute the swing period data.

[0125] For each detected swing cycle, find the maximum (peak) A_peak and minimum (trough) A_trough of the tail fin offset within the cycle. Calculate the peak-to-peak amplitude A_pp = A_peak-A_trough. Calculate the center amplitude A_center = (A_peak-A_trough) / 2. Calculate the average center amplitude A_avg and its standard deviation σ_A within a time window (for example, 10 consecutive cycles). At the same time, for each time point t in the swing filter sequence, calculate its relative phase of the cycle at that moment. If the beginning of each cycle (for example, the zero crossing point from negative to positive) is defined as phase 0 and the end of the cycle is defined as phase 2π, then the moment t is in the cycle [t i ,t i+1 ]The phase φ(t)=2π×(tt i ) / (t i+1 -t i ). These include the mean amplitude, amplitude standard deviation, and relative phase information over time, which constitute the swing amplitude phase characteristics.

[0126] Analyze the difference between the tail fin swinging to the left (for example, the offset is positive) and the tail fin swinging to the right (for example, the offset is negative). Calculate the maximum amplitude of the left swing A_left = max(x_filtered[t]) and the maximum amplitude (absolute value) of the right swing A_right = |min(x_filtered[t])| within a cycle or a time window. Calculate the amplitude asymmetry index I_amp = (A_left-A_right) / (A_left+A_right). Calculate the time Δt_left required for the tail fin to swing from zero offset to the maximum left offset and the time Δt_right required to swing to the maximum right offset. Calculate the time asymmetry index I_time = (Δt_left-Δt_right) / (Δt_left+Δt_right). These asymmetry indices and their changing trends over time constitute the swing asymmetry characteristics.

[0127] The stiffness of the tail fin reflects its ability to resist bending and deformation. Based on the local coordinate data of the tail fin (the caudal peduncle and the locations of the upper and lower tips), the tail fin opening angle (Ψ) can be calculated. This is the angle formed by the three points above the tail tip, the caudal peduncle, and the lower tail tip. During the tail fin's swing, this angle decreases with increasing lateral deflection. A tail fin with higher relative stiffness will experience a smaller decrease in its opening angle when achieving the same lateral deflection, or exhibit less variability in its opening angle as the swing frequency increases. The tail fin stiffness metric K_fin is defined as a measure of the relationship between the tail fin opening angle Ψ and the tail fin's lateral deflection y_avg, or its frequency-dependent response. For example, the average tail fin opening angle at maximum lateral deflection can be calculated, or the regression coefficient of Ψ on |y_avg| can be calculated. Another metric could be the rate of change of the tail fin opening angle at different swing frequencies. These quantitative metrics constitute the tail fin stiffness metric.

[0128] The tail fin thrust data is estimated based on the tail fin stiffness index and the phase characteristics of the swing amplitude (mainly frequency f and amplitude A). The fish tail fin thrust F_thrust is a complex fluid dynamics process, but a simplified empirical or semi-empirical model can be established for estimation. A commonly used simplified model assumes that the thrust is proportional to the square of the tail fin swing frequency and the square of the tail fin tip velocity. The tail fin tip velocity is roughly proportional to the product of the amplitude and frequency. Therefore, F_thrust∝f 2 ×(A×f) 2 =A 2 ×f 3 Considering the influence of tail fin stiffness, a moderate stiffness improves propulsion efficiency. Incorporate the tail fin stiffness index K_fin into the model, such as F_thrust_estimated∝A 2 ×f 3 × g(K_fin), where g(K_fin) is a function that represents the effect of stiffness on propulsion (for example, g(K_fin) increases monotonically with increasing K_fin within a certain range). Estimates of the instantaneous or cycle-averaged propulsion force are calculated based on the amplitude and frequency of each frame or swing cycle. These estimates constitute the tail fin propulsion force data.

[0129] The quantitative indicators related to tail fin swing calculated in the previous steps are integrated to form a multidimensional time series feature vector or structured data. This includes: swing period data (average frequency, frequency standard deviation), swing amplitude phase characteristics (average amplitude, amplitude standard deviation), swing asymmetry characteristics (amplitude asymmetry index, time asymmetry index), tail fin stiffness index, and estimated tail fin propulsion data (average propulsion force, propulsion force standard deviation). These features can be statistically calculated (average value, standard deviation, maximum value, minimum value) at fixed time intervals (for example, every second or every 10 seconds) to form a time series of these indicators changing over time. These time series data constitute the part of the fish gait characteristic spectrum related to tail fin swing, that is, the tail fin swing time series characteristics.

[0130] Preferably, in step S3, extracting turning behavior features from the skeletal motion trajectory data according to the body bending feature data includes:

[0131] Extract the head direction sequence from the skeletal motion trajectory data to obtain the head direction data;

[0132] Perform turning event detection on the head direction data to obtain a turning event list;

[0133] Perform steering trajectory analysis on the skeletal motion trajectory data according to the steering event list to obtain steering trajectory features;

[0134] Calculate turning dynamics parameters based on the turning event list and head direction data;

[0135] Performing body bending and turning correlation analysis on the body bending feature data according to the turning event list to obtain bending and turning correlation data;

[0136] According to the turning event list, the skeletal motion trajectory data is analyzed for speed before and after turning to obtain the turning speed feature;

[0137] Steering energy efficiency is evaluated based on steering trajectory characteristics, steering dynamic parameters, and steering speed characteristics to obtain steering energy efficiency data;

[0138] Classify the steering behavior pattern according to the steering energy efficiency data and the bending steering correlation data to obtain the steering pattern data;

[0139] A steering behavior feature set is generated based on the steering pattern data.

[0140] In an embodiment of the present invention, a sequence of fish head direction is calculated based on the time-series smooth trajectory of the key points of the fish head (for example, the center of the head and the point on the central axis in front of it) in the skeletal motion trajectory data. For each frame, the two-dimensional coordinates of the key point P_head at the center of the head and the point P_front extending a fixed distance in front of it along the central axis (for example, 5% of the length of the fish body) are obtained. Calculate the vector V_direction = P_front-P_head. The direction angle α of this vector = atan2(V_direction_y, V_direction_x) represents the orientation of the head in the current frame, where atan2(y,x) is the angle between the calculation point (x,y) and the line connecting the origin relative to the positive X-axis, in the range of (-π,π] or [0,2π). The head direction angle α calculated for each frame is arranged in chronological order to form head direction data, which is an angle value of a time series.

[0141] Perform turn event detection on the head direction data. Calculate the time rate of change of the head direction data α(t), i.e., angular velocity ω(t) = dα(t) / dt. Use the central difference method. Where Δt is the time interval between frames. Threshold detection is applied to the angular velocity sequence: when |ω(t)| exceeds a preset angular velocity threshold ω_threshold_start (e.g., 15 degrees / frame) for N1 consecutive frames (e.g., N1=5), it is considered to be the start time of a potential turning event t_start. When |ω(t)| is lower than another preset angular velocity threshold ω_threshold_end (e.g., 5 degrees / frame) for N2 consecutive frames (e.g., N2=5), the turning event is considered to end at t_end. Adjacent turning events with a time interval less than T_merge (e.g., 0.5 seconds) are further merged into one event. Each detected turning event consists of its start time t_start, end time t_end, and the total change in head direction during the time period Δα=α(t_end)-α(t_start). The list of these events constitutes the turning event list.

[0142] The turning trajectory of the fish body center point (e.g., head center or center of mass) in the skeletal motion trajectory data is analyzed based on the turning event list. For each event in the turning event list, the two-dimensional trajectory coordinate sequence (x(t), y(t)) of the fish body center point within the event time period [t_start, t_end] is extracted. The total path length of the trajectory is calculated. where i is from t_start to t_end-Δt. Estimating the turning radius R_turn: There are many methods that can be used, for example, fitting a circle to the trajectory points during the turning process and calculating the radius of the circle; or calculating the instantaneous curvature of each point on the trajectory κ_instantaneous = |x'y"-y'x"| / (x' 2 +y' 2 )^(3 / 2), where x', y', x", and y" are the time derivatives of the trajectory coordinates (velocity and acceleration). The average instantaneous curvature, κ_avg, is then calculated as (∑κ_instantaneous) / N_points, resulting in R_turn ≈ 1 / κ_avg. These calculated quantitative metrics, such as path length and estimated turning radius, constitute the turning trajectory characteristics.

[0143] Steering dynamics parameters are calculated based on the steering event list and head direction data. For each event [t_start, t_end] in the steering event list, the head direction data α(t) and angular velocity data ω(t) within that time period are extracted. The maximum angular velocity ω_max = max(|ω(t)|) and the average angular velocity ω_avg = |Δα| / (t_end-t_start) during the event are calculated. The time-varying pattern of angular velocity is analyzed, for example, the time required from the start of steering to reaching the maximum angular velocity (the duration of the angular acceleration phase) and the time required from the maximum angular velocity to the end of steering (the duration of the angular deceleration phase) are calculated. These parameters, such as ω_max, ω_avg, and the duration of angular acceleration, constitute the steering dynamics parameters.

[0144] According to the turning event list, the body bending feature data is analyzed for the association between body bending and turning. For each event [t_start, t_end] in the turning event list, the body bending feature data within this time period (for example, the medial axis curvature function κ(s, t) or the node angle sequence θ) is extracted. i(t)). Analyze the relationship between the body bending pattern and the turning direction. For example, for a left turn, the left side of the fish body will bend inward, resulting in a larger positive curvature (or a smaller node angle), while the right side will bend outward, resulting in a negative curvature or a smaller positive curvature. Quantify the bending asymmetry: Calculate the maximum curvature κ_left_max on the left side (relative to the forward direction) and the maximum curvature (absolute value) κ_right_max on the right side of the fish body's central axis during the turning process. Calculate the bending asymmetry index I_bend_async = (κ_left_max-κ_right_max) / (κ_left_max+κ_right_max) (assuming that κ_left_max>κ_right_max when turning left). Analyze the position of the maximum bending position along the body axis s_bend_max and its occurrence time t_bend_max. Calculate the time difference Δt_bend_angular = t_bend_max-t_angular_max, the time when the maximum bending occurs relative to the time when the maximum angular velocity occurs, where t_angular_max is the time to reach ω_max. These asymmetry indices, maximum bending positions and time differences constitute the bending-steering correlation data.

[0145] Based on the turning event list, analyze the linear velocity of the fish center point in the skeletal motion trajectory data before and after the turn. For each event [t_start, t_end] in the turning event list, extract the data of the fish center point in the linear velocity sequence V_linear(t) within the time period [t_start-T_pre, t_end+T_post], where T_pre and T_post are the preset analysis time windows before and after the turn (for example, T_pre = 1 second, T_post = 1 second). Calculate the average velocity V_pre_avg within the time window [t_start-T_pre, t_start] before the turn. Calculate the minimum velocity V_du ring_min = min(V_linear(t)) during the turn process [t_start, t_end]. Calculate the average velocity V_post_avg within the time window [t_end, t_end+T_post] after the turn. Calculate the average deceleration before the turn, a_pre_avg = (V_start-V_pre_avg) / T_pre, where V_start is the velocity at t_start. Calculate the average acceleration after turning: a_post_avg = (V_post_avg - V_end) / T_post, where V_end is the speed at time t_end. These speed and acceleration indicators constitute the turning speed characteristics.

[0146] The steering energy efficiency is evaluated based on the steering trajectory characteristics (such as turning radius), steering dynamics parameters (such as average angular velocity) and steering speed characteristics (such as speed loss V_pre_avg-V_during_min). A steering energy efficiency index E_turn is defined. For example, a simple energy efficiency index can measure how much angular steering is completed with a certain loss of linear speed: E_turn = |Δα| / (V_pre_avg-V_during_min+ε), where ε is a small positive number to avoid division by zero. A more complex evaluation can take into account bending energy consumption data (if estimated), such as E_turn = |Δα|2 / (estimated total bending energy consumption + estimated total linear motion energy consumption change). The estimated total linear motion energy consumption change is related to the speed change, for example, proportional to (V_pre_avg 2 -V_post_avg 2 ). These calculated energy efficiency values ​​constitute the steering energy efficiency data.

[0147] The steering behavior is classified into patterns based on the steering energy efficiency data and the bending steering association data (e.g., bending asymmetry, maximum bending position). The steering energy efficiency data, bending steering association data, and some steering trajectory features (e.g., turning radius) and steering dynamics parameters (e.g., maximum angular velocity) of each steering event are combined into a feature vector. Using a clustering algorithm, such as K-means clustering, all detected steering events are clustered into a preset number K (e.g., K=3) of different patterns. The clustering results represent different steering strategies, such as: "sharp turn" (small radius, high angular velocity, large asymmetric bend), "slow turn" (large radius, low angular velocity, small asymmetric bend), and "on-the-spot turn" (extremely small radius, speed close to zero). Each steering event is assigned to the pattern category to which it belongs. These classification results constitute the steering pattern data.

[0148] Based on the turning pattern data and the turning behavior related data obtained in the previous steps, a turning behavior feature set of the fish body during the observation period is generated. The frequency of occurrence of different turning patterns (for example, "sharp turn" and "slow turn") is counted (for example, the number of occurrences per minute). Calculate the average turning radius, average total angle change, average maximum angular velocity, average turning energy efficiency and other statistics of all turning events. Calculate the average value of the deceleration before turning and the acceleration after turning. Calculate the average value and standard deviation of the bending asymmetry index. These statistics are combined into a multidimensional vector or structured data, which describes the typical turning behavior characteristics of the fish body over a period of time, forming a turning behavior feature set.

[0149] Preferably, step S4 includes the following steps:

[0150] Step S41: performing feature space normalization on the fish body morphology feature vector and the fish body gait feature spectrum to obtain normalized feature data;

[0151] Step S42: performing multimodal feature fusion on the standardized feature data to obtain a fused feature representation;

[0152] Step S43: performing individual identification matching on the fused feature representation to obtain an individual identification result set;

[0153] Step S44: extracting growth parameters from the individual identification result set to obtain a growth parameter analysis table;

[0154] Step S45: Analyze the behavior pattern changes of the individual identification result set to obtain a behavior change trend graph;

[0155] Step S46: performing health anomaly detection on the behavior change trend chart according to the growth parameter analysis table to obtain health anomaly mark data;

[0156] Step S47: Perform health status assessment on the health abnormality mark data to obtain health status assessment data.

[0157] The fish morphological feature vector (for example, including dimensions such as scale point density, number of scars, number of fin bifurcations, average network side length, etc.) and the fish gait feature spectrum (for example, a vector composed of indicators such as average swimming speed, standard deviation of tail fin swing frequency, turning frequency, average turning radius, etc. extracted from time series data by statistical methods) are normalized in feature space. For the morphological feature vector V_morph=[f_m1,f_m2,...,f_m_k] and the gait feature vector V_gait=[f_g1,f_g2,...,f_g_l], where k and l are vector dimensions, the Z-score normalization method is used. This method converts each feature dimension into a distribution with a mean of 0 and a standard deviation of 1. For any feature dimension j, its standardized value f' j =(f j -μ j ) / σ j , where f j is the original eigenvalue, μ j is the average value of the feature dimension over a large amount of historical fish data, σ j is the standard deviation of the feature dimension in a large amount of historical fish data. Calculate and store μ of historical data j and σ jThe standardized morphological feature vector V'_morph = [f'_m1, f'_m2, ..., f'_m_k] and the standardized gait feature vector V'_gait = [f'_g1, f'_g2, ..., f'_g_l] are combined together to form standardized feature data, for example, simply concatenated into a vector V'_standardized = [V'_morph, V'_gait].

[0158] Perform multimodal feature fusion on the standardized feature data V'_standardized. Use weighted splicing fusion method. Assign weights to the standardized morphological feature vector V'_morph and the standardized gait feature vector V'_gait according to their respective importance and then splice them. Weight w_m i and w_g j The importance of the morphological feature dimension i and the gait feature dimension j respectively. These weights can be determined by training a recognition model (e.g., a support vector machine or a neural network) in a historical recognition task and analyzing the contribution of each feature or by a feature selection algorithm (e.g., feature sorting based on mutual information). For example, if the morphological feature has a higher discrimination in recognition, the weight of its corresponding dimension will be higher. The fused feature representation V_fused is a new vector with a dimension of k+l, and each component is the original normalized feature value multiplied by the corresponding weight: V_fused=[w_m1·f'_m1,...,w_m_k·f'_m_k,w_g1·f'_g1,...,w_g_l·f'_g_l]. This fused feature representation integrates the morphological and behavioral information of the fish body, and the contribution of each part is adjusted to form a fused feature representation.

[0159] Perform individual identification and matching on the fused feature representation V_fused. Compare the fused feature representation of the current fish body with the fused feature representations of all known individuals stored in the historical fish body feature library. The historical feature library stores the fused feature representations collected for each known individual at different time points. Use cosine similarity (CosineSimilarity) as the similarity measurement function. The similarity between the current fish body feature V_current and the feature V_history_k(t) of the historical individual k at time point t is S_cosine(V_current, V_history_k(t)) = (V_current·V_history_k(t)) / (||V_current||×||V_history_k(t)||), where · represents the vector dot product, and ||V|| represents the Euclidean norm of the vector. Calculate the similarity of all time point features of the current fish body and each individual in the historical library, and take the maximum similarity as the matching score with the historical individual. Set a matching threshold T_match (for example, 0.85). If the highest matching score S_max is higher than T_match, and the difference between S_max and the second highest score S_second_max is greater than a preset margin ΔS (for example, 0.05), the current fish body is considered to be successfully identified as the historical individual corresponding to the highest score. If the highest score is lower than T_match or S_max-S_second_max≤ΔS, the current fish body is marked as a new individual or the identification fails. Record the identification results of each processed fish body (the matched historical individual ID or "NewID"), the highest matching score and the identification timestamp to form an individual recognition result set.

[0160] For each identified historical individual in the individual identification result set, its growth parameters are extracted. The morphological feature vectors collected from the individual at different time points are retrieved from the historical fish body feature library. Although the morphological feature vector is an abstract representation, fine feature area positioning and feature point spatial relationship construction (step S2) are used in its construction process. These data contain fish body size information. For example, the distance between the center point of the head and the caudal peduncle point, the vertical distance between the starting point of the dorsal fin and the starting point of the anal fin, etc. can be extracted from the morphological feature network structure. These distances are linearly correlated with the length and height of the fish body. Through historical data, a regression model is established between these distances and the actual measured body length and height. Using this regression model, the fish body length L(t) and body height H(t) are estimated from the morphological feature vector at each time point. The changes of these estimated values ​​over time are recorded to form an individualized growth curve. Calculate the average growth rate between adjacent collection time points: length growth rate ΔL / Δt = (L(t2) - L(t1)) / (t2 - t1), height growth rate ΔH / Δt = (H(t2) - H(t1)) / (t2 - t1). Compare the individual growth curve to a preset standard growth curve for the species (e.g., a parameter curve based on the Von Bertalanffy model) and calculate the degree of deviation from the standard curve. These estimated length, height, growth rate, and deviation data constitute a growth parameter analysis table.

[0161] For each identified individual in the individual identification result set, analyze changes in its behavioral pattern. Retrieve the gait signature spectrum (or its statistical summary vector) of this individual collected at different time points from the historical fish signature database. The gait signature spectrum includes time series or statistical indicators such as swimming speed, tail fin swing frequency and amplitude, turning frequency, turning radius, and body bending coordination. Apply sliding window averaging smoothing to the time series of these key behavioral indicators. For example, use a sliding window size of 10 minutes and calculate the average within the window to smooth out short-term fluctuations. Use statistical model-based methods, such as the CUSUM algorithm, to detect significant changes in the mean or variance in the smoothed behavioral indicator series. For example, if the average swimming speed is significantly lower than its historical average for multiple consecutive windows, a behavioral change event is marked. For each detected change event, record the time, indicator of change, direction of change (increase or decrease), and magnitude of change. Visualize the smoothed time series of these key behavioral indicators and the detected change points into a chart to form a behavioral change trend graph, such as a line graph of average swimming speed over time, with markers marked at detected speed drops.

[0162] Based on the growth parameter analysis table and behavioral change trend chart, each identified individual is tested for health anomalies. A health indicator system is established, consisting of indicators extracted from the growth parameter analysis table (e.g., current body length, height, length growth rate over the past week, and the degree of deviation from the standard growth curve) and indicators extracted from the behavioral change trend chart (e.g., average swimming speed over the past day, tail fin oscillation frequency, turning frequency, and the presence of behavioral change events). Normal range thresholds are set for each health indicator. For example, a length growth rate less than 50% of the average for fish of the same age is considered abnormal; an average swimming speed less than 70% of its historical average is considered abnormal; and frequent sharp turns are considered abnormal. For each individual, each health indicator is checked to see if it exceeds the normal range or if there has been a significant change in its behavioral pattern (such as the change point detected in S45). If a particular indicator exceeds the threshold or undergoes a significant change, it is marked as abnormal. If the number of abnormal indicators for an individual exceeds a preset threshold (e.g., more than two key indicators are abnormal), the individual is marked as having a health anomaly. The ID of the abnormal individual, the time when the abnormality was detected, and the specific abnormal indicators and their values ​​are recorded to form health abnormality mark data.

[0163] Evaluate the health status of abnormal health marker data. Calculate the comprehensive health status score of each individual based on the abnormal health marker data, the individual's growth parameter analysis table, and the behavior change trend chart. Design a weighted scoring function: HealthScore = ∑ iwixIndicatorScorei, where IndicatorScorei is a sub-score calculated from the specific value or abnormality degree of indicator i, and wi is the weight of this indicator, reflecting its importance to the overall health status. For example, a severe deviation of length growth rate from the standard results in a higher negative score; a significant decrease in the detected behavior pattern results in a negative score; the presence of specific morphological damage (number of scars extracted from the morphological feature vector of S2, etc.) also results in a negative score. The sub-scores are summed up with weights to obtain an original health score. The original score is mapped to a normalized range, for example, 0 to 100 points, where 100 points represent the best health status. The health status level is divided according to the normalized score, for example: Healthy (score ≥ 80), Sub-healthy (60 ≤ score < 80), At Risk (40 ≤ score < 60), Unhealthy (score < 40). At the same time, by comparing the health scores of an individual at consecutive time points, it is determined whether the health status of the individual is improving, deteriorating, or remaining stable. The current health score, health status level, and health trend of each individual are recorded to form the health status evaluation data.

[0164] Therefore, the embodiments should be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the description given above, and all changes which come within the meaning and range of equivalency of the claims are intended to be embraced therein.

[0165] The foregoing is considered as illustrative only of the principles of the application. Numerous modifications and adaptations will be apparent to those skilled in the art in view of the foregoing description, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the invention. Therefore, this application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fish monitoring and management method based on multi-source data analysis, characterized in that: The following steps are involved: Step S1: Capturing and enhancing video images of the target fish school through a multi-angle imaging device to obtain an enhanced video sequence; Extract the fish swimming posture from the enhanced video sequence to obtain the original data set of fish features; Step S2: classifying the surface feature points of the fish body feature original data set to obtain a classification feature point set; constructing a fish body morphological feature vector based on the classification feature point set; Step S3: locating and tracking the fish skeleton according to the fish morphological feature vector to obtain skeleton motion trajectory data; quantifying the body curvature of the skeleton motion trajectory data to obtain body curvature feature data; analyzing the tail fin swing of the skeleton motion trajectory data to obtain the tail fin swing time series feature; extracting the turning behavior feature of the skeleton motion trajectory data according to the body curvature feature data to obtain the turning behavior feature set; The fish swimming gait is analyzed based on the tail fin swing timing characteristics and the turning behavior feature set to obtain the fish gait characteristic spectrum; Step S4: Perform multi-source data fusion and behavior change identification on the fish body morphological feature vector and the fish body gait feature spectrum to obtain a behavior change trend graph; perform health assessment on the behavior change trend graph to obtain health status assessment data.

2. The fish monitoring and management method based on multi-source data analysis according to claim 1 is characterized in that: Step S1 includes the following steps: Step S11: constructing an underwater imaging environment for the fish in the breeding pond to obtain a standardized observation environment parameter table; Step S12: collecting original multi-angle video sequences of the fish body through a camera according to the standardized observation environment parameter table; Step S13: performing image preprocessing and enhancement on the original multi-angle video sequence to obtain an enhanced video sequence; Step S14: performing fish segmentation and tracking on the enhanced video sequence to obtain fish segmentation trajectory data; Step S15: selecting and synthesizing the optimal view angle of the enhanced video sequence according to the fish body segmentation trajectory data to obtain a multi-view optimal image set; Step S16: extracting key frames from the multi-view optimal image set to obtain a fish body key posture frame set; Step S17: performing fine feature region positioning on the fish body key posture frame set to obtain fine image blocks of the feature regions; Step S18: Perform data structuring processing on the fine image blocks of the feature area and the fish body key posture frame set to obtain the original data set of fish body features.

3. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that: In step S2, the classification of the fish body feature original data set includes: Perform feature region enhancement preprocessing on the original fish feature dataset to obtain an enhanced feature image group; Perform rough extraction of body surface key points on the enhanced feature image group to obtain a set of body surface candidate feature points; Screen and accurately locate the candidate feature points on the body surface to obtain accurate feature point data; The precise feature point data is classified into feature point types to obtain a classified feature point set.

4. The fish monitoring and management method based on multi-source data analysis according to claim 1 is characterized in that: Constructing the fish body morphology feature vector according to the classification feature point set in step S2 includes: Construct the spatial relationship of feature points for the classified feature point set to obtain a feature point spatial relationship graph; The fish body morphological feature network is generated based on the feature point spatial relationship diagram and the classification feature point set to obtain the fish body feature network structure; Extract the time-varying pattern of the fish body characteristic network structure to obtain characteristic time-varying data; The morphological feature vector is constructed according to the characteristic time variation data and the fish body feature network structure to obtain the fish body morphological feature vector.

5. The fish monitoring and management method based on multi-source data analysis according to claim 1 is characterized in that: The fish skeleton positioning and tracking in step S3 according to the fish morphological feature vector includes: According to the fish body morphological feature vector, the skeleton key points of the original fish feature data set are located to obtain the fish body skeleton key point set; Perform skeleton initialization marking according to the fish skeleton key point set to obtain the initial skeleton marking; The enhanced video sequence is tracked in time sequence based on the initialized bone markers to obtain the bone motion trajectory data.

6. The fish monitoring and management method based on multi-source data analysis according to claim 1 is characterized in that: The step S3 of quantifying the body curvature of the skeletal motion trajectory data includes: Extract the coordinate point sequence of the central axis from the head to the tail of the fish body from the skeletal motion trajectory data, perform central axis discretization processing, and obtain central axis discrete node data; Calculate the node angles of the discrete node data of the central axis to obtain the node angle sequence; Construct the medial axis curvature function based on the node angle sequence and the discrete node data of the medial axis; Analyze the propagation of bending waves according to the medial axis curvature function to obtain the propagation characteristic data of bending waves; The fish body is divided into three main sections: front, middle and back. The section stiffness coefficient is estimated based on the medial axis curvature function and the node angle sequence to obtain the section stiffness distribution data. The bending energy consumption data is estimated based on the segment stiffness distribution data and the node angle sequence; The muscle contraction strength analysis was performed based on the segment stiffness distribution data and the medial axis curvature function to obtain the muscle contraction strength distribution data; The bending coordination is evaluated based on the muscle contraction intensity distribution data and the bending wave propagation characteristic data to obtain the bending coordination index; Body bending characteristic data is generated according to the bending coordination index.

7. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that: The tail fin swing analysis of the skeletal motion trajectory data in step S3 includes: Extract the tail fin key points from the skeleton motion trajectory data to obtain the tail fin local coordinate data; Calculate the tail fin offset based on the local coordinate data of the tail fin to obtain the original sequence of the tail fin offset; Perform signal filtering optimization on the original sequence of tail fin offset to obtain the tail fin swing filter sequence; Analyze the swing period characteristics of the tail fin swing filter sequence to obtain the swing period data; The amplitude and phase characteristics of the swing period data are extracted according to the tail fin swing filter sequence to obtain the swing amplitude and phase characteristics; The swing asymmetry is analyzed based on the swing amplitude and phase characteristics to obtain the swing asymmetry characteristics; The tail fin stiffness is evaluated based on the swing amplitude phase characteristics and the tail fin local coordinate data to obtain the tail fin stiffness index; The tail fin propulsion force data is estimated based on the tail fin stiffness index; The tail fin swing timing characteristics are generated based on the tail fin propulsion force data and the swing asymmetry characteristics.

8. The fish monitoring and management method based on multi-source data analysis according to claim 1 is characterized in that: Extracting turning behavior features from the skeletal motion trajectory data based on the body bending feature data in step S3 includes: Extract the head direction sequence from the skeletal motion trajectory data to obtain the head direction data; Perform turning event detection on the head direction data to obtain a turning event list; Perform steering trajectory analysis on the skeletal motion trajectory data according to the steering event list to obtain steering trajectory features; Calculate turning dynamics parameters based on the turning event list and head direction data; Performing body bending and turning correlation analysis on the body bending feature data according to the turning event list to obtain bending and turning correlation data; According to the turning event list, the skeletal motion trajectory data is analyzed for speed before and after turning to obtain the turning speed feature; Steering energy efficiency is evaluated based on steering trajectory characteristics, steering dynamic parameters, and steering speed characteristics to obtain steering energy efficiency data; Classify the steering behavior pattern according to the steering energy efficiency data and the bending steering correlation data to obtain the steering pattern data; A steering behavior feature set is generated based on the steering pattern data.

9. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: performing feature space normalization on the fish body morphology feature vector and the fish body gait feature spectrum to obtain normalized feature data; Step S42: performing multimodal feature fusion on the standardized feature data to obtain a fused feature representation; Step S43: performing individual identification matching on the fused feature representation to obtain an individual identification result set; Step S44: extracting growth parameters from the individual identification result set to obtain a growth parameter analysis table; Step S45: Analyze the behavior pattern changes of the individual identification result set to obtain a behavior change trend graph; Step S46: performing health anomaly detection on the behavior change trend chart according to the growth parameter analysis table to obtain health anomaly mark data; Step S47: Perform health status assessment on the health abnormality mark data to obtain health status assessment data.

10. A fish monitoring and management system based on multi-source data analysis, characterized in that: The method for fish monitoring and management based on multi-source data analysis according to claim 1 is used to implement the method, the fish monitoring and management system based on multi-source data analysis comprises: The multi-angle data acquisition module is used to collect and enhance video images of the target fish school through multi-angle imaging equipment to obtain an enhanced video sequence; the enhanced video sequence is used to extract the swimming posture of the fish body to obtain the original data set of fish body features; The morphological feature extraction module is used to classify the surface feature points of the fish body feature original data set to obtain a classification feature point set; and construct a fish body morphological feature vector based on the classification feature point set; The swimming behavior analysis module is used to locate and track the fish skeleton based on the fish morphological feature vector to obtain skeleton motion trajectory data; quantify the body curvature of the skeleton motion trajectory data to obtain body curvature feature data; analyze the tail fin swing of the skeleton motion trajectory data to obtain the tail fin swing timing characteristics; extract the turning behavior characteristics of the skeleton motion trajectory data based on the body curvature feature data to obtain the turning behavior feature set; and analyze the fish swimming gait based on the tail fin swing timing characteristics and the turning behavior feature set to obtain the fish gait feature spectrum; The health status assessment module is used to perform multi-source data fusion and behavioral change identification on the fish body morphological feature vector and the fish body gait feature spectrum to obtain a behavior change trend graph; perform health assessment on the behavior change trend graph to obtain health status assessment data.

Images