A fish monitoring management system and method based on multi-source data analysis

Abstract

The present application relates to the technical field of computer vision, and especially relates to a fish monitoring management system and method based on multi-source data analysis.The method comprises the following steps: video image acquisition and enhancement of a target fish school through a multi-angle imaging device to obtain an enhanced video sequence; fish body swimming posture extraction on the enhanced video sequence to obtain a fish body feature original data set; body surface feature point classification on the fish body feature original data set to obtain a classified feature point set; fish body morphological feature vector construction according to the classified feature point set; fish body skeleton positioning and tracking according to the fish body morphological feature vector to obtain skeleton motion trajectory data; and body bending quantification on the skeleton motion trajectory data to obtain body bending feature data.The present application solves the problem of unmarked individual identification of ordinary farmed fish through "morphological fingerprint" and "behavior gait" dual biological characteristics, and provides technical support for fine management of aquaculture.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a fish monitoring and management system and method based on multi-source data analysis. Background Technology

[0002] Existing computer vision-based automated identification systems still face significant challenges in accurately identifying individual farmed fish under unlabeled conditions. Current technologies mainly rely on obvious markings or color patterns on the fish's body surface, which are insufficient for effectively identifying mainstream farmed species such as bass and tilapia that lack these features. Furthermore, existing monitoring systems have limited ability to extract fine morphological and behavioral gait characteristics of fish, failing to simultaneously capture minute surface features (such as small scars or missing scales) and complex movement patterns (such as tail fin wagging frequency and body curvature), resulting in incomplete feature representation. In addition, traditional aquaculture management methods generally use population average indicators for health assessment, lacking the ability to identify individual differences and long-term tracking capabilities, thus failing to achieve precise and individualized health status monitoring and management.

[0003] In summary, existing technologies have several problems that urgently need to be addressed, including the inability to perform unmarked individual identification of common farmed fish, the lack of multidimensional feature analysis capabilities, and the inability to achieve individualized long-term health monitoring. Summary of the Invention

[0004] Therefore, it is necessary to provide a fish monitoring and management system and method based on multi-source data analysis to solve at least one of the above-mentioned technical problems.

[0005] To achieve the above objectives, a fish monitoring and management method based on multi-source data analysis includes the following steps:

[0006] Step S1: Acquire and enhance video images of the target fish group using a multi-angle imaging device to obtain an enhanced video sequence; extract the swimming posture of the fish from the enhanced video sequence to obtain the original dataset of fish features;

[0007] Step S2: Classify the surface feature points of the original fish body feature dataset to obtain a classification feature point set; construct the fish body morphology feature vector based on the classification feature point set;

[0008] Step S3: Based on the fish body morphology feature vector, locate and track the fish skeleton to obtain skeleton movement trajectory data; quantify the body curvature of the skeleton movement trajectory data to obtain body curvature feature data; analyze the tail fin swing of the skeleton movement trajectory data to obtain tail fin swing temporal features; extract turning behavior features from the skeleton movement trajectory data based on the body curvature feature data to obtain a turning behavior feature set; analyze the fish swimming gait based on the tail fin swing temporal features and the turning behavior feature set to obtain the fish gait feature spectrum.

[0009] Step S4: Perform multi-source data fusion and behavior change identification on the fish body morphology feature vector and fish body gait feature spectrum to obtain a behavior change trend map; perform health assessment on the behavior change trend map to obtain health status assessment data.

[0010] This invention overcomes challenges such as complex underwater environments, rapid fish movement, and difficulty in capturing details by constructing a standardized underwater imaging environment, employing an infrared triggering mechanism to accurately capture the fish's passage, applying multi-angle high-resolution imaging, and performing image enhancement and fish segmentation tracking. The resulting enhanced video sequences and original fish feature datasets contain clear morphological information and continuous dynamic swimming posture data from multiple advantageous viewpoints, providing high-quality, multi-dimensional, and spatiotemporally correlated foundational data for subsequent fine morphological feature extraction and behavioral gait analysis, significantly improving the accuracy and reliability of subsequent analysis. Fine enhancement preprocessing of feature regions clearly reveals surface features that are difficult to continuously track with the naked eye, such as tiny scars, missing scales, and forked fins. Combined with improved corner detection, spot detection, and sub-pixel precise localization techniques, high-precision extraction of these fine feature points is achieved. Multi-view consistency checks and classification algorithms ensure the stability of feature points and the accuracy of type identification. Furthermore, a spatial relationship network of feature points is constructed and its temporal evolution pattern is analyzed, forming a unique, time-varying "morphological fingerprint" for each fish. This morphological feature vector, based on minute surface features, provides crucial and stable biological evidence for the accurate identification of individuals in common farmed fish under label-free conditions, overcoming the limitations of traditional methods that rely on obvious markings. Using morphological feature vectors as a reference, combined with deep learning pose estimation and robust temporal tracking algorithms (including motion prediction, structural constraints, multi-frame smoothing, and occlusion inference), stable and high-precision tracking of key points in the fish skeleton during complex swimming processes is achieved. Furthermore, by quantifying the curvature of the body's central axis, analyzing the propagation characteristics of bending waves, evaluating segment stiffness and muscle contraction intensity, and providing detailed analysis of caudal fin oscillation (including spectrum, asymmetry, stiffness, and propulsion estimation) and turning behavior (including trajectory, dynamics, energy efficiency, and pattern classification), the unique swimming gait of individual fish is comprehensively and meticulously characterized. This dynamic and quantified behavioral gait feature spectrum complements static morphological information, providing rich dynamic biological indicators for distinguishing individuals and assessing their physiological state and activity capabilities. By performing weighted multimodal fusion on standardized morphological feature vectors and behavioral gait feature spectra, a more comprehensive and discriminative individual representation was constructed, significantly improving the accuracy and robustness of individual identification under label-free conditions. Based on the individual identification results, it is possible to track changes in the growth parameters (body length, body height, and growth rate) of a single fish over a long period and detect long-term trends and abrupt changes in its key behavioral patterns (such as swimming speed and turning frequency). By comparing these growth and behavioral change indicators with preset thresholds or their own historical baselines, early anomaly detection and quantitative scoring of individual health status were achieved.This individualized and refined health assessment capability breaks through the limitations of traditional group average monitoring, providing accurate individual health profiles and decision-making basis for aquaculture management. It is conducive to timely detection of problems, taking targeted measures, and improving aquaculture efficiency and fish health.

[0011] Therefore, this invention provides a fish monitoring and management method based on multi-source data analysis. It captures fine morphological features of fish by constructing a multi-dimensional feature acquisition system, and simultaneously introduces innovative skeletal tracking technology to analyze the unique swimming gait of fish. These two types of features are then fused in a multimodal manner to establish a complete system for individual fish identification and health assessment. This comprehensive analysis method based on dual biological characteristics of "morphological fingerprints" and "behavioral gait" not only solves the problem of unmarked individual identification of common farmed fish, but also achieves accurate health assessment based on individual differences, providing technical support for the refined management of aquaculture.

[0012] Preferably, the present invention also provides a fish monitoring and management system based on multi-source data analysis, used to execute the fish monitoring and management method based on multi-source data analysis as described above, the fish monitoring and management system based on multi-source data analysis comprising:

[0013] The multi-angle data acquisition module is used to acquire and enhance video images of the target fish group through multi-angle imaging equipment to obtain an enhanced video sequence; the fish swimming posture is extracted from the enhanced video sequence to obtain the original dataset of fish features.

[0014] The morphological feature extraction module is used to classify the surface feature points of the original fish body feature dataset to obtain a set of classification feature points; and to construct a fish body morphological feature vector based on the set of classification feature points.

[0015] The swimming behavior analysis module is used to locate and track the fish skeleton based on the fish's body morphology feature vectors to obtain skeletal movement trajectory data; to quantify the body curvature of the skeletal movement trajectory data to obtain body curvature feature data; to analyze the tail fin swing of the skeletal movement trajectory data to obtain tail fin swing temporal features; to extract turning behavior features from the skeletal movement trajectory data based on the body curvature feature data to obtain a turning behavior feature set; and to analyze the fish's swimming gait based on the tail fin swing temporal features and the turning behavior feature set to obtain the fish's gait feature spectrum.

[0016] The health status assessment module is used to perform multi-source data fusion and behavior change identification on fish morphological feature vectors and fish gait feature spectra to obtain a behavior change trend map; and to perform health assessment on the behavior change trend map to obtain health status assessment data.

[0017] This fish monitoring and management system, based on multi-source data analysis, achieves precise individual monitoring and health management of common farmed fish under label-free conditions through the collaborative work of its various modules. The multi-angle data acquisition module efficiently acquires high-quality fish morphology and dynamic behavior data, laying the foundation for comprehensive analysis. The morphological feature extraction module innovatively constructs a "morphological fingerprint" based on minute surface features, solving the problem of traditional methods' difficulty in identifying individual fish. The swimming behavior analysis module quantifies the unique movement patterns of individual fish through precise skeletal tracking and multi-dimensional gait analysis, providing dynamic evidence for individual identification and physiological status assessment. The health status assessment module effectively integrates morphological and behavioral characteristics, significantly improving the accuracy of individual identification and enabling long-term tracking of individual growth and behavioral changes, achieving early detection and quantitative assessment of health abnormalities based on individual differences. As a whole, the entire system overcomes the bottlenecks of existing technologies, providing a refined, individualized, and intelligent monitoring and management method for aquaculture, helping to increase yield, reduce disease risk, and optimize resource allocation, demonstrating significant practical application value. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the steps of a fish monitoring and management method based on multi-source data analysis.

[0019] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0021] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0022] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0023] In this embodiment of the invention, reference is made to Figure 1 The diagram shown is a flowchart illustrating the steps of the fish monitoring and management method based on multi-source data analysis according to the present invention. In this example, the fish monitoring and management method based on multi-source data analysis includes the following steps:

[0024] Step S1: Acquire and enhance video images of the target fish group using a multi-angle imaging device to obtain an enhanced video sequence; extract the swimming posture of the fish from the enhanced video sequence to obtain the original dataset of fish features;

[0025] In this embodiment of the invention, firstly, a standardized underwater imaging environment is constructed in the aquaculture pond, including a uniform light source array, stereo coverage by multiple high-resolution cameras, and a water clarification device, and environmental parameters are recorded. When a fish enters the standardized area, an infrared sensor triggers the camera system to acquire original multi-angle video sequences and simultaneously record environmental information. Next, the original video is preprocessed and enhanced using algorithms such as dark channel prior, bilateral filtering, and CLAHE to correct for water body influences and improve details. Then, background subtraction and morphological operations are used to segment the fish outline, and multi-object tracking algorithms such as Kalman filtering are used to establish the spatiotemporal relationship of the fish in consecutive frames. Based on the visibility scores of the fish from different viewpoints, the image with the best viewpoint is selected from the enhanced video. Based on changes in fish posture (such as midline curvature and tail sway), representative key posture frames are extracted from the selected images. Fine feature regions such as the fish head and fins are located in the key frames and locally enhanced. Finally, all acquired key posture frames, feature region image blocks, fish segmentation trajectory data, and environmental parameters are organized and stored in a hierarchical structure to generate a raw dataset of fish features.

[0026] Step S2: Classify the surface feature points of the original fish body feature dataset to obtain a classification feature point set; construct the fish body morphology feature vector based on the classification feature point set;

[0027] In this embodiment of the invention, feature region image patches in the original dataset of fish body features are enhanced using methods such as CLAHE, bilateral filtering, and DoG to highlight minute details. Then, algorithms such as Shi-Tomasi corner detection and Hessian spot detection are applied to coarsely extract candidate feature points from the enhanced feature image. High-quality feature points are selected by reprojecting the candidate points onto images from other perspectives and calculating spatial consistency, and their coordinate accuracy is improved using sub-pixel localization algorithms. Using SIFT descriptors and a pre-trained SVM classifier, precise feature points are classified into different types such as scale edge points, scar points, and pigment spots based on local texture and region information, resulting in a set of categorized feature points. Based on the coordinates and types of the categorized feature point set, local spatial relationships (distance, angle) between feature points are constructed. Stable feature points are selected as anchor points, and Delaunay triangulation and other methods are used to construct the fish body morphology feature network structure, including geometric attributes and topological characteristics. The feature networks of the same fish body at different time points are registered, and the temporal variation patterns of feature point positions, network deformation, and feature point appearance / disappearance are extracted. Finally, the morphological network structure features and temporal change pattern features are encoded, weighted, and concatenated, and then L2 normalized to construct a unique fish morphological feature vector for each fish.

[0028] Step S3: Based on the fish body morphology feature vector, locate and track the fish skeleton to obtain skeleton movement trajectory data; quantify the body curvature of the skeleton movement trajectory data to obtain body curvature feature data; analyze the tail fin swing of the skeleton movement trajectory data to obtain tail fin swing temporal features; extract turning behavior features from the skeleton movement trajectory data based on the body curvature feature data to obtain a turning behavior feature set; analyze the fish swimming gait based on the tail fin swing temporal features and the turning behavior feature set to obtain the fish gait feature spectrum.

[0029] In this embodiment of the invention, a pre-trained deep learning pose estimation algorithm model (such as MaskR-CNN) is used in conjunction with stable point references in the morphological feature vector to locate anatomical key points (head, fin base, caudal peduncle, etc.) on key pose frames of the fish. Based on these key points, skeletal segment connectivity relationships are defined and a midline is fitted, completing the initial skeletal labeling. Next, for the enhanced video sequence, starting from the initial labeling frame, a Kalman filter is used to track the key points. Observational data is provided through adjacent frame matching (e.g., using Lucas-Kanade optical flow) and combined with motion prediction. During tracking, skeletal structure constraints (such as skeletal segment length and joint angle range) are applied, and the key point positions are corrected through iterative optimization. The tracking results are subjected to multi-frame Savitzky-Golay smoothing to improve temporal consistency. For occluded or unreliable tracked key points, position inference is performed using the positions of visible reliable points and skeletal structure, ultimately obtaining complete skeletal motion trajectory data. Then, discrete nodes of the midline are extracted from the skeletal motion trajectory, the node angles are calculated, a midline curvature function is constructed, and the propagation speed, wavelength, and frequency of the curved wave are analyzed. The relative stiffness of the anterior, middle, and posterior segments of the fish body is estimated, and bending energy consumption is estimated based on stiffness, curvature, and rate of change of angle. The coordination between muscle contraction intensity distribution and bending wave propagation is analyzed to generate body bending characteristic data. Simultaneously, local coordinates of key points in the caudal fin are extracted, and the lateral offset of the caudal fin is calculated. After Butterworth filtering, the oscillation period, amplitude, and phase are analyzed. Oscillation asymmetry (amplitude, time) is calculated, caudal fin stiffness is assessed, and caudal fin propulsion force is estimated based on an empirical model, generating caudal fin oscillation temporal characteristics. Furthermore, head orientation is extracted from the skeletal trajectory, and abrupt changes in angular velocity are detected to identify turning events. The correlation between turning trajectory (radius), dynamic parameters (angular velocity), body bending and turning, and velocity changes before and after turning are analyzed. Turning energy efficiency is evaluated, and turning events are classified into different patterns through clustering to generate a turning behavior feature set. Finally, the body bending characteristic data, caudal fin oscillation temporal characteristics, and turning behavior feature set are integrated to construct a multidimensional temporal or statistical feature describing the fish's swimming pattern, forming a fish gait feature spectrum.

[0030] Step S4: Perform multi-source data fusion and behavior change identification on the fish body morphology feature vector and fish body gait feature spectrum to obtain a behavior change trend map; perform a health assessment on the behavior change trend map to obtain health status assessment data;

[0031] In this embodiment of the invention, the fish morphological feature vector and the fish gait feature spectrum are Z-score standardized to scale the features of each dimension to a uniform range. Then, based on the feature importance weights learned from historical recognition tasks, a weighted concatenation method is used to fuse the standardized morphological and gait features into a unified fused feature representation. Next, the fused feature representation of the current fish is matched with the fused feature representations of known individuals in the historical fish feature database using cosine similarity. By setting a matching threshold and a score difference margin, it is determined whether the current fish has been successfully identified as a historical individual, generating an individual recognition result set. For identified individuals, distance indicators related to body length and height are extracted from the historical morphological feature vector. A pre-built regression model is used to estimate body length and height at different time points, individual growth curves are plotted, and growth rate and deviation from the standard growth curve are calculated to form a growth parameter analysis table. Simultaneously, key behavioral indicators (such as swimming speed, swaying frequency, and turning frequency) are extracted from the historical gait feature spectrum, smoothed using a sliding window, and the CUSUM algorithm is applied to detect significant changes in behavioral patterns, generating a behavioral change trend map. Based on the growth parameter analysis table and behavioral change trend chart, each indicator is compared with the preset normal range threshold or its own historical baseline. Indicators that exceed the range or show significant changes are marked. Individuals with multiple abnormal indicators are marked as having health abnormalities, generating health abnormality marker data. Finally, based on the health abnormality marker data, growth parameters, and behavioral change trends, a weighted scoring function is designed to calculate the individual's comprehensive health score. Health status levels (e.g., Healthy, At Risk) are then assigned according to the score, and score change trends are analyzed to generate health status assessment data.

[0032] Most importantly, the fish swimming gait analysis is specifically based on the timing characteristics of tail fin wagging and the feature set of turning behavior:

[0033] The timing characteristics of the tail fin's oscillations are extracted to obtain an oscillation spectrum feature map;

[0034] A set of swimming efficiency indicators was calculated based on skeletal motion trajectory data, body bending characteristic data, and tail fin swing timing characteristics.

[0035] Based on the swimming efficiency index set, individual gait characteristics are summarized from the turning behavior feature set and the oscillation spectrum feature map to obtain the fish gait feature spectrum;

[0036] In this embodiment of the invention, the tail fin wobbling filtering sequence (the time series of tail fin lateral offset x_filtered[t]) contained in the tail fin wobbling timing features is used to perform time-frequency analysis using the Short Time Fourier Transform (STFT) method. The time series x_filtered[t] is divided into a series of overlapping time windows of length N_window (e.g., the window length N_window corresponds to 0.5 seconds of video frames, with an overlap ratio of 50%). A Hanning window function is applied to each window to reduce spectral leakage. The Discrete Fourier Transform (DFT) of each windowed signal segment is calculated. The square of the magnitude of the DFT result is calculated to obtain the power spectral density within that time window. The power spectral density results of consecutive time windows are arranged in chronological order to form a two-dimensional matrix with dimensions of time and frequency. The values ​​in the matrix represent the wobbling energy density at the corresponding time and frequency. This matrix is ​​the wobbling spectrum diagram. Quantization features are extracted from the oscillation spectrogram. For example, within each time window, the frequency with the highest energy is identified as the dominant frequency; the total energy within a specific frequency range (e.g., 0-1Hz, 1-2Hz, 2-3Hz) is calculated; and the rate of change of the dominant frequency over time (frequency modulation) is analyzed. These quantization features extracted from the spectrogram constitute the oscillation spectrogram feature map, which is actually stored as a time series of these statistical features.

[0037] Calculate the instantaneous forward velocity V_forward(t) (the velocity component along the fish's central axis) at the fish's center point (e.g., head center or center of mass) from the skeletal motion trajectory data. Obtain the estimated instantaneous bending energy dissipation rate P_bend(t) from the body bending feature data. Obtain the estimated instantaneous caudal fin propulsion force F_thrust(t) and caudal fin tip velocity V_tip(t) from the caudal fin swing timing features. Estimate the instantaneous total energy dissipation rate P_total(t)≈P_bend(t)+P_fin(t), where P_fin(t) is the power of the caudal fin, which can be approximated as F_thrust(t)×V_tip(t). Calculate the instantaneous swimming efficiency η(t)=(F_thrust_forward(t)×V_forward(t)) / P_total(t), where F_thrust_forward(t) is the component of the propulsion force in the forward direction (this part of the estimation is more complex; a simpler efficiency index can also be used). Use a simplified efficiency metric based on kinematic parameters, such as the Strouhal number (St). For each oscillation cycle or within a short time window, calculate the average forward velocity V_avg, the average oscillation frequency f_avg, and the average oscillation amplitude A_avg. Calculate St = f_avg × A_avg / V_avg. For fish swimming, a Strouhal number between 0.2 and 0.4 generally corresponds to high propulsion efficiency. Calculate the average Strouhal number St_avg and its standard deviation σ_St over a time window (e.g., 30 seconds). Calculate the average forward velocity V_avg within this time window. Compile these average Strouhal numbers, Strouhal number standard deviations, and average forward velocities into a set of swimming efficiency metrics, which are actually stored as time series of these statistical indicators.

[0038] The dynamic characteristic indicators calculated within a recent time window (e.g., the past minute) are statistically summarized to form a fixed-length vector describing the individual's typical swimming gait. From the time series of the swimming efficiency indicator set, the mean Strouhal number, the standard deviation of the Strouhal number, the mean forward speed, and the standard deviation of the forward speed are calculated. From the turning behavior feature set (which is itself a statistical summary or pattern classification result), the turning frequency (e.g., number of turns per minute), the average turning radius, the proportion of different turning patterns (sharp turns, gentle turns), and the average turning energy efficiency are extracted. From the time series of the oscillation spectrum feature map, the mean dominant frequency, the standard deviation of the dominant frequency, the average energy proportion of a specific frequency band (e.g., 0-1Hz, 1-2Hz), and the frequency modulation degree index are calculated. These statistically summarized values ​​are combined into a multi-dimensional vector. For example, a vector can contain: [average forward speed, standard deviation of forward speed, average Strouhal number, standard deviation of Strouhal number, turning frequency, average turning radius, proportion of sharp turn patterns, proportion of gentle turn patterns, average dominant oscillation frequency, standard deviation of dominant oscillation frequency, energy proportion of 0-1Hz, energy proportion of 1-2Hz, average turning efficiency]. This vector comprehensively describes the individual-specific attributes of the fish in terms of movement speed, oscillation patterns, turning strategies, and swimming efficiency, constituting the fish gait feature spectrum.

[0039] Preferably, step S1 includes:

[0040] Step S11: Construct an underwater imaging environment for the fish population in the aquaculture pond and obtain a standardized observation environment parameter table;

[0041] Step S12: Based on the standardized observation environment parameter table, acquire the original multi-angle video sequence of the fish using a camera;

[0042] Step S13: Perform image preprocessing and enhancement on the original multi-angle video sequence to obtain the enhanced video sequence;

[0043] Step S14: Perform fish body segmentation and tracking on the enhanced video sequence to obtain fish body segmentation trajectory data;

[0044] Step S15: Select and synthesize the best viewpoint for the enhanced video sequence based on the fish body segmentation trajectory data to obtain a multi-view optimized image set;

[0045] Step S16: Extract keyframes from the multi-view optimized image set to obtain the key pose frame set of the fish body;

[0046] Step S17: Perform fine feature region localization on the key pose frame set of the fish body to obtain fine image patches of the feature regions;

[0047] Step S18: Perform data structuring processing on the fine image patches of the feature region and the key pose frame set of the fish body to obtain the original dataset of fish body features.

[0048] In this embodiment of the invention, a standardized observation area of ​​2 meters × 2 meters × 1.5 meters (length × width × depth) is defined in the aquaculture pond. A waterproof LED light source array is installed at the top and around the perimeter of this area, with the light source power set to 100 watts / square meter. The angle is adjusted to achieve a light uniformity of less than 10% within the area. At least four high-resolution underwater cameras, such as industrial-grade underwater cameras with a resolution of 3840 × 2160 pixels and a frame rate of 30 frames per second, are installed along the perimeter and bottom of the area. The camera installation positions and angles are calibrated to ensure 360° coverage of the observation area. A water clarification device is installed at the entrance of the observation area. This device integrates physical filtration and ultraviolet sterilization functions, reducing the concentration of suspended solids in the water to less than 5 mg / L and improving water transparency. The system records the camera serial numbers and spatial coordinates. , where i represents the camera number, and environmental parameters such as light intensity (lux) and water temperature (degrees Celsius) are used to generate a standardized observation environment parameter table.

[0049] Based on the observation area defined in the standardized observation environment parameter table, an infrared through-beam sensor array is deployed at the area entrance. This array contains multiple pairs of transmitters and receivers, forming a crisscrossing sensor beam network. When a fish swims through and blocks any one or more pairs of beams, a trigger signal is sent to the control unit. Upon receiving the trigger signal, the control unit immediately activates all camera systems associated with that area to begin synchronously acquiring video sequences. Simultaneously, a guide channel with a width of 0.5 meters and a length of 1 meter is designed to guide the fish to the standardized observation area. After the camera system is triggered and activated, it continuously acquires a 2-second video sequence to ensure that the entire process of the fish passing through the observation area is recorded. The system automatically records the start timestamp of the video sequence (e.g., 2023-10-27, 10:35:12.345) and associates and stores the current environmental parameters such as water temperature and light intensity with the video sequence to generate the original multi-angle video sequence.

[0050] Underwater image enhancement algorithms are applied to the acquired original multi-angle video sequences. For example, a Dark Channel Prior (DCT) method is used to correct for the attenuation and scattering effects of water on light, restoring image contrast and color balance. Next, mean filtering or nonlocal mean filtering is applied to the enhanced images to suppress noise and reduce interference from water turbidity and equipment noise. Adaptive Histogram Equalization (CLAHE) is employed to enhance the contrast of local areas of the fish, making fine features such as scale texture and small scars more prominent. Inter-frame jitter correction is performed by calculating optical flow or feature point matching between consecutive frames to estimate and compensate for image translation and rotation caused by water flow, improving the stability of the video sequence. Finally, all images are uniformly adjusted to a standard size (e.g., 1920×1080 pixels) and converted to a unified color space (e.g., RGB) to ensure consistency in subsequent processing, generating the enhanced video sequence.

[0051] For the enhanced video sequence, a Gaussian Mixture Model (GMM) or background modeling method is used for background subtraction to learn and establish a static background model of the observation area. In each frame, the current pixel value is compared with the background model to identify moving foreground targets that differ significantly from the background, i.e., fish. Morphological operations are applied to the foreground segmentation results, such as erosion to remove small isolated noise points and burrs, and dilation to fill the holes inside the fish contour. Next, multi-target tracking algorithms, such as trajectory association based on the Hungarian algorithm or a combination of Kalman filtering and data association, are used to establish the correspondence of the same fish in consecutive frames. By calculating the change of the center point coordinates of the fish bounding box or contour over time, the motion trajectory of each tracked fish is generated, and its contour pixel set in each frame is recorded to establish fish segmentation trajectory data.

[0052] Based on the contour information of each fish in each frame from the fish segmentation trajectory data, the visibility score of the fish is calculated from each camera's perspective. The visibility score comprehensively considers the fish's area proportion in the frame, contour sharpness (e.g., calculating the contour gradient variance), and fish integrity (e.g., whether it is occluded or partially out of frame). For each time point (i.e., video frame), the system selects the two to three frames with the highest calculated visibility scores as preferred viewpoint images. These preferred image sets preserve the morphological and pose information of the fish captured from different advantageous angles at the current moment. The output of this step is a set of the clearest and most complete images selected for each fish at each time point or keyframe, constituting a multi-view preferred image set.

[0053] Pose analysis is performed on the video sequences corresponding to the multi-view optimized image set to identify key pose points of the fish during swimming. Typical pose events are defined and detected by calculating indices such as the rate of change of curvature along the fish's central axis and the swing angle of the tail key points relative to the central axis. These events include body straightening (rate of change of curvature close to zero), maximum bending (curvature reaching a local extreme), acceleration, deceleration, turning (rate of change of head direction greater than a threshold), and extreme positions of the tail fin swinging to the sides. Representative static image frames are extracted from the corresponding optimized image set based on the time points of these detected key pose events. The extracted keyframes comprehensively cover various angles and main morphological features of the fish in different swimming states, such as close-ups of the head, full-body side views, and the instant of tail swing. Each extracted keyframe is assigned a precise timestamp and corresponding pose label (e.g., "straight body," "left turn," "maximum right tail swing") to generate a set of key pose frames for the fish.

[0054] In each image of the fish's key pose frames, high-information fine-feature regions on the fish's body are located using a pre-trained segmentation model or a geometry-based method. These include the head, pectoral fins, dorsal fin, anal fin, caudal fin, lateral line region, and specific spot regions on the body surface (if any). For these located feature region image patches, local contrast enhancement algorithms, such as contrast-limited adaptive histogram equalization (CLAHE), are applied to further highlight minute details within the regions, such as small scars, missing scales, and fin forks. A sharpness index for each feature region image patch is calculated, for example, using the Tenengrad gradient function or the variance of the Laplacian operator. Regions with sharpness higher than a preset threshold are selected for subsequent processing. A local two-dimensional coordinate system is created for each selected feature region, with its origin at the center of the region and its axes parallel to the image boundaries, facilitating subsequent feature extraction and comparison, generating fine-grained image patches of the feature regions.

[0055] A hierarchical data structure is designed to organize and store all acquired raw data and intermediate processing results. The top-level structural unit is a single acquisition batch, uniquely identified by the acquisition start timestamp. Each batch contains multiple fish instances, each assigned a unique identifier (e.g., the batch timestamp plus the fish's sequence number within that batch). For each fish instance, its corresponding key pose frame set (static features) and fish segmentation trajectory data (dynamic features) are stored. Each image in the key pose frame set is associated with its timestamp and pose label; the fish segmentation trajectory data stores the contour coordinates and motion parameters of each frame, establishing a connection with the key pose frame set through the timestamp. Simultaneously, the acquisition environment parameters (water temperature, lighting, etc.) and system configuration information (camera model, calibration parameters, etc.) recorded in steps S11 and S12 are stored as metadata, associated with the corresponding acquisition batch or fish instance. All data is stored in a structured file format (e.g., HDF5 or Protocol Buffers) in a database or storage system, forming the raw dataset of fish features.

[0056] Preferably, step S2, which involves classifying the body surface feature points of the original fish body feature dataset, includes:

[0057] The original dataset of fish body features was preprocessed with feature region enhancement to obtain a set of enhanced feature images.

[0058] Coarse extraction of key points on the body surface is performed on the enhanced feature image group to obtain a candidate feature point set on the body surface.

[0059] The candidate feature point set on the body surface is filtered and precisely located to obtain accurate feature point data;

[0060] The precise feature point data is classified into feature point types to obtain a categorized feature point set.

[0061] In this embodiment of the invention, enhanced preprocessing is performed on fine image patches of feature regions in the original dataset of fish body features. For example, a contrast-limited adaptive histogram equalization (CLAHE) algorithm is applied to each image patch, dividing the image patch into 8×8 small regions. Each region is independently histogram equalized, and the contrast limit is set to 0.02 to enhance local detail contrast while avoiding excessive noise amplification. Next, a bilateral filter is applied, with the filter kernel size set to 5×5 pixels and the spatial standard deviation set to 0.02. Set to 3, strength standard deviation Setting it to 50 preserves edge information while smoothing the image. For the scale-like region, multi-scale Difference of Gaussians (DoG) decomposition is applied to extract texture features at different scales, for example, using... and , and Different scale combinations are used to separate fine and coarse textures. For transparent or semi-transparent areas such as fins, color- and texture-based enhancement methods are applied, such as adjusting the gamma value of specific color channels and combining it with local contrast stretching. These processed image patches constitute an enhanced feature image set.

[0062] Coarse extraction of key points on the body surface is performed on the enhanced feature image group. On each enhanced feature region image patch, the Shi-Tomasi corner detection algorithm is applied, with a maximum number of corners set to 500, a minimum eigenvalue quality level k of 0.01, and a minimum Euclidean distance of 5 pixels between two corners. Points with high curvature in the image are detected; these points are often located at scale edge junctions and scar contour transitions. For fin image patches, a Hessian matrix-based spot detection algorithm (e.g., using the maximum determinant in scale space) is applied to identify tiny spots or damaged areas on the fin membrane. Simultaneously, the direction of the fin rays is identified by analyzing the image gradient direction, and intersections at fin ray bifurcation or convergence points are detected as candidate feature points. The pixel coordinates (u,v) and corresponding detection response intensities (e.g., the minimum eigenvalue of the Shi-Tomasi algorithm) of all detected candidate feature points are recorded. This set of points constitutes the body surface candidate feature point set.

[0063] Feature point screening and precise localization are performed on the candidate feature point set on the body surface. For each point in the candidate feature point set, using the preferred image set of the same fish at similar time points and different viewpoints obtained in step S15, as well as the known camera calibration parameters and fish pose estimation (from trajectory data in S14), the point is back-projected from the current viewpoint to 3D space, and then reprojected onto other preferred viewpoint images. The distance between the reprojected point and the candidate points detected in other viewpoints is calculated to evaluate the spatial consistency of the point under multiple viewpoints. A consistency threshold is set (e.g., reprojection error less than 3 pixels) to remove candidate points with poor spatial consistency. At the same time, a detection response intensity threshold is set to remove candidate points with excessively low response intensity. For the high-quality candidate points after screening, a sub-pixel precise localization algorithm is applied, such as an iterative method based on image gradients, to calculate the sub-pixel level precise coordinates (u_sub, v_sub) by analyzing the image gradient in the small neighborhood around the point. A unique identifier is assigned to each precise feature point to facilitate subsequent tracking and matching, thereby obtaining precise feature point data.

[0064] The precise feature point data is classified into feature point types. For each precise feature point, a local image patch centered on it is extracted (e.g., 32×32 pixels). A local descriptor, such as a Scale Invariant Feature Transform (SIFT) descriptor, is computed on this local image patch. This descriptor is a vector representing the histogram of gradient directions of the local region. A pre-trained classification model, such as a Support Vector Machine (SVM) classifier, is used, taking the SIFT descriptor as input and outputting the probability or score of the feature point belonging to different body surface feature types. This SVM model uses a Radial Basis Function (RBF) as its kernel function and is trained on a labeled dataset containing various fish bodies and feature types. The classification model categorizes precise feature points into predefined type categories, such as: scale edge points (Type Code 1), pigment spots (Type Code 2), small scar points (Type Code 3), fin bifurcation points (Type Code 4), etc. Each precise feature point is assigned an identified type code and a corresponding classification confidence score (e.g., the probability value output by the SVM). These precise feature point sets, containing type and confidence information, constitute the classification feature point set.

[0065] Preferably, the step S2 of constructing the fish morphological feature vector based on the classification feature point set includes:

[0066] The spatial relationships of the feature points in the classification feature point set are constructed to obtain the feature point spatial relationship graph;

[0067] Based on the spatial relationship diagram of feature points and the set of classification feature points, a fish body morphology feature network is generated to obtain the fish body feature network structure.

[0068] Temporal variation patterns were extracted from the fish body feature network structure to obtain feature temporal variation data.

[0069] Based on the characteristic time change data and the fish body feature network structure, morphological feature vectors are constructed to obtain the fish body morphological feature vectors.

[0070] In this embodiment of the invention, for each precise feature point in the classification feature point set, based on its two-dimensional image coordinates (u_sub, v_sub) and known camera parameters and the approximate fish pose estimation, it is mapped to an approximate local three-dimensional coordinate system on the fish surface. A local neighborhood radius R is selected (e.g., R = 20 pixels on the two-dimensional image plane projection), and for each feature point... Find all other feature points in its neighborhood. ( If point exist If the two are within the neighborhood of each other, then a local spatial relationship is considered to exist between them. Calculation arrive Two-dimensional Euclidean distance At the same time, calculate the vector. and The angle between the principal directions of the local region (e.g., estimated through local principal component analysis). Construct an adjacency list or sparse matrix to store these relationships, where each entry records a pair of feature points with a spatial relationship. and the distance between them. Relative angle And their respective type codes. This structured data representation constitutes a spatial relationship graph of feature points.

[0071] Based on the spatial relationship graph of feature points and the classification feature point set, feature points with high stability (e.g., consecutive occurrence rate above 90%) and high classification confidence (e.g., confidence above 0.8) at multiple time points are selected as anchor points. The fish body feature network skeleton is constructed based on these anchor points. For example, two-dimensional Delaunay triangulation is performed using anchor points to form a basic triangular mesh. For non-anchor points, they are added to the interior of the nearest triangle or described by their distance and relative angle to the nearest anchor point. The area of ​​each triangle in the network is calculated. Where k represents the triangle number and the length of each side. , where 'e' represents the edge number. The topological characteristics of the network are extracted, such as calculating the average node degree, the spatial distribution density of different types of feature points, and whether there are significant connected regions or isolated points in the network. This structured data, including anchor point positions, relative positions of non-anchor points, connectivity (edges), local geometric attributes (side length, angle, triangle area), and global topological attributes, constitutes the fish body feature network structure.

[0072] For the same fish at different time points and Registration is performed on a fish body feature network structure (e.g., at 1-second intervals). Each matching point is calculated by matching feature points with the same unique identifier. from arrive position change And calculate its speed. Analyze the deformation of the network structure, such as calculating the rate of change of the area of ​​the corresponding triangle or the rate of change of the side length. Identify in Existence but Disappearing feature points (indicating scar healing or scale regeneration) and in Newly emerging feature points (representing new damage or pigment changes). The frequency of appearance / disappearance of different types of feature points is statistically analyzed. The average rate and variance of feature point position changes are calculated. These data on position changes, rates of change, deformation rates, appearance / disappearance events, and their frequencies constitute the feature time-varying data.

[0073] The fish body feature network structure (e.g., the structure at the most recent data collection time t0) and feature temporal variation data (e.g., changes within a recent time period ΔT) are encoded into numerical vectors. These vectors contain several dimensions: global features such as the total number of feature points and the proportion of different types of feature points (scale edge points, scar points, pigment spots, fin forking points, etc.); local geometric features such as a histogram of feature point spacing (e.g., divided into 10 intervals) and a histogram of relative angles between adjacent points (e.g., divided into 12 directional intervals); and network topology features such as average node degree and the number of connected components. Temporal features are extracted from the feature temporal variation data, such as the average point velocity amplitude over the past ΔT, the standard deviation of the point velocity amplitude, the average occurrence rate (per minute) of different types of feature points, the average disappearance rate (per minute) of different types of feature points, and the average network area change rate. For each dimension of the vector... Assign a weight The weights are obtained based on the feature's inter-individual discriminative power and intra-individual stability. The final morphological feature vector V consists of the weighted features of each dimension: , where n is the vector dimension. Perform vector V... Normalization is performed to obtain the final fish body morphology feature vector. ,in .

[0074] Preferably, step S3, which involves locating and tracking the fish skeleton based on the fish's morphological feature vector, includes:

[0075] Based on the fish body morphology feature vector, the skeletal key points of the original fish body feature dataset are located to obtain the fish body skeletal key point set.

[0076] Initialize the bone markers by performing bone initialization marking based on the key point set of the fish skeleton.

[0077] Based on the initialized bone markers, the enhanced video sequence is subjected to temporal tracking of the bone structure to obtain bone motion trajectory data.

[0078] In this embodiment of the invention, a deep learning pose estimation algorithm model, such as a model based on MaskR-CNN or AlphaPose architecture, is used. This model is pre-annotated with anatomical key points (e.g., head center, left / right pectoral fin base, dorsal fin origin, anal fin origin, caudal peduncle, and caudal fin tip top / bottom) on a large number of fish images and trained. The model takes a single image from a set of key pose frames as input and outputs the pixel coordinates (u_kp, v_kp) and detection confidence scores of each key point in the image, where kp represents the key point type. The output of the pose estimation algorithm model is post-processed and validated using stable feature point information contained in the fish morphological feature vector. For example, the relative positional relationship between the detected key points and known stable feature points in the morphological feature vector is calculated. If the positional relationship between the detected key point and the morphological feature point deviates from the typical pattern learned through the morphological vector (e.g., the distance between the head center and a certain stable scale feature point is outside the statistical distribution), the confidence score of the key point is reduced or its position is corrected. The final output is verified and corrected data containing the type, pixel coordinates, and confidence level of each keypoint, forming a keypoint set for the fish skeleton.

[0079] In a selected keyframe from the fish's skeletal keypoint set (e.g., the frame where the fish's posture is closest to an extended state), a simplified skeletal model of the fish is constructed using the coordinates of the keypoints detected in that frame. The connection relationships between bone segments are defined as follows: head center connects to the base of the left and right pectoral fins; head center connects to the origin of the dorsal fin; dorsal fin origin connects to the caudal peduncle; head center connects to the origin of the anal fin; anal fin origin connects to the caudal peduncle; and caudal peduncle connects to the upper / lower tip of the caudal fin. These connections constitute the skeletal structure. Simultaneously, based on the keypoints such as the head center, dorsal fin origin, anal fin origin, and caudal peduncle, a curve passing through the central axis of the fish is fitted (e.g., using cubic spline interpolation). This curve represents the main body of the fish and is used for subsequent body bending analysis. The initial coordinates of these keypoints, their connection relationships, and the central axis representation are stored as initial skeletal markers.

[0080] Starting from the video frame where the skeleton markers are initialized, a multi-object tracking algorithm is used to perform inter-frame tracking of skeletal keypoints. A Kalman filter-based multi-object tracking method is employed, establishing an independent Kalman filter for each skeletal keypoint. The state vector of each filter contains the pixel coordinates (u,v) of the keypoint and its velocity on the image plane. ,Right now The prediction step predicts the keypoint positions in the current frame based on the state and motion model of the previous frame (e.g., uniform linear motion model). The update step then refines the predicted positions using image information from the current frame. Image information is obtained by performing local feature matching (e.g., calculating the motion vector of the region using optical flow) or appearance-based matching within a small region (e.g., 15×15 pixels) around the predicted position. During tracking, skeletal structure constraints are introduced: the length changes of adjacent bone segments should be within biologically reasonable limits over short periods, and the relative positional relationships between keypoints should maintain a certain topological structure. If a keypoint is occluded (determined through fish body segmentation data), its position depends entirely on the prediction of the Kalman filter and the motion inference of adjacent visible keypoints. The coordinates and confidence scores of all tracked skeletal keypoints in each frame are stored chronologically to form skeletal motion trajectory data.

[0081] Most importantly, the temporal tracking of the skeletal structure in the enhanced video sequence based on the initialized skeletal markers is specifically performed as follows:

[0082] Based on the initialized skeleton markers, adjacent frame matching search is performed on the enhanced video sequence to obtain the inter-frame matching results;

[0083] Motion state prediction is performed on the inter-frame matching results to obtain motion prediction data;

[0084] Skeletal structure constraints are applied based on motion prediction data to obtain the constrained optimal position;

[0085] Multi-frame consistency checks are performed on the constrained optimization positions to obtain a temporally smoothed trajectory.

[0086] Based on the temporal smoothing trajectory, the occlusion point positions of the enhanced video sequence are inferred to obtain the skeletal motion trajectory data;

[0087] In this embodiment of the invention, adjacent frame matching search is performed on the enhanced video sequence based on the initialized skeletal markers (including keypoint type, initial coordinates, and skeletal connection relationships). For each frame t in the video sequence, the positions of the skeletal keypoints already tracked in frame t-1 are used as the starting point. A local search window (e.g., window size 21 pixels × 21 pixels) is defined in frame t, centered on the position of each keypoint in frame t-1. Within this search window, the motion vector of the pixel is calculated using the Lucas-Kanade sparse optical flow algorithm. The motion vector of the pixel at the keypoint position in frame t-1 or its small neighboring pixels is used as the observation data for the predicted position of the keypoint in frame t. A minimum quality threshold for optical flow tracing is set (e.g., optical flow points less than 0.01 are not considered) and a maximum number of iterations (e.g., 30 times). For each keypoint, its potential matching position in frame t and the corresponding tracking confidence (e.g., tracking quality based on feature points or optical flow residuals) are calculated. These potential matching positions and confidences constitute the inter-frame matching results.

[0088] Using inter-frame matching results and the tracking history of previous frames, the precise position of each skeletal keypoint in the current frame (frame t) is predicted. A simple linear motion model is used for prediction: it is assumed that the keypoints move at a constant velocity in a straight line over a short period of time. The predicted position of keypoint i in frame t is... ,in This is the key point. At the final tracking position in frame t-1, It is its velocity in frame t-1 (by...) (Estimation), where Δt is the inter-frame time interval (e.g., for 30fps video, Δt = 1 / 30 second). If a Kalman filter is used, the Kalman filter prediction step is performed here, predicting the current state (position and velocity) based on the state estimate from the previous time step and the system dynamics model. These predicted positions constitute the motion prediction data.

[0089] The keypoint locations in the motion prediction data are considered as initial estimates. Constraints are imposed using knowledge of the fish's anatomical and skeletal structure. Key constraints include: 1. Skeletal segment length constraints: The distance between connected keypoint pairs (e.g., the center of the head and the origin of the dorsal fin) should be close to their initial length determined in the initialization markers or the relative length estimated from the total fish length, allowing a small fluctuation range (e.g., ±5%). 2. Joint angle constraints: The joint angle formed by three connected keypoints (e.g., the angle formed by the origin of the dorsal fin, the caudal fin peduncle, and the tip of the caudal fin at the caudal peduncle) should be within a biologically reasonable range (e.g., 0 to 180 degrees). 3. Relative position constraints: Stable relative positional relationships exist between certain keypoints (e.g., the bases of the left and right pectoral fins should be approximately symmetrically distributed on both sides of the midline). An optimization problem is constructed to find a set of keypoint locations that are both close to the motion prediction data and maximally satisfy the skeletal structure constraints. This optimization problem can be expressed as minimizing an energy function E = E_data + E_structure, where E_data measures the difference between the predicted and optimized locations, and E_structure measures the degree to which the optimized locations violate structural constraints. The problem is solved using an iterative optimization algorithm (e.g., the Levenberg-Marquardt algorithm or gradient-based optimization methods) to obtain the locations of the key optimization points that satisfy the constraints, which constitute the constrained optimization locations.

[0090] Constrained optimization position data from the most recent frames (e.g., the most recent 5 frames) are collected to form short-term trajectories for each keypoint. Savitzky-Golay smoothing filters are applied to these short-term trajectories. A quadratic polynomial is fitted to each coordinate component (u and v) in the trajectory data using a smoothing window of 5 frames. The smoothed position of the center frame of the filter output window reduces random noise and local jitter. Simultaneously, instantaneous anomalous jumps in velocity or acceleration are detected in the trajectory. If the rate of change of velocity (acceleration) of a keypoint in a frame exceeds a preset threshold (e.g., more than 3 times the average acceleration), the point is marked as a potential anomaly and corrected by interpolation using the smoothed trajectories of the preceding and following frames. The sequence of keypoint positions after multi-frame consistency checks and corrections constitutes the temporally smoothed trajectory.

[0091] In the current frame, if a keypoint is determined to be outside the fish body outline, occluded by other fish bodies, or exhibits anomalies in its multi-frame consistency check results based on the fish body segmentation trajectory data (from S14), then the keypoint is considered occluded or unreliable in tracking. For these unreliable keypoints, their performance in previous frames is used in the temporally smoothed trajectory, combined with the positions of other visible and reliable keypoints in the current frame and skeletal structure constraints, to infer their precise position in the current frame. For example, if the caudal fin tip is occluded, but the caudal peduncle position is reliable, the position of the caudal fin tip can be inferred from the caudal peduncle position in the current frame based on the average distance and relative angle between the caudal peduncle and the caudal fin tip in previous frames. Another inference method is to use a Kalman filter, where, for occluded keypoints, the current frame's observation data is not used in the update step; instead, the state is updated solely based on the prediction step (based on the motion model and the state estimate from the previous time step). The tracked reliable keypoint positions are then merged with the inferred unreliable keypoint positions to form a complete set of skeletal keypoint positions for the current frame. The skeletal keypoint position data of all video frames are stored in chronological order to form skeletal motion trajectory data.

[0092] Preferably, step S3, which involves quantifying the body curvature of the skeletal motion trajectory data, includes:

[0093] Extract the sequence of midline coordinate points from the head to the tail of the fish from the skeletal motion trajectory data, and perform midline discretization to obtain discrete node data of the midline.

[0094] The node angles are calculated from the discrete node data of the central axis to obtain a sequence of node angles;

[0095] Construct the central axis curvature function based on the node angle sequence and discrete node data of the central axis;

[0096] Bending wave propagation analysis was performed based on the central axis curvature function to obtain bending wave propagation characteristic data;

[0097] The fish body is divided into three main sections: front, middle and rear. The stiffness coefficients of the sections are estimated based on the central axis curvature function and the nodal angle sequence to obtain the stiffness distribution data of the sections.

[0098] Bending energy consumption data were estimated based on the section stiffness distribution data and the node angle sequence.

[0099] Muscle contraction intensity analysis was performed based on segment stiffness distribution data and central axis curvature function to obtain muscle contraction intensity distribution data.

[0100] The bending coordination was evaluated using muscle contraction intensity distribution data and bending wave propagation characteristics data to obtain bending coordination indices.

[0101] Body bending characteristic data are generated based on bending coordination index.

[0102] In this embodiment of the invention, the time-series smoothed trajectory data of key points such as the head center, dorsal fin origin, anal fin origin, and caudal peduncle obtained from the skeletal key point localization in step S3 are used as control points. A smooth curve is fitted through these control points using cubic spline interpolation, representing the midline of the fish. The parameterized form of this curve is C(s)=(x(s),y(s)), where s is a parameter varying from 0 to 1. The fitted midline curve is discretized into a fixed number of nodes, for example, 20 nodes, with equal arc lengths from the head (s=0) to the tail (s=1). Each node... The position is determined by its parameter value on the curve. Sure, Where i ranges from 1 to N, and N=20. The two-dimensional coordinate sequence of these discrete nodes constitutes the discrete node data of the central axis, recording the shape of the fish's body along the central axis in each frame.

[0103] For each frame of the discrete node data of the central axis, select three consecutive nodes. (i ranges from 2 to N-1). Calculate the vector. and Calculate the angle between these two vectors. included angle It can be obtained by calculating the vector dot product and the magnitude: .in , included angle Reflects the nodes The degree of local curvature at each point is determined. The calculated angles of all internal nodes (from node 2 to node N-1) are arranged in order to form a node angle sequence, which records the distribution of local curvature angles of the fish body along the central axis in each frame.

[0104] Curvature κ is a quantity describing the degree of curvature of a curve; it is the rate of change of the tangent direction angle with respect to arc length. For discrete nodes, the curvature can be approximated using the node angles. Curvature It can be approximated as Divide by the average length of two adjacent line segments: Here Indicates a straight line state (with an included angle of ). The deviation angle of the calculated curvature value. Arc length position along the central axis (e.g., using normalized arc length) Interpolation is performed to construct a continuous function. , where s represents the normalized arc length along the central axis, ranging from 0 to 1. This function describes the curvature distribution of the fish body along the central axis, forming the central axis curvature function, which is actually stored as a sequence of curvature values ​​at discrete nodes.

[0105] The variation of the axial curvature function over time was analyzed. Local maxima and minima (i.e., crests and troughs of the flexural wave) were identified on the axial curvature function κ(s) in each frame. The trajectories of these crests and troughs along the axial axis (s-direction) were tracked over time. The propagation speed of the flexural wave, v_wave = Δs / Δt, was estimated in body length per second by calculating the positional change Δs of the crests / troughs in consecutive frames and the inter-frame time interval Δt. The wavelength λ of the flexural wave was obtained by calculating the distance between crests in body length. The frequency of the flexural wave, f_wave = 1 / T_pass, was obtained by calculating the time required for the flexural wave to travel the entire length of the fish, where T_pass is the period. The stability of the wave speed, wavelength, and frequency, as well as the number of flexural waves present on the fish at any given time, were analyzed. These calculated parameters constitute the propagation characteristics data of the flexural wave.

[0106] The fish body is divided into three main segments: anterior, middle, and posterior. For example, the normalized arc length *s* of the central axis from head to tail is divided into: anterior segment (0 ≤ *s* < 0.3), middle segment (0.3 ≤ *s* < 0.7), and posterior segment (0.7 ≤ *s* ≤ 1.0). This is based on the central axis curvature function. and node angle sequence Estimate the relative "stiffness coefficient" of each segment. Stiffness can be indirectly estimated by analyzing the degree of bending of the segment under muscle-driven (manifested by the propagation of bending waves) conditions. For example, if the average curvature change of a segment is smaller than that of other segments when a bending wave passes through, that segment is considered to have relatively high stiffness. Calculate the standard deviation of the nodal angles or curvature values ​​within each segment. or This serves as a measure of the bending flexibility of that section. The stiffness coefficient K_section can be negatively correlated with flexibility, for example... These estimated values, representing the relative ease of bending in the front, middle, and rear sections, constitute the section stiffness distribution data.

[0107] Bending energy loss is related to the work done by muscles to overcome the internal stiffness of the fish body and water resistance. A simplified energy loss model is adopted: instantaneous bending energy loss rate. ,in It is the local stiffness of node i at time t, estimated based on the segment stiffness distribution. It is the amount by which the bending angle of node i deviates from the straight line. This is the rate of change of the bending angle at node i. The rate of change of angle is obtained by taking the time derivative of the node angle sequence. The stiffness coefficient of the segment to which each node belongs is then assigned to that node as... The instantaneous energy consumption rate of all nodes i is summed to obtain the total instantaneous energy consumption rate of the fish. The instantaneous energy consumption rate is then integrated over a time window (e.g., a complete oscillation cycle) to obtain the total bending energy consumption over that time period. These estimated energy consumption values ​​constitute the bending energy consumption data.

[0108] Fish body curvature is primarily produced by muscle contractions along both sides of the body. Assume that achieving a certain local curvature κ requires a corresponding muscle contraction strength. The required strength depends on the magnitude of the local curvature and the tissue stiffness at that location. A simplified model: instantaneous muscle contraction intensity distribution. Here, K(s,t) is the local stiffness at time t along the central axis at position s (obtained through interpolation of segmental stiffness distribution), and κ(s,t) is the curvature at that position. By analyzing the spatial distribution and temporal variation of the central axis curvature function, and combining it with the estimated local stiffness, the relative muscle contraction intensity distribution along the fish's central axis is estimated. For example, the estimated muscle strength will be higher on the side of the body in front of the flexural crest. These estimated time-varying relative muscle contraction intensity distribution curves along the fish's central axis constitute the muscle contraction intensity distribution data.

[0109] Assess the spatiotemporal coordination between the muscle contraction wave (represented by the time-shifting peak of the muscle contraction intensity distribution) and the flexural wave (represented by the time-shifting peak of the axial curvature function). Calculate the phase difference (e.g., in normalized arc length *s* or in angles) between the peak positions of muscle contraction intensity and curvature along the axial direction. For efficient propulsion, the muscle contraction wave should typically lead the flexural wave by a small phase difference. Analyze the mean and stability of this phase difference. Simultaneously, assess the smoothness and continuity of flexural wave propagation, for example, by calculating the coefficients of variation of wave velocity and wavelength over consecutive oscillation periods. A smaller, stable leading phase difference and a lower coefficient of variation of wave parameters indicate better flexural coordination. These quantitative indicators, such as mean phase difference, standard deviation of phase difference, and coefficient of variation of wave velocity, constitute the flexural coordination indices.

[0110] Based on the bending coordination index and the body bending-related data obtained in the previous steps (such as the statistical characteristics of the central axis curvature function, including maximum curvature, average curvature, and curvature variance; bending wave propagation characteristics, including average wave velocity, frequency, and wavelength; statistical characteristics of segment stiffness distribution data; statistical characteristics of bending energy consumption data, including average energy consumption rate; and statistical characteristics of muscle contraction intensity distribution data), these quantitative indicators are integrated into a multidimensional vector or structured dataset. For example, the vector may include: average central axis curvature, maximum curvature, average stiffness coefficient of the anterior, middle, and posterior segments, average bending wave velocity, average bending wave frequency, average bending coordination phase difference, standard deviation of the bending coordination phase difference, and estimated average bending energy consumption rate. These values ​​collectively describe the pattern, efficiency, and coordination of the fish's body bending, constituting the body bending characteristic data.

[0111] Preferably, step S3, which involves analyzing the tail fin oscillation of the skeletal motion trajectory data, includes:

[0112] The caudal fin key points were extracted from the skeletal motion trajectory data to obtain the local coordinate data of the caudal fin;

[0113] The caudal fin offset is calculated from the local coordinate data of the caudal fin to obtain the original sequence of caudal fin offset;

[0114] The original tail fin offset sequence was optimized by signal filtering to obtain the tail fin swing filtered sequence.

[0115] The oscillation period characteristics of the tail fin oscillation filter sequence were analyzed to obtain oscillation period data;

[0116] The amplitude and phase of the oscillation period data are extracted based on the tail fin oscillation filter sequence to obtain the oscillation amplitude and phase characteristics.

[0117] The asymmetry of the oscillation is analyzed based on the amplitude and phase characteristics of the oscillation, and the asymmetry features of the oscillation are obtained.

[0118] The caudal fin stiffness is evaluated based on the swing amplitude and phase characteristics and local coordinate data of the caudal fin, and the caudal fin stiffness index is obtained.

[0119] The tail fin thrust data was estimated based on the tail fin stiffness index;

[0120] Tail fin oscillation timing features are generated based on caudal fin propulsion data and oscillation asymmetry characteristics.

[0121] In this embodiment of the invention, the temporal smoothed trajectory of key points belonging to the caudal fin is extracted from the skeletal motion trajectory data. These key points include the caudal peduncle, the upper tail fin tip, and the lower tail fin tip. For each frame, the two-dimensional pixel coordinates of these key points are obtained. Based on the fish's central axis (provided by the body curvature quantization step) or the caudal peduncle position, a local coordinate system is established with the caudal peduncle as the origin, the X-axis approximately along the fish's forward direction, and the Y-axis perpendicular to the X-axis. The pixel coordinates of the upper and lower tail fin tips are transformed into this local coordinate system to obtain their local two-dimensional coordinates (x_tip_upper, y_tip_upper) and (x_tip_lower, y_tip_lower) relative to the caudal peduncle. These time-varying local coordinate data constitute the caudal fin local coordinate data.

[0122] In the local coordinate system of the caudal fin, the fish's forward direction roughly corresponds to the local X-axis. The lateral swaying of the caudal fin is reflected in the offset on the local Y-axis. Calculate the average Y-coordinate of the caudal fin tip: y_avg = (y_tip_upper + y_tip_lower) / 2. This average Y-coordinate represents the lateral position of the caudal fin body. Use this average Y-coordinate as the lateral offset of the caudal fin relative to the local X-axis (roughly representing the end of the fish's midline). Record the sequence of this offset over time to form the original sequence of caudal fin offsets. For example, if the positive direction of the local Y-axis points to the left side of the fish, a positive offset indicates the caudal fin sways to the left, and a negative offset indicates it sways to the right.

[0123] The raw tail fin offset sequence contains noise caused by tracking errors or water flow. A low-pass digital filter is applied to this sequence to remove high-frequency noise above the typical tail fin wiggle frequency of fish. For example, a 4th-order Butterworth low-pass filter with a cutoff frequency of 10 Hz can be used. This frequency is higher than the typical tail fin wiggle frequency range of common farmed fish such as bass and tilapia (typically 0.5-5 Hz), but lower than the noise frequency. Applying the filter to the raw offset sequence x_raw[t] yields the smoothed tail fin wiggle filtered sequence x_filtered[t].

[0124] Periodic analysis is performed on the tail fin oscillation filtered sequence x_filtered[t]. The oscillation period is determined by detecting zero-crossing points (points where the signal changes from negative to positive or from positive to negative) or local extrema (peaks and troughs) in the sequence. For example, the moment when the signal changes from a negative value to a positive value by detecting the crossing of zero points. The time interval between two consecutive zero crossings in the same direction. This is one oscillation cycle. The duration of each detected cycle is calculated. And calculate the average period T_avg and its standard deviation σ_T over a time window (e.g., 10 consecutive cycles). Oscillation frequency Calculate the average frequency f_avg and its standard deviation σ_f. These statistics of period and frequency constitute the oscillation period data.

[0125] For each detected oscillation cycle, find the maximum (peak) A_peak and the minimum (trough) A_trough of the tail fin offset within that cycle. Calculate the peak-to-peak amplitude A_pp = A_peak - A_trough. Calculate the center amplitude A_center = (A_peak - A_trough) / 2. Calculate the average center amplitude A_avg and its standard deviation σ_A over a time window (e.g., 10 consecutive cycles). Simultaneously, for each time point t in the oscillation filter sequence, calculate its relative phase within the cycle at that moment. If the start of each cycle (e.g., zero-crossing from negative to positive) is defined as phase 0, and the end of the cycle is defined as phase 2π, then time t is within the cycle... Phase within These include the average amplitude, the standard deviation of the amplitude, and the relative phase information that changes over time, which constitute the phase characteristics of the oscillation amplitude.

[0126] Analyze the difference between leftward (e.g., positive offset) and rightward (e.g., negative offset) tail fin oscillations. Calculate the maximum amplitude of the leftward oscillation A_left = max(x_filtered[t]) and the maximum amplitude (absolute value) of the rightward oscillation A_right = |min(x_filtered[t])| within one period or time window. Calculate the amplitude asymmetry index I_amp = (A_left - A_right) / (A_left + A_right). Calculate the time required for the tail fin to oscillate from zero offset to the maximum leftward offset Δt_left and the time required to oscillate to the maximum rightward offset Δt_right. Calculate the time asymmetry index I_time = (Δt_left - Δt_right) / (Δt_left + Δt_right). These asymmetry indices and their trends over time constitute the oscillation asymmetry characteristic.

[0127] Caudal fin stiffness reflects its ability to resist bending and deformation. Based on local coordinate data of the caudal fin (positions of the caudal peduncle and upper and lower tips), the caudal fin's opening angle can be calculated, which is the angle Ψ formed by the three points: the upper tip of the caudal fin, the caudal peduncle, and the lower tip of the caudal fin. During caudal fin oscillation, this opening angle decreases with increasing lateral offset. A relatively stiff caudal fin will exhibit a smaller decrease in opening angle for the same lateral offset, or less variability in opening angle as oscillation frequency increases. The caudal fin stiffness index K_fin is defined as a measure of the relationship between the caudal fin opening angle Ψ and the lateral offset y_avg, or its response to frequency variations. For example, calculating the average caudal fin opening angle at maximum lateral offset, or calculating the regression coefficient of Ψ on |y_avg|. Another index can be the rate of change of the caudal fin opening angle at different oscillation frequencies. These quantitative indicators constitute the caudal fin stiffness index.

[0128] The propulsive force of the caudal fin is estimated based on its stiffness parameters and the phase characteristics of its oscillation amplitude (mainly frequency f and amplitude A). The propulsive force F_thrust of a fish's caudal fin is a complex hydrodynamic process, but it can be estimated using simplified empirical or semi-empirical models. A commonly used simplified model assumes that the propulsive force is proportional to the square of the caudal fin oscillation frequency and the square of the caudal fin tip velocity. The caudal fin tip velocity is approximately proportional to the product of amplitude and frequency. Therefore, Considering the influence of tail fin stiffness, a moderate stiffness improves propulsion efficiency. The tail fin stiffness index K_fin is incorporated into the model, for example... Here, g(K_fin) is a function representing the effect of stiffness on thrust (e.g., g(K_fin) monotonically increases with K_fin within a certain range). Estimated instantaneous or periodically averaged thrust is calculated based on the amplitude and frequency of each frame or each oscillation cycle. These estimated values ​​constitute the tail fin thrust data.

[0129] The quantitative indicators related to caudal fin oscillation obtained from the preceding steps are integrated to form a multidimensional temporal feature vector or structured data. This includes: oscillation period data (mean frequency, frequency standard deviation), oscillation amplitude phase characteristics (mean amplitude, amplitude standard deviation), oscillation asymmetry characteristics (amplitude asymmetry index, time asymmetry index), caudal fin stiffness indicators, and estimated caudal fin propulsion force data (mean propulsion force, propulsion force standard deviation). These features can be statistically calculated at fixed time intervals (e.g., every second or every 10 seconds) (mean, standard deviation, maximum value, minimum value) to form a time series of these indicators changing over time. This time series data constitutes the caudal fin oscillation portion of the fish gait feature spectrum, i.e., the caudal fin oscillation temporal feature.

[0130] Preferably, step S3, which involves extracting turning behavior features from the skeletal motion trajectory data based on body bending feature data, includes:

[0131] Head orientation sequence is extracted from skeletal motion trajectory data to obtain head orientation data;

[0132] Perform steering event detection on the head orientation data to obtain a list of steering events;

[0133] Based on the list of turning events, the skeletal motion trajectory data is analyzed to obtain turning trajectory characteristics;

[0134] Steering dynamics parameters are calculated based on the list of steering events and head direction data;

[0135] Based on the list of steering events, a correlation analysis between body bending and steering is performed on the body bending feature data to obtain bending-steering correlation data.

[0136] Based on the list of steering events, the speed before and after steering is analyzed from the skeletal motion trajectory data to obtain steering speed characteristics;

[0137] Steering energy efficiency is evaluated by analyzing steering trajectory characteristics, steering dynamics parameters, and steering speed characteristics to obtain steering energy efficiency data.

[0138] Steering behavior patterns are classified based on steering efficiency data and bending steering correlation data to obtain steering pattern data;

[0139] A steering behavior feature set is generated based on steering pattern data.

[0140] In this embodiment of the invention, the fish head orientation sequence is calculated based on the temporally smoothed trajectory of key points on the fish head (e.g., the center of the head and points on the central axis in front of it) in the skeletal motion trajectory data. For each frame, the two-dimensional coordinates of the head center key point P_head and the point P_front extending a fixed distance (e.g., 5% of the fish length) along the central axis in front of it are obtained. The vector V_direction = P_front - P_head is calculated. The orientation angle α of this vector α = atan2(V_direction_y, V_direction_x) represents the orientation of the head in the current frame, where atan2(y,x) is the angle between the line connecting the calculation point (x,y) and the origin relative to the positive X-axis, ranging from (-π,π] or [0,2π]. The head orientation angle α calculated for each frame is arranged in chronological order to form head orientation data, which is a time-series angle value.

[0141] Turning event detection is performed on the head orientation data. The time rate of change of the head orientation data α(t) is calculated, i.e., the angular velocity ω(t) = dα(t) / dt. The central difference method is used. Where Δt is the inter-frame time interval. Threshold detection is applied to the angular velocity sequence: when |ω(t)| exceeds a preset angular velocity threshold ω_threshold_start (e.g., 15 degrees / frame) for N1 consecutive frames (e.g., N1=5), it is considered the start time t_start of a potential turning event. Frames (e.g.) If the angular velocity is below another preset threshold ω_threshold_end (e.g., 5 degrees / frame), the turning event is considered to end at t_end. Adjacent turning events with a time interval less than T_merge (e.g., 0.5 seconds) are further merged into a single event. Each detected turning event consists of its start time t_start, end time t_end, and the total change in head direction during that time interval Δα = α(t_end) - α(t_start). This list of events constitutes the turning event list.

[0142] Based on the list of turning events, a turning trajectory analysis is performed on the trajectory of the fish's center point (e.g., head center or centroid) in the skeletal motion trajectory data. For each event in the list of turning events, the two-dimensional trajectory coordinate sequence (x(t), y(t)) of the fish's center point within the event occurrence time period [t_start, t_end] is extracted. The total path length of this trajectory is calculated. Where i ranges from t_start to t_end-Δt. The turning radius R_turn can be estimated using various methods. For example, during the turning process, a circle can be fitted to the trajectory point, and the radius of the circle can be calculated; or the instantaneous curvature κ_instantaneous=|x'y''-y'x''| / (x'²+y'²)^(3 / 2) can be calculated, where x',y',x'',y'' are the derivatives of the trajectory coordinates with respect to time (velocity and acceleration). Then, the average instantaneous curvature κ_avg=(∑κ_instantaneous) / N_points is calculated, resulting in R_turn≈1 / κ_avg. These calculated path lengths, estimated turning radii, and other quantitative indicators constitute the characteristics of the turning trajectory.

[0143] Steering dynamics parameters are calculated based on the steering event list and head direction data. For each event [t_start, t_end] in the steering event list, the head direction data α(t) and angular velocity data ω(t) for that time period are extracted. The maximum angular velocity ω_max = max(|ω(t)|) and the average angular velocity ω_avg = |Δα| / (t_end - t_start) during the event are calculated. The variation pattern of angular velocity over time is analyzed, for example, calculating the time required from the start of the steering to reach the maximum angular velocity (angular acceleration phase duration) and the time required from the maximum angular velocity to the end of the steering (angular deceleration phase duration). These parameters, such as ω_max, ω_avg, and angular acceleration duration, constitute the steering dynamics parameters.

[0144] A correlation analysis between body bending and turning is performed on the body bending feature data based on the turning event list. For each event [t_start, t_end] in the turning event list, the body bending feature data (e.g., the central axis curvature function κ(s,t) or the nodal angle sequence θᵢ(t)) within that time period is extracted. The relationship between the body bending pattern and the turning direction is analyzed. For example, for a left turn, the left side of the fish bends inward, producing a larger positive curvature (or a smaller nodal angle), while the right side bends outward, producing a negative curvature or a smaller positive curvature. The bending asymmetry is quantified: the maximum curvature κ_left_max on the left side of the fish's central axis (relative to the direction of travel) and the maximum curvature (absolute value) κ_right_max on the right side are calculated during the turning process. The bending asymmetry index I_bend_async = (κ_left_max - κ_right_max) / (κ_left_max + κ_right_max) (assuming κ_left_max > κ_right_max when turning left). The analysis focuses on the position of maximum bending along the body axis, s_bend_max, and its occurrence time, t_bend_max. The time difference between the occurrence time of maximum bending and the occurrence time of maximum angular velocity, Δt_bend_angular = t_bend_max - t_angular_max, is calculated, where t_angular_max is the time to reach ω_max. These asymmetry indices, maximum bending position, and time difference constitute the bending and steering correlation data.

[0145] Based on the list of turning events, the linear velocity of the fish's center point in the skeletal motion trajectory data is analyzed before and after turning. For each event [t_start, t_end] in the list of turning events, the data of the fish's center point in the linear velocity sequence V_linear(t) within the time period [t_start - T_pre, t_end + T_post] is extracted, where T_pre and T_post are the preset analysis time window durations before and after turning (e.g., T_pre = 1 second, T_post = 1 second). The average velocity V_pre_avg within the time window [t_start - T_pre, t_start] before turning is calculated. The minimum velocity V_during_min = min(V_linear(t)) during the turning process [t_start, t_end] is calculated. The average velocity V_post_avg within the time window [t_end, t_end + T_post] after turning is calculated. The average deceleration a_pre_avg = (V_start - V_pre_avg) / T_pre before turning is calculated, where V_start is the velocity at time t_start. Calculate the average acceleration after turning: a_post_avg = (V_post_avg - V_end) / T_post, where V_end is the velocity at time t_end. These velocity and acceleration indices constitute the turning speed characteristics.

[0146] Steering efficiency is evaluated based on steering trajectory characteristics (such as steering radius), steering dynamics parameters (such as mean angular velocity), and steering speed characteristics (such as speed loss V_pre_avg - V_during_min). A steering efficiency index, E_turn, is defined. For example, a simple efficiency index measures how many angles of steering are completed with a certain loss of linear speed: E_turn = |Δα| / (V_pre_avg - V_during_min + ε), where ε is a small positive number to avoid division by zero. More complex evaluations can consider bending energy consumption data (if estimated), for example, E_turn = |Δα|² / (estimated total bending energy consumption + estimated total linear motion energy consumption change). The estimated total linear motion energy consumption change is related to speed change, for example, proportional to (V_pre_avg² - V_post_avg²). These calculated efficiency values ​​constitute the steering efficiency data.

[0147] Steering behavior is classified into patterns based on steering efficiency data and bending steering correlation data (e.g., bending asymmetry, maximum bending location). For each steering event, the steering efficiency data, bending steering correlation data, and some steering trajectory features (e.g., steering radius) and steering dynamics parameters (e.g., maximum angular velocity) are combined into a feature vector. Using a clustering algorithm, such as K-means clustering, all detected steering events are clustered into different patterns of a predetermined number K (e.g., K=3). The clustering results represent different steering strategies, such as: "sharp turn" (small radius, high angular velocity, large asymmetric bending), "gentle turn" (large radius, low angular velocity, small asymmetric bending), and "stationary turn" (very small radius, near-zero velocity). Each steering event is assigned to its corresponding pattern category. These classification results constitute the steering pattern data.

[0148] Based on the steering pattern data and the steering behavior data obtained in the previous steps, a steering behavior feature set for the fish within the observation period is generated. The frequency of different steering patterns (e.g., "sharp turn," "gentle turn") is statistically analyzed (e.g., number of occurrences per minute). Statistics such as the average turning radius, average total angle change, average maximum angular velocity, and average steering efficiency are calculated for all steering events. The average values ​​of pre-turn deceleration and post-turn acceleration are calculated. The average and standard deviation of the bending asymmetry index are calculated. These statistics are then aggregated into a multidimensional vector or structured data set, which describes the typical steering behavior characteristics of the fish over a period of time, constituting the steering behavior feature set.

[0149] Preferably, step S4 includes the following steps:

[0150] Step S41: Standardize the fish morphological feature vector and the fish gait feature spectrum in the feature space to obtain standardized feature data;

[0151] Step S42: Perform multimodal feature fusion on the standardized feature data to obtain the fused feature representation;

[0152] Step S43: Perform individual identification matching on the fused feature representation to obtain an individual identification result set;

[0153] Step S44: Extract growth parameters from the individual identification result set to obtain a growth parameter analysis table;

[0154] Step S45: Perform behavioral pattern change analysis on the individual identification result set to obtain a behavioral change trend map;

[0155] Step S46: Detect health abnormalities in the behavioral change trend graph based on the growth parameter analysis table to obtain health abnormality marker data;

[0156] Step S47: Perform a health status assessment on the health anomaly marker data to obtain health status assessment data.

[0157] The fish morphological feature vector (e.g., including dimensions such as scale dot density, number of scars, number of fin forks, and average network side length) and the fish gait feature spectrum (e.g., a vector composed of indicators such as average swimming speed, standard deviation of tail fin wagging frequency, turning frequency, and average turning radius extracted from time-series data using statistical methods) are standardized in feature space. For the morphological feature vector... and gait feature vector , where k and l are the vector dimensions, and Z-score normalization is used. This method transforms each feature dimension into a distribution with a mean of 0 and a standard deviation of 1. For any feature dimension... Its standardized value ,in These are the original eigenvalues. It is the average value of this feature dimension across a large amount of historical fish data. This is the standard deviation of this feature dimension over a large amount of historical fish data. The historical data is calculated and stored. and The standardized morphological feature vector and standardized gait feature vectors These are combined to form standardized feature data, for example, simply concatenated into a vector V'_standardized=[V'_morph,V'_gait].

[0158] Multimodal feature fusion is performed on the standardized feature data V'_standardized. A weighted concatenation fusion method is used. The standardized morphological feature vector V'_morph and the standardized gait feature vector V'_gait are concatenated after being weighted according to their respective importance. Weights and These weights correspond to the importance of morphological feature dimension i and gait feature dimension j, respectively. These weights can be determined by training a recognition model (e.g., a support vector machine or neural network) on historical recognition tasks and analyzing the contribution of each feature, or by using feature selection algorithms (e.g., feature ranking based on mutual information). For example, if a morphological feature has higher discriminative power in recognition, its corresponding dimension will have a higher weight. The fused feature representation V_fused is a new vector with dimension k+l, where each component is the original standardized feature value multiplied by its corresponding weight. This fusion feature representation integrates the morphological and behavioral information of the fish, and the contribution of each part is adjusted to form the fusion feature representation.

[0159] Individual identification matching is performed on the fused feature representation V_fused. The fused feature representation of the current fish is compared with the fused feature representations of all known individuals stored in the historical fish feature database. The historical feature database stores the fused feature representations of each known individual collected at different time points. Cosine similarity is used as the similarity measurement function. The similarity between the current fish feature V_current and the feature V_history_k(t) of the historical individual k at time point t is S_cosine(V_current,V_history_k(t))=(V_current·V_history_k(t)) / (||V_current||×||V_history_k(t)||), where · represents the vector dot product and ||V|| represents the Euclidean norm of the vector. Calculate the similarity of the current fish body with the features of each individual in the historical database across all time points, and take the maximum similarity as the matching score with that historical individual. Set a matching threshold T_match (e.g., 0.85). If the highest matching score S_max is higher than T_match, and the difference between S_max and the second highest score S_second_max is greater than a preset margin ΔS (e.g., 0.05), then the current fish body is considered to have been successfully identified as the historical individual corresponding to the highest score. If the highest score is lower than T_match or S_max - S_second_max ≤ ΔS, then the current fish body is marked as a new individual or the identification fails. Record the identification result (matched historical individual ID or "NewID"), the highest matching score, and the identification timestamp for each processed fish body to form an individual identification result set.

[0160] For each identified historical individual in the individual identification result set, its growth parameters are extracted. Morphological feature vectors of this individual collected at different time points are retrieved from the historical fish feature database. Although morphological feature vectors are abstract representations, their construction process utilizes fine-grained feature region localization and spatial relationship construction of feature points (step S2). These data contain fish size information. For example, the distance between the head center point and the caudal peduncle point, and the vertical distance between the lines connecting the origin of the dorsal fin and the origin of the anal fin can be extracted from the morphological feature network structure. These distances are linearly correlated with fish length and height. A regression model is established between these distances and the actual measured body length and height using historical data. Using this regression model, the fish length L(t) and body height H(t) are estimated from the morphological feature vectors at each time point. The changes of these estimated values ​​over time are recorded to form an individualized growth curve. The average growth rate between adjacent collection time points is calculated: length growth rate. Body height growth rate The individual growth curve is compared with a pre-defined standard growth curve for the fish species (e.g., a parametric curve based on the Von Bertalanffy model) to calculate the degree of deviation of the individual from the standard curve. These estimated body length, body height, growth rate, and deviation data constitute a growth parameter analysis table.

[0161] For each identified historical individual in the individual identification result set, analyze changes in their behavioral patterns. Retrieve the individual's gait feature spectrum (or its statistical summary vector) from the historical fish feature database at different time points. The gait feature spectrum includes time-series or statistical indicators such as swimming speed, tail fin wagging frequency, amplitude, turning frequency, turning radius, and body bending coordination. Apply sliding window averaging to the time series of these key behavioral indicators, for example, using a sliding window of 10 minutes, and calculate the average value within the window to smooth short-term fluctuations. Use statistical model-based methods, such as the CUSUM algorithm, to detect significant changes in the mean or variance in the smoothed behavioral indicator series. For example, if the average swimming speed is significantly lower than its historical average for several consecutive windows, mark a behavioral change event. Record the time, the indicator of change, the direction of change (increase or decrease), and the magnitude of change for each detected change event. Visualize the smoothed time series of these key behavioral indicators and the detected change points into graphs to form behavioral change trend charts, such as plotting a line graph of average swimming speed over time, and marking the detected speed decrease points.

[0162] Based on the growth parameter analysis table and behavioral change trend chart, health anomaly detection is performed on each identified individual. A health indicator system is established, which includes indicators extracted from the growth parameter analysis table (e.g., current body length, body height, length growth rate over the past week, degree of deviation from the standard growth curve) and indicators extracted from the behavioral change trend chart (e.g., average swimming speed over the past day, caudal fin wagging frequency, turning frequency, presence of behavioral change events). Normal range thresholds are set for each health indicator. For example, a length growth rate below 50% of the average level of fish of the same age for this species is considered abnormal; an average swimming speed below 70% of its own historical average level is considered abnormal; frequent sharp turns are detected as abnormal. For each individual, its various health indicators are checked to see if they exceed the normal range or if there are significant changes in behavioral patterns (e.g., changes detected by S45). If an indicator exceeds a threshold or changes significantly, that indicator is marked as abnormal. If the number of indicators marked as abnormal for an individual exceeds a preset threshold (e.g., more than 2 key indicators are abnormal), the individual is marked as having a health anomaly. The ID of the abnormal individual, the time of detection of the anomaly, and the specific abnormal indicator and its value are recorded to form health anomaly marking data.

[0163] A health status assessment is performed on the health anomaly marker data. Based on the health anomaly marker data, individual growth parameter analysis tables, and behavioral change trend graphs, a comprehensive health status score is calculated for each individual. A weighted scoring function is designed: ,in It is a sub-score calculated based on the specific value or degree of abnormality of indicator i. This refers to the weight of the indicator, reflecting its importance to overall health status. For example, a significant deviation from the standard in length growth rate leads to a high negative score; a detected significant decline in behavioral patterns leads to a negative score; the presence of specific morphological damage (such as the number of scars extracted from the morphological feature vector of S2) also leads to a negative score. The weighted sum of the scores for each component yields a raw health score. This raw score is then mapped to a standardized range, e.g., 0 to 100, where 100 represents the optimal health status. Health status levels are then categorized based on the standardized scores, for example: Healthy (score ≥ 80), Sub-healthy (60 ≤ score < 80), AtRisk (40 ≤ score < 60), and Unhealthy (score < 40). Simultaneously, by comparing an individual's health scores at consecutive time points, it is determined whether their health status is improving, deteriorating, or remaining stable. The current health score, health status level, and health trend for each individual are recorded to form health status assessment data.

[0164] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0165] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A fish monitoring management method based on multi-source data analysis, characterized in that, Includes the following steps: Step S1: Acquire and enhance video images of the target fish group using a multi-angle imaging device to obtain an enhanced video sequence; Fish swimming postures were extracted from the enhanced video sequence to obtain the original dataset of fish features. Step S2: Classify the surface feature points of the original fish body feature dataset to obtain a classification feature point set; construct the fish body morphology feature vector based on the classification feature point set; Step S3: Based on the fish body morphology feature vector, locate and track the fish skeleton to obtain skeleton movement trajectory data; quantify the body curvature of the skeleton movement trajectory data to obtain body curvature feature data; analyze the tail fin swing of the skeleton movement trajectory data to obtain tail fin swing temporal features; extract turning behavior features from the skeleton movement trajectory data based on the body curvature feature data to obtain a turning behavior feature set. Fish swimming gait analysis was performed based on the timing characteristics of tail fin swing and the feature set of turning behavior to obtain the fish gait feature spectrum; Step S4: Perform multi-source data fusion and behavior change identification on the fish body morphology feature vector and fish body gait feature spectrum to obtain a behavior change trend map; perform health assessment on the behavior change trend map to obtain health status assessment data.

2. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: Construct an underwater imaging environment for the fish population in the aquaculture pond and obtain a standardized observation environment parameter table; Step S12: Based on the standardized observation environment parameter table, acquire the original multi-angle video sequence of the fish using a camera; Step S13: Perform image preprocessing and enhancement on the original multi-angle video sequence to obtain the enhanced video sequence; Step S14: Perform fish body segmentation and tracking on the enhanced video sequence to obtain fish body segmentation trajectory data; Step S15: Select and synthesize the best viewpoint from the enhanced video sequence based on the fish body segmentation trajectory data to obtain a multi-view image set; Step S16: Extract keyframes from the multi-view image set to obtain a set of key pose frames of the fish. Step S17: Perform fine feature region localization on the key pose frame set of the fish body to obtain fine image patches of the feature regions; Step S18: Perform data structuring processing on the fine image patches of the feature region and the key pose frame set of the fish body to obtain the original dataset of fish body features.

3. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that, Step S2 involves classifying the surface feature points of the original fish body feature dataset, including: The original dataset of fish body features was preprocessed with feature region enhancement to obtain a set of enhanced feature images. Coarse extraction of key points on the body surface is performed on the enhanced feature image group to obtain a candidate feature point set on the body surface. The candidate feature point set on the body surface is filtered and precisely located to obtain accurate feature point data; The precise feature point data is classified into feature point types to obtain a categorized feature point set.

4. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that, Step S2, which involves constructing a fish morphological feature vector based on the set of classification feature points, includes: The spatial relationships of the feature points in the classification feature point set are constructed to obtain the feature point spatial relationship graph; Based on the spatial relationship diagram of feature points and the set of classification feature points, a fish body morphology feature network is generated to obtain the fish body feature network structure. Temporal variation patterns were extracted from the fish body feature network structure to obtain feature temporal variation data. Based on the characteristic time change data and the fish body feature network structure, morphological feature vectors are constructed to obtain the fish body morphological feature vectors.

5. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that, Step S3, which involves locating and tracking the fish skeleton based on the fish's morphological feature vector, includes: Based on the fish body morphology feature vector, the skeletal key points of the original fish body feature dataset are located to obtain the fish body skeletal key point set. Initialize the bone markers by performing bone initialization marking based on the key point set of the fish skeleton. Based on the initialized bone markers, the enhanced video sequence is subjected to temporal tracking of the bone structure to obtain bone motion trajectory data.

6. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that, Step S3, which involves quantifying the body curvature of the skeletal motion trajectory data, includes: Extract the sequence of midline coordinate points from the head to the tail of the fish from the skeletal motion trajectory data, and perform midline discretization to obtain discrete node data of the midline. The node angles are calculated from the discrete node data of the central axis to obtain a sequence of node angles; Construct the central axis curvature function based on the node angle sequence and discrete node data of the central axis; Bending wave propagation analysis was performed based on the central axis curvature function to obtain bending wave propagation characteristic data; The fish body is divided into three main sections: front, middle and rear. The stiffness coefficients of the sections are estimated based on the central axis curvature function and the nodal angle sequence to obtain the stiffness distribution data of the sections. Bending energy consumption data were estimated based on the section stiffness distribution data and the node angle sequence. Muscle contraction intensity analysis was performed based on segment stiffness distribution data and central axis curvature function to obtain muscle contraction intensity distribution data. The bending coordination was evaluated using muscle contraction intensity distribution data and bending wave propagation characteristics data to obtain bending coordination indices. Body bending characteristic data are generated based on bending coordination index.

7. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that, Step S3 involves analyzing the tail fin oscillation of the skeletal motion trajectory data, including: The caudal fin key points were extracted from the skeletal motion trajectory data to obtain the local coordinate data of the caudal fin; The caudal fin offset is calculated from the local coordinate data of the caudal fin to obtain the original sequence of caudal fin offset; The original tail fin offset sequence was optimized by signal filtering to obtain the tail fin swing filtered sequence. The oscillation period characteristics of the tail fin oscillation filter sequence were analyzed to obtain oscillation period data; The amplitude and phase of the oscillation period data are extracted based on the tail fin oscillation filter sequence to obtain the oscillation amplitude and phase characteristics. The asymmetry of the oscillation is analyzed based on the amplitude and phase characteristics of the oscillation, and the asymmetry features of the oscillation are obtained. The caudal fin stiffness is evaluated based on the swing amplitude and phase characteristics and local coordinate data of the caudal fin, and the caudal fin stiffness index is obtained. The tail fin thrust data was estimated based on the tail fin stiffness index; Tail fin oscillation timing features are generated based on caudal fin propulsion data and oscillation asymmetry characteristics.

8. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that, Step S3 involves extracting turning behavior features from the skeletal motion trajectory data based on body bending feature data, including: Head orientation sequence is extracted from skeletal motion trajectory data to obtain head orientation data; Perform steering event detection on the head orientation data to obtain a list of steering events; Based on the list of turning events, the skeletal motion trajectory data is analyzed to obtain turning trajectory characteristics; Steering dynamics parameters are calculated based on the list of steering events and head direction data; Based on the list of steering events, a correlation analysis between body bending and steering is performed on the body bending feature data to obtain bending-steering correlation data. Based on the list of steering events, the speed before and after steering is analyzed from the skeletal motion trajectory data to obtain steering speed characteristics; Steering energy efficiency is evaluated by analyzing steering trajectory characteristics, steering dynamics parameters, and steering speed characteristics to obtain steering energy efficiency data. Steering behavior patterns are classified based on steering efficiency data and bending steering correlation data to obtain steering pattern data; A steering behavior feature set is generated based on steering pattern data.

9. The fish monitoring and management method based on multi-source data analysis according to claim 1, characterized in that, Step S4 includes the following steps: Step S41: Standardize the fish morphological feature vector and the fish gait feature spectrum in the feature space to obtain standardized feature data; Step S42: Perform multimodal feature fusion on the standardized feature data to obtain the fused feature representation; Step S43: Perform individual identification matching on the fused feature representation to obtain an individual identification result set; Step S44: Extract growth parameters from the individual identification result set to obtain a growth parameter analysis table; Step S45: Perform behavioral pattern change analysis on the individual identification result set to obtain a behavioral change trend map; Step S46: Detect health abnormalities in the behavioral change trend graph based on the growth parameter analysis table to obtain health abnormality marker data; Step S47: Perform a health status assessment on the health anomaly marker data to obtain health status assessment data.

10. A fish monitoring and management system based on multi-source data analysis, characterized in that, For executing the fish monitoring and management method based on multi-source data analysis as described in claim 1, the fish monitoring and management system based on multi-source data analysis includes: The multi-angle data acquisition module is used to acquire and enhance video images of the target fish group through multi-angle imaging equipment to obtain an enhanced video sequence; the fish swimming posture is extracted from the enhanced video sequence to obtain the original dataset of fish features. The morphological feature extraction module is used to classify the surface feature points of the original fish body feature dataset to obtain a set of classification feature points; and to construct a fish body morphological feature vector based on the set of classification feature points. The swimming behavior analysis module is used to locate and track the fish skeleton based on the fish's body morphology feature vectors to obtain skeletal movement trajectory data; to quantify the body curvature of the skeletal movement trajectory data to obtain body curvature feature data; to analyze the tail fin swing of the skeletal movement trajectory data to obtain tail fin swing temporal features; to extract turning behavior features from the skeletal movement trajectory data based on the body curvature feature data to obtain a turning behavior feature set; and to analyze the fish's swimming gait based on the tail fin swing temporal features and the turning behavior feature set to obtain the fish's gait feature spectrum. The health status assessment module is used to perform multi-source data fusion and behavior change identification on fish morphological feature vectors and fish gait feature spectra to obtain a behavior change trend map; and to perform health assessment on the behavior change trend map to obtain health status assessment data.

Images