A fish feeding behavior recognition method and system

Abstract

The application discloses a fish feeding behavior recognition method and system, and particularly relates to the technical field of intelligent monitoring of aquaculture, and is used for solving the problems of fuzzy feature extraction and low behavior judgment accuracy caused by the complexity of underwater environment and the instantaneousness of feeding action of the existing visual recognition method; through obtaining underwater continuous video frames, fish head movement continuity parameters are extracted and candidate key frames are screened; nonlinear dynamics characteristics of mouth deformation displacement are analyzed to generate time sequence correlation parameters, and the coupling strength of head and mouth movement is dynamically weighted and fused; finally, the cumulative effect and distribution mode statistics of the motion coupling strength are used to determine the feeding behavior state; through the multi-dimensional time sequence feature fusion and dynamic weight distribution mechanism, the application realizes accurate capture and reliable discrimination of the transient feeding action of fish in a complex underwater scene, and provides effective technical support for optimizing the feeding strategy of aquaculture.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent monitoring of aquaculture, more particularly, the present application relates to a fish feeding behavior recognition method and system. BACKGROUND

[0002] In the field of aquaculture and ecological monitoring, fish behavior recognition technology based on computer vision can optimize feeding strategies and assess fish health status. Existing technologies usually analyze visual features such as fish movement trajectories and body posture changes to identify feeding behavior. Related methods rely on general algorithms such as target detection and action segmentation in video data to extract behavior patterns. However, the complexity of underwater environments and the inherent characteristics of fish feeding motions pose significant limitations to existing methods in practical applications.

[0003] The recognition accuracy of fish feeding behavior in existing technologies is limited by the weak saliency of temporal features. Since fish feeding motions are short in duration and highly similar to regular swimming and breathing behaviors in visual representation, existing methods struggle to effectively distinguish the temporal key features of feeding behavior from consecutive video frames, leading to ambiguity in behavior classification results and directly affecting the accuracy and reliability of feeding state identification. SUMMARY

[0004] To overcome the above-mentioned defects of the prior art, embodiments of the present application provide a fish feeding behavior recognition method and system to solve the problems raised in the background art.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0006] A fish feeding behavior recognition method, comprising the following steps:

[0007] S1. Obtain underwater continuous video frames containing fish, and extract the motion continuity parameters of the head region of the fish body in each frame of image;

[0008] S2. Extract the curvature mutation features of the head movement trajectory based on the motion continuity parameters and analyze the direction aggregation degree of the head region motion vector. Filter the video frames that simultaneously have curvature mutation features and direction aggregation degree exceeding the direction aggregation degree threshold as the candidate key frame sequence;

[0009] S3. Perform local deformation motion analysis on the mouth opening and closing region of the fish body in the candidate key frame sequence, and generate the deformation displacement amount of the mouth region between adjacent key frames;

[0010] S4. Perform phase space reconstruction and fractal dimension analysis on the deformation displacement amount, extract the nonlinear dynamics features of the mouth motion, and generate the temporal correlation parameters of the mouth motion;

[0011] S5, dynamically weighting the motion coupling strength between the head region and the mouth region of the fish body based on the distribution characteristics of the time sequence correlation parameters on the time axis;

[0012] S6, determining the occurrence state of the fish feeding behavior according to the cumulative effect and distribution pattern of the motion coupling strength within a preset time window.

[0013] In a preferred embodiment, underwater continuous video frames containing fish are acquired, and motion continuity parameters of the head region of the fish body in each frame image are extracted, including:

[0014] Image preprocessing operations are performed on the underwater continuous video frames to suppress noise and equalize illumination, obtaining a sequence of preprocessed video frames;

[0015] The head region of the fish body in the sequence of preprocessed video frames is located based on a target detection algorithm, and the boundary integrity of the head region is optimized through morphological closing operation;

[0016] The optical flow field of the head region of the fish body between adjacent frames is calculated, and a set of motion vectors of the pixel points in the optical flow field is extracted;

[0017] The motion continuity parameters of the head region of the fish body are calculated according to the set of motion vectors, including the average motion speed, the motion direction angle variance, and the motion trajectory curvature change rate between adjacent frames.

[0018] In a preferred embodiment, the curvature mutation features of the head motion trajectory of the fish body are extracted based on the motion continuity parameters, and the direction aggregation degree of the head region motion vector is analyzed, and the video frames that simultaneously exist curvature mutation features and have a direction aggregation degree exceeding a direction aggregation degree threshold are selected as a candidate key frame sequence, including:

[0019] The trajectory curve is generated by cubic spline interpolation method for the head motion trajectory of the fish body, the trajectory curvature change rate is calculated, and the video frames with a curvature change rate exceeding a curvature change rate threshold are marked as candidate curvature mutation frames;

[0020] The covariance matrix is constructed for the set of head region motion vectors, and the ratio of the maximum eigenvalue to the minimum eigenvalue of the covariance matrix is calculated as a direction aggregation degree index;

[0021] The video frames with a direction aggregation degree index exceeding a direction aggregation degree threshold in the candidate curvature mutation frames are merged into a candidate key frame sequence.

[0022] In a preferred embodiment, the curvature change rate threshold is dynamically adjusted according to the average value and standard deviation of the curvature change rate statistically calculated from historical data, and the direction aggregation degree threshold is set based on the direction consistency empirical value of different fish species feeding actions.

[0023] In a preferred embodiment, local morphing motion analysis is performed on the mouth opening and closing region of the fish body in the candidate key frame sequence to generate the morphing displacement of the mouth region between adjacent key frames, including:

[0024] Key points of the mouth opening and closing region of the fish body are located in the candidate key frame sequence, and the key points are adaptively selected based on anatomical features of the mouth of the fish body;

[0025] An optical flow tracking algorithm is used to calculate the motion vector of the key points between adjacent key frames;

[0026] The motion vector is separated into rigid and non-rigid morphing, and the displacement vector corresponding to the non-rigid morphing is extracted;

[0027] The displacement of the mouth region is calculated based on the amplitude and direction distribution of the displacement vector.

[0028] In a preferred embodiment, the iteration termination condition of the optical flow tracking algorithm is that the change amount of the motion vector between adjacent iterations is less than a preset precision threshold, and the separation of rigid and non-rigid morphing is achieved by motion vector field decomposition.

[0029] In a preferred embodiment, the morphing displacement is reconstructed in phase space and fractal dimension analysis is performed to extract the nonlinear dynamics characteristics of the mouth movement, and the time sequence correlation parameters of the mouth movement are generated, including:

[0030] The morphing displacement is reconstructed in phase space to construct a trajectory dynamics model, and the embedding dimension and time delay parameter of the phase space reconstruction are determined by the pseudo-nearest neighbor method and the autocorrelation function;

[0031] Fractal dimension analysis is performed on the reconstructed phase space trajectory, and the box counting method is used to calculate the fractal dimension of the trajectory;

[0032] The nonlinear dynamics characteristics of the mouth movement are extracted based on the fractal dimension and the maximum Lyapunov exponent of the phase space trajectory;

[0033] The nonlinear dynamics characteristics and the time domain statistics of the morphing displacement are weighted and fused to generate the time sequence correlation parameters of the mouth movement.

[0034] In a preferred embodiment, based on the distribution characteristics of the time sequence correlation parameters on the time axis, the motion coupling strength of the head region and the mouth region of the fish body is dynamically weighted and generated, including:

[0035] The time sequence correlation parameters are divided into equal length windows according to the time axis, and the mean, variance and autocorrelation coefficient of the time sequence correlation parameters in each window are calculated;

[0036] The window dynamic weight coefficient is generated based on the ratio of the autocorrelation coefficient to the variance;

[0037] Performing sliding window covariance analysis on the motion parameters of the head region and the mouth region of the fish body, calculating the principal eigenvalue of the covariance matrix;

[0038] Fusing the principal eigenvalue and the window dynamic weight coefficient according to the exponential weighted moving average algorithm to generate the motion coupling strength;

[0039] The decay factor of the exponential weighted moving average algorithm is dynamically adjusted according to the autocorrelation coefficient of the time series correlation parameter, and the window size of the covariance analysis is aligned with the equal-length window.

[0040] In a preferred embodiment, according to the cumulative effect and distribution pattern of the motion coupling strength within a preset time window, the occurrence state of the fish feeding behavior is determined, including:

[0041] Cumulative effect calculation is performed on the motion coupling strength within a preset time window, and the cumulative effect is obtained by sliding window integration, and the integration window size is aligned with the preset time window;

[0042] Statistical feature extraction is performed on the distribution pattern of the motion coupling strength within the window, including skewness, kurtosis and interquartile range;

[0043] A decision vector is constructed based on the combination of the cumulative effect and the statistical features, and a support vector machine classifier is used to classify the decision vector into a feeding behavior state;

[0044] The kernel function type and the penalty factor of the support vector machine classifier are optimized according to the cross-validation of historical feeding behavior data, and the classification threshold is dynamically set based on the distribution overlap degree of the decision vectors of the feeding behavior and the non-feeding behavior.

[0045] On the other hand, the present application provides a fish feeding behavior recognition system, comprising the following modules:

[0046] Video acquisition module: acquiring underwater continuous video frames containing fish, and extracting motion continuity parameters of the head region of the fish body in each frame image;

[0047] Curvature screening module: based on the motion continuity parameters, extracting the curvature mutation feature of the head motion trajectory of the fish body and analyzing the direction aggregation degree of the head region motion vector, screening the video frames with both curvature mutation feature and direction aggregation degree exceeding the direction aggregation degree threshold as the candidate key frame sequence;

[0048] Deformation analysis module: performing local deformation motion analysis on the mouth opening and closing region of the fish body in the candidate key frame sequence, and generating the deformation displacement amount of the mouth region between adjacent key frames;

[0049] Nonlinear analysis module: performing phase space reconstruction and fractal dimension analysis on the deformation displacement amount, extracting the nonlinear dynamics feature of the mouth action, and generating the time series correlation parameter of the mouth action;

[0050] The dynamic weighting module: based on the distribution characteristics of the time sequence correlation parameters on the time axis, dynamically weights the motion coupling strength of the fish head region and the mouth region;

[0051] The state determination module: according to the cumulative effect and distribution mode of the motion coupling strength in the preset time window, determines the occurrence state of the fish feeding behavior.

[0052] Compared with the prior art, the present application has the following beneficial effects:

[0053] The present application effectively enhances the time sequence feature significance of the fish feeding behavior through multi-stage feature collaborative analysis. First, the candidate key frame is screened based on the curvature mutation and direction aggregation degree of the head movement trajectory, solving the weak distinguishability problem of the motion feature caused by underwater environmental interference, accurately positioning the key time sequence segment of the feeding action. Second, combined with the nonlinear dynamics feature extraction of the mouth deformation displacement and the head-mouth motion coupling strength analysis, the limitation of single motion parameter in representing complex feeding action is broken through, and the discrimination ability of the behavior mode is enhanced through multi-dimensional feature fusion;

[0054] The technical scheme of the present application significantly improves the robustness of the feeding behavior recognition through the dynamic weighting mechanism and the statistical distribution mode criterion. On the one hand, the adaptive weight distribution based on the time sequence correlation parameters fully considers the dynamic change of the feature contribution degree of different feeding stages, avoiding the misjudgment caused by fixed threshold. On the other hand, through the joint analysis of the cumulative effect and the distribution mode, the local transient feature and the global statistical law are combined, effectively suppressing the influence of underwater noise on single-frame detection results, ensuring the continuity and reliability of the feeding behavior state determination. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The flowchart of the fish feeding behavior recognition method of the present application;

[0056] Figure 2 The structural schematic diagram of the fish feeding behavior recognition system of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0058] Embodiment 1: Figure 1 A fish feeding behavior recognition method is given, which comprises the following steps:

[0059] S1, acquire underwater continuous video frames containing fish, extract the motion continuity parameters of the head region of the fish in each frame image;

[0060] S2, extract the curvature mutation features of the fish head motion trajectory based on the motion continuity parameters, and analyze the direction aggregation degree of the head region motion vector, screen the video frames with both curvature mutation features and direction aggregation degree exceeding the direction aggregation degree threshold as the candidate key frame sequence;

[0061] S3, perform local deformation motion analysis on the mouth opening and closing region of the fish in the candidate key frame sequence, and generate the deformation displacement amount of the mouth region between adjacent key frames;

[0062] S4, perform phase space reconstruction and fractal dimension analysis on the deformation displacement amount, extract the nonlinear dynamics features of the mouth action, and generate the time sequence correlation parameters of the mouth action;

[0063] S5, based on the distribution features of the time sequence correlation parameters on the time axis, dynamically generate the motion coupling strength of the head region and the mouth region of the fish;

[0064] S6, according to the cumulative effect and distribution mode of the motion coupling strength in the preset time window, determine the occurrence state of the fish feeding behavior.

[0065] S1, acquire underwater continuous video frames containing fish, extract the motion continuity parameters of the head region of the fish in each frame image, and the specific implementation is:

[0066] After acquiring the underwater continuous video frames containing fish, first, image preprocessing operation is performed on the video frames. The image preprocessing operation includes noise suppression and illumination equalization processing, wherein the noise suppression is realized by Gaussian filtering, the size of the convolution kernel of Gaussian filtering is dynamically adjusted according to the density of suspended particles in the underwater image, and the density of suspended particles is determined by calculating the energy value of the high-frequency noise component of the image, specifically, the energy proportion of the high-frequency band is calculated after Fourier transform of the image, and when the energy proportion exceeds 60%, the convolution kernel size is selected as 5x5 pixels, otherwise 3x3 pixels; the illumination equalization processing adopts the limited contrast adaptive histogram equalization method, the image is divided into 8x8 pixel sub-blocks, each sub-block is independently processed by histogram equalization, and the contrast limiting parameter is used to prevent local over-enhancement, and the contrast limiting parameter is set to 1.2 to 1.5 times of the standard deviation of the gray value of the sub-block.

[0067] Based on the pre-processed video frame sequence, a target detection algorithm is used to locate the fish head region. The target detection algorithm is a convolutional neural network based on the Faster R-CNN framework, and the training data set contains head images of different fish species, covering front, side and overhead views, and contains annotated data of underwater light attenuation and bubble interference scenes; the model inference outputs the boundary box coordinates of the head region, and the boundary box confidence threshold is set to 0.7, and the detection result below the threshold is considered invalid; the morphological closing operation is performed on the binary mask in the boundary box to optimize the morphological closing operation, and the structure element is circular, the ratio of the diameter of the structure element to the average width of the head region is 1:5 to 1:3, and the boundary gap caused by motion blur or occlusion is filled by the inflation and then corrosion operation to ensure the closure of the head region contour.

[0068] The optical flow field of the fish head region between adjacent frames is calculated, the optical flow field is calculated by the Lucas-Kanade algorithm, the pixel neighborhood window size in the algorithm is set to 15x15 pixels, the maximum iteration number is 20 times, and the iteration termination condition is that the optical flow vector change of adjacent two iterations is less than 0.01 pixel; when extracting the motion vector set of the pixel points in the optical flow field, the motion vector of the background region is filtered, the background region is defined as a ring-shaped region with an extension of 10 to 20 pixels outside the head region boundary box, and the motion vector amplitude threshold is set to 1.5 times the average amplitude of the motion vector of the background region. The motion vector exceeding the threshold is considered as valid head motion data.

[0069] The motion continuity parameter of the fish head region is calculated according to the motion vector set. The average motion speed between adjacent frames is obtained by calculating the arithmetic mean of the lengths of all valid motion vectors, and the length unit is pixel / frame; the motion direction angle variance is calculated by the following steps: the direction angle of all valid motion vectors is counted, the direction angle is expressed in radians and the range is 0 to 2π, the circular variance of the direction angle is calculated, and the circular variance formula is 1 minus the average length of all direction angle unit vectors; the motion trajectory curvature change rate is calculated by fitting the motion trajectory curve of the head region centroid in consecutive frames, the centroid coordinates are determined by the geometric center of the head region mask, the trajectory curve is generated by the cubic spline interpolation method, the interpolation node interval matches the video frame rate, the video frame rate is 30 frames per second, the node interval is 0.033 seconds, the curvature change rate is the ratio of the curvature difference between adjacent nodes to the time interval, and the curvature unit is 1 / pixel.

[0070] For extreme scenarios, such as underwater light illumination changes dramatically resulting in histogram equalization failure, enable the backup light compensation strategy: convert the preprocessed video frame to HSV color space, segment the linear stretch of the brightness component V, and adjust the stretch interval according to the 5% to 95% quantile of the current frame brightness histogram; when the target detection algorithm cannot locate the head region for three consecutive frames, trigger the motion vector backtracking mechanism, predict the head region position of the current frame according to the motion vector set of the previous five frames, and the prediction model is a Kalman filter based on linear regression, the process noise covariance is set to 0.1, and the observation noise covariance is set to 0.01.

[0071] Hardware dependencies include at least a GPU with 4GB of video memory for real-time computation of optical flow field, and software dependencies include image processing functions in OpenCV4.5.4 library and target detection model inference interface under PyTorch 1.9.0 framework.

[0072] S2, based on the motion continuity parameter, extract the curvature mutation feature of the fish head motion trajectory and analyze the direction aggregation degree of the head region motion vector, select the video frames with both curvature mutation feature and direction aggregation degree exceeding the direction aggregation degree threshold as the candidate key frame sequence, and the specific implementation is:

[0073] When extracting the curvature mutation feature of the fish head motion trajectory based on the motion continuity parameter, first generate the trajectory curve by cubic spline interpolation method. The node interval of cubic spline interpolation method strictly matches the video frame rate, and when the video frame rate is 30 frames per second, the node interval is set to 0.033 seconds, ensuring that each video frame corresponds to an interpolation node; the interpolation curve satisfies the second derivative continuity condition between adjacent nodes, so that the smoothness of trajectory curvature calculation is not affected by frame skipping; the trajectory curvature change rate is obtained by calculating the difference between the curvatures of adjacent nodes divided by the time interval, and the curvature unit is 1 / pixel.

[0074] The curvature calculation is realized by deriving the parametric equation, specifically, the horizontal direction parameter equation x(u) and the vertical direction parameter equation y(u) of the trajectory curve are derived respectively, where u is the normalized arc length parameter of the trajectory curve. The formula of curvature K is:

[0075]

[0076] Where x(u) represents the parameter equation of the trajectory curve in the horizontal direction, u∈[0,1] is the normalized arc length parameter; y(u) represents the parameter equation of the trajectory curve in the vertical direction, u shares the same parameter as x(u); The first derivative of the parameter u represents the instantaneous change rate in the horizontal direction; The first derivative of the parameter u for y(u) represents the instantaneous change rate in the vertical direction; The second derivative of the parameter u, representing the instantaneous acceleration of the horizontal direction change rate; The second derivative of the parameter u for y(u), representing the instantaneous acceleration of the vertical direction change rate; the numerator part The curvature numerator term, quantifying the geometric strength of the trajectory turning; the denominator part The curvature denominator term, a normalization term, eliminating the influence of the parameterized velocity on the curvature.

[0077] The curvature change rate threshold is dynamically adjusted according to the average value and standard deviation of the curvature change rate statistically calculated from historical data, specifically, the curvature change rate threshold is equal to the average value plus twice the standard deviation, when the real-time calculated curvature change rate exceeds the threshold, the corresponding video frame is marked as a candidate curvature mutation frame. For example, the historical data analysis of a certain type of carnivorous fish feeding action shows that its average value of curvature change rate is 0.15 1 / pixel·second, and the standard deviation is 0.05, then the threshold is 0.15+2×0.05=0.25 1 / pixel·second, when the real-time curvature change rate exceeds 0.25, it is determined as a candidate frame of feeding action.

[0078] When constructing the covariance matrix for the head region motion vector set, the motion vector set is derived from the effective motion vector data extracted in step S1, and the effective motion vector is filtered by the amplitude threshold to retain the motion data within the head region.

[0079] The dimension of the covariance matrix is 2x2, and the matrix elements are derived from the horizontal component h k and the vertical component v k of the motion vector, where k=1,2,…,N, N is the total number of motion vectors. The formula of the covariance matrix C is:

[0080]

[0081] Wherein each element is defined as follows:

[0082] The horizontal direction variance σ hh :

[0083]

[0084] The vertical direction variance σ vv :

[0085]

[0086] The horizontal-vertical covariance σ hv (symmetry ensures σ hv =σ vh ):

[0087]

[0088] wherein, h k denotes the displacement component of the kth motion vector in the horizontal direction (unit: pixel), which is calculated by the optical flow method; v k denotes the displacement component of the kth motion vector in the vertical direction (unit: pixel), which is calculated by the optical flow method; N denotes the total number of motion vectors, corresponding to the number of effective motion vectors in the candidate key frame sequence screened in step S2; σ hh denotes the variance of the horizontal displacement component, representing the fluctuation intensity of head movement in the horizontal direction; σ vv denotes the variance of the vertical displacement component, representing the fluctuation intensity of head movement in the vertical direction; σ hv denotes the covariance of the horizontal and vertical displacement components, quantifying the correlation of movement in the two directions; denotes the normalization factor in mean calculation, ensuring that the result is independent of the number of motion vectors.

[0089] The ratio of the maximum eigenvalue to the minimum eigenvalue of the covariance matrix is calculated as the direction aggregation degree index. The larger the ratio, the higher the consistency of the motion vector direction. The direction aggregation degree threshold is set based on the consistency of the feeding action direction of different fish species. For example, the direction aggregation degree threshold for carnivorous fish is set to 5.0, and for filter-feeding fish, it is set to 3.0. The threshold is set based on statistical analysis of the direction aggregation degree index in the feeding stage and non-feeding stage of historical video data. When the direction aggregation degree index exceeds the set threshold, it is determined that the frame direction aggregation degree meets the requirements. For example, the head motion vector direction aggregation degree index of a certain carnivorous fish during a sudden attack feeding can reach 6.0 or more, while the index is less than 3.0 during regular swimming. By setting the threshold to 5.0, the feeding and swimming states can be effectively distinguished.

[0090] When merging the video frames with direction aggregation degree index exceeding the direction aggregation degree threshold in the candidate curvature mutation frame into the candidate key frame sequence, a time window sliding matching mechanism is adopted. The window size is set to 5 frames, and the window sliding step is 1 frame. When there are at least 3 frames with direction aggregation degree index exceeding the threshold in the window, all candidate curvature mutation frames in the window are included in the candidate key frame sequence. If the number of frames exceeding the threshold in the window is insufficient, all candidate curvature mutation frames in the window are discarded. For example, when the feeding action lasts for 10 frames and the direction aggregation degree index of the 3rd to 7th frames all exceed the threshold, the window sliding covers the generated candidate key frame sequence containing the 1st to 9th frames, ensuring the integrity of the starting and ending stages of the feeding action. For scenes with feeding action duration shorter than the window size, such as a 3-frame fast feeding, if all 3 frames in the window exceed the threshold, all 3 frames are retained, otherwise they are discarded to avoid false detection.

[0091] For extreme scenarios, such as insufficient node data for trajectory interpolation resulting in failure of cubic spline interpolation, linear interpolation method is enabled as a backup solution, the linear interpolation node interval is expanded to 0.1 seconds, and after the interpolation is completed, the curvature rate of change is smoothed and filtered, the filter window size is 3 frames, and the filtering method is the sliding average algorithm; When the motion vector set is empty, the covariance matrix cannot be constructed, skip the current frame and use the direction aggregation index of the previous frame to interpolate and compensate, the interpolation and compensation logic is to predict the current frame index according to the weighted average value of the direction aggregation index of the previous five frames, the weight coefficient is set according to the time decay model, the weight of the nearest frame is 0.5, and the weight of each previous frame decreases by 0.1. For example, when the current frame is missing, the weighted average value of the direction aggregation index of the previous five frames 0.5, 0.4, 0.3, 0.2, 0.1 is 0.5x0.5+0.4x0.4+0.3x0.3+0.2x0.2+0.1x0.1=0.55, which is used as the interpolation result of the current frame.

[0092] Hardware dependencies include GPUs that support floating-point operation acceleration to process covariance matrix eigenvalue decomposition in real time, and GPU memory needs at least 6GB to support parallel computing of high-resolution video streams; Software dependencies include linear algebra computing interfaces in NumPy 1.21.0 library and interpolation functions in SciPy 1.7.0 library, wherein the interpolation function of SciPy needs to be configured to use the default smoothing parameter (smoothing factor=0.5) of cubic spline interpolation method.

[0093] S3, local deformation motion analysis is performed on the mouth opening and closing area of the fish body in the candidate key frame sequence to generate the deformation displacement amount of the mouth area between adjacent key frames, which is specifically implemented as:

[0094] In the local deformation motion analysis of the fish mouth opening and closing area in the candidate key frame sequence, first, the key points of the fish mouth opening and closing area in the candidate key frame sequence are located. The key points are adaptively selected based on the anatomical features of the fish mouth, including the corner of the mouth, the lip edge and the gill cover connection point. Specifically, the mouth contour is extracted by an image edge detection algorithm, and the key point position is determined by matching the mouth feature template in the fish dissection database. The image edge detection algorithm uses Canny operator, and the threshold is set to the first 30% to 50% of the gradient amplitude, to ensure the continuity of the contour extraction; the mouth feature template in the fish dissection database is constructed by labeled samples in historical video data, and the normalized cross-correlation algorithm is used for template matching, and the matching threshold is set to 0.75, and the matching result below the threshold is regarded as invalid. For example, for the sharp mouth of a bass, the corner of the mouth is located at the intersection of the upper and lower jaws, the lip edge is distributed along the lip with an interval of 10 to 15 pixels, and the gill cover connection point is located at the geometric center of the joint between the gill cover and the head. In the key point positioning process, if the image is blurred and the edge detection fails, the key point position of the adjacent frame is interpolated to compensate, the interpolation method is linear extrapolation, the extrapolation step is not more than 3 frames, and when the step limit is exceeded, the key point data of the current frame is marked as missing and the abnormal processing mechanism is triggered.

[0095] The optical flow tracking algorithm is used to calculate the motion vector of the key points between adjacent key frames. The optical flow tracking algorithm is Lucas-Kanade algorithm, and the algorithm parameters include pixel neighborhood window size and iteration termination condition. The pixel neighborhood window size is set according to the resolution of the mouth area, and the resolution is 1920x1080 pixels, and the window size is 15x15 pixels, and the resolution is 1280x720 pixels, and the window size is 10x10 pixels; the iteration termination condition is that the change of the motion vector between the adjacent two iterations is less than the preset accuracy threshold, and the preset accuracy threshold is set according to the convergence characteristics of the motion vector in the historical data, specifically, the average error when the optical flow tracking converges in the historical data is counted, and its double value is taken as the threshold, for example, when the historical average error is 0.01 pixel, the preset accuracy threshold is 0.02 pixel; in the optical flow tracking process, if the key point exceeds the image boundary or the occlusion rate exceeds 50%, the point is marked as invalid and interpolation compensation is triggered, and the interpolation compensation logic is to predict the motion vector of the current frame according to the weighted average value of the previous five valid motion vectors, and the weight coefficient is attenuated according to time, and the weight of the nearest frame is 0.5, and the weight decreases by 0.1 for each previous frame.

[0096] The motion vector is separated into rigid and non-rigid deformation, and the displacement vector corresponding to the non-rigid deformation is extracted. The rigid and non-rigid deformation separation is realized by motion vector field decomposition. Specifically, the motion vector field is decomposed by principal component analysis, and the first two principal components are taken as the rigid motion component, and the remaining components are taken as the non-rigid deformation component. The rigid motion component describes the overall translation and rotation of the fish head, and the non-rigid deformation component focuses on the opening and closing action of the mouth. In the principal component analysis, the principal component contribution rate threshold is set to 85%. If the cumulative contribution rate of the first two principal components is lower than the threshold, the neighborhood window size is expanded to 20x20 pixels and recalculated until the contribution rate requirement is met. For example, when the local deformation is complex due to the rapid opening and closing of the mouth, expanding the window size can improve the stability of the motion vector field decomposition.

[0097] The deformation displacement of the mouth region is calculated based on the amplitude and direction distribution of the displacement vector. The amplitude of the displacement vector is the arithmetic mean of the modulus length of the non-rigid deformation component, and the modulus length unit is pixel; the direction distribution is quantified by the circular variance of the direction angle of the displacement vector, and the circular variance calculation formula is 1 minus the average modulus length of all direction angle unit vectors, the direction angle is expressed in radians and the range is 0 to 2π; the deformation displacement is the weighted sum of the amplitude and the direction variance, and the weight coefficient is set according to the feeding action characteristics of fish species, and the weight coefficient setting rule is: the feeding action of carnivorous fish is mainly rapid attack, the amplitude weight is set to 0.7, and the direction variance weight is 0.3; the feeding action of filter-feeding fish is mainly continuous filtering, the amplitude weight is set to 0.5, and the direction variance weight is 0.5; if the displacement vector set is empty, the deformation displacement of the previous frame is used and marked as an estimated value, and the effective period of the estimated value does not exceed 5 frames, and the key points are forcibly reinitialized after the effective period is exceeded.

[0098] For extreme scenarios, such as when the key point positioning failure rate exceeds 70%, switch to a backup key point detection model based on deep learning. The model is a pre-trained ResNet-18 network, the input is a mouth region image block with a size of 64x64 pixels, the output is a key point heat map, and the peak point of the heat map is taken as the key point position. When the optical flow tracking fails continuously for more than 5 frames, enable the historical motion vector prediction mechanism, the prediction model is a sliding average algorithm, the window size is 5 frames, the prediction value is the weighted average of the motion vectors in the window, and the weight coefficient decays over time, with the weight of the most recent frame being 0.5 and decreasing by 0.1 for each previous frame. For example, when the optical flow tracking fails for 5 consecutive frames, the prediction value of the sixth frame is the fifth frame motion vector x 0.5 + the fourth frame x 0.4 + the third frame x 0.3 + the second frame x 0.2 + the first frame x 0.1, and the weighted average is taken as the interpolation result of the current frame.

[0099] The hardware dependence includes at least a GPU with 8GB display memory to support real-time calculation of optical flow tracking, and the GPU model needs to support parallel computing of CUDA architecture; the software dependence includes the optical flow calculation interface in OpenCV4.5.4 library, the principal component analysis function in NumPy1.21.0 library and the deep learning model inference interface under PyTorch1.9.0 framework.

[0100] S4, phase space reconstruction and fractal dimension analysis are performed on the deformation displacement amount, nonlinear dynamics characteristics of the mouth movement are extracted, and time sequence correlation parameters of the mouth movement are generated, and the specific implementation is as follows:

[0101] When the phase space reconstruction is performed on the deformation displacement amount to construct the trajectory dynamics model, the embedding dimension and the time delay parameter of the phase space reconstruction are first determined. The embedding dimension is determined by the false nearest neighbor method, and the specific implementation steps of the false nearest neighbor method include: starting from the initial dimension 1, gradually increasing the embedding dimension, increasing by 1 each time, calculating the proportion of false nearest neighbor points under the current dimension, the false nearest neighbor points are defined as points that are adjacent in high-dimensional space but not adjacent in low-dimensional projection; stop increasing when the false nearest neighbor proportion is lower than 5%, and select the minimum embedding dimension at this time as the optimal embedding dimension. The time delay parameter is determined by the autocorrelation function, and the calculation method of the autocorrelation function is to calculate the correlation coefficient between the time sequence of the deformation displacement amount and its delayed version, and select the delay time corresponding to the first time when the autocorrelation function decreases to 37% of the initial value as the time delay parameter. For example, the time sequence of the deformation displacement amount of a certain fish feeding action contains 1000 frames of data, the autocorrelation function first decreases to 37% of the initial value at a delay of 10 frames, so the time delay parameter is set to 10 frames; the false nearest neighbor method analysis shows that the false nearest neighbor proportion decreases to 4% when the embedding dimension is 3, so the embedding dimension is set to 3. After the phase space reconstruction is completed, the trajectory dynamics model is represented by a three-dimensional vector sequence, each vector contains the displacement amount at the current time and the delayed time, the displacement amount unit is pixel, and the time delay parameter unit is frame.

[0102] When analyzing the fractal dimension of the reconstructed phase space trajectory, the box-counting method is used to calculate the trajectory fractal dimension. The specific steps of the box-counting method include: dividing the phase space into multiple cubic boxes, starting with one-tenth of the maximum range of the trajectory space, gradually reducing to 1%, and each time reducing the step size by half of the previous box size; under each box size, count the minimum number of boxes required to cover the trajectory, and record the corresponding relationship between the logarithm of the box size and the logarithm of the number of boxes; through linear regression fitting of the slope of the double logarithmic data points, the correlation coefficient is required to be not less than 0.95, if the correlation coefficient is lower than the threshold, then reselect the box size range and repeat the calculation, and finally the fitting slope is taken as the fractal dimension. For example, when the maximum range of the trajectory space is 100 pixels x 100 pixels x 100 pixels, the initial box size is 10 pixels x 10 pixels x 10 pixels, and 100 boxes are required to cover the trajectory; when reduced to 5 pixels x 5 pixels x 5 pixels, 800 boxes are required, and when further reduced to 2.5 pixels x 2.5 pixels x 2.5 pixels, 6400 boxes are required, and the fitting slope of the double logarithmic line is 1.2, then the fractal dimension is 1.2. The fractal dimension is a dimensionless parameter, reflecting the spatial filling complexity of the trajectory.

[0103] The nonlinear dynamics features of the mouth movement are extracted based on the fractal dimension and the maximum Lyapunov exponent of the phase space trajectory. The calculation steps of the maximum Lyapunov exponent include: selecting several pairs of adjacent trajectory points with an initial distance less than 1% of the trajectory range in the phase space, tracking their divergence process over time, and calculating the instantaneous divergence rate of each pair of trajectory points; linearly regressing the logarithm of the divergence rate, and the slope of the regression line is the maximum Lyapunov exponent, with a unit of 1 / second; if the regression error exceeds 10% or the data length is less than 100 frames, then the current frame Lyapunov exponent is invalid and interpolation compensation is triggered. The combination of the fractal dimension and the Lyapunov exponent is the nonlinear dynamics feature vector, for example, the feature vector of a certain frame is [fractal dimension = 1.2, Lyapunov exponent = 0.15]. The interpolation compensation logic is to predict the current frame feature value based on the weighted average of the previous five valid feature vectors, with the weight coefficient decaying over time, the weight of the most recent frame being 0.5, and the weight decreasing by 0.1 for each previous frame.

[0104] The nonlinear dynamics feature and the time domain statistics of the deformation displacement are weighted and fused to generate the time sequence correlation parameter of the mouth movement. The time domain statistics includes the mean, variance and peak factor of the deformation displacement. The mean and variance are calculated based on the displacement sequence in the time window. The window size matches the average duration of the feeding action, for example, the window size of carnivorous fish is 30 frames, and the window size of filter-feeding fish is 60 frames. The peak factor is the ratio of the peak displacement to the root mean square displacement. The peak displacement is the maximum displacement in the window, and the root mean square displacement is the square root of the square mean of the displacement. The weight coefficients of the weighted fusion are dynamically adjusted according to the differences between the chaotic characteristics and the periodicity of the feeding actions of different fish species. The specific rules are as follows: Calculate the mutual information between the nonlinear dynamics feature and the feeding behavior label in the historical data, and give a higher weight to the feature with higher mutual information. For example, the mutual information between the Lyapunov exponent of carnivorous fish and the feeding behavior is 0.85, the fractal dimension is 0.65, and the time domain statistics is 0.45. The weight coefficient is normalized to 0.85 / (0.85+0.65+0.45) = 0.44, the weight of the fractal dimension is 0.65 / 1.95 = 0.33, and the weight of the time domain statistics is 0.45 / 1.95 = 0.23. The weight coefficients are recalibrated every 24 hours according to the latest 100 feeding action data. The calibration method is the sliding window average method.

[0105] For extreme scenarios, such as incomplete trajectories due to insufficient data length for phase space reconstruction, a dimension reduction strategy is enabled, with the embedding dimension forced to 2 and a default time delay parameter of 5 frames. The reduced trajectory model is represented by a two-dimensional vector sequence. If the Lyapunov exponent calculation fails consecutively for more than 3 frames, switch to a simplified model based on time domain statistics. The simplified model only uses the mean and variance of the deformation displacement to generate the time sequence correlation parameter, with the weight coefficients fixed at 0.6 for the mean and 0.4 for the variance. Hardware dependencies include a GPU (such as NVIDIA Tesla V100) that supports double-precision floating-point operations, and the video memory needs to be at least 16 GB to process high-dimensional phase space data. Software dependencies include the nonlinear dynamics analysis toolkit in SciPy 1.7.0 library, the linear regression function in NumPy 1.21.0 library, and the custom weight calibration script.

[0106] S5, based on the distribution characteristics of the time sequence correlation parameter on the time axis, dynamically weighted to generate the motion coupling strength between the head region and the mouth region of the fish body, which is specifically implemented as:

[0107] When the motion coupling strength is generated based on the distribution characteristics of the time series correlation parameters on the time axis, the time series correlation parameters are first divided into equal length windows according to the time axis. The length of the equal length window is set according to the average duration of the feeding action. The specific rule is to calculate the duration distribution of the feeding action in the historical data, and take the 75th percentile of the duration as the window length. For example, the 75th percentile of the duration of the feeding action of a certain fish species is 35 frames, and the window length is set to 35 frames. The mean, variance and autocorrelation coefficient of the time series correlation parameters in each window are calculated. The mean reflects the average level of the parameters, and the calculation method is the arithmetic mean of all time series correlation parameters in the window. The variance represents the fluctuation intensity of the parameters, and the calculation method is the sum of the squares of the differences between each parameter and the mean divided by the window length minus one. The autocorrelation coefficient is calculated by lagging 1 frame, and the calculation formula is the covariance of the time series correlation parameter sequence and its lag sequence in the window divided by the product of their standard deviations. For example, the time series correlation parameter sequence in a certain window is [0.8, 0.7, 0.9, 0.6], the lag sequence is [0.7, 0.9, 0.6, 0.5], the covariance is 0.0125, the standard deviations are 0.1291 and 0.1581 respectively, and the autocorrelation coefficient is 0.0125 / (0.1291x0.1581) = 0.62. If the window length is less than 5 frames, the autocorrelation coefficient calculation is considered invalid and an exception handling is triggered.

[0108] The window dynamic weight coefficient is generated based on the ratio of the autocorrelation coefficient to the variance. The calculation formula of the ratio of the autocorrelation coefficient to the variance is the absolute value of the autocorrelation coefficient divided by the variance. The larger the ratio, the higher the stability and information content of the time series correlation parameters in the window. The threshold interval of the ratio is set to [0.5, 2.0]. When the ratio is less than 0.5, the weight coefficient is set to 0.2, between 0.5 and 1.0, the weight coefficient is set to 0.5, and higher than 1.0, the weight coefficient is set to 0.8. The threshold interval is determined according to the ratio distribution statistics of the feeding behavior and non-feeding behavior in the historical data. For example, the median of the ratio of the feeding behavior window is 1.2, and the non-feeding behavior is 0.3, so the lower limit of the threshold is set to 0.5 to distinguish the two types of behavior. For example, the autocorrelation coefficient of a certain window is 0.62, and the variance is 0.04, so the ratio is 0.62 / 0.04 = 15.5, which exceeds the upper limit of the threshold interval, and the weight coefficient is set to 0.8. If the data in the window is insufficient to calculate the ratio, the weight coefficient of the previous valid window is used.

[0109] The motion parameters of the head region and the mouth region of the fish body are analyzed by using a sliding window covariance analysis. The motion parameters include the average motion speed of the head region and the deformation displacement of the mouth region. The head motion speed is calculated by using an optical flow method to calculate the displacement of the centroid of the head region in each frame and then dividing the displacement by the frame interval time, and the unit is pixel / s. The mouth deformation displacement is derived from the output data of step S3, and the unit is pixel. The sliding window size is aligned with the equal-length window (for example, 35 frames), and the window sliding step is 1 frame. The covariance matrix is constructed by calculating the covariance of the head motion speed and the mouth displacement and the variances of the two parameters. The covariance calculation formula is the average of the product of the two parameters in the window minus the product of the average values of the two parameters, and the variance is the average of the square of the difference between the parameter and the average value. For example, the average value of the head speed in a certain window is 0.5 pixel / s, and the average value of the mouth displacement is 0.6 pixel. The covariance is (0.8*0.7+0.7*0.9+0.9*0.6+0.6*0.5) / 4-(0.5*0.6)=0.0325. The covariance matrix is [[0.15, 0.0325], [0.0325, 0.12]]. The principal eigenvalue of the covariance matrix is extracted by eigenvalue decomposition. The eigenvalue decomposition method is to solve the characteristic equation of the matrix, and the largest root is taken as the principal eigenvalue. The principal eigenvalue reflects the variance intensity of the principal component of the coupling between the head and the mouth motion.

[0110] The principal eigenvalue and the window dynamic weight coefficient are fused by using an exponentially weighted moving average algorithm to generate the motion coupling strength. The decay factor of the exponentially weighted moving average algorithm is dynamically adjusted according to the autocorrelation coefficient of the time series correlation parameter. The specific rules are as follows: when the autocorrelation coefficient is higher than 0.8, the decay factor is set to 0.1 to enhance the weight of recent data; when the autocorrelation coefficient is between 0.5 and 0.8, the decay factor is set to 0.3; and when the autocorrelation coefficient is lower than 0.5, the decay factor is set to 0.5. The decay factor adjustment is based on the positive correlation between the autocorrelation coefficient and the data stability. High autocorrelation indicates strong time series continuity, and the decay needs to be reduced to retain historical trends. The calculation formula of the motion coupling strength is: the principal eigenvalue of the current window multiplied by the weight coefficient plus the coupling strength of the last window multiplied by (1 minus the decay factor). For example, the principal eigenvalue of the current window is 0.25, the weight coefficient is 0.8, the decay factor is 0.3, and the coupling strength of the last window is 0.18. Then the current coupling strength is 0.25*0.8+0.18*0.7=0.20+0.126=0.326. If the current window data is invalid, the coupling strength remains the value of the last window and is marked as an estimated result.

[0111] For extreme scenarios, such as insufficient data in the window leading to the covariance matrix being unable to be calculated, a historical data interpolation mechanism is enabled, with a linear extrapolation method, and the extrapolation step is no more than 3 frames. Specifically, the current value is predicted according to the linear equation fitted according to the last 3 frames of valid data. If there are insufficient data for 5 consecutive frames, the sliding window is reset and the coupling strength is reinitialized, with the initialization value being the mean coupling strength of the last valid window. Hardware dependencies include at least a GPU (such as NVIDIA A100) with 16 GB of video memory, and the GPU needs to support CUDA 11.0 or later to accelerate covariance matrix decomposition. Software dependencies include the covariance matrix function in NumPy 1.21.0 library, the eigenvalue decomposition interface in SciPy 1.7.0 library, and the custom exponential weighting algorithm script.

[0112] S6. According to the cumulative effect and distribution pattern of the motion coupling strength in the preset time window, the occurrence state of the fish feeding behavior is determined, which is specifically implemented as:

[0113] When determining the occurrence state of the fish feeding behavior according to the cumulative effect and distribution pattern of the motion coupling strength in the preset time window, the cumulative effect of the motion coupling strength in the preset time window is first calculated. The length of the preset time window is set according to the shortest duration of the feeding behavior, and the shortest duration is determined by the interval between the start frame and the end frame of the feeding action in the historical video data. For example, the shortest duration of the feeding action of a carnivorous fish is 20 frames, and the window length is set to 20 frames, and the duration of the feeding action of a filter feeder is 30 frames. The cumulative effect is obtained by the sliding window integration method, and the integration window size is aligned with the preset time window. The integration method is to add each motion coupling strength value in the window in time order, and the weight coefficient decays with time. The decay factor is 0.9, that is, the weight of the current frame is 0.9 raised to the power of n (n is the time difference between the current frame and the start frame of the window, in frames), and the accumulation result is the sum of the intensity values of each frame multiplied by the corresponding weight. The unit of the intensity value is dimensionless. If the window length is insufficient to complete the integration, the window is expanded to the nearest complete window in the historical data and marked as an interpolation result. The interpolation logic is to set the intensity value of the missing frame to 0.8 times the intensity of the previous valid frame.

[0114] Statistical feature extraction is performed on the distribution pattern of the motion coupling strength in the window, including skewness, kurtosis, and interquartile range.

[0115] Skewness calculation: Based on the distribution data of all motion coupling strength values within the window, first calculate the arithmetic mean of all strength values (i.e. the sum of all values divided by the number), while calculating the median (the value located in the middle position after sorting the values by size), and calculating the standard deviation (the square root of the mean of the square of the difference between each value and the mean). The skewness is obtained by subtracting the median from the arithmetic mean and dividing by the standard deviation, which measures the asymmetry of the distribution; if the result is greater than zero, it indicates that the distribution is skewed to the right, and less than zero, it is skewed to the left.

[0116] Kurtosis calculation: First calculate the fourth central moment (the mean of the fourth power of the difference between each value and the mean), then divide it by the fourth power of the standard deviation, and finally subtract the constant three; kurtosis is used to describe the sharpness of the distribution shape, and the result greater than zero indicates that the distribution is sharper than the normal distribution, and less than zero indicates that it is more gentle.

[0117] Interquartile range calculation: Sort all motion coupling strength values within the window from small to large, determine the 25th percentile (i.e. the value at the one-fourth position) and the 75th percentile (i.e. the value at the three-fourth position). If the percentile position is not an integer, determine the specific value by linear interpolation of the adjacent two values. The interquartile range is the 75th percentile minus the 25th percentile, which directly reflects the dispersion of the data distribution.

[0118] For example, the intensity value sequence in a window is [0.3, 0.5, 0.7, 0.6, 0.4], after sorting it is [0.3, 0.4, 0.5, 0.6, 0.7], the 25th percentile is 0.4, the 75th percentile is 0.6, and the interquartile range is 0.6-0.4=0.2. If the number of data points in the window is less than 10, the statistical characteristics are considered invalid and the history data interpolation is triggered, and the interpolation method is to take the mean of the same characteristics of the nearest 3 valid windows.

[0119] Based on the combination of cumulative effect and statistical characteristics, a judgment vector is constructed. The judgment vector contains four dimensions: cumulative effect value, skewness, kurtosis and interquartile range, and each dimension value is processed by Z-score standardization, the standardization formula is (original value-mean) / standard deviation, and the mean and standard deviation are calculated based on the first 1000 valid windows of the historical feeding behavior data. For example, the mean of the cumulative effect in the historical data is 0.5, the standard deviation is 0.1, and the current window cumulative effect value is 0.6, then the standardized value is (0.6-0.5) / 0.1=1.0. If the data of a certain dimension is missing, it is filled by the weighted average of other dimensions, and the weight coefficient is set according to the Pearson correlation coefficient between the dimensions in the historical data, for example, the correlation coefficient between skewness and cumulative effect is 0.7, and the correlation coefficient between kurtosis is 0.5, then the weight is 0.7 / (0.7+0.5)=0.58 and 0.5 / (0.7+0.5)=0.42.

[0120] The decision vector is classified into feeding behavior state by a support vector machine classifier. The kernel function type of the support vector machine classifier is selected as a radial basis function (RBF), and the kernel function parameter γ and the penalty factor C are optimized by grid search cross-validation, the grid search range is γ ∈ [0.1, 10.0] (step size 0.5), C ∈ [0.1, 5.0] (step size 0.5), and the optimal parameter combination is selected according to the principle of the highest accuracy of five-fold cross-validation; the classification threshold is dynamically set based on the overlap degree of the decision vectors of feeding behavior and non-feeding behavior, and the overlap degree is calculated by the reciprocal of the Mahalanobis distance of the decision vectors of the two types of samples.

[0121] The calculation formula of the Mahalanobis distance is:

[0122]

[0123] wherein D M represents the Mahalanobis distance, which measures the standardized distance between the distributions of the two types of samples;

[0124] represents the mean vector of the feeding behavior samples, wherein represents the mean of the first dimension (cumulative effect) of the decision vector, represents the mean of the skewness, represents the mean of the kurtosis, represents the mean of the interquartile range;

[0125] represents the mean vector of the non-feeding behavior samples, and the dimension meanings are consistent with μ (A) ;

[0126] Σ pool represents the combined covariance matrix, and the calculation formula is:

[0127]

[0128] wherein n A represents the number of feeding behavior samples; n B represents the number of non-feeding behavior samples; Σ A represents the covariance matrix of the feeding behavior samples; Σ B represents the covariance matrix of the non-feeding behavior samples; represents the inverse matrix of the combined covariance matrix; T represents the matrix transposition operator, which converts a row vector into a column vector.

[0129] For extreme scenarios, such as when the support vector machine classification fails continuously for more than 3 times, switch to the nearest neighbor classifier based on the Euclidean distance of the decision vector, with the nearest neighbor number set to 5 and the distance weight as the reciprocal; if the historical data is insufficient to cause the parameter optimization to fail, enable the default parameters (γ = 1.0, C = 1.0) and mark it as a temporary mode, which is valid for no more than 24 hours. Hardware dependencies include at least a GPU (such as NVIDIA A100) equipped with 16GB of video memory to accelerate support vector machine training and prediction, and the GPU needs to support CUDA version 11.0 or later; software dependencies include the support vector machine module in the scikit-learn 1.0.2 library, the matrix operation function in the NumPy 1.21.0 library, and the custom standardization script.

[0130] Embodiment 2: Figure 2 A structural diagram of a fish feeding behavior recognition system is given, and the fish feeding behavior recognition system comprises the following modules:

[0131] The video acquisition module acquires underwater continuous video frames containing fish, and extracts the motion continuity parameters of the fish head region in each frame image;

[0132] The curvature screening module extracts the curvature mutation feature of the fish head motion trajectory based on the motion continuity parameters and analyzes the direction aggregation degree of the head region motion vector, and screens the video frames that simultaneously exist curvature mutation feature and direction aggregation degree exceeding the direction aggregation degree threshold as the candidate key frame sequence;

[0133] The deformation analysis module performs local deformation motion analysis on the fish mouth opening and closing region in the candidate key frame sequence, and generates the deformation displacement of the mouth region between adjacent key frames;

[0134] The nonlinear analysis module performs phase space reconstruction and fractal dimension analysis on the deformation displacement, extracts the nonlinear dynamics feature of the mouth action, and generates the time sequence correlation parameter of the mouth action;

[0135] The dynamic weighting module dynamically weights the motion coupling strength of the fish head region and the mouth region based on the distribution feature of the time sequence correlation parameter on the time axis;

[0136] The state determination module determines the occurrence state of the fish feeding behavior according to the cumulative effect and distribution mode of the motion coupling strength in the preset time window.

[0137] The calculations involved in the embodiments are all dimensionless numerical calculations, and the preset parameters and threshold values in the calculations are set by those skilled in the art according to the actual situation.

[0138] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product.

[0139] Those skilled in the art can realize that the modules and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed by hardware or software depends on the specific application and the constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0140] In addition, each functional module in each embodiment of the present application can be integrated into one processing module, or each module can exist physically independently, or two or more modules can be integrated into one module.

[0141] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are merely schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0142] The above is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0143] Finally: the above is only the preferred embodiment of the present application, and is not used to limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A fish feeding behavior recognition method, characterized in that, The method comprises the following steps: S1, obtaining underwater continuous video frames containing fish, and extracting motion continuity parameters of the head region of the fish in each frame image; S2, extracting curvature mutation features of the head motion trajectory of the fish based on the motion continuity parameters, and analyzing the direction aggregation degree of the motion vector of the head region, and screening video frames with both curvature mutation features and direction aggregation degrees exceeding a direction aggregation degree threshold as a candidate key frame sequence; S3, performing local deformation motion analysis on the mouth opening and closing region of the fish in the candidate key frame sequence, and generating deformation displacement of the mouth region between adjacent key frames; S4, performing phase space reconstruction and fractal dimension analysis on the deformation displacement to extract nonlinear dynamics features of the mouth action, and generating time sequence correlation parameters of the mouth action; S5, based on the distribution features of the time sequence correlation parameters on the time axis, dynamically weighting to generate the motion coupling strength of the head region and the mouth region of the fish, comprising: dividing the time sequence correlation parameters into equal length windows according to the time axis, calculating the mean, variance and autocorrelation coefficient of the time sequence correlation parameters in each window; generating window dynamic weight coefficients based on the ratio of autocorrelation coefficients to variances; performing sliding window covariance analysis on the motion parameters of the head region and the mouth region of the fish, calculating the principal eigenvalue of the covariance matrix; fusing the principal eigenvalue and the window dynamic weight coefficient according to the exponential weighted moving average algorithm to generate the motion coupling strength; the decay factor of the exponential weighted moving average algorithm is dynamically adjusted according to the autocorrelation coefficient of the time sequence correlation parameters, and the window size of the covariance analysis is aligned with the equal length window; S6, according to the cumulative effect and distribution mode of the motion coupling strength in the preset time window, determining the occurrence state of the fish feeding behavior, comprising: calculating the cumulative effect of the motion coupling strength in the preset time window, the cumulative effect is obtained by sliding window integration method, and the integration window size is aligned with the preset time window; extracting statistical features of the distribution mode of the motion coupling strength in the window, including skewness, kurtosis and interquartile range; constructing a judgment vector based on the combination of the cumulative effect and the statistical features, and classifying the judgment vector into the feeding behavior state through a support vector machine classifier; the kernel function type and the penalty factor of the support vector machine classifier are optimized according to the cross validation of historical feeding behavior data, and the classification threshold is dynamically set based on the distribution overlap degree of the judgment vectors of the feeding behavior and the non-feeding behavior.

2. The fish feeding behavior recognition method according to claim 1, characterized in that, The method comprises the following steps: obtaining underwater continuous video frames containing fish, and extracting motion continuity parameters of the head region of the fish in each frame image, comprising: performing image preprocessing operations on the underwater continuous video frames to suppress noise and balance illumination, obtaining a preprocessed video frame sequence; locating the head region of the fish in the preprocessed video frame sequence based on a target detection algorithm, and optimizing the boundary integrity of the head region through morphological closing operation; calculating the optical flow field of the head region of the fish between adjacent frames and extracting the motion vector set of the pixel points in the optical flow field; calculating the motion continuity parameters of the head region of the fish according to the motion vector set, the motion continuity parameters including the average motion speed, the motion direction angle variance and the motion trajectory curvature change rate between adjacent frames.

3. The fish feeding behavior recognition method of claim 1, wherein, The curvature mutation feature of the fish head motion trajectory is extracted based on the motion continuity parameter, and the direction aggregation degree of the head region motion vector is analyzed, video frames with both the curvature mutation feature and the direction aggregation degree exceeding a direction aggregation degree threshold are selected as a candidate key frame sequence, including: The trajectory curve is generated by cubic spline interpolation method for the fish head motion trajectory, the trajectory curvature change rate is calculated, and the video frame with the curvature change rate exceeding a curvature change rate threshold is marked as a candidate curvature mutation frame; The covariance matrix is constructed for the head region motion vector set, and the ratio of the maximum eigenvalue to the minimum eigenvalue of the covariance matrix is calculated as the direction aggregation degree index; The video frame with the direction aggregation degree index exceeding the direction aggregation degree threshold in the candidate curvature mutation frame is merged into the candidate key frame sequence.

4. The fish feeding behavior recognition method according to claim 3, characterized in that, The curvature change rate threshold is dynamically adjusted according to the average value and standard deviation of the curvature change rate statistically calculated from historical data, and the direction aggregation degree threshold is set based on the direction consistency empirical value of different fish species feeding actions.

5. The fish feeding behavior recognition method according to claim 1, characterized in that, Local deformation motion analysis is performed on the fish mouth opening and closing area in the candidate key frame sequence to generate the deformation displacement of the mouth area between adjacent key frames, including: The key points of the fish mouth opening and closing area are located in the candidate key frame sequence, and the key points are adaptively selected based on the anatomical features of the fish mouth; The motion vector of the key points between adjacent key frames is calculated by using an optical flow tracking algorithm; The motion vector is separated into rigid and non-rigid deformation, and the displacement vector corresponding to the non-rigid deformation is extracted; The deformation displacement of the mouth area is calculated based on the amplitude and direction distribution of the displacement vector.

6. The fish feeding behavior recognition method according to claim 5, characterized in that, The iteration termination condition of the optical flow tracking algorithm is that the motion vector change between adjacent iterations is less than a preset precision threshold, and the separation of rigid and non-rigid deformation is realized by motion vector field decomposition.

7. The fish feeding behavior recognition method of claim 1, wherein, The deformation displacement is reconstructed in phase space and analyzed for fractal dimension to extract the nonlinear dynamics feature of the mouth action and generate the time sequence correlation parameter of the mouth action, including: The trajectory dynamics model is constructed by phase space reconstruction of the deformation displacement, and the embedding dimension and time delay parameter of the phase space reconstruction are determined by joint pseudo-neighbor method and autocorrelation function; The fractal dimension of the trajectory is calculated by using the box counting method for fractal dimension analysis of the reconstructed phase space trajectory; The nonlinear dynamics feature of the mouth action is extracted based on the fractal dimension and the maximum Lyapunov exponent of the phase space trajectory; The nonlinear dynamics feature and the time domain statistical quantity of the deformation displacement are weighted and fused to generate the time sequence correlation parameter of the mouth action.

8. A fish feeding behavior recognition system for implementing the fish feeding behavior recognition method of any one of claims 1-7, characterized in that, The following modules are included: Video acquisition module: obtain underwater continuous video frames containing fish, and extract the motion continuity parameter of the fish head region in each frame of image; Curvature aggregation screening module: extract the curvature mutation feature of the fish head motion trajectory based on the motion continuity parameter, and analyze the direction aggregation degree of the head region motion vector, select the video frame with both the curvature mutation feature and the direction aggregation degree exceeding the direction aggregation degree threshold as the candidate key frame sequence; Deformation analysis module: local deformation motion analysis is performed on the fish mouth opening and closing area in the candidate key frame sequence to generate the deformation displacement of the mouth area between adjacent key frames; The nonlinear analysis module: the deformation displacement is reconstructed in phase space and the fractal dimension analysis is carried out, nonlinear dynamics characteristics of the mouth action are extracted, and time sequence correlation parameters of the mouth action are generated; The dynamic weighting module: based on the distribution characteristics of the time sequence correlation parameters on the time axis, the motion coupling strength of the head region and the mouth region of the fish body is dynamically weighted and generated; The state determination module: according to the cumulative effect and the distribution mode of the motion coupling strength in the preset time window, the occurrence state of the fish feeding behavior is determined.

Images