Method and device for recognizing fish behavior

Abstract

The application provides a fish behavior recognition method and device, the method comprising: acquiring a fish motion image; inputting the fish motion image into a target detection model to obtain a fish target detection result; analyzing the fish target detection result to obtain a fish tracking trajectory; analyzing the fish tracking trajectory to obtain a fish motion feature; and inputting the fish motion feature into a behavior recognition model to obtain a fish behavior recognition result. The fish behavior recognition method and device can solve the defects of complex parameters and difficult recognition in the prior art by constructing a model framework integrating motion feature extraction and behavior state recognition functions, thereby improving the fish behavior recognition accuracy and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a fish behavior recognition method and device. Background Art

[0002] In aquaculture, monitoring and analyzing fish behavior guides daily feeding, disease analysis, and diagnosis. Common fish behaviors in aquaculture include feeding and stress behaviors. Common characteristic indicators can be categorized as motion characteristics, physical characteristics, and environmental characteristics. Motion characteristics are the most direct indicator of changes in fish behavior.

[0003] Initial studies of fish behavior relied on direct observation and recording by human observers. This method was subject to subjective influences and could not accurately identify fish behavior. With technological advancements, various methods have been proposed to extract information about fish behavior, including machine vision, acoustics, and bioindicators. Bioindicators often require the implantation of biorecorders into animals, thereby affecting their normal behavior. Acoustics are susceptible to surface and wave noise. Machine vision, a new technology developed based on a variety of techniques, including image processing, artificial intelligence, and signal processing, has gradually become the preferred method for aquaculture practitioners and researchers due to its non-destructive, accurate, and easy-to-deploy capabilities.

[0004] Currently, fish behavior recognition methods based on machine vision often suffer from complex structures, large number of parameters, and high computational complexity when extracting fish motion features. These algorithms cannot meet the needs of real-world scenarios. Furthermore, there are difficulties in accurately quantifying fish motion characteristics and combining multiple motion features to identify and classify their behavior states. These issues make current fish behavior recognition methods difficult to ensure accuracy and efficiency.

[0005] Therefore, it is an urgent problem to propose a model framework that can integrate motion feature extraction and behavioral state recognition functions, effectively reducing the model size and inference time while ensuring accuracy. Summary of the Invention

[0006] The present invention provides a fish behavior recognition method and device, which are used to solve the defects of complex parameters and difficult recognition in the prior art of fish behavior recognition, and to improve the accuracy and efficiency of fish behavior recognition.

[0007] The present invention provides a fish behavior recognition method, comprising:

[0008] Acquire images of fish movement;

[0009] Inputting the fish motion image into a target detection model to obtain a fish target detection result;

[0010] Analyzing the fish target detection result to obtain the fish tracking trajectory;

[0011] Analyzing the fish tracking trajectory to obtain fish movement characteristics;

[0012] Inputting the fish movement characteristics into a behavior recognition model to obtain a fish behavior recognition result;

[0013] Among them, the target detection model is trained based on fish motion image samples and the fish target detection results corresponding to the fish motion image samples as labels; the behavior recognition model is trained based on fish motion feature samples and the fish behavior recognition results corresponding to the fish motion feature samples.

[0014] According to a fish behavior recognition method provided by the present invention, the training step of the target detection model includes:

[0015] Acquiring fish motion image samples and performing data enhancement processing on the fish motion image samples;

[0016] Marking the fish target detection result corresponding to the fish motion image sample as a label;

[0017] Dividing the fish motion image samples into a fish motion image sample training set, a fish motion image sample verification set, and a fish motion image sample test set;

[0018] The target detection model is obtained by training the fish motion image sample training set, the fish target detection result of the target detection model is verified by the fish motion image sample verification set, and after obtaining the network weight of the target detection model, the target detection model is tested by the fish motion image sample test set.

[0019] According to a fish behavior recognition method provided by the present invention, the target detection model is obtained by training based on the LDW-YOLOv8 model. Before the target detection model is obtained by training the fish motion image sample training set, the method further includes:

[0020] Adjust the basic network feature fusion structure of the LDW-YOLOv8 model to reduce the learning of redundant features;

[0021] Replace the backbone downsampling convolutions of the LDW-YOLOv8 model with full-dimensional dynamic convolutions;

[0022] The bounding box loss function CIoU_Loss of the LDW-YOLOv8 model is replaced by Wise-IoUv3 Loss.

[0023] According to a fish behavior recognition method provided by the present invention, analyzing the fish target detection result to obtain the fish tracking trajectory includes:

[0024] Divide the fish target detection result into a high-score detection frame and a low-score detection frame according to a confidence threshold in the target detection result, put the high-score detection frame into a high-score detection frame set, and put the low-score detection frame into a low-score detection frame set;

[0025] Performing trajectory matching on the high-scoring detection frame set, placing unsuccessfully matched trajectories into a first remaining trajectory set, and placing unsuccessfully matched high-scoring detection frames in the high-scoring detection frame set into a remaining detection frame set;

[0026] Matching the low-scoring detection frame set with the first remaining trajectory set, placing unsuccessfully matched trajectories into a second remaining trajectory set, and deleting the unsuccessfully matched low-scoring detection frame set from the low-scoring detection frame set;

[0027] Update the Kalman filter of the successfully matched trajectory;

[0028] Generate a new track based on the high-scoring detection frames in the remaining detection frame set whose confidence is greater than the tracking score threshold, and delete tracks longer than 30 frames in the second remaining track set;

[0029] A Kalman filter and an extended Kalman filter are constructed, and the new position of the trajectory with acceleration less than the acceleration threshold is predicted by the Kalman filter, and the new position of the trajectory with acceleration greater than the acceleration threshold is predicted by the extended Kalman filter to obtain the fish tracking trajectory.

[0030] According to a fish behavior recognition method provided by the present invention, before analyzing the fish tracking trajectory, the method further includes: performing Gaussian interpolation processing on the fish tracking trajectory, and the specific steps include:

[0031] The observation value of the i-th track in the x-frame of the fish tracking track is calculated as follows: for:

[0032]

[0033] in, is the coordinate variable of trajectory i at the x frame, The mean is 0 and the variance is Gaussian distribution, f (i) ∈GP(0,k(x,x′)), l is a hyperparameter;

[0034] Through the known frame x1x2x3······x n and the frame to be predicted x *Compute the covariance matrix:

[0035]

[0036] K * =[k(x * ,x1) k(x * ,x2) … k(x * ,x n )]

[0037] The best estimate of Among them, y is The n×1 dimensional matrix composed of

[0038] By setting the hyperparameter σ to represent the maximum number of frame intervals for performing Gaussian interpolation, Gaussian interpolation is performed when the interval number is less than the hyperparameter σ.

[0039] According to a fish behavior recognition method provided by the present invention, the fish movement characteristics include: swimming speed, swimming acceleration and discreteness. In the process of analyzing the fish tracking trajectory, the image is corrected through inverse perspective transformation.

[0040] According to a fish behavior recognition method provided by the present invention, the training step of the behavior recognition model includes:

[0041] Obtain samples of fish movement characteristics;

[0042] Marking the fish behavior recognition result corresponding to the fish movement feature sample as a label;

[0043] Dividing the fish motion feature samples into a fish motion feature sample training set and a fish motion feature sample test set;

[0044] A behavior recognition model is obtained by training the fish motion feature sample training set, and the behavior recognition model is evaluated by using the fish motion feature sample test set.

[0045] The present invention also provides a fish behavior recognition device, comprising:

[0046] An image acquisition module, used for acquiring fish movement images;

[0047] A target detection module is used to input the fish motion image into a target detection model to obtain a fish target detection result;

[0048] A trajectory analysis module is used to analyze the fish target detection results to obtain fish tracking trajectories;

[0049] A feature analysis module is used to analyze the fish tracking trajectory to obtain fish movement characteristics;

[0050] A behavior recognition module is used to input the fish movement characteristics into a behavior recognition model to obtain fish behavior recognition results;

[0051] Among them, the target detection model is trained based on fish motion image samples and the fish target detection results corresponding to the fish motion image samples as labels; the behavior recognition model is trained based on fish motion feature samples and the fish behavior recognition results corresponding to the fish motion feature samples.

[0052] The present invention also provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described fish behavior recognition methods when executing the program.

[0053] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the above-described fish behavior recognition methods.

[0054] The present invention provides a method and device for identifying fish behavior. This method obtains fish motion images and inputs them into a target detection model to generate fish target detection results, thereby identifying individual fish targets from the raw data of the fish motion images. The fish target detection results are analyzed to obtain fish tracking trajectories, i.e., the fish's motion trajectory in its current state. The fish tracking trajectories are further analyzed, and fish motion features are extracted from the fish tracking trajectories through computational quantification to reflect the various motion data of the fish in its current state. The fish motion features are then input into a behavior recognition model, and a comprehensive analysis is performed based on the various fish motion features to obtain fish behavior recognition results, i.e., the fish's current state of behavior, providing an effective reference for fish husbandry or diagnosis. The target detection model is trained using fish motion image samples and the corresponding fish target detection results as labels; the behavior recognition model is trained using fish motion feature samples and the corresponding fish behavior recognition results. Based on machine vision and deep learning, the present invention constructs a model framework that integrates motion feature extraction and behavioral state recognition, effectively reducing model size and inference time while ensuring accuracy. The method solves the defects of complex parameters and difficult identification in the existing fish behavior recognition technology, and improves the accuracy and efficiency of fish behavior recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 1 is a flow chart of the fish behavior recognition method provided by the present invention;

[0057] Figure 2 It is a structural diagram of the target detection model of the fish behavior recognition method provided by the present invention;

[0058] Figure 3 It is a structural schematic diagram of the fish behavior recognition device provided by the present invention;

[0059] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0061] The following combination Figure 1 The fish behavior recognition method according to the first embodiment of the present invention is described.

[0062] like Figure 1 As shown, a fish behavior recognition method of this embodiment specifically includes the following steps (the numbering of each step in the present invention is only used to distinguish the steps and does not limit the specific execution order of each step):

[0063] Step S1: Acquire fish motion images.

[0064] In the process of acquiring fish motion images, video data of free motion of fish in actual breeding scenes is collected as fish motion images. In this embodiment, the fish motion images collected are sea bass motion images.

[0065] Step S2: input the fish motion image into the target detection model to obtain the fish target detection result.

[0066] The fish motion image is input into the preset target detection model to obtain the fish target detection result, so as to identify the fish target individual from the raw data of the fish motion image and obtain the fish's location and behavior information.

[0067] Step S3: Analyze the fish target detection result to obtain the fish tracking trajectory.

[0068] Analyze the fish target detection results, track the fish target detection results, and obtain the fish tracking trajectory, that is, the movement trajectory of the fish in the current state.

[0069] Step S4: Analyze the fish tracking trajectory to obtain fish movement characteristics.

[0070] Continue to analyze the fish tracking trajectory, and extract the fish movement characteristics from the fish tracking trajectory through computational quantification to reflect the various movement data of the fish in the current state and obtain the long-term free movement information of the target fish.

[0071] Step S5: inputting the fish movement characteristics into the behavior recognition model to obtain the fish behavior recognition result.

[0072] The fish movement characteristics are input into the behavior recognition model, and a comprehensive analysis is performed based on the various fish movement characteristics to obtain the fish behavior recognition results, that is, the fish behavior in the current state, which provides an effective reference for fish breeding or diagnosis.

[0073] Among them, the target detection model is trained based on fish motion image samples and the fish target detection results corresponding to the fish motion image samples as labels; the behavior recognition model is trained based on fish motion feature samples and the fish behavior recognition results corresponding to the fish motion feature samples.

[0074] The target detection model is trained using fish motion image samples and the corresponding fish target detection results as labels. The behavior recognition model is trained using fish motion feature samples and the corresponding fish behavior recognition results. Through deep learning, accurate fish target detection and behavior recognition results are achieved.

[0075] A first embodiment of the present invention provides a fish behavior recognition method. This method obtains fish motion images and inputs them into a target detection model to obtain fish target detection results, thereby identifying individual fish targets from the raw data of the fish motion images. The fish target detection results are analyzed to obtain fish tracking trajectories, i.e., the fish's motion trajectory in its current state. The fish tracking trajectories are further analyzed, and fish motion features are extracted from the fish tracking trajectories through computational quantification to reflect the various motion data of the fish in its current state. The fish motion features are then input into a behavior recognition model, and a comprehensive analysis is performed based on the various fish motion features to obtain fish behavior recognition results, i.e., the fish's behavior in its current state, providing an effective reference for fish husbandry or diagnosis. The target detection model is trained using fish motion image samples and the corresponding fish target detection results as labels; the behavior recognition model is trained using fish motion feature samples and the corresponding fish behavior recognition results. Based on machine vision and deep learning, this invention constructs a model framework that integrates motion feature extraction and behavioral state recognition, effectively reducing model size and inference time while ensuring accuracy. The method solves the defects of complex parameters and difficult identification in the existing fish behavior recognition technology, and improves the accuracy and efficiency of fish behavior recognition.

[0076] In this embodiment, the training steps of the target detection model include:

[0077] Acquiring fish motion image samples and performing data enhancement processing on the fish motion image samples;

[0078] Marking the fish target detection result corresponding to the fish motion image sample as a label;

[0079] Dividing the fish motion image samples into a fish motion image sample training set, a fish motion image sample verification set, and a fish motion image sample test set;

[0080] The target detection model is obtained by training the fish motion image sample training set, the fish target detection result of the target detection model is verified by the fish motion image sample verification set, and after obtaining the network weight of the target detection model, the target detection model is tested by the fish motion image sample test set.

[0081] By controlling the water quality of the aquaculture environment, we collected video data of fish in free-motion under conditions of low temperature and hypoxia. We also installed an automatic feeder to capture video data of fish in free-motion during feeding. These data served as samples of fish motion images. Using the image data enhancement library (Albumentation), the image data annotation tool (Labelme), and the multi-target tracking and annotation tool (DarkLabel), we filtered, enhanced, annotated, and classified these fish motion image samples to form a valid dataset.

[0082] First, fish motion image samples were screened and data augmented using methods including horizontal flipping, vertical flipping, blurring, random brightness contrast, RGB shifting, random 90-degree rotation, Gaussian noise, and Gaussian blurring. The fish motion image samples were annotated with the corresponding fish target detection results as labels. After processing, the processed data was saved as a dataset for target detection and verification of multi-target tracking. The fish motion image samples were then divided into a training set, a validation set, and a test set using a 6:2:2 ratio.

[0083] The target detection model is obtained by training with the fish motion image sample training set, the fish target detection results of the target detection model are verified with the fish motion image sample validation set, and after obtaining the network weights of the target detection model, the target detection model is tested with the fish motion image sample test set.

[0084] In this embodiment, the target detection model is obtained based on the LDW-YOLOv8 model training. Before the target detection model is obtained by training the fish motion image sample training set, the following steps are further included:

[0085] Adjust the basic network feature fusion structure of the LDW-YOLOv8 model to reduce the learning of redundant features;

[0086] Replace the backbone downsampling convolutions of the LDW-YOLOv8 model with full-dimensional dynamic convolutions;

[0087] The bounding box loss function CIoU_Loss of the LDW-YOLOv8 model is replaced by Wise-IoUv3 Loss.

[0088] The target detection model of this embodiment is based on the LDW-YOLOv8 improved network model. The LDW-YOLOv8 model is improved based on the YOLOv8 model. L stands for Light, D stands for the use of full-dimensional dynamic convolution, and W stands for Wiou v3_Loss loss function. According to the characteristics of the data set, the size of the feature map is adjusted. First, the four downsamplings in the YOLOv8 backbone network are reduced to three, so that the output feature map size is expanded from 20×20×512 to 40×40×512. At the same time, the one upsampling of the neck network and the feature fusion module are reduced. The detection head part is redesigned, and the large target detection head of the smallest layer of feature map is removed. By adjusting the basic network feature fusion structure of the target detection model, the learning of redundant features is reduced.

[0089] The downsampling module in the YOLOv8 backbone network is improved, and the ordinary convolution of the three downsampling modules is replaced with full-dimensional dynamic convolution ODConv. The full-dimensional dynamic convolution adopts a multi-dimensional attention mechanism along the four dimensions of the kernel space through a parallel strategy to learn complementary attention.

[0090] Two-dimensional convolution is described as: Output(x) = W*x, where x represents the input feature, W represents the convolution layer, and * represents the convolution operation. Dynamic convolution is described as: Output(x) = (α1w1+…+α n W n )*x, where W i represents the i-th group of convolution filters, α i W i The weight parameter of the full-dimensional dynamic convolution is Output(x)=(α w1 ⊙α f1 ⊙α c1 ⊙α s1 ⊙W1+…+α wn ⊙α fn ⊙α cn ⊙α sn ⊙W n )*x, where α wi Equivalent to α i , the newly introduced α fi , α ci and α si , which represent the learnable weights of the output dimension, input dimension, and spatial dimension respectively.

[0091] The bounding box loss function of the YOLOv8 object detection network is optimized, and the Wise-IoUv3 Loss based on the dynamic non-monotonic focusing mechanism is used instead of the original CIoU_Loss. This method can focus more on anchor boxes of normal quality and improve the overall performance of the detector.

[0092] Among them, β and r are controlled by hyperparameters α and δ; in, W g and H g Represents the width and height of the minimum bounding box respectively; the superscript * represents W g and H g It is separated from the computational graph; The final structural diagram of the target detection model in this embodiment is as follows Figure 2 shown.

[0093] The specific training process of the target detection model is:

[0094] The image size was scaled to 640*640, and the Stochastic Gradient Descent (SGD) algorithm was used as the optimizer. The basic parameter learning rate was set to 0.01. All experiments were performed on a GPU for 200 rounds. Based on the collected and prepared datasets, experiments were conducted and compared with other models. The number of parameters, computational complexity, model size, FPS, inference time, accuracy, recall rate, AP50, and AP50-95 were used as the overall evaluation indicators of the model. The ablation experiment is shown in Table 1:

[0095] Table 1

[0096]

[0097]

[0098] The comparative experiments are shown in Table 2:

[0099] Table 2

[0100]

[0101]

[0102] In this embodiment, analyzing the fish target detection result to obtain the fish tracking trajectory includes:

[0103] Divide the fish target detection result into a high-score detection frame and a low-score detection frame according to a confidence threshold in the target detection result, put the high-score detection frame into a high-score detection frame set, and put the low-score detection frame into a low-score detection frame set;

[0104] Performing trajectory matching on the high-scoring detection frame set, placing unsuccessfully matched trajectories into a first remaining trajectory set, and placing unsuccessfully matched high-scoring detection frames in the high-scoring detection frame set into a remaining detection frame set;

[0105] Matching the low-scoring detection frame set with the first remaining trajectory set, placing unsuccessfully matched trajectories into a second remaining trajectory set, and deleting the unsuccessfully matched low-scoring detection frame set from the low-scoring detection frame set;

[0106] Update the Kalman filter of the successfully matched trajectory;

[0107] Generate a new track based on the high-scoring detection frames in the remaining detection frame set whose confidence is greater than the tracking score threshold, and delete tracks longer than 30 frames in the second remaining track set;

[0108] A Kalman filter and an extended Kalman filter are constructed, and the new position of the trajectory with acceleration less than the acceleration threshold is predicted by the Kalman filter, and the new position of the trajectory with acceleration greater than the acceleration threshold is predicted by the extended Kalman filter to obtain the fish tracking trajectory.

[0109] The target detection results output by the LDW-YOLOv8 model are calculated according to the confidence threshold τ high and τ lww Divide into high-score detection frames and low-score detection frames, and divide the confidence level greater than τ high The high-score detection frame is put into the high-score detection frame set D high In the example, the confidence level is less than τ high and greater than τ low The low-score detection frame is put into the low-score detection frame set D low middle.

[0110] The high-score detection box set D high Match with existing trajectories (when the algorithm detects in the first frame, there is no trajectory at this time, so it will create trajectory objects for all target detection boxes and store them. Starting from the second frame, the detection box will be matched with the trajectory created in the first frame, and then the next frame will be matched with the corresponding trajectory of the previous frame), calculate its IoU distance matrix, match it using the Hungarian algorithm, and put the trajectories that failed to match successfully into the first remaining trajectory set T remain , put the high-score detection frames that failed to match successfully in the high-score detection frame set into the remaining detection frame set D remain . Then the low score detection box set D low With the first remaining trajectory set T remain Match again and put the unmatched trajectories into the second set of remaining trajectories T lost Delete the low-score detection frames that failed to match successfully in the low-score detection frame set. For all the above successfully matched trajectories, update their Kalman filters and put them into the current trajectory set.

[0111] For the remaining detection box set D remainThe confidence level is greater than the tracking score threshold τ track Create a new track for the high-score detection frame and delete the second remaining track set T lost Trajectories with more than 30 frames.

[0112] Construct Kalman filter (KF) and extended Kalman filter (EKF), and use Kalman filter to predict that the acceleration is less than the acceleration threshold τ a The new position of the trajectory is predicted by the extended Kalman filter to be greater than the acceleration threshold τ a The new position of the trajectory is obtained to obtain the fish tracking trajectory.

[0113] Specifically, when the acceleration threshold of the trajectory is greater than τ a The extended Kalman filter is used to predict its new position. The extended Kalman filter performs Taylor expansion on the nonlinear function, linearizes the nonlinear function, and then predicts the fish tracking trajectory.

[0114] This embodiment solves the problem of the tracker being unable to correctly predict the trajectory due to mutual occlusion between freely moving fish, incorrect ID conversion, and sudden acceleration of the fish through the above method, thereby ensuring the accuracy of the fish tracking trajectory.

[0115] In this embodiment, before analyzing the fish tracking trajectory, the method further includes: performing Gaussian interpolation processing on the fish tracking trajectory, and the specific steps include:

[0116] The observation value of the i-th track in the x-frame of the fish tracking track is calculated as follows: for:

[0117]

[0118] in, is the coordinate variable of trajectory i at the x frame, The mean is 0 and the variance is Gaussian distribution, f (i) ∈GP(0,k(x,x′)), l is a hyperparameter;

[0119] Through the known frame x1x2x3...x n and the frame to be predicted x * Compute the covariance matrix:

[0120]

[0121] K * =[k(x * ,x1) k(x *, x2) … k(x * , x n )]

[0122] The best estimate of Among them, y is The n×1 dimensional matrix composed of

[0123] By setting the hyperparameter σ to represent the maximum number of frame intervals for performing Gaussian interpolation, Gaussian interpolation is performed when the interval number is less than the hyperparameter σ.

[0124] is the observation value corresponding to the i-th trajectory at the x-th frame (x1x2x3……x n ), now we need to get x * The corresponding predicted value, This is the predicted value. The hyperparameter setting represents a constraint; only when the conditions are met will the Gaussian smoothing interpolation strategy be executed. By applying Gaussian interpolation to fish trajectories before analyzing them, the results of analyzing and identifying fish motion characteristics can be improved.

[0125] The specific process of evaluating the target tracking results based on the fish tracking trajectory:

[0126] The fish target detection results of 6 videos of fish in different states of free movement are selected as the test set. The high-order tracking accuracy (HOTA), mean identification fraction (IDF1), majority tracking (MT), major missing targets (ML), number of identity switches (IDs), multi-target tracking accuracy (MOTA) and multi-target tracking precision (MOTP) are used for comprehensive evaluation to ensure the accuracy of the behavior recognition model. The evaluation results are shown in Table 3:

[0127] Table 3

[0128]

[0129] Only when the tracking accuracy is high enough can the error caused by extracting motion features through calculation be reduced.

[0130] In this embodiment, the fish movement characteristics include swimming speed, swimming acceleration and discreteness. In the process of analyzing the fish tracking trajectory, the image is corrected through inverse perspective transformation.

[0131] In the process of acquiring and analyzing the movement characteristics of various fish, due to the perspective effect of the monocular camera, the image is corrected through inverse perspective mapping (IPM) to ensure that the pixel distance is evenly distributed in the entire image.

[0132] Fish movement characteristics include swimming speed, swimming acceleration and discreteness. Through inverse perspective transformation, the two-dimensional pixel point (u, v) is mapped to the corresponding world coordinate (X W ,Y W ,Z W ):

[0133] [uv 1] T =KRT[X w Y w Z w 1] T , where R is the rotation matrix, T is the translation matrix, K is the intrinsic matrix, h is the camera height, f is the focal length, ku and kv are the measurement calibration values ​​in horizontal and vertical pixel units respectively, and c x , c y To correct the principal point displacement of the image plane optical axis.

[0134]

[0135]

[0136]

[0137] The discrete degree of the i-th frame can be expressed as: D i =det(cov(x, y)), where det(.) represents the determinant calculation of the center coordinates of the fish target detection frame, and cov(.) is the covariance matrix operation of the matrix. i The smaller the size, the denser the fish distribution, and vice versa, the more dispersed the fish distribution.

[0138] The swimming speed of the individual with ID number i in the k+Δt frame and assuming that the total number of fish individuals in the video is N, the average swimming speed of the whole in the k+1 frame is:

[0139]

[0140] The swimming acceleration of the individual with ID number i at time t2 and assuming that the total number of fish individuals in the video is N, the average swimming acceleration of the population at time t2 is:

[0141]

[0142] In this embodiment, the training steps of the behavior recognition model include:

[0143] Obtain samples of fish movement characteristics;

[0144] Marking the fish behavior recognition result corresponding to the fish movement feature sample as a label;

[0145] Dividing the fish motion feature samples into a fish motion feature sample training set and a fish motion feature sample test set;

[0146] A behavior recognition model is obtained by training the fish motion feature sample training set, and the behavior recognition model is evaluated by using the fish motion feature sample test set.

[0147] The behavior recognition model is trained based on the LTSM (Long Short-Term Memory) model. During the training process, fish motion feature samples are obtained, and the corresponding fish behavior recognition results are annotated as labels. After obtaining three fish motion features (swimming speed, swimming acceleration, and discreteness), the fish behavior recognition results under low temperature, hypoxia, normal, and feeding conditions are annotated. The fish motion feature samples are divided into a training set of fish motion feature samples and a test set of fish motion feature samples. 70% of the fish motion feature sample dataset is randomly selected as the training set, and 30% is randomly selected as the test set. The behavior recognition model is trained on the training set of fish motion feature samples, and the behavior recognition model is evaluated on the test set of fish motion feature samples.

[0148] The fish behavior recognition device provided by the present invention is described below. The fish behavior recognition device described below and the fish behavior recognition method described above can be referenced to each other.

[0149] like Figure 3 As shown, the second embodiment of the present invention further provides a fish behavior recognition device, comprising:

[0150] An image acquisition module 210 is used to acquire fish movement images;

[0151] The target detection module 220 is used to input the fish motion image into the target detection model to obtain the fish target detection result;

[0152] A trajectory analysis module 230 is used to analyze the fish target detection results to obtain fish tracking trajectories;

[0153] A feature analysis module 240 is used to analyze the fish tracking trajectory to obtain fish movement characteristics;

[0154] A behavior recognition module 250 is used to input the fish movement characteristics into a behavior recognition model to obtain a fish behavior recognition result;

[0155] Among them, the target detection model is trained based on fish motion image samples and the fish target detection results corresponding to the fish motion image samples as labels; the behavior recognition model is trained based on fish motion feature samples and the fish behavior recognition results corresponding to the fish motion feature samples.

[0156] A second embodiment of the present invention provides a fish behavior recognition device. This device captures fish motion images and inputs them into a target detection model to obtain fish target detection results, thereby identifying individual fish targets from the raw data of the fish motion images. The fish target detection results are analyzed to obtain fish tracking trajectories, i.e., the fish's motion trajectory in its current state. The fish tracking trajectories are further analyzed, and fish motion features are extracted from the fish tracking trajectories through computational quantification to reflect the various motion data of the fish in its current state. The fish motion features are then input into a behavior recognition model, and a comprehensive analysis is performed based on the various fish motion features to obtain fish behavior recognition results, i.e., the fish's behavior in its current state, providing an effective reference for fish husbandry or diagnosis. The target detection model is trained using fish motion image samples and the corresponding fish target detection results as labels; the behavior recognition model is trained using fish motion feature samples and the corresponding fish behavior recognition results. Based on machine vision and deep learning, this invention constructs a model framework that integrates motion feature extraction and behavioral state recognition, effectively reducing model size and inference time while ensuring accuracy. The method solves the defects of complex parameters and difficult identification in the existing fish behavior recognition technology, and improves the accuracy and efficiency of fish behavior recognition.

[0157] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the fish behavior recognition method, which includes:

[0158] Acquire images of fish movement;

[0159] Inputting the fish motion image into a target detection model to obtain a fish target detection result;

[0160] Analyzing the fish target detection result to obtain the fish tracking trajectory;

[0161] Analyzing the fish tracking trajectory to obtain fish movement characteristics;

[0162] Inputting the fish movement characteristics into a behavior recognition model to obtain a fish behavior recognition result;

[0163] Among them, the target detection model is trained based on fish motion image samples and the fish target detection results corresponding to the fish motion image samples as labels; the behavior recognition model is trained based on fish motion feature samples and the fish behavior recognition results corresponding to the fish motion feature samples.

[0164] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0165] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the fish behavior recognition method provided by the above methods, which includes:

[0166] Acquire images of fish movement;

[0167] Inputting the fish motion image into a target detection model to obtain a fish target detection result;

[0168] Analyzing the fish target detection result to obtain the fish tracking trajectory;

[0169] Analyzing the fish tracking trajectory to obtain fish movement characteristics;

[0170] Inputting the fish movement characteristics into a behavior recognition model to obtain a fish behavior recognition result;

[0171] Among them, the target detection model is trained based on fish motion image samples and the fish target detection results corresponding to the fish motion image samples as labels; the behavior recognition model is trained based on fish motion feature samples and the fish behavior recognition results corresponding to the fish motion feature samples.

[0172] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the fish behavior recognition method provided by the above methods, the method comprising:

[0173] Acquire images of fish movement;

[0174] Inputting the fish motion image into a target detection model to obtain a fish target detection result;

[0175] Analyzing the fish target detection result to obtain the fish tracking trajectory;

[0176] Analyzing the fish tracking trajectory to obtain fish movement characteristics;

[0177] Inputting the fish movement characteristics into a behavior recognition model to obtain a fish behavior recognition result;

[0178] Among them, the target detection model is trained based on fish motion image samples and the fish target detection results corresponding to the fish motion image samples as labels; the behavior recognition model is trained based on fish motion feature samples and the fish behavior recognition results corresponding to the fish motion feature samples.

[0179] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0180] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0181] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A fish behavior recognition method, characterized in that: include: Acquire images of fish movement; Inputting the fish motion image into a target detection model to obtain a fish target detection result; Analyzing the fish target detection results to obtain fish tracking trajectories includes: Construct Kalman filter and extended Kalman filter to obtain fish tracking trajectory; Analyzing the fish tracking trajectory to obtain fish movement characteristics; Inputting the fish movement characteristics into a behavior recognition model to obtain a fish behavior recognition result; The target detection model is trained based on fish motion image samples and the fish target detection results corresponding to the fish motion image samples as labels; the behavior recognition model is trained based on fish motion feature samples and the fish behavior recognition results corresponding to the fish motion feature samples; The target detection model is trained based on the LDW-YOLOv8 model, which is obtained based on the following method: The four downsampling steps in the backbone network of the YOLOv8 model were reduced to three, while one upsampling step and feature fusion module in the neck network were reduced. The detection head was redesigned, and the large object detection head with the smallest feature map layer was removed. Replace the YOLOv8 model's backbone network downsampling convolution with full-dimensional dynamic convolution; The LDW-YOLOv8 model is obtained by replacing the bounding box loss function CIoU_Loss of the YOLOv8 model with Wise-IoUv3 Loss.

2. The fish behavior recognition method according to claim 1, characterized in that: The training steps of the target detection model include: Acquiring fish motion image samples and performing data enhancement processing on the fish motion image samples; Marking the fish target detection result corresponding to the fish motion image sample as a label; Dividing the fish motion image samples into a fish motion image sample training set, a fish motion image sample verification set, and a fish motion image sample test set; The target detection model is obtained by training the fish motion image sample training set, the fish target detection result of the target detection model is verified by the fish motion image sample verification set, and after obtaining the network weight of the target detection model, the target detection model is tested by the fish motion image sample test set.

3. The fish behavior recognition method according to claim 1, characterized in that: The analyzing the fish target detection result to obtain the fish tracking trajectory includes: Divide the fish target detection result into a high-score detection frame and a low-score detection frame according to a confidence threshold in the target detection result, put the high-score detection frame into a high-score detection frame set, and put the low-score detection frame into a low-score detection frame set; Performing trajectory matching on the high-scoring detection frame set, placing unsuccessfully matched trajectories into a first remaining trajectory set, and placing unsuccessfully matched high-scoring detection frames in the high-scoring detection frame set into a remaining detection frame set; Matching the low-scoring detection frame set with the first remaining trajectory set, placing unsuccessfully matched trajectories into a second remaining trajectory set, and deleting the unsuccessfully matched low-scoring detection frame set from the low-scoring detection frame set; Update the Kalman filter of the successfully matched trajectory; Generate a new track based on the high-scoring detection frames in the remaining detection frame set whose confidence is greater than the tracking score threshold, and delete tracks longer than 30 frames in the second remaining track set; A Kalman filter and an extended Kalman filter are constructed, and the new position of the trajectory with acceleration less than the acceleration threshold is predicted by the Kalman filter, and the new position of the trajectory with acceleration greater than the acceleration threshold is predicted by the extended Kalman filter to obtain the fish tracking trajectory.

4. The fish behavior recognition method according to claim 3, characterized in that: Before analyzing the fish tracking trajectory, the method further includes: performing Gaussian interpolation processing on the fish tracking trajectory, the specific steps of which include: The observation value of the i-th track in the x-frame of the fish tracking track is calculated as follows: for: in, is the coordinate variable of trajectory i at the x frame, The mean is 0 and the variance is Gaussian distribution, , , is a hyperparameter; Through known frames and the frame to be predicted Compute the covariance matrix: The best estimate of ,in, for the reason The n×1 dimensional matrix composed of By setting hyperparameters Represents the maximum number of frame intervals for performing Gaussian interpolation. Gaussian interpolation is performed in the case of .

5. The fish behavior recognition method according to claim 1, characterized in that: The fish movement characteristics include swimming speed, swimming acceleration and discreteness. In the process of analyzing the fish tracking trajectory, the image is corrected through inverse perspective transformation.

6. The fish behavior recognition method according to any one of claims 1 to 5, characterized in that: The training steps of the behavior recognition model include: Obtain samples of fish movement characteristics; Marking the fish behavior recognition result corresponding to the fish movement feature sample as a label; Dividing the fish motion feature samples into a fish motion feature sample training set and a fish motion feature sample test set; A behavior recognition model is obtained by training the fish motion feature sample training set, and the behavior recognition model is evaluated by using the fish motion feature sample test set.

7. A fish behavior recognition device, characterized in that: The method for performing the fish behavior recognition method according to claim 1 comprises: An image acquisition module, used for acquiring fish movement images; A target detection module is used to input the fish motion image into a target detection model to obtain a fish target detection result; A trajectory analysis module is used to analyze the fish target detection results to obtain fish tracking trajectories; A feature analysis module is used to analyze the fish tracking trajectory to obtain fish movement characteristics; A behavior recognition module is used to input the fish movement characteristics into a behavior recognition model to obtain fish behavior recognition results; Among them, the target detection model is trained based on fish motion image samples and the fish target detection results corresponding to the fish motion image samples as labels; the behavior recognition model is trained based on fish motion feature samples and the fish behavior recognition results corresponding to the fish motion feature samples.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the fish behavior recognition method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the fish behavior recognition method according to any one of claims 1 to 6 is implemented.

Images