A method for detecting vitality status of fish products

Abstract

The present invention relates to the technical field of aquaculture, and in particular to a method for detecting the vitality status of fish products, comprising obtaining an image data set of a school of fish through a detection device, preprocessing the image data set, inputting the preprocessed image data set into a YOLOv8 network, generating a target detection frame and a detection score corresponding to the target detection frame, processing the historical trajectory of the image data set through a Kalman filter estimation algorithm, generating a predicted trajectory of the fish, ByteTrack judging the similarity between the target detection frame and the predicted trajectory, matching the target detection frame with the predicted trajectory, and obtaining the position information of the school of fish; based on the position information of the school of fish, establishing motion information parameters of the school of fish, and inputting the motion information parameters into an ANNS model, and the ANNS model outputting the activity of the school of fish. The present invention can provide the detection stability and accuracy of the activity of fish products, and reduce the influence of the environment on the detection process.

Description

Technical Field

[0001] The invention relates to the technical field of aquaculture, and in particular to a method for detecting the vitality state of fish products. Background Art

[0002] As an important way of food production, aquaculture has developed rapidly. According to data from the Food and Agriculture Organization (FAO) (2022), the average annual output of aquaculture has continued to grow by about 2.95%, making an important contribution to meeting people's demand for seafood and aquatic products. The output of China's aquaculture industry has continued to grow in the past few decades, including marine aquaculture, freshwater aquaculture and fishing. To this end, the country has increased its investment in scientific research in the field of aquaculture, supported scientific research and technological innovation, improved the scientific and technological content and competitiveness of the aquaculture industry, and promoted the development and upgrading of the aquaculture industry through the transformation and application of scientific research results.

[0003] In aquaculture, the activity assessment of fish is crucial to aquaculture practices. Accurate assessment of fish activity can help farmers better control the amount of feed, improve fish growth efficiency, reduce feed costs, and thus improve the economic benefits of aquaculture. In addition, the activity status of fish is also one of the important indicators for assessing fish welfare and health status.

[0004] For a long time, the traditional measurement method has been mainly manual contact methods, such as manual weighing, measuring the size with a tape measure, or judging with the naked eye. This method is easily affected by subjective factors of aquaculture managers, such as workers' habits, experience, preferences and external environment. Moreover, the judgment process is time-consuming, labor-intensive, subjective, inconsistent, and has a high error rate, and the monitoring results are difficult to quantify. In addition, aquatic animals are often sensitive and easily stressed. This type of manual contact measurement can easily damage them and even cause disease and death in fish, affecting the healthy growth of fish. Therefore, it is of great significance to study an objective, convenient, fast, accurate, and contactless automatic monitoring technology to realize the automatic and accurate identification of fish swimming behavior, attack behavior, and feeding behavior, so that aquaculture managers can formulate better production and marketing plans in a timely manner, which can promote the healthy and sustainable development of the aquaculture industry.

[0005] With the rapid development of artificial intelligence technology, especially the application of deep learning technology, aquaculture research has made significant progress. The application of computer vision technology in aquaculture has become more and more common, especially image recognition and analysis technology based on deep learning. This technology can not only help identify the species and color of fish, but also realize functions such as fish counting and activity monitoring, providing farmers with more management methods. Among them, the activity status monitoring of fish has important guiding significance for actual aquaculture production. By analyzing the movement behavior and activity level of fish, activity monitoring technology can help farmers detect health problems and abnormal conditions of fish in a timely manner, and take timely measures to adjust and intervene, thereby ensuring aquaculture benefits and fish welfare.

[0006] At present, the mainstream activity monitoring technology is mainly based on computer vision technology and deep learning algorithms. These technologies can realize real-time monitoring and evaluation of fish behavior and movement status by processing and analyzing fish image data. For example, the target tracking algorithm can be used to track and record the predicted trajectory of fish; the deep learning algorithm can be used to identify and analyze the activity level and behavior pattern of fish. However, there are still some problems and challenges in the current activity monitoring technology. For example, in a real industrial aquaculture environment, occlusion and stacking often occur between fish, which will lead to the performance degradation of the tracking algorithm, such as data loss and ID switching. In addition, the movement behavior of fish is affected by many factors, such as environmental factors, group behavior and individual differences, which also brings certain challenges to the research and application of activity monitoring technology. To improve the stability and accuracy of activity detection, it is urgent to propose a method for detecting the vitality status of fish products. Summary of the invention

[0007] In order to solve the above problems, the present invention provides a method for detecting the vitality status of fish products. The present invention can provide stability and accuracy in detecting the activity of fish products and reduce the impact of the environment on the detection process.

[0008] To achieve the above object, the technical solution adopted by the present invention is:

[0009] A method for detecting the vitality status of fish products, comprising:

[0010] Acquire an image data set of a school of fish through a detection device, preprocess the image data set, and mark a format of the image data set, adding angle information to the format;

[0011] Inputting the preprocessed and annotated image data set into a YOLOv8 network, wherein the YOLOv8 network generates a target detection frame of a current frame and a detection score corresponding to the target detection frame according to the image data set;

[0012] Obtaining the target detection frame and the detection score through a detector of ByteTrack, classifying the target detection frame according to the detection score, and dividing the target detection frame into a high-confidence frame group and a low-confidence frame group;

[0013] Processing the historical trajectory of the image data set by a Kalman filter estimation algorithm to generate a predicted trajectory of the fish;

[0014] The similarity between the target detection frame and the predicted trajectory is determined by ByteTrack, and the target detection frame is matched with the predicted trajectory by using the Hungarian algorithm based on the similarity to obtain the position information of the school of fish;

[0015] Based on the position information of the school of fish, motion information parameters of the school of fish are established, and the motion information parameters are input into an ANNS model, and the activity of the school of fish is output through the ANNS model.

[0016] Preferably, associating the target detection frame with the predicted trajectory to obtain the position information of the school of fish includes:

[0017] Using the IoU or Re-ID feature distance as a measure of the similarity of the first match, matching the target detection frame with the predicted trajectory, retaining the predicted trajectory and the low-confidence detection frame remaining from the first match, and obtaining the result of the first match;

[0018] Based on the result of the first matching, a second matching is performed, the distance between the IoU features is used as a measure of the similarity of the second matching, the target detection frame is matched with the predicted trajectory, the predicted trajectory that is not matched to the target detection frame after the second matching is retained, the low-confidence detection frame in the low-confidence frame group that is not found to correspond to the predicted trajectory after the second matching is deleted, and the result of the second matching is obtained;

[0019] The high confidence detection frame that is not matched to the predicted trajectory is used as a new predicted trajectory, and the new predicted trajectory and the result of the second matching are saved in a file as the position information of the school of fish.

[0020] Preferably, the motion information parameters include fish school dispersion, average swimming speed, swimming acceleration and average swimming acceleration.

[0021] Preferably, the expression of the fish school discreteness is:

[0022] ;

[0023] in represents the determinant calculation of the center coordinate matrix of the position of the i-th fish target detection box, is the center coordinate of the i-th fish target detection frame, is the covariance matrix operation of the matrix.

[0024] Preferably, the average swimming speed of the fish school in the k+1 frame is expressed as:

[0025] ;

[0026] The speed of the i-th fish in the k+1 frame is expressed as:

[0027] ;

[0028] in, is the speed sum of all fishes, i is the fish number, k is the frame number, N is the total number of fishes, is the sampling interval, is the center coordinate of the target detection frame of the i-th fish in the k+1th frame, is the center coordinate of the target detection box of the i-th fish in the k-th frame.

[0029] Preferably, the expression of the swimming acceleration of the i-th fish in the k+1-th frame is:

[0030] ;

[0031] in , are the swimming speeds of the i-th fish at the K+1th frame and the kth frame respectively.

[0032] Preferably, the expression of the average swimming acceleration of the fish school in the k+1th frame is:

[0033] ;

[0034] in Represents the total acceleration of the school of fish.

[0035] Preferably, the image data set is processed by a Kalman filter estimation algorithm to generate a predicted trajectory of the fish, comprising:

[0036] Predicting the next frame of the predicted trajectory by using Kalman filtering on the historical frames of the image data set, recorded as a predicted frame;

[0037] Using IoU or Re-ID feature distance as a measure of similarity, the current frame matching the predicted frame is matched by the Hungarian algorithm;

[0038] The predicted trajectory of the school of fish is generated by combining the current frame and the historical frames.

[0039] Preferably, the preprocessing of the image data set includes unifying the size of the image data set and normalizing the image data set.

[0040] Preferably, the location information of the school of fish is in the form of coordinates.

[0041] The beneficial effects of the present invention are:

[0042] This application processes the historical trajectory of the image data set through the Kalman filter estimation algorithm to generate the predicted trajectory of the fish, identifies the fish school video data based on YOLOv8, generates the target detection frame, and matches and tracks the individual trajectories in the fish school by using ByteTrack's multi-target tracking algorithm. ByteTrack uses high-performance YOLO to perform MOT on the video by matching all detection frames without considering their confidence, and uses BYTE to perform the association between the detection frame and the trajectory. ByteTrack uses motion models and similarity indicators to effectively restore the real target and filter out background noise, thereby improving tracking accuracy and robustness. Therefore, it has a more reliable tracking effect for actual underwater environments such as mutual occlusion of high-density fish schools and bubble occlusion. Finally, ANNs are used to establish the association between trajectory motion information and fish school activity and abnormal conditions, so as to achieve effective evaluation of the overall activity and abnormal conditions of the fish school. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of a method for detecting the vitality status of fish products in a specific embodiment of the present invention;

[0044] Figure 2 A flow chart of obtaining the location information of a school of fish in a specific embodiment of the present invention;

[0045] Figure 3 The figure is a flow chart of generating predicted trajectories of fish in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] See also Figure 1-Figure 3 As shown, the present invention relates to a method for detecting the vitality status of fish products.

[0048] Example 1

[0049] refer to Figure 1 As shown, a method for detecting the vitality status of fish products comprises:

[0050] 1. Image acquisition: Use underwater cameras to capture images of the fish schools that need to be monitored in the environment, obtain video stream data of the fish schools and pre-process the images, unify the image size and annotate the images, and add angle information to the annotation format.

[0051] Preprocessing includes normalizing the image, which includes the following:

[0052] Coordinate centering: Move the coordinate origin of the image to the center of the image, which can eliminate the influence of the image position on subsequent processing.

[0053] x-shearing normalization: Perform a shearing transformation on the image in the x-axis direction to align the image in the x-axis direction.

[0054] Scaling Normalization: Scale the image to a specified size as needed, usually to make the images have the same size for subsequent processing.

[0055] Rotation Normalization: Rotate the images so that they have the same orientation or angle.

[0056] 2. Trajectory generation: predicting the next frame of the predicted trajectory by using Kalman filtering on the historical frames of the image data set, recorded as the predicted frame;

[0057] Using IoU or Re-ID feature distance as a measure of similarity, the current frame matching the predicted frame is matched by the Hungarian algorithm;

[0058] The predicted trajectory of the school of fish is generated by combining the current frame and the historical frames.

[0059] IoU is a key metric in object detection, which is used to measure the degree of overlap between the predicted bounding box and the ground truth box. It calculates the ratio of the intersection area of ​​the predicted box and the ground truth box to the union area, that is, IoU = (intersection area) / (union area).

[0060] Re-ID refers to the task of re-identifying the same target at different times, different angles or different cameras. In Re-ID, the distance between features is usually used to measure the similarity or difference between two feature vectors. Common distance measurement methods include Euclidean distance, cosine similarity, etc.

[0061] 3. Multi-target tracking: The multi-target tracking algorithm (MOT) is used to track the target in the current frame. Many tracking algorithms prioritize matching the identity of high-confidence detection frames and ignore the identity of low-confidence detection frames. This practice may lead to the loss of occluded target detection frames, which is a common phenomenon, especially in fish farms.

[0062] Therefore, the scheme adopts the ByteTrack tracking method, which matches the identities of all target detection boxes regardless of their confidence. ByteTrack uses high-performance YOLO to perform MOT on the video and uses BYTE to perform the association between target detection boxes and tracks.

[0063] ByteTrack uses motion models and similarity indicators to effectively restore the true target and filter out background noise, thereby improving tracking accuracy and robustness. BYTE retains all detection boxes and divides them into high-confidence box groups and low-confidence box groups, and then uses a Kalman filter to predict the new position in the current frame of each track. Therefore, ByteTrack has an advantage in solving the tracking challenges brought by fish schools and is superior to other algorithms.

[0064] ByteTrack is based on deep learning models, especially the tracking-by-detection paradigm, which first uses a deep learning network (such as the YOLO series) for target detection, and then uses an association algorithm (such as Kalman filtering and Hungarian algorithm) to establish associations between targets, thereby achieving target tracking.

[0065] ByteTrack first obtains the detection box and the corresponding detection score through the detector, and classifies the detection box. If the score is higher than T_high, the detection box is classified into the high confidence group. When the score is lower than T_high and higher than T_low, the detection box is classified into the low confidence group.

[0066] Step 2: Use ByteTrack to determine the similarity between the target detection box and the predicted trajectory, and use the Hungarian algorithm based on the similarity to match the target detection box with the predicted trajectory. Here, IoU or Re-ID feature distance can be used as the similarity metric. Then use the Hungarian algorithm to match based on the similarity, and retain those high-confidence detection boxes that are not matched to the trajectory and the trajectory that is not matched to the detection box.

[0067] Step 3: Use the IoU or Re-ID feature distance as a measure of the similarity of the first match, match the target detection box with the predicted trajectory, retain the predicted trajectory and low-confidence detection box remaining from the first match, and obtain the result of the first match;

[0068] Based on the result of the first matching, a second matching is performed, the distance between the IoU features is used as a measure of the similarity of the second matching, the target detection frame is matched with the predicted trajectory, the predicted trajectory that is not matched to the target detection frame after the second matching is retained, the low-confidence detection frame in the low-confidence frame group that is not found to correspond to the predicted trajectory after the second matching is deleted, and the result of the second matching is obtained;

[0069] Step 4: Save those high-confidence bounding boxes that are not matched to corresponding tracks as newly appeared tracks.

[0070] Repeat the above steps to obtain the trajectory information of the fish school, and record the positions of all fish in each frame in a file.

[0071] 4. Motion tracking: After obtaining the fish position information, it is expressed in coordinate form. On the basis of achieving fish target detection and motion tracking, by calculating the feeding behavior parameters such as fish discreteness, average swimming speed of fish schools, average swimming acceleration of fish schools and average swimming height of fish schools, and referring to the different fluctuation trends of these parameters, the activity of fish is evaluated in real time. The quantification method for fish group motion parameters is as follows:

[0072] In the process of tracking fish, the center of individual fish targets is selected for tracking, and the fish group convergence degree of the i-th frame can be expressed as

[0073] ;

[0074] in, represents the determinant calculation of the center coordinate matrix of the i-th fish target detection box position, is the covariance matrix operation of the matrix, The smaller the value, the denser the distribution of fish in the i-th frame; conversely, the fish distribution is more dispersed.

[0075] Assume that in a video of fish feeding, the center coordinates of the fish with ID i in the kth frame and the k+1th frame image are and , the interval between image frames is , then the swimming speed of the fish with ID i in the k+1th frame can be expressed as:

[0076] ;

[0077] Assuming that the total number of fish individuals in the video image is N, the average swimming speed of the fish in frame k + 1 is:

[0078] ;

[0079] Assume that in a video, a fish with ID i is tracked by the fish target trajectory at the kth frame and the k+th frame. The swimming speeds of the frames are and , then the acceleration of the fish with ID i in the k+1th frame is:

[0080] ;

[0081] Assuming that the total number of fish in the captured video image is N, the average swimming acceleration of the fish in frame k+1 is:

[0082] ;

[0083] 5. Estimation of the overall activity of fish schools: Data is collected from abnormal and normal fish schools, and then the data is labeled. The labeled data is input into the ANN model for training. This method allows the model to learn the hidden relationship between motion information parameters and activity status.

[0084] The trained ANNs model is used to evaluate new data. ANNs is used to establish corresponding models for individual and overall average motion information of fish. The motion information parameters are used as input, and activity estimation or abnormal behavior estimation is used as network output, ultimately achieving the evaluation of activity and abnormal conditions of farmed fish populations.

[0085] The ANNs model is a complex network composed of multiple simple processing units (neurons) that interact through connections with variable weights. ANNs can learn and extract features from input data, and then make predictions or classifications based on these features. They can handle complex nonlinear relationships and have strong generalization capabilities. The following is the workflow of the ANNs model:

[0086] Input data The fish school image dataset is fed into the input layer of the ANNs model.

[0087] Data propagates forward through the hidden layers of the model, with neurons in each layer receiving outputs from neurons in the previous layer and computing their own outputs.

[0088] The neurons in the output layer produce the final output of the model, which is the model's prediction or classification result for the input data.

[0089] Compare the model's output with the true label (i.e. the correct answer to the training data) and calculate the value of the loss function (such as mean squared error, cross entropy, etc.). The loss function measures the difference between the model's prediction and the true result.

[0090] According to the gradient information of the loss function, the error term (i.e., gradient) of each neuron is calculated layer by layer starting from the output layer.

[0091] Use the gradient descent algorithm (or its variants such as Adam, RMSprop, etc.) to update the weights and bias terms in the model to minimize the loss function.

[0092] Repeating the process of forward propagation, calculating loss, and backpropagation, through multiple iterative training, the model gradually learns and optimizes its weights and bias terms to improve the accuracy of prediction or classification.

[0093] Use validation sets or test sets to evaluate the performance of the trained model. These datasets are not seen during the training process and are used to verify the generalization ability of the model.

[0094] Adjust the model structure, hyperparameters, or training strategy based on the evaluation results to further optimize model performance.

[0095] Example 2

[0096] Based on Example 1, the present invention uses ByteTrack as a tracking method. ByteTrack shows strong robustness in the tracking process. It can effectively cope with complex scenes such as target occlusion, scale change, and illumination change, and maintain a stable tracking effect. This makes ByteTrack have good adaptability in various practical applications, and can well solve the current real industrial aquaculture environment. The phenomenon of occlusion and stacking often occurs between fish, which will cause the performance of the tracking algorithm to deteriorate, such as data loss and ID switching.

[0097] ByteTrack focuses on reducing computational costs while maintaining high performance. It uses efficient algorithm design and optimization strategies to enable the tracking process to run under lower hardware requirements, reducing the cost of the entire detection process.

[0098] This application processes the image data set through the Kalman filter estimation algorithm to generate the predicted trajectory of the fish, identifies the fish school video data based on YOLOv8, generates the target detection frame, and matches and tracks the individual trajectories in the fish school by using ByteTrack's multi-target tracking algorithm. ByteTrack uses high-performance YOLO to perform MOT on the video by matching all detection frames without considering their confidence, and uses BYTE to perform the association between the detection frame and the trajectory. ByteTrack uses motion models and similarity indicators to effectively restore the real target and filter out background noise, thereby improving tracking accuracy and robustness. Therefore, it has a more reliable tracking effect for actual underwater environments such as mutual occlusion of high-density fish schools and bubble occlusion. Finally, ANNs are used to establish the association between trajectory motion information and fish school activity and abnormal conditions, so as to achieve effective evaluation of the overall activity and abnormal conditions of the fish school.

[0099] The above implementation modes are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering and technical personnel in the field shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for detecting the vitality status of fish products, characterized in that: include: Acquire an image data set of a school of fish through a detection device, preprocess the image data set, and mark a format of the image data set, adding angle information to the format; Inputting the preprocessed and annotated image data set into a YOLOv8 network, wherein the YOLOv8 network generates a target detection frame of a current frame and a detection score corresponding to the target detection frame according to the image data set; Obtaining the target detection frame and the detection score through a detector of ByteTrack, classifying the target detection frame according to the detection score, and dividing the target detection frame into a high-confidence frame group and a low-confidence frame group; Processing the historical trajectory of the image data set by a Kalman filter estimation algorithm to generate a predicted trajectory of the fish; The similarity between the target detection frame and the predicted trajectory is determined by ByteTrack, and the target detection frame is matched with the predicted trajectory by using the Hungarian algorithm based on the similarity to obtain the position information of the school of fish; Based on the position information of the school of fish, motion information parameters of the school of fish are established, and the motion information parameters are input into an ANNS model, and the activity of the school of fish is output through the ANNS model.

2. The method for detecting the vitality status of fish products according to claim 1, characterized in that: The step of associating the target detection frame with the predicted trajectory to obtain the position information of the school of fish includes: Using the IoU or Re-ID feature distance as a measure of the similarity of the first match, matching the target detection frame with the predicted trajectory, retaining the predicted trajectory and the low-confidence detection frame remaining from the first match, and obtaining the result of the first match; Based on the result of the first matching, a second matching is performed, the distance between the IoU features is used as a measure of the similarity of the second matching, the target detection frame is matched with the predicted trajectory, the predicted trajectory that is not matched to the target detection frame after the second matching is retained, the low-confidence detection frame in the low-confidence frame group that is not found to correspond to the predicted trajectory after the second matching is deleted, and the result of the second matching is obtained; The high confidence detection frame that is not matched to the predicted trajectory is used as a new predicted trajectory, and the new predicted trajectory and the result of the second matching are saved in a file as the position information of the school of fish.

3. The method for detecting the vitality status of fish products according to claim 1, characterized in that: The motion information parameters include fish school dispersion, average swimming speed, swimming acceleration and average swimming acceleration.

4. The method for detecting the vitality status of fish products according to claim 3, characterized in that: The expression of the fish school dispersion is: ； in represents the determinant calculation of the center coordinate matrix of the i-th fish target detection box position, is the center coordinate of the i-th fish target detection frame, is the covariance matrix operation of the matrix.

5. The method for detecting the vitality status of fish products according to claim 3, characterized in that: The average swimming speed of the fish school in the k+1th frame is expressed as: ； The speed of the i-th fish in the k+1 frame is expressed as: ； in, is the speed sum of all fishes, i is the fish number, k is the frame number, N is the total number of fishes, is the sampling interval, is the center coordinate of the target detection box of the i-th fish in the k+1th frame, is the center coordinate of the target detection box of the i-th fish in the k-th frame.

6. The method for detecting the vitality status of fish products according to claim 3, characterized in that: The expression of the swimming acceleration of the i-th fish in the k+1-th frame is: ； in , are the swimming speeds of the i-th fish at the K+1th frame and the kth frame respectively.

7. The method for detecting the vitality status of fish products according to claim 3, characterized in that: The expression of the average swimming acceleration of the fish school in the k+1th frame is: ； in Represents the sum of the accelerations of the school of fish.

8. The method for detecting the vitality status of fish products according to claim 1, characterized in that: Processing the image data set by a Kalman filter estimation algorithm to generate a predicted trajectory of the fish includes: Predicting the next frame of the predicted trajectory by using Kalman filtering on the historical frames of the image data set, recorded as a predicted frame; Using IoU or Re-ID feature distance as a measure of similarity, the current frame matching the predicted frame is matched by the Hungarian algorithm; The predicted trajectory of the school of fish is generated by combining the current frame and the historical frames.

9. The method for detecting the vitality status of fish products according to claim 1, characterized in that: The preprocessing of the image data set includes unifying the size of the image data set and normalizing the image data set.

10. The method for detecting the vitality status of fish products according to claim 1, characterized in that: The position information of the fish school is in the form of coordinates.

Images