An end-to-end method for tracking red claw crayfish behavior trajectories

Abstract

The present invention discloses an end-to-end method for tracking the behavioral trajectory of red claw crayfish. The method comprises: obtaining and preprocessing a dataset of red claw crayfish image samples; training a preset detector using the preprocessed dataset to obtain a trained detector; filtering the output image frames of the preset detector using a preset filtering module, and constructing a tracker based on the filtering results; connecting the trained detector, the filtering module, and the tracker in series to obtain a red claw crayfish behavioral trajectory tracking model; and tracking the red claw crayfish's behavioral trajectory using the model to obtain a corresponding total distance traveled and a behavioral trajectory map. The present invention achieves high-precision and high-efficiency red claw crayfish behavioral trajectory tracking.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an end-to-end red claw crayfish behavior trajectory tracking method. Background Art

[0002] Red claw crayfish, also known as Australian freshwater crayfish, are a species of freshwater shrimp with significant economic value and widespread market demand. Their meat is firm, crisp, and deliciously sweet, similar to that of saltwater lobsters. Furthermore, they are highly adaptable and disease-resistant, tolerate low oxygen levels, have a high survival rate, and survive a wide temperature range. Consequently, they enjoy strong market demand and command very high prices in the domestic market.

[0003] However, red claw crayfish are aggressive by nature, and fighting often occurs in aquaculture ponds, increasing mortality and causing significant economic losses to farmers. Further research into red claw crayfish behavior has shown that relying solely on traditional visual inspection is insufficient, as this method is prone to visual fatigue and is inefficient.

[0004] With the continuous development of image recognition technology and the need for biological behavior monitoring, image recognition technology is being used more and more widely in biometrics. It has been applied to fish classification and individual pig tracking. The JDE type of multi-target tracking algorithm has high data annotation costs, and the model is difficult to tune and has poor generalization. The SDE type of multi-target tracking algorithm divides the model into two parts: the detector and the tracker. These two parts can be tuned separately, but require a detector model with extremely high accuracy. However, traditional target detection algorithms such as optical flow and particle filtering are prone to missed detections and false detections when processing complex scenes, and are easily affected by the lighting and angle of the shooting. In addition, the number of anchor frames generated by deep learning algorithms such as the YOLO series and Faster R-CNN based on the anchor frame method is large and redundant, which is not conducive to the construction of subsequent trackers. With the introduction of the end-to-end design concept and the increasing application of transformer models in the field of vision, a series of anchor-free and transformer-integrated models have been created, such as DETR and RTDETR, which have shown their prowess in the field of target detection. Therefore, the present invention adopts an end-to-end red claw crayfish behavior trajectory tracking method, which has the characteristics of high accuracy, high precision, and fast operation speed.

[0005] The Chinese invention patent with application number 202410362230.6 discloses "A fish tracking method based on Kalman filter", and its technical solution includes: S1. Use the target detector to detect multiple first fish targets in the current video frame and obtain a first rectangular frame corresponding to each first fish target; S2. According to the time sequence of the video frames, use the Kalman filter to estimate the position of the fish target in the current video frame based on the first rectangular frame in the previous video frame to obtain the second rectangular frame; S3. Use the Hungarian algorithm to match the first rectangular frame, the second rectangular frame in the current video frame or the rectangular frame in the last frame where the fish target appears three times according to the IoU value; S4. Update the Kalman filter according to the matching results and assign a number; S5. Process each video frame according to steps S1 to S4 until all video frames are processed, and output the tracking information of the fish target, which includes the position information and number of the fish target in each video frame. Summary of the Invention

[0006] In order to solve the technical problems of low accuracy, insufficient precision and slow operation speed of the existing multi-target tracking algorithm, the present invention provides an end-to-end red claw crayfish behavior trajectory tracking method. The technical solution adopted by the present invention is:

[0007] A first aspect of the present invention provides an end-to-end method for tracking the behavior trajectory of red claw crayfish, the method comprising:

[0008] Obtain a dataset of red claw crayfish images and perform preprocessing;

[0009] A pre-set detector is trained using the pre-processed red claw crayfish image sample dataset to obtain a trained detector;

[0010] Filtering the output image frames of the preset detector by a preset filtering module, and constructing a tracker according to the filtering results;

[0011] The trained detector, the screening module and the tracker are connected in series to obtain a red claw crayfish behavior trajectory tracking model;

[0012] The behavior trajectory of the red claw crayfish is tracked by the model to obtain the total movement distance and behavior trajectory diagram of the red claw crayfish.

[0013] As a preferred solution, the method for obtaining and preprocessing a redclaw crayfish image sample dataset includes:

[0014] Use an image sensor to collect images of red claw crayfish in each scene to obtain a red claw crayfish image sample dataset;

[0015] The red claw crayfish image sample dataset is uniformly cropped and scaled to a preset resolution;

[0016] The red claw crayfish in the image is annotated using a preset image annotation tool to obtain a red claw crayfish image sample dataset containing annotation box information;

[0017] The red claw crayfish image sample dataset is divided into a training set, a validation set and a test set according to a preset ratio.

[0018] As a preferred solution, a method for annotating red claw crayfish in an image using a preset image annotation tool to obtain a red claw crayfish image sample dataset containing annotation box information includes:

[0019] The red claw crayfish in the image was annotated through the Makesense website to obtain a red claw crayfish image sample dataset containing annotation box information.

[0020] As a preferred solution, the method of dividing the red claw crayfish image sample dataset into a training set, a validation set, and a test set according to a preset ratio includes:

[0021] The red claw crayfish image sample dataset is divided into training set, validation set and test set in the ratio of 7:2:1.

[0022] As a preferred solution, a method for training a preset detector using a preprocessed red claw crayfish image sample dataset to obtain a trained detector includes:

[0023] The training set is input into the preset detector for training, the performance of the detector is evaluated using the validation set, and the detector with the best performance is retained to obtain a trained detector.

[0024] As a preferred solution, the method of filtering the output image frames of the preset detector by a preset filtering module and constructing a tracker according to the filtering results includes:

[0025] Set a confidence threshold, and use the detector output image frame with a confidence greater than the threshold as the tracker input image frame;

[0026] Initialize the number of currently tracked individuals and a dictionary containing the positions of different individuals in each frame based on the number of red claw crayfish individuals in the tracker input image frame. The key of the dictionary is the ID of each red claw crayfish, and the value is its position information.

[0027] In the subsequent frame detection process, the dictionary is updated in real time, specifically:

[0028] If the number of red claw crayfish in the current tracker input image frame is greater than that in the previous frame, a new ID is assigned to the newly detected red claw crayfish and its position information is set to None;

[0029] Traverse the dictionary and assign the key-value pairs with location information None to the unintended dictionary, and the rest to the intended dictionary;

[0030] Traversing the intentional dictionary, performing Kalman filter prediction on all historical frame position information of each red claw crayfish ID, and obtaining predicted position information of each ID in the current tracker input image frame;

[0031] Match the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame, and update the position information of the corresponding ID in the dictionary according to the actual position information of the red claw crayfish that has been successfully matched;

[0032] The actual location information of unmatched red claw crayfish is assigned to the ID in the unintentional dictionary and moved into the intentional dictionary.

[0033] As a preferred solution, the method of matching the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame includes:

[0034] The predicted position information of each ID is respectively compared with the actual position information of each red claw crayfish in the current tracker input image frame for IOU calculation. If the calculation result is greater than the preset threshold, the match is successful. If the calculation result is less than the preset threshold, the match fails, and the ID that fails to match is moved from the intentional dictionary to the unintentional dictionary.

[0035] As a preferred solution, the method of tracking the behavior trajectory of the red claw crayfish using the model to obtain the total movement distance and behavior trajectory map corresponding to the red claw crayfish includes:

[0036] Collect videos of red claw crayfish of preset duration;

[0037] Inputting the red claw crayfish video into a red claw crayfish behavior trajectory tracking model;

[0038] Take out the dictionary containing the position information in the tracker, divide it into a series of detection frame coordinates according to different IDs, and take the midpoint of the detection frame coordinates as the centroid coordinates;

[0039] The trajectory graph is drawn using the data visualization library based on the centroid coordinates, and the Euclidean distance between the centroid coordinates of each two frames is calculated. Finally, all Euclidean distances are summed to obtain the total distance traveled by a single red claw crayfish in the time period. The specific formula is as follows:

[0040]

[0041] Where X represents the total distance traveled by a single red claw crayfish during this time period, x i Represents the horizontal coordinate of the center of mass of the red claw crayfish at the i-th frame, yi represents the vertical coordinate of the center of mass of the red claw crayfish at the i-th frame, and n represents the total number of frames of the processed video.

[0042] A second aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the steps of the aforementioned end-to-end red claw crayfish behavior trajectory tracking method.

[0043] A third aspect of the present invention provides a computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor. When the computer program is executed by the processor, the steps of the aforementioned end-to-end red claw crayfish behavior trajectory tracking method are implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] This method feeds video frames into an efficient end-to-end object detector. The detector output is then fed into a tracker based on IoU calculation and Kalman filtering. This generates a series of center-of-mass coordinates of the red claw crayfish within the image. This is then rendered into a motion trajectory map and the total distance traveled is calculated. This method boasts high accuracy and high speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of an end-to-end red claw crayfish behavior trajectory tracking method provided in this embodiment. DETAILED DESCRIPTION

[0047] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting the present invention;

[0048] It should be clear that the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the embodiments of the present application.

[0049] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the embodiments of the present application. The singular forms "a," "the," and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0050] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0051] In addition, in the description of this application, unless otherwise specified, "plurality" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship. The present invention is further described below with reference to the accompanying drawings and examples.

[0052] The present invention is further described below with reference to the accompanying drawings and embodiments.

[0053] Example 1

[0054] Please refer to Figure 1 This embodiment provides an end-to-end method for tracking the behavior trajectory of red claw crayfish, the method comprising:

[0055] S1: Obtain a dataset of red claw crayfish images and perform preprocessing;

[0056] In a specific embodiment, a method for obtaining and preprocessing a redclaw crayfish image sample dataset includes:

[0057] Use an image sensor to collect images of red claw crayfish in each scene to obtain a red claw crayfish image sample dataset;

[0058] Specifically, several red claw crayfish were placed in a water tank, and a camera was used to shoot downward from the top of the tank to collect a series of pictures.

[0059] The red claw crayfish image sample dataset is uniformly cropped and scaled to a preset resolution;

[0060] Specifically, the images are compressed to the same size, specifically 640*640 pixels.

[0061] The red claw crayfish in the image is annotated using a preset image annotation tool to obtain a red claw crayfish image sample dataset containing annotation box information;

[0062] The red claw crayfish image sample dataset is divided into a training set, a validation set and a test set according to a preset ratio.

[0063] In a specific embodiment, a method for annotating red claw crayfish in an image using a preset image annotation tool to obtain a red claw crayfish image sample dataset containing annotation box information includes:

[0064] The red claw crayfish in the image was annotated through the Makesense website to obtain a red claw crayfish image sample dataset containing annotation box information.

[0065] In a specific embodiment, the method of dividing a red claw crayfish image sample dataset into a training set, a validation set, and a test set according to a preset ratio includes:

[0066] The red claw crayfish image sample dataset is divided into training set, validation set and test set in the ratio of 7:2:1.

[0067] S2: training a preset detector using the preprocessed red claw crayfish image sample dataset to obtain a trained detector;

[0068] Specifically, a detector is built based on the RTDETR model in the PaddleDetection target detection framework of PaddlePaddle, model training is performed, and the trained model is saved in onnx format.

[0069] In a specific embodiment, a method for training a preset detector using a preprocessed redclaw crayfish image sample dataset to obtain a trained detector includes:

[0070] The training set is input into the preset detector for training, the performance of the detector is evaluated using the validation set, and the detector with the best performance is retained to obtain a trained detector.

[0071] S3: filtering the output image frames of the preset detector through a preset filtering module, and constructing a tracker according to the filtering results;

[0072] In a specific embodiment, the output image frames of the preset detector are filtered by a preset filtering module, and the method for constructing a tracker according to the filtering results includes:

[0073] Set a confidence threshold, and use the detector output image frame with a confidence greater than the threshold as the tracker input image frame;

[0074] Specifically, the threshold is set to 0.9.

[0075] Initialize the number of currently tracked individuals and a dictionary containing the positions of different individuals in each frame based on the number of red claw crayfish individuals in the tracker input image frame. The key of the dictionary is the ID of each red claw crayfish, and the value is its position information.

[0076] In the subsequent frame detection process, the dictionary is updated in real time, specifically:

[0077] If the number of red claw crayfish in the current tracker input image frame is greater than that in the previous frame, a new ID is assigned to the newly detected red claw crayfish and its position information is set to None;

[0078] Traverse the dictionary and assign the key-value pairs with location information None to the unintended dictionary, and the rest to the intended dictionary;

[0079] Traversing the intentional dictionary, performing Kalman filter prediction on all historical frame position information of each red claw crayfish ID, and obtaining predicted position information of each ID in the current tracker input image frame;

[0080] Match the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame, and update the position information of the corresponding ID in the dictionary according to the actual position information of the red claw crayfish that has been successfully matched;

[0081] The actual location information of unmatched red claw crayfish is assigned to the ID in the unintentional dictionary and moved into the intentional dictionary.

[0082] In a specific embodiment, the method of matching the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame includes:

[0083] The predicted position information of each ID is calculated with the actual position information of each red claw crayfish in the current tracker input image frame. If the calculation result is greater than 0.9, the match is successful. If the calculation result is less than 0.9, the match fails, and the ID that fails to match is moved from the intentional dictionary to the unintentional dictionary.

[0084] S4: connecting the trained detector, the screening module, and the tracker in series to obtain a red claw crayfish behavior trajectory tracking model;

[0085] S5: Tracking the behavior trajectory of the red claw crayfish using the model to obtain a total movement distance and a behavior trajectory map corresponding to the red claw crayfish;

[0086] In a specific embodiment, the method of tracking the behavior trajectory of the red claw crayfish using the model to obtain the total movement distance and behavior trajectory map corresponding to the red claw crayfish includes:

[0087] Collect videos of red claw crayfish of preset duration;

[0088] Specifically, place the crayfish in a flat water tank with the water in the tank covering the crayfish and 2 cm higher. The camera needs to shoot from directly above the water tank, and the field of view needs to include the edge of the water tank.

[0089] Inputting the red claw crayfish video into a red claw crayfish behavior trajectory tracking model;

[0090] Take out the dictionary containing the position information in the tracker, divide it into a series of detection frame coordinates according to different IDs, and take the midpoint of the detection frame coordinates as the centroid coordinates;

[0091] The trajectory graph is drawn using the data visualization library based on the centroid coordinates, and the Euclidean distance between the centroid coordinates of each two frames is calculated. Finally, all Euclidean distances are summed to obtain the total distance traveled by a single red claw crayfish in the time period. The specific formula is as follows:

[0092]

[0093] Where X represents the total distance traveled by a single red claw crayfish during this time period, x i Represents the horizontal coordinate of the center of mass of the red claw crayfish at the i-th frame, y i represents the vertical coordinate of the center of mass of the red claw crayfish at the i-th frame, and n represents the total number of frames of the processed video.

[0094] Example 2

[0095] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of an end-to-end red claw crayfish behavior trajectory tracking method described in Example 1.

[0096] Example 3

[0097] A computer device includes a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, wherein when the computer program is executed by the processor, the steps of the end-to-end red claw crayfish behavior trajectory tracking method described in Example 1 are implemented.

[0098] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for tracking the behavior trajectory of red claw crayfish based on end-to-end, characterized in that: The method comprises: Obtain a dataset of red claw crayfish images and perform preprocessing; A pre-set detector is trained using the pre-processed red claw crayfish image sample dataset to obtain a trained detector; Filtering the output image frames of the preset detector by a preset filtering module, and constructing a tracker according to the filtering results; The trained detector, the screening module and the tracker are connected in series to obtain a red claw crayfish behavior trajectory tracking model; The behavior trajectory of the red claw crayfish is tracked by the model to obtain the total movement distance and behavior trajectory diagram corresponding to the red claw crayfish; The method of filtering the output image frames of the preset detector by a preset filtering module and constructing a tracker according to the filtering results includes: Set a confidence threshold, and use the detector output image frame with a confidence greater than the threshold as the tracker input image frame; Initialize the number of currently tracked individuals and a dictionary containing the positions of different individuals in each frame based on the number of red claw crayfish individuals in the tracker input image frame. The key of the dictionary is the ID of each red claw crayfish, and the value is its position information. In the subsequent frame detection process, the dictionary is updated in real time, specifically: If the number of red claw crayfish in the current tracker input image frame is greater than that in the previous frame, a new ID is assigned to the newly detected red claw crayfish and its position information is set to None; Traverse the dictionary and assign the key-value pairs with location information None to the unintended dictionary, and the rest to the intended dictionary; Traversing the intentional dictionary, performing Kalman filter prediction on all historical frame position information of each red claw crayfish ID, and obtaining predicted position information of each ID in the current tracker input image frame; Match the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame, and update the position information of the corresponding ID in the dictionary according to the actual position information of the red claw crayfish that has been successfully matched; The actual location information of unmatched red claw crayfish is assigned to the ID in the unintentional dictionary and moved into the intentional dictionary.

2. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 1, characterized in that: The method for obtaining a red claw crayfish image sample dataset and performing preprocessing includes: Use an image sensor to collect images of red claw crayfish in each scene to obtain a red claw crayfish image sample dataset; The red claw crayfish image sample dataset is uniformly cropped and scaled to a preset resolution; The red claw crayfish in the image is annotated using a preset image annotation tool to obtain a red claw crayfish image sample dataset containing annotation box information; The red claw crayfish image sample dataset is divided into a training set, a validation set and a test set according to a preset ratio.

3. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 2 is characterized in that: The method for labeling red claw crayfish in an image using a preset image labeling tool to obtain a red claw crayfish image sample dataset containing labeling box information includes: The red claw crayfish in the image was annotated through the Makesense website to obtain a red claw crayfish image sample dataset containing annotation box information.

4. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 2, characterized in that: The method of dividing the red claw crayfish image sample dataset into a training set, a validation set, and a test set according to a preset ratio includes: The red claw crayfish image sample dataset is divided into training set, validation set and test set in the ratio of 7:2:

1.

5. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 2, characterized in that: The method of training a preset detector using the preprocessed red claw crayfish image sample dataset to obtain a trained detector includes: The training set is input into the preset detector for training, the performance of the detector is evaluated using the validation set, and the detector with the best performance is retained to obtain a trained detector.

6. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 1, characterized in that: The method of matching the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame includes: The predicted position information of each ID is respectively compared with the actual position information of each red claw crayfish in the current tracker input image frame for IOU calculation. If the calculation result is greater than the preset threshold, the match is successful. If the calculation result is less than the preset threshold, the match fails, and the ID that fails to match is moved from the intentional dictionary to the unintentional dictionary.

7. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 1, characterized in that: The method for tracking the behavior trajectory of the red claw crayfish by using the model to obtain the total movement distance and behavior trajectory map corresponding to the red claw crayfish includes: Collect videos of red claw crayfish of preset duration; Inputting the red claw crayfish video into a red claw crayfish behavior trajectory tracking model; Take out the dictionary containing the position information in the tracker, divide it into a series of detection frame coordinates according to different IDs, and take the midpoint of the detection frame coordinates as the centroid coordinates; The trajectory graph is drawn using the data visualization library based on the centroid coordinates, and the Euclidean distance between the centroid coordinates of each two frames is calculated. Finally, all Euclidean distances are summed to obtain the total distance traveled by a single red claw crayfish in the time period. The specific formula is as follows: Where X represents the total distance traveled by a single red claw crayfish during this time period. represents the horizontal coordinate of the center of mass of the red claw crayfish at the i-th frame, represents the vertical coordinate of the center of mass of the red claw crayfish at the i-th frame, and n represents the total number of frames of the processed video.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the end-to-end red claw crayfish behavior trajectory tracking method according to any one of claims 1 to 7 are implemented.

9. A computer device, characterized in that: The method comprises a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, wherein when the computer program is executed by the processor, the steps of the end-to-end red claw crayfish behavior trajectory tracking method according to any one of claims 1 to 7 are implemented.

Images