End-to-end-based cherax quadricarinatus behavior trajectory tracking method

Abstract

The invention discloses an end-to-end cherax quadricarinatus behavior trajectory tracking method. The method comprises the following steps: acquiring a cherax quadricarinatus image sample data set and performing preprocessing; training a preset detector through the preprocessed cherax quadricarinatus image sample data set to obtain a trained detector; screening the output image frames of the preset detector through a preset screening module, and constructing a tracker according to a screening result; the trained detector, the screening module and the tracker are connected in series and bridged, and a cherax quadricarinatus behavior trajectory tracking model is obtained; and performing behavior trajectory tracking on the cherax quadricarinatus through the model to obtain a total movement distance and a behavior trajectory chart corresponding to the cherax quadricarinatus. The cherax quadricarinatus behavior trajectory tracking method realizes high-precision and high-efficiency cherax quadricarinatus behavior trajectory tracking.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an end-to-end red claw crayfish behavior trajectory tracking method. Background Art

[0002] Red claw crayfish, also known as Australian freshwater crayfish, is a freshwater shrimp with important economic value and wide market demand. The meat of red claw crayfish is firm, smooth and crisp, and tastes delicious and sweet, similar to the meat of sea lobster. In addition, red claw crayfish has strong adaptability and disease resistance, is resistant to low oxygen, has a high survival rate, and survives in a wide temperature range. Therefore, the market demand is strong and it can be sold at a very good price in the domestic market.

[0003] However, red claw crayfish are aggressive by nature and often fight with each other in the breeding pond, which increases the mortality rate and causes considerable economic losses to farmers. With further research on the behavior of red claw crayfish, it is difficult to meet the needs by relying solely on traditional human eye inspection. This traditional method easily causes visual fatigue and is inefficient.

[0004] With the continuous development of image recognition technology and the need for biological behavior monitoring, image recognition technology is increasingly widely used in biometrics. It has been applied to fish classification and individual pig tracking. The JDE type of multi-target tracking algorithm has high data annotation costs, and the model is not easy to tune and has poor generalization. The SDE type of multi-target tracking algorithm divides the model into two parts: the detector and the tracker. The two parts can be tuned separately, but a detector model with particularly high accuracy is required. However, traditional target detection algorithms such as optical flow and particle filtering are prone to missed detection and false detection when dealing with complex scenes, and are easily affected by the light and angle of the shooting; and the number of anchor frames generated by the YOLO series algorithm based on the anchor frame method in deep learning, the Faster R-CNN algorithm, etc. is huge and redundant, which is not conducive to the construction of subsequent trackers. With the proposal of the end-to-end design concept and the increasing application of transformer models in the field of vision, a series of anchor-free and transformer-combined models have been created, such as DETR and RTDETR, which have been very effective in the field of target detection. Therefore, the present invention adopts an end-to-end red claw crayfish behavior trajectory tracking method, which has the characteristics of high accuracy, high precision and fast running speed.

[0005] The Chinese invention patent with application number 202410362230.6 discloses "A fish tracking method based on Kalman filter", and its technical solution includes: S1. Use the target detector to detect multiple first fish targets in the current video frame, and obtain the first rectangular frame corresponding to each first fish target; S2. According to the time sequence of the video frames, use the Kalman filter to estimate the position of the fish target in the current video frame according to the first rectangular frame in the previous video frame to obtain the second rectangular frame; S3. Use the Hungarian algorithm to match the first rectangular frame, the second rectangular frame in the current video frame or the rectangular frame of the fish target in the last frame where it appears three times according to the IoU value; S4. Update the Kalman filter according to the matching results and assign a number; S5. Process each video frame according to steps S1 to S4 until all video frames are processed, and output the tracking information of the fish target, and the tracking information includes the position information and number of the fish target in each video frame. Summary of the invention

[0006] In order to solve the technical problems of low accuracy, insufficient precision and slow running speed of the prior art multi-target tracking algorithm, the present invention provides an end-to-end red claw crayfish behavior trajectory tracking method. The technical solution adopted by the present invention is:

[0007] The first aspect of the present invention provides an end-to-end red claw crayfish behavior trajectory tracking method, the method comprising:

[0008] Obtain a sample dataset of red claw crayfish images and perform preprocessing;

[0009] The preset detector is trained by using the preprocessed red claw crayfish image sample data set to obtain a trained detector;

[0010] The output image frames of the preset detector are screened by a preset screening module, and a tracker is constructed according to the screening result;

[0011] The trained detector, the screening module and the tracker are connected in series to obtain a red claw crayfish behavior trajectory tracking model;

[0012] The behavior trajectory of the red claw crayfish is tracked by the model to obtain the total moving distance and behavior trajectory diagram of the red claw crayfish.

[0013] As a preferred solution, the method for obtaining and preprocessing a red claw crayfish image sample dataset includes:

[0014] Using an image sensor to collect images of red claw crayfish in each scene, and obtaining a red claw crayfish image sample dataset;

[0015] The red claw crayfish image sample dataset is uniformly cropped and scaled to a preset resolution;

[0016] Annotate the red claw crayfish in the image using a preset image annotation tool to obtain a red claw crayfish image sample dataset containing annotation box information;

[0017] The red claw crayfish image sample dataset is divided into a training set, a validation set and a test set according to a preset ratio.

[0018] As a preferred solution, the method of annotating the red claw crayfish in the image by using a preset image annotation tool to obtain a red claw crayfish image sample data set containing annotation box information includes:

[0019] The red claw crayfish in the image is labeled through the makesense website to obtain a red claw crayfish image sample dataset containing the annotation box information.

[0020] As a preferred solution, the method of dividing the red claw crayfish image sample dataset into a training set, a validation set and a test set according to a preset ratio includes:

[0021] The red claw crayfish image sample dataset is divided into training set, validation set and test set in the ratio of 7:2:1.

[0022] As a preferred solution, a method for training a preset detector by using a preprocessed red claw crayfish image sample data set to obtain a trained detector includes:

[0023] The training set is input into the preset detector for training, the performance of the detector is evaluated through the verification set, the detector with the best performance is retained, and a trained detector is obtained.

[0024] As a preferred solution, the output image frame of the preset detector is screened by a preset screening module, and the method for constructing a tracker according to the screening result includes:

[0025] A confidence threshold is set, and the detector output image frames with confidence greater than the threshold are used as the tracker input image frames;

[0026] Initialize the number of currently tracked individuals and a dictionary containing the positions of different individuals in each frame according to the number of red claw crayfish individuals in the input image frame of the tracker. The key of the dictionary is the ID of each red claw crayfish, and the value is its position information.

[0027] In the subsequent frame detection process, the dictionary is updated in real time, specifically:

[0028] If the number of red claw crayfish in the current tracker input image frame is greater than that in the previous frame, a new ID is assigned to the newly detected red claw crayfish, and its position information is set to None;

[0029] Traverse the dictionary and classify the key-value pairs with location information None into the unintended dictionary, and the rest into the intended dictionary;

[0030] Traversing the intentional dictionary, performing Kalman filter prediction on all historical frame position information of each red claw crayfish ID, and obtaining predicted position information of each ID in the current tracker input image frame;

[0031] Match the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame, and update the position information of the corresponding ID in the dictionary according to the actual position information of the red claw crayfish that has been successfully matched;

[0032] The actual location information of unmatched red claw crayfish is assigned to the ID in the unintentional dictionary and moved into the intentional dictionary.

[0033] As a preferred solution, the method of matching the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame includes:

[0034] The predicted position information of each ID is respectively calculated by IOU with the actual position information of each red claw crayfish in the current tracker input image frame. If the calculated result is greater than the preset threshold, the match is successful. If the calculated result is less than the preset threshold, the match fails, and the ID that fails to match is moved from the intentional dictionary to the unintentional dictionary.

[0035] As a preferred solution, the method of tracking the behavior trajectory of the red claw crayfish by using the model to obtain the total movement distance and behavior trajectory map corresponding to the red claw crayfish includes:

[0036] Collect red claw crayfish videos of preset duration;

[0037] Inputting the red claw crayfish video into a red claw crayfish behavior trajectory tracking model;

[0038] Take out the dictionary containing the position information in the tracker, divide it into a series of detection frame coordinates according to different IDs, and take the midpoint of the detection frame coordinates as the centroid coordinates;

[0039] According to the centroid coordinates, a trajectory diagram is drawn through a data visualization library, and the centroid coordinates between every two frames are used to calculate the Euclidean distance. Finally, all Euclidean distances are summed to obtain the total distance traveled by a single red claw crayfish in the time period. The specific formula is as follows:

[0040]

[0041] Where X represents the total distance traveled by a single red claw crayfish during this period of time, x i represents the horizontal coordinate of the center of mass of the red claw crayfish at the i-th frame, yi represents the ordinate of the center of mass of the red claw crayfish at the i-th frame, and n represents the total number of frames of the processed video.

[0042] A second aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the aforementioned end-to-end red claw crayfish behavior trajectory tracking method are implemented.

[0043] A third aspect of the present invention provides a computer device, comprising a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the aforementioned end-to-end red claw crayfish behavior trajectory tracking method.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention inputs video frames into an efficient end-to-end target detector, and the output of the detector is then input into a tracker based on IOU calculation and Kalman filtering principle to generate a series of centroid coordinates of red claw crayfish in the image, and finally renders a motion trajectory map and calculates the total motion distance. The method of the present invention has the characteristics of high accuracy and fast running speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flow chart of an end-to-end red claw crayfish behavior trajectory tracking method provided in this embodiment. DETAILED DESCRIPTION

[0047] The accompanying drawings are only used for illustrative purposes and are not to be construed as limiting the present invention;

[0048] It should be clear that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the embodiments of the present application.

[0049] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0050] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0051] In addition, in the description of this application, unless otherwise specified, "multiple" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0052] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0053] Example 1

[0054] Please refer to Figure 1 This embodiment provides an end-to-end red claw crayfish behavior trajectory tracking method, the method comprising:

[0055] S1: Obtain a dataset of red claw crayfish images and perform preprocessing;

[0056] In a specific embodiment, the method of obtaining and preprocessing a red claw crayfish image sample dataset includes:

[0057] Using an image sensor to collect images of red claw crayfish in each scene, and obtaining a red claw crayfish image sample dataset;

[0058] Specifically, several red claw crayfish were placed in a water tank, and a camera was used to shoot downward from the top of the tank to collect a series of pictures.

[0059] The red claw crayfish image sample dataset is uniformly cropped and scaled to a preset resolution;

[0060] Specifically, the images are uniformly compressed to the same size, specifically 640*640 pixels.

[0061] Annotate the red claw crayfish in the image using a preset image annotation tool to obtain a red claw crayfish image sample dataset containing annotation box information;

[0062] The red claw crayfish image sample dataset is divided into a training set, a validation set and a test set according to a preset ratio.

[0063] In a specific embodiment, a method for annotating red claw crayfish in a picture by using a preset image annotation tool to obtain a red claw crayfish image sample dataset containing annotation box information includes:

[0064] The red claw crayfish in the image is labeled through the makesense website to obtain a red claw crayfish image sample dataset containing the annotation box information.

[0065] In a specific embodiment, the method of dividing the red claw crayfish image sample dataset into a training set, a validation set and a test set according to a preset ratio includes:

[0066] The red claw crayfish image sample dataset is divided into training set, validation set and test set in the ratio of 7:2:1.

[0067] S2: training a preset detector using the preprocessed red claw crayfish image sample data set to obtain a trained detector;

[0068] Specifically, a detector is built based on the RTDETR model in the paddleDetection target detection framework of PaddlePaddle, model training is performed, and the trained model is saved in onnx format.

[0069] In a specific embodiment, a method for obtaining a trained detector by training a preset detector with a preprocessed red claw crayfish image sample dataset includes:

[0070] The training set is input into the preset detector for training, the performance of the detector is evaluated through the verification set, the detector with the best performance is retained, and a trained detector is obtained.

[0071] S3: filtering the output image frames of the preset detector through a preset filtering module, and constructing a tracker according to the filtering results;

[0072] In a specific embodiment, the output image frame of the preset detector is screened by a preset screening module, and the method for constructing a tracker according to the screening result includes:

[0073] A confidence threshold is set, and the detector output image frames with confidence greater than the threshold are used as the tracker input image frames;

[0074] Specifically, the threshold is set to 0.9.

[0075] Initialize the number of currently tracked individuals and a dictionary containing the positions of different individuals in each frame according to the number of red claw crayfish individuals in the input image frame of the tracker. The key of the dictionary is the ID of each red claw crayfish, and the value is its position information.

[0076] In the subsequent frame detection process, the dictionary is updated in real time, specifically:

[0077] If the number of red claw crayfish in the current tracker input image frame is greater than that in the previous frame, a new ID is assigned to the newly detected red claw crayfish, and its position information is set to None;

[0078] Traverse the dictionary and classify the key-value pairs with location information None into the unintended dictionary, and the rest into the intended dictionary;

[0079] Traversing the intentional dictionary, performing Kalman filter prediction on all historical frame position information of each red claw crayfish ID, and obtaining predicted position information of each ID in the current tracker input image frame;

[0080] Match the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame, and update the position information of the corresponding ID in the dictionary according to the actual position information of the red claw crayfish that has been successfully matched;

[0081] The actual location information of unmatched red claw crayfish is assigned to the ID in the unintentional dictionary and moved into the intentional dictionary.

[0082] In a specific embodiment, the method of matching the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame includes:

[0083] The predicted position information of each ID is calculated by IOU with the actual position information of each red claw crayfish in the current tracker input image frame. If the calculated result is greater than 0.9, the match is successful. If the calculated result is less than 0.9, the match fails, and the ID that fails to match is moved from the intentional dictionary to the unintentional dictionary.

[0084] S4: connecting the trained detector, the screening module and the tracker in series to obtain a red claw crayfish behavior trajectory tracking model;

[0085] S5: Tracking the behavior trajectory of the red claw crayfish by using the model to obtain a total moving distance and a behavior trajectory diagram corresponding to the red claw crayfish;

[0086] In a specific embodiment, the method of tracking the behavior trajectory of the red claw crayfish by using the model to obtain the total movement distance and behavior trajectory map corresponding to the red claw crayfish includes:

[0087] Collect red claw crayfish videos of preset duration;

[0088] Specifically, place the crayfish in a flat water tank with the water in the tank covering the crayfish and 2 cm higher. The camera needs to shoot from directly above the water tank, and the field of view needs to include the edge of the water tank.

[0089] Inputting the red claw crayfish video into a red claw crayfish behavior trajectory tracking model;

[0090] Take out the dictionary containing the position information in the tracker, divide it into a series of detection frame coordinates according to different IDs, and take the midpoint of the detection frame coordinates as the centroid coordinates;

[0091] According to the centroid coordinates, a trajectory diagram is drawn through a data visualization library, and the centroid coordinates between every two frames are used to calculate the Euclidean distance. Finally, all Euclidean distances are summed to obtain the total distance traveled by a single red claw crayfish in the time period. The specific formula is as follows:

[0092]

[0093] Where X represents the total distance traveled by a single red claw crayfish during this period of time, x i represents the horizontal coordinate of the center of mass of the red claw crayfish at the i-th frame, y i represents the ordinate of the center of mass of the red claw crayfish at the i-th frame, and n represents the total number of frames of the processed video.

[0094] Example 2

[0095] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of an end-to-end red claw crayfish behavior trajectory tracking method described in Example 1 are implemented.

[0096] Example 3

[0097] A computer device comprises a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, wherein when the computer program is executed by the processor, the steps of an end-to-end red claw crayfish behavior trajectory tracking method described in Example 1 are implemented.

[0098] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for tracking the behavior trajectory of red claw crayfish based on end-to-end, characterized in that: The method comprises: Obtain a sample dataset of red claw crayfish images and perform preprocessing; The preset detector is trained by using the preprocessed red claw crayfish image sample data set to obtain a trained detector; The output image frames of the preset detector are screened by a preset screening module, and a tracker is constructed according to the screening result; The trained detector, the screening module and the tracker are connected in series to obtain a red claw crayfish behavior trajectory tracking model; The behavior trajectory of the red claw crayfish is tracked by the model to obtain the total moving distance and behavior trajectory diagram of the red claw crayfish.

2. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 1 is characterized in that: The method of obtaining a red claw crayfish image sample dataset and performing preprocessing includes: Using an image sensor to collect images of red claw crayfish in each scene, and obtaining a red claw crayfish image sample dataset; The red claw crayfish image sample dataset is uniformly cropped and scaled to a preset resolution; Annotate the red claw crayfish in the image using a preset image annotation tool to obtain a red claw crayfish image sample dataset containing annotation box information; The red claw crayfish image sample dataset is divided into a training set, a validation set and a test set according to a preset ratio.

3. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 2 is characterized in that: The method for annotating the red claw crayfish in the image by using a preset image annotation tool to obtain a red claw crayfish image sample dataset containing annotation box information includes: The red claw crayfish in the image is labeled through the makesense website to obtain a red claw crayfish image sample dataset containing the annotation box information.

4. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 2 is characterized in that: The method of dividing the red claw crayfish image sample dataset into a training set, a validation set and a test set according to a preset ratio includes: The red claw crayfish image sample dataset is divided into training set, validation set and test set in the ratio of 7:2:

1.

5. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 2 is characterized in that: The method of training a preset detector by using the preprocessed red claw crayfish image sample data set to obtain a trained detector includes: The training set is input into the preset detector for training, the performance of the detector is evaluated through the verification set, the detector with the best performance is retained, and a trained detector is obtained.

6. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 1 is characterized in that: The method of filtering the output image frame of the preset detector by a preset filtering module and constructing a tracker according to the filtering result includes: A confidence threshold is set, and the detector output image frames with confidence greater than the threshold are used as the tracker input image frames; Initialize the number of currently tracked individuals and a dictionary containing the positions of different individuals in each frame according to the number of red claw crayfish individuals in the tracker input image frame. The key of the dictionary is the ID of each red claw crayfish, and the value is its position information. In the subsequent frame detection process, the dictionary is updated in real time, specifically: If the number of red claw crayfish in the current tracker input image frame is greater than that in the previous frame, a new ID is assigned to the newly detected red claw crayfish, and its position information is set to None; Traverse the dictionary and classify the key-value pairs with location information None into the unintended dictionary, and the rest into the intended dictionary; Traversing the intentional dictionary, performing Kalman filter prediction on all historical frame position information of each red claw crayfish ID, and obtaining predicted position information of each ID in the current tracker input image frame; Match the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame, and update the position information of the corresponding ID in the dictionary according to the actual position information of the red claw crayfish that has been successfully matched; The actual location information of unmatched red claw crayfish is assigned to the ID in the unintentional dictionary and moved into the intentional dictionary.

7. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 6 is characterized in that: The method of matching the predicted position information of each ID with the actual position information of each red claw crayfish in the current tracker input image frame includes: The predicted position information of each ID is respectively calculated by IOU with the actual position information of each red claw crayfish in the current tracker input image frame. If the calculated result is greater than the preset threshold, the match is successful. If the calculated result is less than the preset threshold, the match fails, and the ID that fails to match is moved from the intentional dictionary to the unintentional dictionary.

8. The end-to-end red claw crayfish behavior trajectory tracking method according to claim 1 is characterized in that: The method of tracking the behavior trajectory of the red claw crayfish by using the model to obtain the total moving distance and behavior trajectory diagram corresponding to the red claw crayfish includes: Collect videos of red claw crayfish of preset duration; Inputting the red claw crayfish video into a red claw crayfish behavior trajectory tracking model; Take out the dictionary containing the position information in the tracker, divide it into a series of detection frame coordinates according to different IDs, and take the midpoint of the detection frame coordinates as the centroid coordinates; According to the centroid coordinates, a trajectory diagram is drawn through a data visualization library, and the centroid coordinates between every two frames are used to calculate the Euclidean distance. Finally, all Euclidean distances are summed to obtain the total distance traveled by a single red claw crayfish in the time period. The specific formula is as follows: Where X represents the total distance traveled by a single red claw crayfish during this period of time, x i represents the horizontal coordinate of the center of mass of the red claw crayfish at the i-th frame, y i represents the ordinate of the center of mass of the red claw crayfish at the i-th frame, and n represents the total number of frames of the processed video.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the end-to-end red claw crayfish behavior trajectory tracking method as described in any one of claims 1 to 8 are implemented.

10. A computer device, characterized in that: The invention comprises a storage medium, a processor and a computer program stored in the storage medium and executable by the processor, wherein when the computer program is executed by the processor, the steps of the end-to-end red claw crayfish behavior trajectory tracking method as claimed in any one of claims 1 to 8 are implemented.

Images