A method and device for detecting and tracking captive target animals
The depth camera is used to obtain the comparison of depth maps and template maps, and combined with the boundary tracking algorithm and position information tracking algorithm, the problem of manual labor and inaccurate detection of artificial marking in captivity animals is solved, and animal tracking and behavioral analysis without artificial marking is realized , improves detection efficiency and accuracy and reduces detection errors.
Patent Information
- Application Number
- CN202110283960.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-17
- Publication Date
- 2024-12-10
- Estimated Expiration
- 2041-03-17
AI Technical Summary
In the detection of captive animals, the prior art has problems such as manual labeling, inaccurate detection, and easy to be affected by light changes. Especially in the case of pigs crowding, the detection error is large, which reduces the detection efficiency and accuracy.
Depth cameras are used to obtain the depth map, and the target animal is detected and tracked by comparison with the animal pen template map. The boundary tracking algorithm and position information tracking algorithm are used to realize the contour recognition and behavioral analysis of animals, avoiding artificial marking and inter-frame differential methods. insufficient.
It realizes animal tracking and behavioral analysis without artificial marking, reduces detection errors, can accurately handle pig crowding, improves detection efficiency and accuracy, timely monitors abnormal behaviors, and enhances disease prevention and control and production management.
Smart Images
Figure CN112767435B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of digital video image processing, and in particular to a method and device for detecting and tracking captive target animals based on video images. Background Art
[0002] In recent years, with the continuous development of large-scale and intensive breeding and the increase in the number of breeding farms, especially under the influence of the epidemic in recent years, the importance of animal detection has become more evident. Real-time detection of animal behavior and timely analysis can effectively improve efficiency and avoid the occurrence of diseases, greatly promoting the development of the breeding industry.
[0003] At present, the detection of target animals in China requires manual marking of seed pixels, and cannot be fully automatically detected. This method consumes manpower and material resources. At the same time, the current video image detection and computer processing cannot distinguish and deal with the phenomenon of pigs gathering together, which reduces the accuracy of detection and greatly increases the detection error.
[0004] The key point in today's behavioral analysis methods is to track pigs. Currently, pigments are commonly used for manual marking, which is labor-intensive and not durable. Inter-frame difference detection is easily affected by sudden changes in light, resulting in inaccurate detection. This series of methods has large implementation errors, inaccurate results, and increased production costs. Summary of the invention
[0005] The object of the present invention is to provide a method and device for detecting and tracking captive target animals based on video images, so as to overcome the above-mentioned technical shortcomings or at least solve part of the above-mentioned problems; the present invention obtains a depth map by using a depth camera, detects the position of the pig by using the depth map information, tracks the pig according to the relationship between the previous and next frames of the detection results, and finally realizes the behavior analysis of the target animal.
[0006] The technical solution adopted by the present invention is as follows:
[0007] In one aspect, the present invention provides a method for detecting and tracking a captive target animal, comprising the following steps:
[0008] Step 1: Animal pen calibration: Use a camera to collect an image of an animal pen as the animal pen template image B. The distance value of each pixel in the image is the distance from the camera plane to the ground plane of the animal pen.
[0009] Step 2: Detect the target animal and obtain the depth map F, where each pixel value is the distance value from the point to the camera plane;
[0010] Step 3: Compare the pixels of the depth map F and the template map B to obtain the segmentation map G. For pixels at the same position, if the pixel in the depth map F is smaller than the template map B, the pixel in the segmentation map G at the corresponding position is 1, otherwise it is 0.
[0011] Step 4: Perform boundary tracking algorithm on the segmentation map G to obtain the outline of the target animal;
[0012] Step 5: Count the number of complete contours. If the number of complete contours is less than the actual number of target animals, subtract 1 from all pixels in template image B, and repeat steps 3 and 4 until the number of contours is equal to the actual number of target animals.
[0013] Step 6: Track the test results;
[0014] Step 7: Conduct behavioral analysis.
[0015] Furthermore, in step 1 and step 2, a depth camera is used to obtain the template image B and the depth image F.
[0016] Furthermore, in step 4, after obtaining the segmentation map, a boundary tracking algorithm is performed on the segmentation map G to obtain the outline of the target animal. The specific steps are as follows:
[0017] (1) Set the contour point set C;
[0018] (2) Scan the segmentation map G from bottom to top and from left to right until pixel 1 is found. This point is added to the contour point set C and used as the starting point of a contour.
[0019] (3) Continue to visit pixels according to the following rules until the current pixel returns to the starting point;
[0020] (4) If the current pixel is 0, access the point on the right; if the current pixel is 1, access the point on the left, and add the current pixel to the contour point set C;
[0021] (5) After finding a contour, set the contour points and the points inside it on the segmentation graph G to 0.
[0022] Furthermore, in step 6, after the detection is completed, the position information of the detection result is tracked, and the position information tracking algorithm is as follows:
[0023] (1) Initialize tracking: assign increasing IDs to the current detection result D = {D1, D2, D3, ...} in order from left to right and from top to bottom, where D1, D2, D3, ... are the location information of the detection box, and save the current tracking result as T = {T1, T2, T3, ...}, where T1, T2, T3, ... contain the location information and ID of each detection box;
[0024] (2) Obtain the detection result of a new frame D' = {D1', D2', D3', ...}, and calculate the Euclidean distance D from each detection box in D' to each detection box in T ij =sqrt[(X i -X j ) 2 +(Y i -Y j ) 2 ], where (X i ,Y i ) is the coordinate of the center point of the i-th detection box in D', (X j , Y j ) is the coordinate of the center point of the detection frame with id j in T, and the minimum minDij is obtained. Then the id of the i-th detection frame is j. Traverse all the detection frames in D' until all the detection frames can obtain the correct id number according to the minimum minDij; then update T with this tracking result;
[0025] (3) Repeat step (2).
[0026] Furthermore, in step seven, the behavior analysis includes trajectory analysis and static behavior classification of the target animal, and the specific method is as follows:
[0027] Trajectory analysis: Using the tracking results, we can get the movement trajectory of each pig, and thus get the distance traveled by each pig in a period of time, the time it stays in the trough, the time it stays in the water trough, and the time it stays in place;
[0028] Classification of stationary behavior of target animals: When the target animal is stationary, all pixels in the depth tracking result box are averaged. Where P ij is the depth pixel value of the i-th row and j-th column, N is the number of points in the tracking result box; due to the different heights of the target animal when lying down and standing up, the Havg of the two states will also be significantly different. Based on experience, a threshold Hr can be determined. When Havg is greater than Hr, the target animal is in a standing state, and when Havg is less than Hr, the target animal is in a lying state.
[0029] On the other hand, the present invention also provides a captive target animal detection and tracking device, which comprises:
[0030] An image acquisition module is used to acquire a template image B and a depth image F;
[0031] The image processing module (computer) is used to receive the acquired template image B and depth image F, and analyze and process them to achieve behavioral analysis of the target animal.
[0032] Furthermore, the image acquisition module uses a depth camera suspended just above the center of the animal pen, the lens of the depth camera faces vertically downward, the camera field of view covers the entire animal pen, and the camera image is transmitted to the image processing module via a wired or wireless network.
[0033] Furthermore, the image processing module includes a processor and a memory, the memory stores computer instructions, and the processor executes the steps described in any one of claims 1-5 by executing the computer instructions, and the computer instructions are used to enable the processor to execute the method described in any one of claims 1-5.
[0034] The beneficial effects of the present invention are as follows: according to the method and device provided by the present invention, it is possible to track and analyze the behavior of captive animals without artificial marking, and at the same time correctly track, distinguish and deal with the crowding of livestock such as pigs, reduce detection errors, monitor abnormal behaviors of livestock in a timely manner, do a good job in disease prevention and control and production management, improve efficiency, and increase economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 This is a schematic diagram of the camera installation position of the present invention; in the figure: 1-depth camera, 2-animal pen;
[0036] Figure 2 A background diagram B is an example of the present invention;
[0037] Figure 3 A depth map F is an example of the present invention;
[0038] Figure 4 A segmentation graph G is an example of the present invention;
[0039] Figure 5 A depth map F is another example of the present invention;
[0040] Figure 6 A segmentation graph G is another example of the present invention;
[0041] Figure 7 The segmentation graph G after updating of another example B of the present invention;
[0042] Figure 8 The following are the specific steps for implementing the behavior analysis method. DETAILED DESCRIPTION
[0043] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0044] A method and device for detecting and tracking a captive target animal comprises the following steps:
[0045] Step 1: Device installation;
[0046] Step 2: Pigpen calibration, i.e. collecting an empty pigpen picture as the pigpen template picture B;
[0047] Step 3: Detect the target animal using the depth map and obtain the depth map F.
[0048] Step 4: Compare F and B pixel by pixel to obtain the segmentation map;
[0049] Step 5: Perform boundary tracking algorithm on the segmentation map G to obtain the outline of the target animal;
[0050] Step 6: Count the number of complete contours. If the number of complete contours is less than the actual number of target animals, subtract 1 from all pixels in B and repeat step 5 until the number of contours is equal to the actual number of target animals.
[0051] Step 7: Track the test results after the test is completed;
[0052] Step 8: Conduct behavioral analysis
[0053] In the specific implementation, the steps are as follows Figure 1 , Figure 1 This is a schematic diagram of the camera installation position. The depth camera is hung directly above the center of the pigpen, with the lens facing vertically downward and the camera's field of view covering the entire pigpen.
[0054] In the specific implementation, step 2 is to calibrate the pigpen after the depth camera is installed, that is, to collect an empty pigpen picture as the pigpen template picture B.
[0055] like Figure 2 The distance value of each pixel in the image is the distance from the camera plane to the ground plane of the pigpen. In the example, the distance value of each pixel in the hollow pigpen is 200.
[0056] In the specific implementation, a depth camera is used to collect data from the pig pens, such as Figure 3 ,The depth map F is obtained by acquisition. In the example picture, the distance value of each pixel at the location of the pig is less than 200.
[0057] In the specific implementation process, the F and B pixels are compared to obtain the segmentation map G. The F and B pixels are compared. For pixels at the same position, if the pixel in F is smaller than that in B, the pixel in the segmentation map G at the corresponding position is 1, otherwise it is 0. In the example, Figure 4 ,for Figure 2 and Figure 3 The comparison between Figure 3 150 and 160 are both less than Figure 2 Therefore, the points less than 200 on the corresponding segmentation graph G are marked as 1, and the other points are marked as 0.
[0058] After obtaining the segmentation map, perform boundary tracking algorithm on the segmentation map G to obtain the outline of the target animal. The steps are as follows:
[0059] (1) Set the contour point set C.
[0060] (2) Scan the segmentation map G from bottom to top and from left to right until pixel 1 is found. Add this point to the contour point set C and use it as the starting point of a contour.
[0061] (3) Continue to access pixels according to the following rules until the current pixel returns to the starting point.
[0062] (4) If the current pixel is 0, access the point on the right; if the current pixel is 1, access the point on the left, and add the current pixel to the contour point set C.
[0063] (5) After finding a contour, set the contour points and the points inside it on the segmentation graph G to 0.
[0064] During the specific implementation process, there will be a phenomenon of pigs gathering together. The present invention also has a corresponding solution to this phenomenon. Count the number of complete contours in the segmentation map. If the number of complete contours is less than the actual number of target animals, subtract 1 from all pixels in B, and then re-compare the depth map F and the template map B based on the updated data to obtain another segmentation map. If the number of complete contours is still less than the actual number, repeat the above process until the number of contours is equal to the actual number of target animals. Figure 5 , Figure 6 , Figure 7 As shown by Figure 5 The obtained depth map is compared with the B map to obtain the segmentation Figure 6 , then according to the segmentation Figure 6 As shown in the figure, the number of animal outlines obtained is less than the actual number of animals. At this time, each pixel of B is subtracted by 1, and then re-combined with Figure 5 Compare and get a new segmentation map Figure 7 .like Figure 7 The number of contours shown was counted and the number obtained corresponds to the actual number of animals.
[0065] In the specific implementation process, after the detection is completed, the location information of the detection result is tracked. The location information tracking algorithm is as follows:
[0066] (1) Initialize tracking: assign increasing IDs to the current detection result D = {D1, D2, D3, ...} in order from left to right and from top to bottom. D1, D2, D3, ... are the location information of the detection box. And save the current tracking result as T = {T1, T2, T3, ...}, where T1, T2, T3, ... contain the location information and ID of each detection box.
[0067] (2) Obtain the detection result of a new frame D' = {D1', D2', D3', ...}, and calculate the Euclidean distance D from each detection box in D' to each detection box in T ij =sqrt[(X i -X j ) 2 +(Y i -Y j ) 2 ], where (X i ,Y i ) is the coordinate of the center point of the i-th detection box in D', (X j , Y j ) is the coordinate of the center point of the detection frame with id j in T. Find the minimum minDij, then the id of the i-th detection frame is j. Traverse all detection frames in D' until all detection frames can obtain the correct id number according to the minimum minDij. Update T with this tracking result.
[0068] (3) Repeat step (2).
[0069] In the specific implementation, the detection results are subjected to behavioral analysis, which includes trajectory analysis and static behavior classification of the target animal. The specific method is as follows:
[0070] Trajectory analysis: Using the above tracking results, we can obtain the movement trajectory of each pig, and thus obtain the distance traveled by each pig in a period of time, the time it stays in the trough, the time it stays in the drinking trough, and the time it stays in place.
[0071] Classification of the static behavior of the target animal. When the target animal is static, all pixels in the depth tracking result box are averaged. Where P ij is the depth pixel value of the i-th row and j-th column, and N is the number of points in the tracking result box. Since the target animal is at different heights when lying down and standing up, the Havg of the two states will also be significantly different. Based on experience, a threshold Hr can be determined. When Havg is greater than Hr, the target animal is in a standing state, and when Havg is less than Hr, the target animal is in a lying state.
[0072] The above examples are only used to illustrate the present invention, but not to limit the present invention. Although the present invention is described in detail with reference to the examples, it should be understood by those skilled in the art that various modifications and variations of the present invention made by those skilled in the art do not deviate from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for detecting and tracking a captive target animal, characterized in that The following steps are involved: Step 1: Animal pen calibration: collect an empty animal pen picture as the animal pen template picture B; Step 2: Detect the target animal and obtain a depth map F; Step 3: Compare the pixels of the depth map F and the template map B to obtain the segmentation map G. For pixels at the same position, if the pixel in the depth map F is smaller than the template map B, the pixel in the segmentation map G at the corresponding position is 1, otherwise it is 0. Step 4: Perform boundary tracking algorithm on the segmentation map G to obtain the outline of the target animal; Step 5: Count the number of complete contours. If the number of complete contours is less than the actual number of target animals, subtract 1 from all pixels in template image B, and repeat steps 3 and 4 until the number of contours is equal to the actual number of target animals. Step 6: Track the test results; Step 7: Conduct behavioral analysis.
2. A method for detecting and tracking captive target animals according to claim 1, characterized in that: In the step 1 and the step 2, a depth camera is used to obtain the template image B and the depth image F.
3. The method for detecting and tracking a captive target animal according to claim 1, characterized in that: In step 4, after obtaining the segmentation map, a boundary tracking algorithm is performed on the segmentation map G to obtain the outline of the target animal. The specific steps are as follows: (1) Set the contour point set C; (2) Scan the segmentation map G from bottom to top and from left to right until pixel 1 is found. This point is added to the contour point set C and used as the starting point of a contour. (3) Continue to visit pixels according to the following rules until the current pixel returns to the starting point; (4) If the current pixel is 0, access the point on the right; if the current pixel is 1, access the point on the left, and add the current pixel to the contour point set C; (5) After finding a contour, set the contour points and the points inside it on the segmentation graph G to 0.
4. The method for detecting and tracking a captive target animal according to claim 1, characterized in that: In step 6, after the detection is completed, the position information of the detection result is tracked, and the position information tracking algorithm is as follows: (1) Initialize tracking: assign increasing IDs to the current detection result D = {D1, D2, D3, ...} in order from left to right and from top to bottom, where D1, D2, D3, ... are the location information of the detection box, and save the current tracking result as T = {T1, T2, T3, ...}, where T1, T2, T3, ... contain the location information and ID of each detection box; (2) Obtain the detection result of a new frame D' = {D1', D2', D3', ...}, and calculate the Euclidean distance D from each detection box in D' to each detection box in T ij =sqrt[(X i -X j ) 2 +(Y i -Y j ) 2 ], where (X i ,Y i ) is the coordinate of the center point of the i-th detection box in D', (X j , Y j ) is the coordinate of the center point of the detection frame with id j in T, and the minimum minDij is obtained. Then the id of the i-th detection frame is j. Traverse all the detection frames in D' until all the detection frames can obtain the correct id number according to the minimum minDij; then update T with this tracking result; (3) Repeat step (2).
5. The method for detecting and tracking a captive target animal according to claim 1, characterized in that: In step 7, the behavior analysis includes trajectory analysis and static behavior classification of the target animal, and the specific method is as follows: Trajectory analysis: Using the tracking results, we can get the movement trajectory of each animal, and thus get the distance traveled by each animal in a period of time, the time it stays at the food trough, the time it stays at the water trough, and the time it stays in place; Classification of stationary behavior of target animals: When the target animal is stationary, all pixels in the depth tracking result box are averaged. Where P ij is the depth pixel value of the i-th row and j-th column, N is the number of points in the tracking result box; due to the different heights of the target animal when lying down and standing up, the Havg of the two states will also be significantly different. Based on experience, a threshold Hr can be determined. When Havg is greater than Hr, the target animal is in a standing state, and when Havg is less than Hr, the target animal is in a lying state.
6. A device for detecting and tracking captive target animals, characterized in that include: An image acquisition module is used to acquire a template image B and a depth image F; An image processing module is used to receive the acquired template image B and depth image F, and analyze and process them to achieve behavioral analysis of the target animal; the image processing module includes a processor and a memory, the memory stores computer instructions, the processor executes the steps described in any one of claims 1-5 by executing the computer instructions, and the computer instructions are used to enable the processor to execute the method described in any one of claims 1-5.
7. A captive target animal detection and tracking device according to claim 6, characterized in that: The image acquisition module uses a depth camera suspended just above the center of the animal pen. The lens of the depth camera faces vertically downward, and the camera field of view covers the entire animal pen. The camera image is transmitted to the image processing module via a wired or wireless network.