A method for estimating the amount of movement of primates in groups based on target detection

Abstract

The present invention proposes a method for estimating the amount of movement of a group of primates based on target detection, comprising the following steps: Step 1: design a multi-view camera shooting device that can shoot the daily behavior of primates in an iron cage without blind spots; Step 2: collect video data of the daily behavior of primates in the cage through the shooting device, and pre-process and annotate the collected video data; Step 3: train a rhesus monkey head detector based on a semi-supervised learning method of a target detection model; Step 4: calculate the amount of movement of rhesus monkeys in actual scenes through the trained model. The present invention designs a multi-view camera shooting device suitable for shooting the daily behavior of primates without blind spots; by preparing a primate target detection data set, a deep learning algorithm can be used for target detection; and a method for calculating the amount of movement of rhesus monkeys in cages based on target detection is proposed.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a method for estimating the movement amount of a group of primates based on target detection. Background Art

[0002] Primate target detection is a new research direction. By establishing a primate target detection dataset and using a series of deep learning methods for training, the trained model can predict the specific location and type of primates in a specific scene (such as five rhesus monkeys in a cage). The detected primate location information can effectively determine its current location and movement distance, which has high application value in the field of new drug evaluation.

[0003] Object detection is a basic task in the field of computer vision and has developed rapidly in recent years, including data sets and deep learning algorithms. The RCNN series of models is a multi-stage method that uses a series of pre-set anchors to predict the final bounding box area. The YOLO series of methods abandons the two-stage idea and directly uses neural networks to predict the center point and width and height of the target box. The DETR series of methods use the encoding and decoding modules in the Transformer to pre-set multiple target entities, directly predict these target entities in the decoding stage, and then match them one by one with the target box of the real label to obtain the best prediction result. CornerNet and others treat the target box as a point and directly predict the upper left and lower right corners of the target box or the center point of the box to achieve the purpose of regressing the entire box.

[0004] Currently, few people have studied primate target detection and exercise estimation, and there is no suitable data set for primate target detection (such as daily behavior data of primates in cages). Based on the prior knowledge of human target detection, this paper further explores the feasibility of primate target detection and exercise estimation. Summary of the invention

[0005] The technical problem to be solved by the present invention is to explore the feasibility of using deep learning methods for primate target detection. The deep learning model is trained using the established primate target detection data set, and the amount of motion is calculated based on the target detection results.

[0006] A method for estimating the amount of movement of a group of primates based on target detection, characterized in that the method for estimating the amount of movement of primates comprises the following steps:

[0007] Step 1: Design a multi-angle camera device that can capture the daily behavior of primates in cages without blind spots;

[0008] Step 2: Collect video data of the daily behavior of primates in cages through a camera, and pre-process and annotate the collected video data;

[0009] Step 3: Train the rhesus monkey head detector based on the semi-supervised learning method of the target detection model;

[0010] Step 4: Use the trained model to calculate the amount of motion of the rhesus monkey in the actual scene.

[0011] Furthermore, in step 1, the multi-view camera shooting device includes a rectangular iron cage and cameras arranged on the front, side and top of the iron cage.

[0012] Furthermore, in step 1, the multi-view camera shooting device model is:

[0013]

[0014] The cage space is V, V = {(x, y, z) | 0 ≤ x ≤ x0, 0 ≤ y ≤ y0, 0 ≤ z ≤ z0}; the camera width angle is 2θ; the camera position coordinates for the front, top, and side views are C1 (x0, y C1 ,z C1 ), C2(x C2 ,y C2 ,z0),C3(x C3 ,y0,z C3 ); the V C1 , V C2 , V C3 Respectively represent the blind areas of cameras C1, C2, and C3.

[0015] Furthermore, in step 1, when (1-3cos 2 When θ)>0, x=x0, y=y0, z=z0, the volume of the blind area is 0, and this value must be the minimum value.

[0016] Furthermore, in step 2, the following steps are also included:

[0017] Step 21, collecting video data of primates in iron cages;

[0018] Step 22: preprocess and label the video data.

[0019] Furthermore, in step 21, the primates are all provided with collars of different colors for differentiation.

[0020] Furthermore, in step 22, part of the video data is labeled, and the labeled video frames are used as a labeled head target detection data set, and the unlabeled video frames are used as an unlabeled head target detection data set.

[0021] Furthermore, in step 3, the rhesus monkey head detector includes a model A using a YOLO-S algorithm and a model B using a YOLO-L algorithm.

[0022] Furthermore, in step 3, the labeled image is input into model A for gradient backpropagation to update the parameters of model A; then the unlabeled image is input into model A and model B at the same time; the output of model A can be regarded as a pseudo-label of the unlabeled image, and the pseudo-label is regarded as the label output by model B to perform gradient backpropagation on model B to update the parameters of model B; model B obtained in step 31 is used as the initial value of the model parameters in this step, and model B is fine-tuned using labeled data to obtain the final rhesus monkey head object detector.

[0023] Furthermore, in step S6, the following steps are also included:

[0024] Step 61, using the face detection and recognition results of any two viewing angles of the front view, the side view and the top view, the three-dimensional movement distance of each individual can be obtained;

[0025] Step 62, taking the first frame as the starting point, in the front view or side view of each frame, the IDs of different individuals are detected, and finally the movement amount is matched with the animal ID.

[0026] The beneficial effects of the present invention include:

[0027] The present invention designs a multi-view camera shooting device suitable for shooting the daily behaviors of primates without blind spots; by preparing a primate target detection data set, a deep learning algorithm can be used for target detection; and a method for calculating the amount of exercise of caged rhesus monkeys based on target detection is proposed.

[0028] The trained model in the present invention takes only 1 / 40 second to detect a picture with a resolution of 736x1280, meeting the real-time requirements, and its performance on the labeled test set is better than that of the model trained with labeled data directly using YOLO-S or YOLO-L. It solves the problem of automatic labeling of unlabeled data and uses unlabeled data to enhance the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of a method for estimating the movement amount of a group of primates based on target detection provided by an embodiment of the present invention;

[0030] Figure 2It is a schematic structural diagram of an iron cage and a lens in a method for estimating the movement amount of a group of primates based on target detection provided by an embodiment of the present invention;

[0031] Figure 3 It is a flow chart of model training in a method for estimating the movement amount of a group of primates based on target detection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention is described in more detail below in conjunction with the accompanying drawings. The present invention includes but is not limited to the following embodiments.

[0033] As attached Figure 1 As shown, the present invention provides a method for estimating the amount of movement of a group of primates based on target detection, and the method for estimating the amount of movement of primates comprises the following steps:

[0034] Step 1: Design a multi-angle camera device that can capture the daily behavior of primates in cages without blind spots;

[0035] Step 2: Collect video data of the daily behavior of primates in cages through a camera, and pre-process and annotate the collected video data;

[0036] Step 3: Train the rhesus monkey head detector based on the semi-supervised learning method of the target detection model;

[0037] Step 4: Use the trained model to calculate the amount of motion of the rhesus monkey in the actual scene.

[0038] Step 1, design a multi-view camera shooting device that can capture the daily behavior of primates (rhesus monkeys) in an iron cage without blind spots.

[0039] The evaluation of new drugs requires experiments on some primates, such as rhesus monkeys and crab-eating monkeys. Since primates are more flexible, they need to be placed in iron cages. At the same time, primates may be in any position in the cage, and a multi-angle camera shooting system needs to be designed to ensure that the daily behavior of primates in the iron cage can be shot without blind spots. Based on these requirements, the present invention proposes a shooting device suitable for estimating the amount of exercise of primates.

[0040] As attached Figure 2 As shown, in one embodiment, the size of the actual iron cage is x0*y0*z0. Placing a camera on the front, side and top of the rectangular column can realize a three-view camera shooting device, which can shoot primates in the cage without any blind spots. Secondly, the placement positions of the three cameras need to be considered, and the calculation process is as follows.

[0041] V represents the internal space of the cage, V = {(x, y, z)|0≤x≤x0,0≤y≤y0,0≤z≤z0}; C1(x0,y C1 ,z C1 ),C2(x C2 ,y C2 ,z0),C3(x C3 ,y0,z C3 ) represent the camera position coordinates for the front view, top view, and side view, respectively; V C1 ,V C2 ,V C3 Respectively represent the blind areas of cameras C1, C2, and C3.

[0042] Assume V C1 The coordinates of the midpoint are P(x,y,z), represents the vector with C1 camera and P in the corresponding blind area as the starting and ending points; the normal vector of the plane where C1 is located is The wide angle of the camera is 2θ, then:

[0043]

[0044] (1-cos 2 θ)(x-x0) 2 -cos 2 θ[(yy C1 ) 2 +(zz C1 ) 2 ]≤0

[0045] Cosθ represents the cosine value of the angle between the straight line C1P and the normal vector of the plane where C1 is located. Since P is a point in the blind spot, the cosine value must be smaller than the cosine value corresponding to the wide angle of the camera.

[0046] Similarly, launch:

[0047] (1-cos 2 θ)(z-z0) 2 -cos 2 θ[(xx C2 ) 2 +(yy C2 ) 2 ]≤0

[0048] (1-cos 2 θ)(y-y0) 2 -cos 2 θ[(xx C3 ) 2 +(zz C3 ) 2 ]≤0

[0049] Blind spots of the three cameras:

[0050]

[0051] The problem turns into:

[0052] Find y C1 ,z C1 ,x C2 ,y C2 ,x C3 ,z C3 , so that V = ∫∫∫ Ω 1dxdydz is the smallest. Let X be the optimal solution (y C1 ,z C1 ,x C2 ,y C2 ,x C3 ,z C3 ), that is, X = argmin D V, where D = {(y C1 ,z C1 ,x C2 ,y C2 ,x C3 ,z C3 )|0≤x C2 ,x C3 ≤x0,0≤y C1 ,y C2 ≤y0,0≤z C1 ,z C3 ≤z0}, D represents the feasible domain of the unknown quantity.

[0053] Let x C2 =x C3 =x0,y c1 =y c2 =y0, let z C1 =z C3 =z0, then the area Ω is expressed as:

[0054]

[0055] Adding the first three inequalities gives:

[0056] (1-3cos 2 θ)[(x-x0) 2 +(y-y0) 2 +(z-z0) 2 ]≤0

[0057] When (1-3cos 2 When θ)>0, we get: x=x0, y=y0, z=z0, the volume of the blind area is 0, and this value must be the minimum value.

[0058] In step 2, the device in step 1 is used to collect video data of daily behaviors of caged primates of different genders, breeds, and ages, and the collected video data is preprocessed and labeled, including the following steps:

[0059] Step 21, collecting video data of primates in iron cages;

[0060] After the device was fixed to the iron cage, the five primates in the cage were replaced regularly. The age range included infants and young adults. In medical experiments, primates were kept as young as possible to maintain vitality. The genders included male and female. The species included rhesus monkeys and crab-eating monkeys. The total filming time was about 2 months. The primates in the cages were all wearing collars of different colors to distinguish them.

[0061] Step 22: video data preprocessing and labeling;

[0062] After obtaining the original video data of the daily behavior of primates, 1) first manually delete low-quality video clips with poor lighting, such as video scenes on cloudy days and at night; 2) secondly, filter the transcoded video by similar frames to obtain the video frames to be labeled, that is, delete the continuous frames with small optical flow changes; 3) use labelme software to annotate the target box of the head + collar, and the annotation format is consistent with the public dataset COCO.

[0063] Due to the large amount of data, it is impossible to annotate all the collected videos. The final amount of annotated data is about 80% of the collected data. The annotated video frames are used as the labeled head target detection dataset, which is divided into a training set and a test set; the unlabeled video frames are used as the unlabeled head target detection dataset. These two datasets are used for the next step of neural network training.

[0064] Step 3: Based on the target detection model YOLO, a semi-supervised learning method is used to train a primate head detector.

[0065] The YOLO series of methods abandon the two-stage approach and directly use the neural network to predict the center point, width and height of the target box, achieving the best trade-off between running speed and prediction accuracy. YOLOv5 is close to the Faster-RCNN network in performance, and has made special processing on small target detection, which is suitable for primate detection.

[0066] As attached Figure 3 As shown, the primate head detector used in the present invention includes a model A using the YOLO-S algorithm and a model B using the YOLO-L algorithm. Compared with YOLO-L, YOLO-S has fewer parameters and higher generalization performance.

[0067] Training a primate head detector consists of the following steps:

[0068] Step 31: Select YOLO-S with fewer parameters as model A, and select YOLO-L with more parameters as model B. Input the labeled image into model A for gradient backpropagation to update the parameters of model A. Then input the unlabeled image into model A and model B. Since model A has better generalization performance, the output result after inputting the unlabeled data into model A can be regarded as the pseudo-label of the unlabeled image. The pseudo-label is regarded as the label of the image input into model B, and gradient backpropagation is performed on model B to update the parameters of model B.

[0069] Step 32: Use model B obtained in step 31 as the initial value of the model parameters in this step, and fine-tune model B using labeled data to obtain the final rhesus monkey head object detector.

[0070] The trained model in the present invention takes only 1 / 40 second to detect a picture with a resolution of 736x1280, meeting the real-time requirements, and its performance on the labeled test set is better than that of the model trained with labeled data directly using YOLO-S or YOLO-L. The problem of automatic labeling of unlabeled data is solved, and the generalization ability of the model is enhanced by using unlabeled data. The experimental results are shown in Table 1.

[0071] method mAP@.5 mAP@.5:.95 YOLO-S 0.964 0.604 YOLO-L 0.966 0.591 YOLO-S+L 0.970 0.624

[0072] Table 1

[0073] In Table 1, mAP represents the average of all category recognition accuracies, mAP.5 represents the average accuracy when the calculation threshold is set to 0.5, and mAP.5:.95 represents the result of calculating the average mAP corresponding to all thresholds after taking a threshold every 0.05 from 0.5 to 0.95. Through experimental comparison, the accuracy of the head detector used in the present invention is higher than that of using the YOLO-S or YOLO-L model alone.

[0074] Step 4, using the model obtained in step 3 to calculate the amount of motion of the rhesus monkey in the actual scene, includes the following steps:

[0075] Step 41, the movement amount of the rhesus monkey is a three-dimensional information amount. When calculating, the detection results of any two perspectives in the front view, side view and top view can be used to obtain the three-dimensional movement distance. Assuming that in the front view, the target detector obtains that the box center moves from (y1, z1) to (y2, z2), and in the side view, the target detector obtains that the box center moves from (x1, z1) to (x2, z2), then the absolute distance in space is

[0076] Step 42, taking the first frame as the starting point, calculate the motion amount of the front view and the side view in each frame, detect the IDs of different individuals, and finally correspond the motion amount to the animal ID; obtain the motion amount of different types of rhesus monkeys in the target time period.

[0077] The present invention is not limited to the above-mentioned specific implementation modes. A person skilled in the art may implement the present invention in various other specific implementation modes according to the embodiments and the disclosure of the drawings. Therefore, any design that adopts the design structure and concept of the present invention and makes some simple transformations or changes shall fall within the scope of protection of the present invention.

Claims

1. A method for estimating the amount of movement of a group of primates based on target detection, characterized in that: The method for estimating the amount of exercise of primates comprises the following steps: Step 1: Design a multi-angle camera device that can capture the daily behavior of primates in cages without blind spots; Step 2: Collect video data of the daily behavior of primates in cages through a camera, and pre-process and annotate the collected video data; Step 3: Train the rhesus monkey head detector based on the semi-supervised learning method of the target detection model; Step 4: Calculate the amount of movement of the rhesus monkey in the actual scene through the trained model; In step 1, the multi-view camera shooting device includes a rectangular iron cage and cameras arranged on the front, side and top of the iron cage; V represents the internal space of the cage, V = {(x, y, z)|0≤x≤x0,0≤y≤y0,0≤z≤z0}; C1(x0,y C1 ,z C1 ),C2(x C2 ,y C2 ,z0),C3(x C3 ,y0,z C3 ) represent the camera position coordinates for the front view, top view, and side view, respectively; V C1 ,V C2 ,V C3 Respectively represent the blind areas of cameras C1, C2, and C3; Assume V C1 The coordinates of the midpoint are P(x,y,z), represents the vector with C1 camera and P in the corresponding blind area as the starting and ending points; the normal vector of the plane where C1 is located is The wide angle of the camera is 2θ, then: (1-cos 2 θ)(x−x0) 2 -cos 2 θ[(yy C1 ) 2 +(zz C1 ) 2 ]≤0 Cosθ represents the cosine value of the angle between the straight line C1P and the normal vector of the plane where C1 is located. Since P is a point in the blind spot, the cosine value must be smaller than the cosine value corresponding to the wide angle of the camera. Similarly, launch: (1-cos 2 θ)(z-z0) 2 -cos 2 θ[(xx C2 ) 2 +(yy C2 ) 2 ]≤0 (1-cos 2 θ)(y-y0) 2 -cos 2 θ[(xx C3 ) 2 +(zz C3 ) 2 ]≤0 Blind spots of the three cameras: The problem turns into: Find y C1 ,z C1 ,x C2 ,y C2 ,x C3 ,z C3 , so that V = ∫∫∫ Ω 1dxdydz smallest; Let X be the optimal solution (y C1 ,z C1 ,x C2 ,y C2 ,x C3 ,z C3 ), that is, X = argmin D V, where D = {(y C1 ,z C1 ,x C2 ,y C2 ,x C3 ,z C3 )|0≤x C2 ,x 03 ≤x0,0≤y C1 ,y C2 ≤y0,0≤z C1 ,z C3 ≤z0}, D represents the feasible domain of the unknown quantity; Let x C2 =x C3 =x0,y C1 =y C2 =y0, let z C1 =z C3 =z0, then the area Ω is expressed as: Adding the first three inequalities gives: (1–3cos 2 θ)[(x−x0) 2 +(y-y0) 2 +(z-z0) 2 ]≤0 When (1-3cos 2 When θ)>0, we get: x=x0, y=y0, z=z0, the volume of the blind area is 0, and this value must be the minimum value.

2. The method for estimating the amount of exercise of primates according to claim 1, characterized in that: In step 1, when (1-3cos 2 When θ)>0, x=x0, y=y0, z=z0, the volume of the blind area is 0, and this value must be the minimum value.

3. The method for estimating the amount of exercise of a primate according to claim 1, characterized in that: In step 2, the following steps are also included: Step 21, collecting video data of primates in iron cages; Step 22: preprocess and label the video data.

4. The method for estimating the amount of exercise of primates according to claim 3, characterized in that: In step 21, the primates are each provided with collars of different colors for identification.

5. The method for estimating the amount of exercise of primates according to claim 3, characterized in that: In step 22, part of the video data is labeled, and the labeled video frames are used as a labeled head target detection data set, and the unlabeled video frames are used as an unlabeled head target detection data set.

6. The method for estimating the amount of exercise of a primate according to claim 5, characterized in that: In step 3, the rhesus monkey head detector includes a model A using the YOLO-S algorithm and a model B using the YOLO-L algorithm.

7. The method for estimating the amount of exercise of a primate according to claim 6, characterized in that: In step 3, the labeled image is input into model A for gradient backpropagation to update the parameters of model A; then the unlabeled image is input into model A and model B at the same time; the output of model A can be regarded as a pseudo-label of the unlabeled image, and the pseudo-label is regarded as the label output by model B to perform gradient backpropagation on model B to update the parameters of model B; model B obtained in step 31 is used as the initial value of the model parameters in this step, and model B is fine-tuned using labeled data to obtain the final rhesus monkey head object detector.

8. The method for estimating the amount of exercise of a primate according to claim 1, characterized in that: In step S6, the following steps are also included: Step 61, using the face detection and recognition results of any two viewing angles of the front view, the side view and the top view, the three-dimensional movement distance of each individual can be obtained; Step 62, taking the first frame as the starting point, in the front view or side view of each frame, the IDs of different individuals are detected, and finally the movement amount is matched with the animal ID.

Images