Animal head posture estimation method fusing computer vision and target detection

Abstract

The invention discloses an animal head posture estimation method fusing computer vision and target detection, and is applied to the technical field of computer vision. Firstly, animal head images at different angles in various scenes are collected; screening the collected animal head images and marking in the images to form a data set; secondly, constructing a high-precision and lightweight improved YOLOv8 different-angle animal head key point recognition model; training and evaluating the model by using a data set; integrating a PNP algorithm with the output of the YOLOv8 model, and calculating final attitude estimation; deploying the weight file on an edge device; and finally, transmitting the to-be-detected picture into the edge device to complete prediction. The method is suitable for animal head posture recognition in different scenes on the edge device, animal behaviors can be effectively observed in real time, accurate temperature measurement detection and health condition observation are achieved, and the communication resource cost of manual and remote recognition observation is remarkably reduced.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to an animal head pose estimation method that integrates computer vision and object detection. Background Technology

[0002] The development of modern agriculture and the popularization of smart farming technologies have made real-time monitoring of animal health and behavior increasingly important. However, animal husbandry management largely remains at the stage of manual inspection, temperature measurement, and supervision based on information technology, leading to increased labor costs and low management efficiency. This is especially true in large-scale farms, where timely detection and handling of abnormal conditions in pigs are crucial for preventing disease transmission, improving farming efficiency, and ensuring animal welfare. However, due to environmental changes, individual differences, and behavioral habits, animal head postures vary considerably, posing significant challenges to accurate temperature measurement. High-precision, real-time posture recognition methods are essential for timely capture of animal frontal postures, enabling rapid temperature measurement and frontal identification and observation. Furthermore, lightweight methods deployed on edge devices are crucial for low-cost, high-efficiency intelligent management of livestock. Existing posture recognition methods involve numerous computational parameters, making direct deployment on edge devices impossible. Therefore, providing a lightweight method that integrates computer vision and object detection for animal head posture estimation is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0003] In view of this, the present invention provides an animal head pose estimation method that integrates computer vision and object detection. It uses an improved YOLOv8 model for animal head pose estimation, reduces the number of training parameters, improves detection accuracy, and achieves lightweight algorithm, making it suitable for deployment at the edge.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] An animal head pose estimation method integrating computer vision and object detection includes the following steps:

[0006] S1. Obtain images of animal heads from various angles in different scenes, classify and label the images, select ten key points to represent the main features of the animal head, form an animal head dataset, and divide it into training and validation sets.

[0007] S2, Improved convolutional layer Conv module of YOLOv8 pose estimation model, reconstructed traditional convolutional layer Conv module, improved C2f module and improved detection head PoseHead;

[0008] S3. PNP algorithm integrated based on improved YOLOv8 attitude estimation model;

[0009] S4. Train the improved YOLOv8 pose estimation model after integrating the PNP algorithm using the training set and validation set, and obtain the optimal weights.

[0010] S5. Export the selected weight file in ONNX format and deploy it to the edge device to test the recognition effect. Then, adjust the hyperparameters based on the test results. If the hyperparameters need to be adjusted, retrain the device. If no adjustment is needed, output the detection image directly.

[0011] S6. Use the trained improved YOLOv8 pose estimation model based on optimal weights to estimate the animal's head pose, output the detection result image and label the animal's category confidence, ten key points and the confidence of each key point, and the head Euler angles.

[0012] Optionally, S1 is as follows:

[0013] S11. Collect images of animal heads of various animal categories from various angles in different scenes. Then, filter the collected images, remove blurry images and images where the ten key points are not fully exposed, and organize the remaining images to form a dataset for animal head pose estimation.

[0014] S12. Using the same object detection labeling method as the YOLO dataset, label the various animal head datasets in different scenes obtained by the dataset labeling software Labelimg, and finally obtain a set of labeled txt files and corresponding datasets of images;

[0015] S13. Divide the labeled dataset into a training set and a validation set in a 7:3 ratio. During the training process, the weights of the training set and the validation set are continuously adjusted.

[0016] Optionally, the Conv module of the improved YOLOv8 pose estimation model in S2 specifically includes:

[0017] First, an average pooling layer is used to reduce the spatial dimension of the feature map. The pooled feature map is divided into two parts. One part goes through a 3*3 convolutional layer to capture local context information, and the other part goes through a max pooling layer followed by a 1*1 convolutional layer, which helps to maintain more global structural information. Then, the outputs of the two branches are concatenated in the depth direction to obtain the improved lightweight downsampling DDConv module.

[0018] Optionally, the reconstructed traditional convolutional layer Conv module in S2 specifically includes:

[0019] The traditional isomorphic convolution is improved by using convolution kernels of different sizes, including 1*1 and 3*3 kernels, to maintain accuracy while reducing computation, resulting in an improved lightweight CHetConv module.

[0020] Optionally, the improved C2f module in S2 is specifically as follows:

[0021] In the cross-stage local feature fusion module C2f in the original YOLOv8 network, an improved CHetConv module is introduced into the backbone, which uses convolutional kernels of different sizes to replace the original convolution operation. At the same time, the CSP module is introduced to obtain a lightweight multi-scale module CSYHet.

[0022] Optionally, the improved detection head PoseHead in S2 is specifically:

[0023] By combining spatial attention mechanisms with convolution, bicubic interpolation is used to reduce the resolution of feature maps and upsample them to a higher resolution. The receptive field spatial features are dynamically generated according to the size of the convolution kernel to improve performance, resulting in a detection head CFAHead that enhances the model's feature extraction capabilities.

[0024] Optionally, S3 specifically refers to:

[0025] S31. Define a 3D model and create a three-dimensional coordinate system with the center of the animal's head as the origin, including the X-axis, Y-axis, and Z-axis. The 3D model includes 3D key points representing the features of the animal's head.

[0026] S32. Detect the coordinates of 2D key points in the image based on the improved YOLOv8 pose estimation model, and match the coordinates of 2D key points with the 3D key points in the 3D model.

[0027] S33. The PNP algorithm is integrated end-to-end with the improved YOLOv8 pose estimation model. The PNP algorithm calculates the rotation and translation vectors describing the camera pose by combining the 3D model key points and the corresponding 2D image key points with the camera intrinsic parameters and distortion coefficients. This completes the integration of the improved YOLOv8 pose estimation model with the PNP algorithm. Then, the rotation vector is converted into a rotation matrix, and the rotation and translation vectors are combined into a homogeneous transformation matrix. The translation matrix is ​​represented by the homogeneous transformation matrix. Therefore, the rotation and translation matrices together define the pose of the camera relative to the 3D model points.

[0028] S34. Based on the rotation matrix and translation matrix, the Euler angles are further calculated using the PNP algorithm, where the rotation matrix contains the rotation information of the animal's head around the X, Y, and Z axes.

[0029] S35. Using the calculated rotation matrix, translation matrix, intrinsic parameter matrix, distortion coefficient, Euler angles, and 3D keypoint coordinate pose information, draw a 3D bounding box and coordinate axes on the image to display the animal's head pose.

[0030] Optionally, S34 specifically includes:

[0031] Euler angles include the yaw angle about the X-axis, the pitch angle about the Y-axis, and the roll angle about the Z-axis. The calculation formula is as follows:

[0032]

[0033] In the formula, yaw is the yaw angle, pitch is the pitch angle, roll is the roll angle, and r is the pitch angle. 21 r 11 r 31 r 32 r 33 Let r be an element in the rotation matrix, where r 11 r represents the direction cosine of the rotated X-axis relative to the original X-axis. 21 r represents the direction cosine of the rotated Y-axis relative to the original X-axis. 31 r represents the direction cosine of the rotated Z-axis relative to the original X-axis. 32 r represents the direction cosine of the rotated Z-axis relative to the original Y-axis. 33 This represents the direction cosine of the rotated Z-axis relative to the original Z-axis.

[0034] As can be seen from the above technical solutions, compared with the prior art, this invention provides an animal head pose estimation method that integrates computer vision and object detection, with the following beneficial effects: This invention introduces multiple convolutions and pooling operations into the convolutional layers of the original YOLOv8 model, reducing the number of parameters to lower model complexity and improve keypoint detection, forming a new lightweight downsampling DDConv module; in the Head part, the original detection head introduces bicubic interpolation to upsample low-resolution feature maps to high resolution, forming a new detection head CFAHead with improved model feature extraction capabilities, ultimately achieving a reduction in the number of parameters and computational complexity while maintaining or improving accuracy; integrating the PNP algorithm converts the 2D keypoint coordinates detected by the improved YOLOv8 model into 3D point coordinates for pose estimation and displays 3D bounding boxes, making the results more intuitive. This invention uses an improved YOLOv8 pose estimation model for estimation, reducing the number of computational parameters by 30.1% with almost no impact on detection accuracy, greatly reducing the storage space requirements of the algorithm model on deployment devices, facilitating deployment at edge devices. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0036] Figure 1 This is a flowchart of the animal head pose estimation method of the present invention;

[0037] Figure 2 This is a schematic diagram of the improved YOLOv8 attitude estimation model structure of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] This invention discloses an animal head pose estimation method that integrates computer vision and object detection, such as... Figure 1 As shown, it includes the following steps:

[0040] S1. Obtain images of animal heads from various angles in different scenes, classify and label the images, select ten key points to represent the main features of the animal head, form an animal head dataset, and divide it into training and validation sets.

[0041] S2. Improved convolutional layer Conv module of YOLOv8 pose estimation model, reconstructed traditional convolutional layer Conv module, improved C2f module, and improved PoseHead detection head; the improved model structure is as follows. Figure 2 As shown;

[0042] S3. PNP algorithm integrated based on improved YOLOv8 attitude estimation model;

[0043] S4. Train the improved YOLOv8 pose estimation model after integrating the PNP algorithm using the training set and validation set, and obtain the optimal weights.

[0044] S5. Export the selected weight file in ONNX format and deploy it to the edge device to test the recognition effect. Then, adjust the hyperparameters based on the test results. If the hyperparameters need to be adjusted, retrain the device. If no adjustment is needed, output the detection image directly.

[0045] S6. Use the trained improved YOLOv8 pose estimation model based on optimal weights to estimate the animal's head pose, output the detection result image and label the animal's category confidence, ten key points and the confidence of each key point, and the head Euler angles.

[0046] Furthermore, S1 specifically refers to:

[0047] S11. Collect images of animal heads of various animal categories from various angles in different scenes. Then, filter the collected images, remove blurry images and images where the ten key points are not fully exposed, and organize the remaining images to form a dataset for animal head pose estimation.

[0048] S12. Using the same object detection labeling method as the YOLO dataset, label the various animal head datasets in different scenes obtained by the dataset labeling software Labelimg, and finally obtain a set of labeled txt files and corresponding datasets of images;

[0049] In this embodiment of the invention, the head category of a pig is labeled "pig", the head category of a cat is labeled "cat", and the head category of a dog is labeled "dog". Key points include: upper left ear, lower left ear, left eye, left nose, lower nose, right nose, upper nose, right eye, upper right ear, and lower right ear. The upper left ear is labeled "learu", the lower left ear is labeled "leard", the left eye is labeled "leye", the left nose is labeled "lnose", the lower nose is labeled "dnose", the right nose is labeled "rnose", the upper nose is labeled "unose", the right eye is labeled "reye", the upper right ear is labeled "rearu", and the lower right ear is labeled "reard".

[0050] S13. Divide the labeled dataset into a training set and a validation set in a 7:3 ratio. During the training process, the weights of the training set and the validation set are continuously adjusted to obtain the position information of the true bounding boxes as much as possible.

[0051] In this embodiment of the invention, the corresponding image files are placed in the train and val folders under the images directory, and the label files are placed in the train and val folders under the labels directory. Finally, the label files and the corresponding image paths are saved in the datasets file, so that the network model can obtain the image paths by reading the datasets file for training.

[0052] Furthermore, the Conv module of the improved YOLOv8 pose estimation model in S2 is specifically as follows:

[0053] First, an average pooling layer is used to reduce the spatial dimension of the feature map. The pooled feature map is divided into two parts. One part is first passed through a 3*3 convolutional layer to capture local context information, and the other part is passed through a max pooling layer followed by a 1*1 convolutional layer, which helps to maintain more global structural information. Then, the outputs of the two branches are concatenated in the depth direction to obtain the improved lightweight downsampling DDConv module, which solves the problem of large computational cost of the original module without significantly reducing the module performance.

[0054] In this embodiment of the invention, the DDConv module is used in layers 1, 3, 5, and 7 of the Backbone part and layers 15 and 18 of the Head part in the YOLOv8 pose estimation model. During the feature extraction process, it gradually reduces the resolution of the feature map while increasing the number of channels in the feature map.

[0055] Furthermore, the reconstruction of the traditional convolutional layer Conv module in S2 is specifically as follows:

[0056] Considering that the model needs to be deployed at the edge, four new convolutional layers are used in the backbone of the YOLOv8 pose estimation model to improve the traditional isomorphic convolution. Different sizes of convolution kernels are used for convolution operations, including 1*1 and 3*3 convolution kernels, to maintain accuracy while reducing computation, forming an improved lightweight CHetConv module.

[0057] Furthermore, the improved C2f module in S2 is specifically as follows:

[0058] In the cross-stage local feature fusion module C2f in the original YOLOv8 network, an improved CHetConv module is introduced into the backbone, which uses convolutional kernels of different sizes to replace the original convolution operation. At the same time, the CSP module is introduced to obtain a lightweight multi-scale module CSYHet, while retaining the richness of gradient flow in C2f.

[0059] Furthermore, the improved detection head PoseHead in S2 is specifically as follows:

[0060] Considering that there are ten key points in the dataset, the spatial attention mechanism and convolution are combined. Bicubic interpolation is used to reduce the resolution of the feature map and upsample it to a high resolution. The receptive field spatial features are dynamically generated according to the size of the convolution kernel to improve performance. This results in a detection head CFAHead that increases the model's feature extraction capability and improves the model's performance.

[0061] Furthermore, S3 specifically refers to:

[0062] S31. Define a 3D model and create a three-dimensional coordinate system with the center of the animal's head as the origin, including the X-axis, Y-axis, and Z-axis. The 3D model includes 3D key points representing the features of the animal's head.

[0063] In this embodiment of the invention, the X-axis is horizontal, pointing to the right side of the animal's head; the Y-axis is vertical, pointing to the top of the animal's head; and the Z-axis is forward and backward, pointing directly in front of the animal's head.

[0064] S32. Detect the coordinates of 2D key points in the image based on the improved YOLOv8 pose estimation model, and match the coordinates of 2D key points with the 3D key points in the 3D model.

[0065] S33. The PNP algorithm is integrated end-to-end with the improved YOLOv8 pose estimation model. The PNP algorithm calculates the rotation and translation vectors describing the camera pose by combining the 3D model key points and the corresponding 2D image key points with the camera intrinsic parameters and distortion coefficients. This completes the integration of the improved YOLOv8 pose estimation model with the PNP algorithm. Then, the rotation vector is converted into a rotation matrix, and the rotation and translation vectors are combined into a homogeneous transformation matrix. The translation matrix is ​​represented by the homogeneous transformation matrix. Therefore, the rotation and translation matrices together define the pose of the camera relative to the 3D model points.

[0066] In this embodiment of the invention, the `calibrateCamera` function is introduced to calculate the camera's intrinsic parameter matrix and distortion coefficients, addressing the lack of these parameters in the YOLOv8 algorithm. These parameters are essential for accurate pose estimation; uncorrected images will suffer from lens distortion, affecting the accuracy of keypoint coordinates. The `solvePnP` function from OpenCV is used to calculate rotation and translation vectors, and then the `Rodrigues` function from OpenCV is called to convert the rotation vectors into a rotation matrix (R). `solvePnP` further optimizes the rotation and translation vectors based on the initial values, thus improving the accuracy of pose estimation. Since YOLOv8 only provides the location and category of the detected object, not its pose information, this method can estimate the target's pose based on the provided information.

[0067] S34. Based on the rotation matrix and translation matrix, the Euler angles are further calculated using the PNP algorithm, where the rotation matrix contains the rotation information of the animal's head around the X, Y, and Z axes.

[0068] S35. Using the calculated rotation matrix, translation matrix, intrinsic parameter matrix, distortion coefficient, Euler angles, and 3D keypoint coordinate pose information, draw a 3D bounding box and coordinate axes on the image to display the animal's head pose.

[0069] In this embodiment of the invention, the OpenCV `projectPoints` function is used to project 3D point coordinates, rotation matrix, translation vector, intrinsic parameter matrix, and distortion coefficients onto a 2D image plane. `polylines` and `line` are introduced to draw 3D bounding boxes and coordinate axes in the image. The `visualize` function is used to define a set of points for a 3D bounding box. The values ​​of `rear_size` and `front_size` are defined to determine the size of the 3D bounding box on the X and Y axes. `rear_depth` and `front_depth` define the size of the bounding box on the Z axis, set at the origin in the Z direction. These three parameters are set according to the size of the object. A total of eight vertices are defined: four points for the rear bounding box and four points for the front bounding box. The four vertices are the bottom left, top left, top right, and bottom right corners. Then, the OpenCV `cv2.polylines` function is used to draw the outline of these points, thus completing the 3D bounding box. This 3D bounding box can visually display the pose estimation results provided by YOLOv8.

[0070] The original YOLOv8 network's convolutional module consists of convolutional layers and activation functions. However, for animal head pose estimation, there are several key points that are difficult to identify when the head is tilted. Therefore, the improved lightweight downsampling module DDConv incorporates pooling operations and increases the number of channels to extract higher-level features. Simultaneously, the improved lightweight CSYHet convolutional neural network structure, by combining Dual convolution and heterogeneous kernel convolution, significantly reduces the number of parameters. This reduction is achieved in layers 2, 4, 6, and 8 of the Backbone and layers 12 and 1 of the Head. Layers 5, 18, and 21 employ upsampling and feature map concatenation (Concat) to connect with other layers. By optimizing convolution operations and feature fusion, the feature extraction capability and multi-scale feature fusion are enhanced, significantly improving the performance of pose estimation. In the improved detection head CFAHead, the CSYHet module is combined with spatial attention, further improving the model's accuracy in detecting keypoints. Furthermore, the detection head CSYHet module is integrated with the PNP algorithm, achieving accurate estimation from 2D keypoints to 3D pose, improving the accuracy of pose estimation recovery, and making the detection results more intuitive.

[0071] To facilitate real-time detection and deployment of algorithm models at the edge, the number of parameters and the size of the weight file are crucial. The reduction in the number of parameters resulting from algorithm improvements greatly facilitates our deployment at the edge.

[0072] Furthermore, S34 specifically refers to:

[0073] Euler angles include the yaw angle about the X-axis, the pitch angle about the Y-axis, and the roll angle about the Z-axis. The calculation formula is as follows:

[0074]

[0075] In the formula, yaw is the yaw angle, pitch is the pitch angle, roll is the roll angle, and r is the pitch angle. 21 r 11 r 31 r 32 r 33 Let r be an element in the rotation matrix, where r 11 r represents the direction cosine of the rotated X-axis relative to the original X-axis. 21 r represents the direction cosine of the rotated Y-axis relative to the original X-axis. 31 r represents the direction cosine of the rotated Z-axis relative to the original X-axis. 32 r represents the direction cosine of the rotated Z-axis relative to the original Y-axis. 33 This represents the direction cosine of the rotated Z-axis relative to the original Z-axis.

[0076] In one embodiment of the present invention, the animal head pose estimation method of the present invention is used to perform detailed detection and recognition processing on an animal head image of size 640*640*3 based on the improved YOLOv8 pose estimation model. The specific steps are as follows:

[0077] Step 1: Input a 640*640*3 image of an animal head;

[0078] Step 2: The image data passes through the Conv layer, which has 64 output channels. A 3*3 convolution kernel with a stride of 2 is used to generate a feature map of size 320*320*64.

[0079] Step 3: After that, the image passes through the first DDConv module, which contains a convolutional layer with a 3*3 kernel and a stride of 2, and a convolutional layer with a 1*1 kernel to adjust the number of channels and use an activation function to increase the number of output channels to 128 and the output feature map size to 160*160*128.

[0080] Step 4: Then, through the CSYHet module with a depth of n=3, further features are extracted, and the size of the output feature map is the same as that in Step 3.

[0081] Step 5: After passing through the second DDConv module, the number of output channels increases to 256, and the size of the output feature map is 80*80*256.

[0082] Step 6: After passing through the CSYHet module with a depth of n=6, further features are extracted, and the output feature map size is 80*80*256. This layer is named feat1.

[0083] Step 7: After the third DDConv module, the number of output channels increases to 512, resulting in a feature layer with a shape of 40*40*512;

[0084] Step 8: After passing through the CSYHet module with n=6, further features are extracted, outputting a 40*40*512 feature map. This layer is then named feat2.

[0085] Step 9: After the fourth DDConv module, the output feature map size is 20*20*1024;

[0086] Step 10: After passing through the CSYHet module with n=3, the output feature map size is 20*20*1024;

[0087] Step 11: Next, the pyramid pooling module SPPF uses five pooling kernels of different sizes to capture richer multi-scale features, resulting in a 20*20*1024 feature map as output. This layer is named feat3. This completes the feature extraction of the backbone network; the next step is feature fusion.

[0088] Step 12: Upsample the output feat3 of the last layer of the backbone feature extraction network to obtain a feature map with a shape of 40*40*1024;

[0089] Step 13: Perform a concat operation on the channel of Step 12 and the feat2 of the backbone network to obtain a feature map of size 40*40*1536;

[0090] Step 14: After passing through the CSYHet module with n=3, a feature map of shape 40*40*512 is obtained.

[0091] And name this layer feat4;

[0092] Step 15: Perform an upsampling operation on Step 14 to obtain a feature map with a shape of 80*80*512;

[0093] Step 16: Perform a channel-wise concat operation on Step 15 and the feat1 of the backbone network to obtain a feature map of shape 80*80*768.

[0094] Step 17: After passing through the CSYHet module with n=3, a feature map of size 80*80*256 is obtained.

[0095] And name this layer P3out;

[0096] Step 18: Input the output feature map from Step 17 into the DDConv module, with convolution kernels of 3*3 and 1*1, stride of 2, and output feature map size of 40*40*256;

[0097] Step 19: Perform a channel-wise concatenation operation between Step 18 and feat4, and output a feature map with a size of 40*40*738;

[0098] Step 20: After passing through the CSYHet module with n=3, output a feature map of size 40*40*512 and name this layer P4out;

[0099] Step 21: Input the feature map output in Step 20 into the DDConv module and perform convolution operations with a kernel size of 3*3 and a stride of 2 and a kernel size of 1*1. The output feature map size is 20*20*512.

[0100] Step 22: Perform a channel-wise concatenation operation between Step 21 and feat3 of the backbone network to obtain a feature map with a size of 20*20*1536;

[0101] Step 23: Obtain a feature map of size 20*20*1024 through a CSYHet module with n=3. Name this layer P5out;

[0102] Step 24: P3out, P4out, and P5out are fed into the CFAHead detection head in the YOLOHead part as the output of the feature fusion network for detection. It is specifically used for pose estimation key point detection and finally outputs the coordinates of the pose estimation key points.

[0103] Step 25: Prepare the corresponding 3D model points from the 2D keypoints detected in YOLOv8. Based on the animal head annotation definition, use the calibrateCamera function to calculate the camera's intrinsic parameter matrix and distortion coefficients.

[0104] Step 26: Call the PNP algorithm, input the 2D key point coordinates and 3D model points, camera matrix and distortion coefficients, and calculate the rotation vector and translation vector. These vectors describe the pose of the camera with respect to the 3D model points.

[0105] Step 27: Using the solved camera pose information, convert the key points in the 2D image into points in 3D space, and output the Euler angles of the head rotation and draw a 3D bounding box in the image to intuitively display the pose estimation results.

[0106] The comparison results between the improved algorithm in this embodiment and the original YOLOv8n network are shown in Table 1. The overall MAP50 is improved by 1.1%. In addition, the number of parameters of the model is reduced from about 3.16 million in YOLOv8n to 2.21 million in the improved network, which is a total reduction of 30.1% in the number of parameters.

[0107] Table 1 Comparison metrics between the improved algorithm and the original YOLOv8n network

[0108]

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0110] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for estimating animal head pose by integrating computer vision and object detection, characterized in that, Includes the following steps: S1. Obtain images of animal heads from various angles in different scenes, classify and label the images, select ten key points to represent the main features of the animal head, form an animal head dataset, and divide it into training and validation sets. S2, Improved convolutional layer Conv module of YOLOv8 pose estimation model, reconstructed traditional convolutional layer Conv module, improved C2f module and improved detection head PoseHead; S3. PNP algorithm integrated based on improved YOLOv8 attitude estimation model; S4. Train the improved YOLOv8 pose estimation model after integrating the PNP algorithm using the training set and validation set, and obtain the optimal weights. S5. Export the selected weight file in ONNX format and deploy it to the edge device to test the recognition effect. Then, adjust the hyperparameters based on the test results. If the hyperparameters need to be adjusted, retrain the device. If no adjustment is needed, output the detection image directly. S6. Use the trained improved YOLOv8 pose estimation model based on optimal weights to estimate the animal's head pose, output the detection result image and label the animal's category confidence, ten key points and the confidence of each key point, and the head Euler angles.

2. The method for estimating animal head pose by integrating computer vision and object detection according to claim 1, characterized in that, S1 specifically refers to: S11. Collect images of animal heads of various animal categories from various angles in different scenes. Then, filter the collected images, remove blurry images and images where the ten key points are not fully exposed, and organize the remaining images to form a dataset for animal head pose estimation. S12. Using the same object detection labeling method as the YOLO dataset, label the various animal head datasets in different scenes obtained by the dataset labeling software Labelimg, and finally obtain a set of labeled txt files and corresponding datasets of images; S13. Divide the labeled dataset into a training set and a validation set in a 7:3 ratio. During the training process, the weights of the training set and the validation set are continuously adjusted.

3. The method for estimating animal head pose by integrating computer vision and object detection according to claim 1, characterized in that, The Conv module of the improved YOLOv8 pose estimation model in S2 is specifically as follows: First, an average pooling layer is used to reduce the spatial dimension of the feature map. The pooled feature map is divided into two parts. One part goes through a 3*3 convolutional layer to capture local context information, and the other part goes through a max pooling layer followed by a 1*1 convolutional layer, which helps to maintain more global structural information. Then, the outputs of the two branches are concatenated in the depth direction to obtain the improved lightweight downsampling DDConv module.

4. The method for estimating animal head pose by integrating computer vision and object detection according to claim 1, characterized in that, The reconstruction of the traditional convolutional layer Conv module in S2 is specifically as follows: The traditional isomorphic convolution is improved by using convolution kernels of different sizes, including 1*1 and 3*3 kernels, to maintain accuracy while reducing computation, resulting in an improved lightweight CHetConv module.

5. The method for estimating animal head pose by integrating computer vision and target detection according to claim 4, characterized in that, The improved C2f module in S2 is specifically as follows: In the cross-stage local feature fusion module C2f in the original YOLOv8 network, an improved CHetConv module is introduced into the backbone, which uses convolutional kernels of different sizes to replace the original convolution operation. At the same time, the CSP module is introduced to obtain a lightweight multi-scale module CSYHet.

6. The method for estimating animal head pose by integrating computer vision and object detection according to claim 1, characterized in that, The improved detection head PoseHead in S2 is specifically as follows: By combining spatial attention mechanisms with convolution, bicubic interpolation is used to reduce the resolution of feature maps and upsample them to a higher resolution. The receptive field spatial features are dynamically generated according to the size of the convolution kernel to improve performance, resulting in a detection head CFAHead that enhances the model's feature extraction capabilities.

7. The method for estimating animal head pose by integrating computer vision and object detection according to claim 1, characterized in that, S3 specifically refers to: S31. Define a 3D model and create a three-dimensional coordinate system with the center of the animal's head as the origin, including the X-axis, Y-axis, and Z-axis. The 3D model includes 3D key points representing the features of the animal's head. S32. Detect the coordinates of 2D key points in the image based on the improved YOLOv8 pose estimation model, and match the coordinates of 2D key points with the 3D key points in the 3D model. S33. The PNP algorithm is integrated end-to-end with the improved YOLOv8 pose estimation model. The PNP algorithm calculates the rotation and translation vectors describing the camera pose by combining the 3D model keypoints and the corresponding 2D image keypoints with the camera intrinsic parameters and distortion coefficients. This completes the integration of the improved YOLOv8 pose estimation model with the PNP algorithm. Then, the rotation vectors are converted into rotation matrices, and the rotation and translation vectors are combined into a homogeneous transformation matrix. The translation matrix is ​​represented by the homogeneous transformation matrix. Therefore, the rotation and translation matrices together define the pose of the camera relative to the 3D model points. S34. Based on the rotation matrix and translation matrix, the Euler angles are further calculated using the PNP algorithm, where the rotation matrix contains the rotation information of the animal's head around the X, Y, and Z axes. S35. Using the calculated rotation matrix, translation matrix, intrinsic parameter matrix, distortion coefficient, Euler angles, and 3D keypoint coordinate pose information, draw a 3D bounding box and coordinate axes on the image to display the animal's head pose.

8. The method for estimating animal head pose by integrating computer vision and object detection according to claim 7, characterized in that, S34 specifically refers to: Euler angles include the yaw angle about the X-axis, the pitch angle about the Y-axis, and the roll angle about the Z-axis. The calculation formula is as follows: In the formula, yaw is the yaw angle, pitch is the pitch angle, roll is the roll angle, and r is the pitch angle. 21 r 11 r 31 r 32 r 33 Let r be an element in the rotation matrix, where r 11 r represents the direction cosine of the rotated X-axis relative to the original X-axis. 21 r represents the direction cosine of the rotated Y-axis relative to the original X-axis. 31 r represents the direction cosine of the rotated Z-axis relative to the original X-axis. 32 r represents the direction cosine of the rotated Z-axis relative to the original Y-axis. 33 This represents the direction cosine of the rotated Z-axis relative to the original Z-axis.

Images