Animal attitude estimation method and system, computer equipment and storage medium

Abstract

The invention provides an animal attitude estimation method and system, computer equipment and a storage medium, and belongs to the crossing field of computer vision, image processing and machine learning, and the method comprises the steps: carrying out key module upgrading on a YOLOv8n-pose model, introducing AK-ADown and C2fkan modules into a backbone network, and enhancing the multi-scale feature extraction capability; an LSKA-SPPF module is adopted in the neck network, multi-scale context information is fused, and a key area is focused; in an attitude estimation Head network, an RCSOSA module, key point positioning precision is improved through re-parameterization and a channel attention mechanism. According to the method, target detection and attitude estimation are integrated in a single-stage end-to-end network, calculation redundancy is effectively reduced, high-precision and high-efficiency attitude estimation on multiple species such as cattle, horses, sheep and dogs is realized, and the method is suitable for scenes such as ecological monitoring and animal husbandry management.

Description

TECHNICAL FIELD

[0001] The present application belongs to the cross field of computer vision, image processing and machine learning, and particularly relates to an animal pose estimation method and system, a computer device and a storage medium. BACKGROUND

[0002] The research of animal pose estimation involves multiple disciplines such as computer vision, image processing, machine learning and cognitive science, and has important theoretical significance. It is also helpful to understand the visual cognitive mechanism of the brain. In addition, animal pose estimation has important application value in ecological environment monitoring, livestock management, pet monitoring, animal behavior science and animal-machine interaction.

[0003] Deep learning technology has made breakthrough progress in target detection, image segmentation, classification and other fields. In addition, 2D human pose estimation based on deep learning has also shown excellent performance. Unlike traditional part-based methods, deep learning methods use convolutional neural networks to automatically learn higher-level and more comprehensive visual features through stacked convolutional layers and pooling layers. Based on deep learning, 2D human pose estimation is regarded as a top-down key point positioning task, which has become the mainstream method in current 2D pose estimation and has shown significant improvement over traditional methods in complex real-world image or video scenes. With the release of human pose datasets containing multiple poses, effective training of deep learning models becomes feasible. Pioneers such as Ain et al. and Toshev et al. were the first to train CNNs to learn global image features and predict human key point positions. Wei et al. introduced the convolutional pose machine network architecture. In the same year, inspired by the hourglass structure, Newell et al. proposed a stacked hourglass structure. Sun et al. proposed a high-resolution network, which achieved a new breakthrough in 2D human pose estimation and laid the foundation for subsequent pose estimation work. In order to simplify the training process of the model, Li et al. also proposed an online knowledge distillation framework OKDHP. Cao et al. introduced PAF to enhance the correlation between human key points based on the work. Thompson et al. combined convolutional neural networks with Markov random fields to achieve end-to-end human pose estimation. Unlike traditional methods, deep learning-based pose estimation achieves end-to-end key point prediction, eliminating the cumbersome process of manual feature design, multi-stage part detection and spatial relationship modeling. Compared with traditional key point detection methods, deep learning-based network architecture is more superior in performance and generalization ability.

[0004] Compared with human pose estimation, animal pose estimation is still in a relatively early stage and faces unique challenges that human pose estimation does not have. For example: due to its limited impact on human medical or military tasks, it leads to lower public interest in animal pose estimation. Animal key point detection tasks are still less in international computer vision conferences and competitions. Due to the difficulty of animals to cooperate, the collection of data is challenging. In network design, Cao et al. solve the domain gap problem between humans and animals and between different species by fine-tuning AlphaPose for animal pose estimation. Li and Lee, and Mu et al. synthesize and real-world data, and use domain adaptation techniques to bridge the gap between the two. In addition, some end-to-end deep neural network pose estimation models have been proposed. Zuffi et al. proposed a skinless multi-animal linear (SMAL) model, which opened up a new way for reconstructing animal models from a single image.

[0005] The "detection-estimation" two-stage paradigm adopted by mainstream methods will produce a large amount of computational redundancy due to repeated feature extraction, and is also prone to error accumulation, making it difficult to meet the real-time requirements of actual applications; most algorithms are only developed for a single animal, lack of adaptation ability for multi-species animals such as cattle, horses, sheep, and dogs with large differences in size and structure, and cannot cover the needs of multiple scenarios such as ecological monitoring and livestock management; in complex environments such as the wild, affected by factors such as changes in light, target occlusion, and the existence of small targets, existing models have the shortcomings of insufficient feature extraction, low key point positioning accuracy, and poor pose continuity in video sequences; at the same time, the model structure is fixed and lacks modular design, not only difficult to flexibly cope with various performance problems, but also difficult to deploy to unmanned aerial vehicles, infrared cameras and other resource-constrained devices due to parameter redundancy. SUMMARY

[0006] In order to be able to realize target detection and pose estimation at the same time, and realize high-precision animal pose estimation in complex scenes, the present application provides an animal pose estimation method, system, computer device and storage medium.

[0007] In order to solve the problems of high computational complexity and low key point positioning accuracy, the present application provides an animal pose estimation method, comprising: Collecting a plurality of image samples of a target animal.

[0008] Extracting initial contour feature maps of the image samples, fusing all the initial contour feature maps to obtain multi-scale fusion features; converting the multi-scale fusion features into a detection box coordinate of the target animal and a key point coordinate and offset of the target animal in the image sample.

[0009] The target animal's detection box coordinates and key point coordinates and offset of the target animal in the image sample are associated by using an IoU overlap and spatiotemporal constraint method to obtain a posture skeleton structure of the target animal; and the posture skeleton structure is visualized and output to estimate the posture of the target animal by using the visualized result.

[0010] Preferably, the image sample of the target animal is processed by using a LARC_YOLOv8n-pose model to obtain the target animal's detection box coordinates and key point coordinates and offset of the target animal in the image sample; the LARC_YOLOv8n-pose model is based on a traditional YOLOv8n-pose model, and a standard convolution stack in a Head network of the traditional YOLOv8n-pose model is replaced by an RCSOSA module; the RCSOSA module includes one RepVGG and two RCS, a Split channel splitting module, a Concat channel splicing module, a ChannelShuffle channel shuffling module and a SExtention enhanced channel attention module; the SExtention enhanced channel attention module contains a global average pooling; after the RCSOSA module extracts features from multi-scale fused features by using three parallel convolution branches, the ChannelShuffle module is used to shuffle and splice the feature maps output by the three branches in the channel dimension, and the target animal's detection box coordinates and key point coordinates and offset of the target animal in the image are output.

[0011] Preferably, the standard down-sampling convolution and part of the C2f module of the backbone network of the traditional YOLOv8n-pose model are replaced by an AK-Adown module and a C2fkan module, respectively; the AK-Adown module splits the input feature map into two channels by the Split channel splitting module, one channel sequentially passes through a Conv standard convolution, an average pooling and a SiLU activation function, and the other channel sequentially passes through an AKConv variable kernel convolution, a BN regularization and a SiLU activation function, and finally the channel splicing is performed by the Concat channel splicing module to output the extracted initial contour base feature map; the C2fkan module includes a Split channel splitting module, a FasKANConv block, a Concat channel splicing module and a Conv standard convolution; the hierarchical feature map is split by the Split channel splitting module, three fast KAN convolution blocks FasKANConv are used to learn nonlinear features in a mode of a traditional convolution and a fixed activation function, then the channel splicing is performed by the Concat channel splicing module, and finally the mixed features after splicing are fused and the channel is adjusted by using the Conv standard convolution to output the initial contour feature map.

[0012] Preferably, the SPPF module in the Neck feature pyramid network of the traditional YOLOv8n-pose model is replaced by an LSKA-SPPF module; the LSKA-SPPF module comprises a Conv standard convolution, three MaxPool2d maximum pooling, a Concat channel splicing module and an LSKA module; the LSKA module is sequentially connected by a Conv convolution, a spatial convolution Conv_Spatial_h, a BN batch normalization and a residual connection; based on the initial contour feature map, the LSKA-SPPF module generates feature maps of different scales by using three MaxPool2d maximum pooling in series, captures the dependency relationship between different scale features by using a separable deep convolution, and obtains multi-scale fusion features.

[0013] Preferably, the Head network of the LARC_YOLOv8n-pose model comprises a target detection branch network and a pose estimation branch network; the target detection branch network outputs the species category of the target animal, and the pose estimation branch network outputs the positional offset of the target animal in the image.

[0014] Preferably, the key point coordinates comprise a plurality of two-dimensional coordinate points and the confidence of each key point , the key point position of which is calculated by the following formula: ; wherein, is an initial estimated position, is an offset.

[0015] Preferably, the visualization result is displayed by superimposing the detection frame coordinates, the key point coordinates and the skeleton connection lines on the original image sample.

[0016] The application also provides an animal pose estimation system, comprising: an image acquisition module, configured to acquire a plurality of image samples of a target animal.

[0017] a feature extraction module, configured to extract initial contour feature maps of the image samples, fuse all the initial contour feature maps to obtain multi-scale fusion features, and convert the multi-scale fusion features into detection frame coordinates of the target animal and key point coordinates and an offset of the target animal in the image sample.

[0018] A model application module is configured to adopt an IoU overlap degree and space-time constraint method to associate the detection box coordinates of the target animal, the key point coordinates and offset amount of the target animal in the image sample, and obtain the posture skeleton structure of the target animal; and visualize the posture skeleton structure and estimate the posture of the target animal by using the visualized result.

[0019] The application further provides a computer device comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the animal posture estimation methods.

[0020] The application further provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program can implement the steps of any one of the animal posture estimation methods when loaded by a processor.

[0021] The animal posture estimation method provided by the application has the following beneficial effects: The method firstly collects target animal image samples, can targetly cover different postures of animals in a target scene, environment (such as illumination, angle), lays a high-quality data foundation for subsequent processing, and reduces estimation deviation caused by insufficient sample generalization; then extracts initial contour features and fuses multi-scale features, not only grasps the core contour information of the animal structure, but also compensates for the problem of insufficient feature capture of different sizes and distances from the target animal by single scale through multi-scale fusion, effectively adapts to small target or long-distance animal scenes, and improves the comprehensiveness of feature representation; subsequently, detection box coordinates, key point coordinates and offset amount are output synchronously, redundant calculation of traditional detection and key point positioning is avoided, and the offset amount can correct the initial position of the key point, significantly improving the positioning accuracy of the key point; then, by associating the detection box and the key point, each animal target and its corresponding posture key point can be accurately matched, key point confusion in a multi-animal scene is avoided, and the accuracy of the posture skeleton structure is ensured; finally, the posture skeleton is visualized and output, which not only intuitively presents the estimation result, is convenient for manual verification or subsequent behavior analysis, but also forms a coherent closed loop from data input to result application, reduces the actual operation threshold, and balances technical performance and practical value. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the application and the design of the embodiments of the application, the drawings required by the embodiments of the application will be briefly introduced as follows. The drawings in the following description are only part of the embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0023] Figure 1 The flowchart of the animal posture estimation method of the embodiments of the application; Figure 2 Figure 1 is a schematic diagram of an improved YOLOv8n-pose model architecture according to an embodiment of the present application. Figure 3 Figure 2 is a schematic diagram of an AK-Adown module structure according to an embodiment of the present application. Figure 4 Figure 3 is a schematic diagram of a C2fkan module structure according to an embodiment of the present application. Figure 5 Figure 4 is a schematic diagram of an LSKA-SPPF module structure according to an embodiment of the present application. Figure 6 Figure 5 is a schematic diagram of an RCSOSA module structure according to an embodiment of the present application. Figure 7 Figure 6 is a comparison diagram of detection effects of the model on an animal pose dataset. DETAILED DESCRIPTION

[0024] In order to better understand the technical embodiments of the present application and to enable one skilled in the art to carry out the technical embodiments of the present application, the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical embodiments of the present application, and cannot be used to limit the protection scope of the present application.

[0025] The present application provides an animal pose estimation method, specifically as shown in Figure 1 comprises: S1, collecting a plurality of image samples of a target animal.

[0026] Collecting animal pose images and performing data preprocessing, first performing Mosaic data enhancement on the input image, and splicing 4 training images into 1; then using adaptive image scaling (LetterBox) to uniformly adjust the input images of different sizes to a fixed size (640x640); then performing random color space transformation (HSV adjustment), including random disturbance of hue, saturation and brightness; finally, performing normalization processing, scaling the pixel value from the range of 0-255 to the range of 0-1, and using the mean and standard deviation to standardize the image.

[0027] S2, extracting initial contour feature maps of the image samples, fusing all initial contour feature maps to obtain multi-scale fusion features; converting the multi-scale fusion features into detection box coordinates of the target animal and key point coordinates and offsets of the target animal in the image sample.

[0028] By optimizing and integrating each module of the YOLOv8n-pose model, the AK-Adown module and the C2fkan module are respectively used to replace the standard down-sampling convolution and part of the C2f module of the backbone main network of the traditional YOLOv8n-pose model; the LSKA-SPPF module is used to replace the SPPF module in the Neck feature pyramid network of the traditional YOLOv8n-pose model; and the RCSOSA module is used to replace the standard convolution stack in the Head network of the traditional YOLOv8n-pose model, to obtain the modified LARC_YOLOv8n-pose model.

[0029] As shown in Figure 2 , the LARC_YOLOv8n-pose model specifically includes introducing the AK-Adown module to replace the traditional convolution down-sampling module. This module can effectively reduce feature space loss and enhance the model's perception of fine-grained features in input images, improve its ability to extract key point features of different scales of animals, and thus enhance feature representation. As shown in Figure 3 , the AK-Adown module splits the input feature map into two channels through the Split channel splitting module. One channel sequentially passes through the Conv standard convolution, average pooling and SiLU activation function, and the other channel sequentially passes through the AKConv variable kernel convolution, BN regularization and SiLU activation function. Finally, the channel splicing module is used for channel splicing, and the final extracted hierarchical feature map is output.

[0030] The C2fkan module is used to introduce a learnable nonlinear activation function in the backbone network, making the convolution process more flexible and efficient. This module enhances the model's adaptability to complex backgrounds and pose changes, as if giving the model stronger "flexibility", thereby improving key point detection accuracy. As shown in Figure 4 , the C2fkan module includes a Split channel splitting module, a FasKANConv block, a Concat channel splicing module and a Conv standard convolution. The Split channel splitting module is used to split the hierarchical feature map, three fast KAN convolution blocks FasKANConv are used to replace the traditional convolution and fixed activation function mode to learn nonlinear features, and then the Concat channel splicing module is used for channel splicing. Finally, the standard convolution Conv is used to fuse and adjust the channels of the spliced mixed features, and the initial contour feature map is output.

[0031] The LSKA-SPPF module integrates large-scale and local feature information, reduces the computational overhead caused by large convolution kernels, improves the attention of the model to the occluded and key point fuzzy area, improves the calculation and storage efficiency in the feature extraction and enhancement process, and ensures that the network can improve the inference speed while maintaining high accuracy. As shown in Figure 5 The LSKA-SPPF module includes a Conv standard convolution, three MaxPool2d maximum pooling, a Concat channel splicing module, and an LSKA module. The LSKA module is connected in sequence by a Conv convolution, a spatial convolution Conv_Spatial_h, a BN batch normalization, and a residual connection. Based on the initial outline feature map, the LSKA-SPPF module uses three MaxPool2d maximum pooling in series to generate feature maps of different scales, uses a separable deep convolution to capture the dependency between different scale features, and obtains multi-scale fusion features.

[0032] The RCSOSA module enhances channel feature mixing and information flow, reduces feature redundancy, improves the feature representation and generalization ability of the model, and provides more comprehensive information support for key point positioning. As shown in Figure 6 The RCSOSA module includes a RepVGG reparameterization convolution kernel and two RCS reparameterization convolution kernels, a Split channel splitting module, a Concat channel splicing module, a Channel Shuffle channel mixing module, and a SExtention enhanced channel attention module. The SExtention enhanced channel attention module includes a global average pooling. After the RCSOSA module uses three parallel convolution branches to extract features from the multi-scale fusion features, it uses the Channel Shuffle module to mix and splice the feature maps output by the three branches in the channel dimension, and outputs the detection box coordinates and key point coordinates of the target animal.

[0033] Through the optimization and integration of the above modules, the model can more effectively extract key point features, reduce the influence of morphological differences between different species, and reduce the dependence on large-scale labeled data in the training process. In testing, the model achieved better accuracy and recall rate than the original YOLOv8n-pose model on multiple public datasets (such as AnimalPose and AP-10K), significantly reducing recognition errors and key point positioning deviations caused by target occlusion and complex morphology, and fundamentally reducing the problem of false positives and false negatives in pose estimation tasks.

[0034] The above technical solutions of the embodiments of the present application have at least the following technical effects or advantages: Mosaic data augmentation can improve the model's ability to recognize small targets and complex scenes, size scaling can reduce the distortion of the set, color space transformation can enhance the model's robustness to light changes, and standardization and normalization can accelerate model convergence. The entire preprocessing process can significantly improve the generalization performance of the model while maintaining high training efficiency.

[0035] Backbone part: input an original RGB image The network first extracts features through multiple convolutions: ; where I is the input image tensor with dimensions (B, C, H, W), where B is the batch size; C is the input channel number (i.e., the RGB channel); H and W are the height and width of the input image, and Conu(·) is a standard two-dimensional convolution operation used to extract local spatial features. is the output feature map after convolution, which serves as the input for subsequent modules.

[0036] After multiple stacked convolution and feature enhancement modules, Conv (standard convolution layer) is used to extract local features. C2f performs channel splitting and residual linking to reduce computational complexity and improve feature expression. AK-Adown uses a convolution module that combines adaptive weights and down-sampling to reduce spatial resolution while enhancing channel features.

[0037] Down-sampling is achieved using channel splitting and adaptive weights on the input feature map X: ; ; ; where X is the input feature map, AvgPool(·) is the average pooling function, which performs spatial down-sampling on the input feature map; Split(X) is the channel splitting operation, which divides the feature map into two parts (Xplit, Xsplitz) in the channel dimension, AKConu(·) is the adaptive kernel convolution (Adaptive Kernel Convolution), which is a dynamic convolution with variable weights used to capture irregular spatial structures, and BN(·) represents the batch normalization layer (Batch Normalization) used to stabilize training, Represents a nonlinear activation, and Concat(·): channel concatenation operation merges two feature maps into an output feature map.

[0038] After the backbone part, three feature maps of different scales are output: ; P3, P4, P5 are low-level features, middle-level features, and high-level features, respectively.

[0039] The technical solutions in the embodiments of the present application have at least the following technical effects or advantages: The ADown module emphasizes multi-scale feature fusion, which is particularly important for animal key point detection, because the posture and key point position of an animal can change at different scales. By effectively fusing features at different scales, the model can more accurately locate and identify key points. In addition, the lightweight nature of the ADown module makes the model less prone to overfitting when processing complex tasks, especially when the dataset is small, which has a positive impact on training results. The specific implementation of ADown includes convolution operations, stride adjustment, and parameter optimization. We further adopt AKConv (variable kernel convolution technology) to replace the traditional convolution in the ADown module.

[0040] Traditional convolution operations localize features through a regular sampling grid, such as the sampling grid R of a standard 3x3 convolution, defined as follows. However, AKConv aims to handle irregularly shaped convolution kernels, where the center position may not be fixed. To allow irregular convolution kernels to have a sampling network, Xin Zhang et al. proposed an algorithm to generate initial sampling coordinates for the convolution kernel Pn. This algorithm generates a regular sampling grid and an irregular grid, and finally connects them to generate the overall sampling grid. After defining the initial coordinates Pn of the irregular convolution, the convolution operation corresponding to the position P0 is defined as follows.

[0041] where R represents the sampling coordinate set of the standard 3x3 convolution. is the convolution center pixel position, The relative position offset of the nth sampling point of the convolution kernel; W is the convolution parameter. In AKConv, the offset of the corresponding kernel is first obtained through convolution operation, and its dimension is (B, 2N, H, W), where N is the size of the convolution kernel. Taking N=5 as an example. Then the modified coordinates are obtained by summing the offset and the original coordinates (P0+Pn). Finally, the features at the corresponding positions are obtained through interpolation and resampling.

[0042] AKConv allows the use of convolution kernels of any shape and quantity, thereby overcoming the limitations of standard convolution. Each neuron in traditional convolution only focuses on a fixed-size local region in the input data, making it difficult to effectively capture information from other windows. Moreover, the fixed size of the traditional convolution kernel is not suitable for capturing features at all scales.

[0043] By introducing AKConv, the ADown module can pay more attention to the information in the image, thereby improving the accuracy of key point detection of the model. Meanwhile, AKConv helps to effectively suppress features irrelevant to the task and reduce noise influence, further improving the robustness of the model.

[0044] Neck part: A feature integration strategy is adopted, including LSKA-SPPF to realize large receptive field feature integration and spatial pyramid pooling process as follows: ; ; ; Finally, the channels are integrated by convolution: ; wherein, (·) is a max pooling operation, and the pooling kernel size is , used to extract different scale features, Concat(·): concatenating the pooling results of different scales in the channel dimension, LSKA(·) is a large kernel attention module (Large Separable Kernel Attention), which realizes equivalent large receptive field by decomposing a large convolution kernel into two-dimensional convolution (horizontal + vertical); The output fusion feature is used as the pyramid fusion input.

[0045] The Upsample+Concat concatenates the features in the top-down feature pyramid structure, and the RCSOSA adopts a reparameterized convolution module (RCS) and an attention mechanism to enhance the channel weight. Finally, the features are integrated to obtain:

[0046] ; ; wherein, Upsample(·) is an up-sampling operation, used to expand the feature map size, , are feature maps from different layers, and Concat(·) represents a concatenation operation for feature fusion; is the final fusion feature, used for the Head prediction stage.

[0047] The above technical solutions of the embodiments of the present application have at least the following technical effects or advantages: Visual attention networks (VAN) equipped with large kernel attention (LKA) modules have shown superior performance in multiple vision-based tasks, outperforming vision Transformers. However, as the size of the convolution kernel increases, the deep convolution layers in these LKA modules cause a quadratic increase in computation and memory usage, severely impacting the practical application of the model. To alleviate the high computational cost of deep convolution with large kernel size in LKA-trivial, M.-H. Guo et al. proposed to decompose the deep convolution of large kernel into two steps: first, use deep convolution with a small kernel, and then use a relatively large kernel through dilated deep convolution. This large kernel decomposition helps to alleviate the problem of quadratic increase in computational cost with the increase of kernel size when deep convolution only uses a large kernel.

[0048] Subsequently, Kin Wai Lau, Lai-Man Po et al. proposed that by splitting the two-dimensional weight kernel of deep convolution into two cascaded one-dimensional separable weight kernels, an equivalent improved configuration of LKA can be obtained. They call the modified configuration of this LKA module LSKA. This decomposition strategy not only significantly reduces the computational complexity and memory usage, but also enables the direct use of deep convolution layers with large kernel in the attention module without the need for additional computational blocks. In addition, studies have shown that the introduction of the LSKA module in VAN can achieve comparable performance to the standard LKA module, while having obvious advantages in computational and memory efficiency. This innovation enables the model to better cope with resource constraints while ensuring performance.

[0049] Based on LSKA, we further introduced a Dropout layer to effectively reduce the risk of overfitting of the model. The Dropout layer randomly discards a certain proportion of neurons, prompting the network to learn more robust feature representations, thereby improving the generalization ability of the model. In addition, we optimized the skip connection in LSKA, allowing information to flow more freely in the network. This design preserves the details and semantic information in the original input data, ensuring that information is not lost during transmission, thereby improving the performance of the model in subsequent layers. The optimized attention mechanism is integrated with the SPPF module in the YOLOv8 structure, further enhancing the effectiveness of target detection.

[0050] The Head part performs joint detection of bounding boxes and key points.

[0051] ; Among them, Boxhead is used for bounding box detection (class, confidence, and bounding box coordinates). Keypointhead is used for key point position regression and confidence prediction.

[0052] S3, adopt the IoU overlap degree and the space-time constraint method, associate the detection box coordinates of the target animal and the key point coordinates and offset of the target animal in the image sample to obtain the posture skeleton structure of the target animal; visualize the posture skeleton structure and estimate the posture of the target animal by using the visualized result.

[0053] In a single frame scene, the association process is based on the output of the double-branch network. First, the detection box output by the target detection branch (including coordinates, category and confidence, which has been filtered by NMS) is used to locate the spatial range of the individual. Then, the key point region (such as the minimum bounding box or the weighted merged region) of the potential individual is constructed by the key points output by the posture estimation branch (extracted by the heat map peak value and corrected by the offset, and the high confidence points are screened). Then, the IoU value of the detection box and the key point region is calculated, and the spatial matching is realized according to the set threshold and the Hungarian algorithm, which ensures that each detection box corresponds to a unique key point subset. At the same time, combined with the large receptive field features of the LSKA-SPPF module and the key point confidence screening, the interference of occlusion and noise on matching is reduced.

[0054] In a video frame sequence, the space-time constraint further optimizes the association result. Based on the assumption of animal motion continuity, the space-time matching score is obtained by calculating the IoU of the detection boxes of adjacent frames and the displacement of the key points, and the correspondence relationship of the individual across frames is determined; for the individual that is occluded or temporarily disappeared, the detection box and the key point distribution are predicted by Kalman filter to realize the re-association after occlusion. Finally, combined with the fixed bone topology of the species (such as the head-neck-trunk connection rule), the reasonableness of the key point connection and the boundary constraint of the detection box are verified, and the IoU spatial matching and the space-time constraint result are fused to finally construct the accurate posture skeleton of each individual, which not only adapts to the lightweight demand of single-stage architecture, but also improves the association robustness in complex scenes.

[0055] The detection box and the key point association process adopts the IoU overlap degree and the space-time constraint method to construct the posture of the corresponding individual, as shown in Figure 7 The detection effect comparison chart of the LARC_YOLOv8n-pose model on the animal posture data set. In order to prove the effectiveness of the method, we carried out experiments on two different public data sets, and the experimental results are shown in Tables 1, 2, and 3. We use accuracy, recall rate and mAP@50 as evaluation indexes. Table 3 is the experimental data on some animals. From the experimental results, our method greatly improves the ability of animal detection and posture estimation:

[0056] Table 1 Experimental data on the public data set animalpose Table 2 Experimental data on the public data set AP-10K Table 3 comparison results of five kinds of animal experiments of cattle, sheep, dogs, cats and horses The embodiment of the application provides an animal posture estimation method based on an improved YOLO algorithm and a corresponding model, and solves the problems of complex calculation and insufficient animal key point detection accuracy in the existing model, and realizes accurate identification of animal posture key points in a complex scene.

[0057] In the technical solutions in the above embodiments of the application, the universal "detection and re-estimation" paradigm can achieve high accuracy, but due to the need to connect multiple models in series, the calculation redundancy is serious, and it is difficult to meet the real-time requirement. The target detection and posture estimation are integrated into a single network for one-time forward propagation, and the calculation amount is reduced by sharing the backbone features.

[0058] The "detection-posture estimation" integrated single-stage network is adopted, the repeated calculation of the traditional two-stage is avoided by sharing the backbone feature extraction, the redundant parameter amount and the calculation amount are greatly reduced, the BoxmAP50 on the AnimalPose dataset reaches 0.963, the PosemAP50 reaches 0.864, which is better than the comparison model Yolov8n-pose, and the efficiency and the high precision are considered to meet the real-time requirement; meanwhile, the embodiment of the application adopts modularized split design, the backbone network, the feature pyramid, the detection / pose branch can be flexibly combined and replaced, such as enhancing feature extraction through C2fkan and AK-Adown, optimizing multi-scale fusion by means of LSKA-SPPF, which can solve the problems of insufficient features and inconsistent scales, and has strong expansibility and maintainability; the improvement of each core module further improves the robustness, AK-Adown reduces the feature space loss to perceive fine-grained features, C2fkan adapts to complex background and posture changes, LSKA-SPPF integrates large-scale and local features to cope with occlusion, and RCSOSA optimizes channel feature mixing to reduce redundancy, so that the model can still work stably in complex environments such as wild environments; and the embodiment of the application supports multiple species such as cattle, horses and sheep through multi-scale fusion and key point topological relationship construction, and naturally supports multiple species such as cattle, horses and sheep, as shown in Table 3, the PosemAP50 of the horse is improved by 3.6% compared with Yolov8n-pose, breaking the single-species limitation and covering multiple scenes such as ecological monitoring and livestock management; meanwhile, it controls the model size by reducing parameter redundancy (such as replacing traditional down-sampling with AK-Adown), and has the characteristics of lightweight, which can be deployed to unmanned aerial vehicles, infrared cameras and other resource-limited devices, and has strong landing performance; in addition, the embodiment of the application realizes an end-to-end process from data input to visual output, without additional intermediate steps, finally displays the original image by superimposing the detection frame, key points and skeleton lines, intuitively presents the results, and reduces the actual application threshold.

[0059] Based on the same inventive concept, the application further provides an animal posture estimation system, comprising: an image acquisition module, configured to acquire a plurality of image samples of the target animal.

[0060] a feature extraction module, configured to extract initial contour feature maps of the image samples, fuse all the initial contour feature maps to obtain multi-scale fusion features, and convert the multi-scale fusion features into a bounding box coordinate of the target animal and a key point coordinate and offset of the target animal in the image sample.

[0061] a model application module, configured to associate the bounding box coordinate of the target animal and the key point coordinate and offset of the target animal in the image sample by using an IoU overlap degree and a spatiotemporal constraint method to obtain a pose skeleton structure of the target animal, and visualize the pose skeleton structure to estimate the pose of the target animal by using a visualized result.

[0062] The application further provides a computer device, which comprises a processor, an internal bus, a network interface, a memory and a nonvolatile memory at a hardware level, and can further comprise other hardware required by a business. The processor reads a corresponding computer program from the nonvolatile memory into the memory and then runs to implement the animal pose estimation method provided above.

[0063] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can be used to execute the animal pose estimation method provided above.

[0064] The specific limitations of the animal pose estimation method computing system can be seen from the limitations of the animal pose estimation method provided above, and will not be described here. Each module in the animal pose estimation system can be realized by software, hardware and a combination thereof in whole or in part. Each module can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0065] Each technical feature of the above embodiments can be combined arbitrarily, and to make the description concise, each technical feature in the above embodiments is not described in all possible combinations, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application. In addition, the above embodiments only express several embodiments of the present application, the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. An animal posture estimation method, characterized in that, include: Collect multiple image samples of the target animal; Extract the initial contour feature map of the image sample, fuse all the initial contour feature maps to obtain multi-scale fused features; The multi-scale fusion features are converted into the detection box coordinates of the target animal and the key point coordinates and offsets of the target animal in the image sample. Using the IoU overlap and spatiotemporal constraint method, the coordinates of the detection box of the target animal and the coordinates and offsets of the key points of the target animal in the image sample are correlated to obtain the pose skeleton structure of the target animal; the pose skeleton structure is visualized and output, and the pose of the target animal is estimated using the visualization results.

2. The animal posture estimation method according to claim 1, characterized in that, The LARC_YOLOv8n-pose model is used to process image samples of the target animal to obtain the bounding box coordinates, keypoint coordinates, and offsets of the target animal within the image sample. The LARC_YOLOv8n-pose model is based on the traditional YOLOv8n-pose model, replacing the standard convolutional stack in the head network with an RCSOSA module. The RCSOSA module includes: a reparameterized convolutional kernel RepVGG, two reparameterized convolutional kernels RCS, a Split module, a Concat module, a ChannelShuffle module, and an SExtention module. The SExtention module incorporates global average pooling. After extracting features from multi-scale fused features using three parallel convolutional branches, the RCSOSA module uses the ChannelShuffle module to shuffle and concatenate the feature maps output from the three branches along the channel dimension, outputting the bounding box coordinates and keypoint coordinates and offsets of the target animal within the image.

3. The animal posture estimation method according to claim 2, characterized in that, This also includes replacing the standard downsampling convolutions and some C2f modules of the backbone network in the traditional YOLOv8n-pose model with AK-Adown and C2fkan modules, respectively. The AK-Adown module splits the input feature map into two channels using a Split channel splitting module. One channel sequentially passes through Conv standard convolution, average pooling, and SiLU activation function, while the other channel sequentially passes through AKConv variable kernel convolution, BN regularization, and SiLU activation function. Finally, the channels are concatenated using a Concat channel concatenation module to output the extracted features. The initial contour feature map is generated by the C2fkan module, which includes a Split channel splitting module, a FasKANConv block, a Concat channel concatenation module, and a Conv standard convolution. The Split channel splitting module splits the hierarchical feature map into channels, and three fast KAN convolution blocks (FasKANConv) are used to learn non-linear features instead of the traditional convolution and fixed activation function. The Concat channel concatenation module then concatenates the channels, and finally, the Conv standard convolution is used to fuse and adjust the channels of the concatenated features to output the initial contour feature map.

4. The animal posture estimation method according to claim 2, characterized in that, It also includes replacing the SPPF module in the Neck feature pyramid network of the traditional YOLOv8n-pose model with the LSKA-SPPF module; the LSKA-SPPF module includes: Conv standard convolution, three MaxPool2d max pooling, Concat channel concatenation module and LSKA module; the LSKA module is composed of Conv convolution, spatial convolution Conv_Spatial_h, BN batch normalization and residual connection connected in sequence; based on the initial contour feature map, the LSKA-SPPF module uses three MaxPool2d max pooling in sequence to generate feature maps of different scales, and uses separable depthwise convolution to capture the dependencies between features of different scales to obtain multi-scale fused features.

5. The animal posture estimation method according to claim 2, characterized in that, The LARC_YOLOv8n-pose model's Head network includes an object detection branch network and a pose estimation branch network; the object detection branch network outputs the species category of the target animal, and the pose estimation branch network outputs the position offset of the target animal in the image.

6. The animal posture estimation method according to claim 1, characterized in that, The key point coordinates include multiple two-dimensional coordinate points. and the confidence level of each key point The key point locations are calculated using the following formula: ； in, For the initial estimated position, This is the offset.

7. The animal posture estimation method according to claim 1, characterized in that, The visualization results are displayed by overlaying the detection box coordinates, key point coordinates, and skeleton lines onto the original image sample.

8. An animal posture estimation system, characterized in that, include: The image acquisition module is used to acquire multiple image samples of the target animal; The feature extraction module is used to extract the initial contour feature map of the image sample, fuse all the initial contour feature maps to obtain multi-scale fusion features, and convert the multi-scale fusion features into the detection box coordinates of the target animal and the key point coordinates and offsets of the target animal in the image sample. The model application module is used to associate the coordinates of the detection box of the target animal with the coordinates and offsets of the key points of the target animal in the image sample using the IoU overlap and spatiotemporal constraint method to obtain the pose skeleton structure of the target animal; the pose skeleton structure is visualized and output, and the pose of the target animal is estimated using the visualization results.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 7.

Images