Skeleton feature-based live pig behavior identification method

Abstract

The invention discloses a live pig behavior recognition method based on skeleton features, and the method comprises the following steps: collecting and constructing a data set; constructing a graph convolutional network model for behavior recognition, and introducing a residual structure into the graph convolutional network model; a Yov8-Pose model is utilized to extract live pig skeleton features from an input video, and the live pig skeleton features are input into the graph convolutional network model; and performing distance weighting on the information of the skeleton characteristics of the live pigs, and outputting an identification result of accurate classification of three behaviors of standing, sitting and lying of the live pigs. The image data enhancement method has the advantages that firstly, the image data enhancement method is provided for the situation that key points in an actual scene may be detected by mistake and missed; the posture of the live pig is estimated by using Yolov8-pose, and AP50 reaches 83%; and thirdly, the average recognition accuracy of the constructed behavior recognition graph convolutional network model on the three behaviors of standing, lying and sitting reaches 87.3%, wherein the sitting posture recognition accuracy reaches up to 94.1%.

Description

Technical Field

[0001] The present invention relates to the technical field of pig behavior recognition, and in particular to a pig behavior recognition method based on skeleton features. Background Art

[0002] As pig farms develop towards scale and intelligence, real-time monitoring of pig health has gradually become an important demand in the production process. Pig behavior is closely related to its health status and is an important basis for evaluating the status of pigs. The traditional way of manual monitoring of pig behavior by breeders is often subjective, inefficient and imprecise, while with the help of sensors or artificial intelligence technology, efficient, low-cost and non-interference behavior monitoring can be achieved. At present, research on pig behavior recognition at home and abroad mainly includes recognition methods based on contact sensors and deep learning methods based on computer vision. Although the deep learning method based on vision performs well in pig behavior recognition, its effect will also be affected by environmental factors such as insufficient light and limited space in the pig house. At the same time, in the process of extracting pig features, due to the non-rigid body and changeable posture of pigs, the existing pig feature extraction method is complex and has low accuracy. Summary of the invention

[0003] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is to provide a pig behavior recognition method based on skeleton features.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a method for pig behavior recognition based on skeleton features, comprising the following steps: collecting and constructing a data set; constructing a graph convolutional network model for behavior recognition, introducing a residual structure into the graph convolutional network model to accelerate the training and optimization process of the model; using the Yolov8-Pose model to estimate the posture of the input video, detect pig key points, extract pig skeleton features, and input the pig skeleton features into the graph convolutional network model; introducing a weighting strategy to perform distance weighting on the information of the pig skeleton features, while retaining the information of distant neighbor nodes, enhancing the feature expression of nearby node; the graph convolutional network model outputs recognition results for accurate classification of three behaviors of pigs: standing, sitting, and lying.

[0007] Preferably, the pig skeleton feature includes the following processing steps in the graph convolutional network model: representing the connection relationship of the skeleton key points as an adjacency graph G=(V, E), where V represents a set of nodes and E is a set of connections or edges between nodes; defining the behavior recognition of pigs as a graph problem; and then connecting the pig skeleton key points to form an adjacency matrix A, where each element Ai of the matrix is j It indicates whether there is a direct connection relationship between node i and node j. The adjacency matrix A is a square matrix of |V|×|V|, and its scale is determined by the number of nodes in the graph. The extracted key points of the pig skeleton are input into the graph convolutional network model to extract feature vectors and predict their posture.

[0008] Preferably, the graph convolutional network model is expressed as:

[0009]

[0010] In the formula, Add the identity matrix to the adjacency matrix A, yes The diagonal matrix of the degree matrix, H (l) is the node feature of the lth layer, W (l) is the weight matrix of the layer and σ is the activation function.

[0011] Preferably, the graph convolutional network model includes introducing a basic convolution block, which captures the local relationship between nodes by aggregating neighborhood node features and combining adjacency matrix information.

[0012] Preferably, the standard convolution operation of the basic convolution block comprises the following steps: given a convolution operator with a kernel size of K×K and an input feature map f with c channels in ; The single-channel output value at the spatial position x is calculated by aggregating the information of the K×K neighborhood around x; For each position x, the output value f out The calculation formula for (x) is:

[0013]

[0014] Where, sampling function s: Z 2 ×Z 2 →Z 2 Enumerates the neighbors of position x, weight function θ: Z 2 →R 2 A weight vector in c-dimensional real space is provided for each convolution kernel position, which is used to calculate the inner product with the sampled c-dimensional input feature vector.

[0015] Preferably, the standard convolution operation includes sampling the direct neighbors of each node and directly selecting the neighbor nodes as the sampling results; marking the neighbor nodes by a fixed number of partitions and weighting them according to the partitions where the nodes are located, and the node v i The neighbor set of i )={v j |d(v j , v i )≤1}, sampling function p: B(v i )→V is p(v i , v j )=v j .

[0016] Preferably, the distance weighting comprises the following steps:

[0017] The weight calculation is as follows:

[0018] weight ij =exp(-||p i -p j ||)

[0019] In the formula, weight ij is the weight between nodes i and j, ||p i -p j || is the Euclidean distance between nodes i and j; Substitute the weight calculation into the output value f out (x), we get the following formula:

[0020]

[0021] In the formula, Z i (v j ) is the normalization factor.

[0022] Preferably, the data set also includes a posture estimation data set and a pig behavior data set, and the pig behavior data set is completed by performing behavior annotation on the posture estimation data set.

[0023] Preferably, the posture estimation data set uses image annotation to divide the skeleton model into four parts: head, back, tail and limbs, with a total of 14 key points, and is divided into a training set and a test set in a ratio of 8:2.

[0024] Preferably, the pig behavior dataset includes the following construction steps: defining three behaviors of pigs: standing, sitting, and lying; eliminating pig data with a visible body range of less than 60%, extracting the skeleton data of each image, and saving it as a cSV file after screening; and enhancing the data by randomly removing 1-3 key points from samples and randomly moving 1-3 key points by 0-8 pixels.

[0025] The beneficial effects of the present invention are as follows: first, by collecting, annotating and generalizing pig images, a data set covering pig target detection, key point extraction and behavior recognition is constructed, a graph convolutional network data set for behavior classification is constructed, and a graph data enhancement method is proposed for situations in which key points may be misdetected or missed in actual scenarios; Yolov8-pose is used to estimate the posture of pigs, and its AP50 reaches 83%; third, the constructed behavior recognition graph convolutional network model has an average recognition accuracy of 87.3% for the three behaviors of standing, lying and sitting, among which the sitting posture recognition accuracy is as high as 94.1%. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0027] Figure 1 This is a schematic diagram of key point markings of a pig skeleton according to the present invention;

[0028] Figure 2 This is a schematic diagram of pig behavior annotation according to the present invention;

[0029] Figure 3 It is a schematic diagram of the overall process of the pig behavior identification method of the present invention;

[0030] Figure 4 It is a schematic diagram of the structure of the Yolov8-Pose network model of the present invention;

[0031] Figure 5 It is a schematic diagram of the structure of the graph convolutional network model described in the present invention;

[0032] Figure 6 A distance weighted schematic diagram of two nodes as an example of the present invention;

[0033] Figure 7 It is a schematic diagram of the pig posture estimation result of the present invention;

[0034] Figure 8 A schematic diagram of an error result of posture estimation according to the present invention;

[0035] Fig. 9 This is a schematic diagram of the pig behavior recognition results under different viewing angles described in the present invention. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0037] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0038] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0039] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0040] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0041] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0042] Example 1

[0043] The daily behavior of pigs is closely related to their individual health status, and the behavior of pigs can be judged by extracting skeleton features. To this end, this embodiment proposes a method for pig behavior recognition based on skeleton features. First, the Yolov8-Pose algorithm is used to detect pig key points, extract pig skeleton features, and input them into the graph convolutional network; then, a residual structure is introduced into the graph convolutional network to accelerate the training and optimization process of the model; finally, a weighting strategy is introduced to perform distance weighting on key point information at different positions, while retaining the information of distant neighbor nodes, the feature expression of neighbor nodes is further enhanced, thereby improving the behavior recognition performance of the model. Specifically, a method for pig behavior recognition based on skeleton features includes the following steps:

[0044] S1: Collect and construct datasets;

[0045] S2: Build a graph convolutional network model for behavior recognition, introduce a residual structure into the graph convolutional network model, and accelerate the training and optimization process of the model;

[0046] S3: using the Yolov8-Pose model to estimate the posture of the input video, detect the key points of the pig, extract the pig skeleton features, and input the pig skeleton features into the graph convolutional network model;

[0047] S4: introducing a weighting strategy to weight the information of the pig skeleton features by distance, thereby enhancing the feature expression of the neighboring nodes while retaining the information of the distant neighboring nodes;

[0048] S5: The graph convolutional network model outputs recognition results that accurately classify the three behaviors of pigs: standing, sitting, and lying.

[0049] To more clearly illustrate the pig behavior recognition method based on skeleton features proposed in this embodiment, the data set in step S1 also includes a posture estimation data set and a pig behavior data set, and the pig behavior data set is completed by annotating the posture estimation data set with behaviors.

[0050] The posture estimation dataset uses image annotation to divide the skeleton model into four parts: head, back, tail and limbs, with a total of 14 key points, and is divided into training set and test set in an 8:2 ratio.

[0051] The pig behavior dataset includes the following construction steps,

[0052] Define the three behaviors of pigs: standing, sitting, and lying;

[0053] The data of pigs with a visible body range of less than 60% were eliminated, and the skeleton data of each image was extracted and saved as a csv file after screening;

[0054] The data is enhanced by randomly removing 1-3 key points from samples and randomly moving 1-3 key points by 0-8 pixels.

[0055] More specifically, pig posture estimation is an important part of the behavior dataset. The video in this embodiment comes from the dataset given by Wang et al. It contains multi-view video collection of pigs from top and side views. The image annotation divides the skeleton model into four parts: head, back, tail and limbs, with a total of 14 key points. The specific annotation method is as follows: Figure 1 As shown, the connection method of the key points is shown in Table 1 below. On the basis of the original data set, this embodiment additionally annotates 829 images for posture estimation, forming an original data set containing 2020 annotated images. In order to improve the generalization ability of the model, the image is enhanced using various methods such as salt and pepper noise, Gaussian blur, brightness adjustment, saturation adjustment, horizontal flipping, etc. of Python's Imague data enhancement library to ensure that the model has a good skeleton extraction effect in a pig house environment with poor lighting and complex environment. The augmented data set totals 8080 images, which are divided into training set and test set in a ratio of 8:2.

[0056] Table 1: Connection method of skeleton key points.

[0057]

[0058] The pig behavior dataset is completed by annotating the behavior of the posture estimation dataset. The construction of the behavior dataset first requires manual observation of the behavior of pigs in the image. The definitions of the three behaviors of standing, sitting, and lying are shown in Table 2. Figure 2 The diagram of behavior annotation is shown. To prevent the incomplete data caused by occlusions such as feeding troughs and waterers from affecting the model, the data of pigs with a visible range of less than 60% of the body are removed, and the skeleton data of each image is extracted and saved as a csv file after screening. In addition, in order to deal with the possibility of misdetection or omission of key points in actual scenes, the following two enhancement methods are used for the data:

[0059] 1. Randomly remove 1-3 key points from the sample;

[0060] 2. Randomly move 1-3 key points by 0-8 pixels;

[0061] Based on the above method, a multimodal dataset for pig behavior recognition was constructed, with a total of 2394 samples. The number of samples for standing, lying, and sitting were 1222, 845, and 327, respectively. 10% of the total dataset was selected as the validation set, and the remaining samples were divided into training and test sets at a ratio of 8:2. To avoid sample imbalance, a total of 783 samples were retained for each category after processing.

[0062] Table 2: Definitions of pig behaviors.

[0063]

[0064] Furthermore, in steps S2-S5, a two-stage architecture is adopted to identify the behavior of pigs by analyzing their skeleton features. The overall process is as follows: Figure 3 First, the Yolov8-pose model is used to estimate the posture of the input video and extract the key points of the pig skeleton; then, the key point data is input into the graph convolutional network to extract features for behavior recognition.

[0065] The posture estimation proposed in this embodiment is a key technology for identifying different postures after detecting the key points of the target in the image or video. This embodiment uses Yolov8-Pose to estimate the posture of the pig. Yolov8-Pose is a model based on Yolov8 released to detect human posture information. It not only inherits the high-efficiency characteristics of the Yolo series, but also can accurately detect the key points of the human body and realize posture estimation. Compared with Yolov5, Yolov8 designs a C2F module on the basis of adding convolutional layers to enhance feature connectivity and network expression capabilities. By adopting the Anchor-Free design, Yolov8 simplifies hyperparameter adjustment and improves training and reasoning efficiency. The use of EfficientRep as the backbone network significantly reduces the number of model parameters and calculations, and reduces deployment costs and resource consumption. In addition, the FPN+PAN structure improves the hierarchical flow of features in the network and enhances the detection capabilities of targets of different sizes. The end-to-end training method not only improves model performance, but also simplifies the application process. The network structure diagram of Yolov8-pose is shown below. Figure 4 shown.

[0066] Use skeleton information combined with graph convolutional network (GCN) for behavior recognition. The connection relationship of skeleton key points can be naturally represented as an adjacency graph, which can effectively capture the key information in the behavior. GCN expands the application scope of convolutional neural networks and is suitable for graph data of arbitrary structure. Compared with traditional CNN, GCN can process graph structure data with more complex relationships between nodes. It should be noted that GCN is a graph convolutional network model, which is also simplified as a graph convolutional network in this embodiment.

[0067] In this embodiment, in GCN, data is represented as a graph, in which nodes represent entities (such as key points of a pig skeleton) and edges represent the relationship or connection between nodes (such as the connection relationship between key points of the skeleton). For a graph G = (V, E), V represents a set of nodes, and E defines the connection between nodes or a set of edges. The connectivity of the graph can be described by the adjacency matrix A, and each element Aij of the matrix indicates whether there is a direct connection relationship between node i and node j. Typically, the adjacency matrix is ​​a square matrix of |V|×|V|, and its size is determined by the number of nodes in the graph. Traditional convolutional neural networks (CNNs) are suitable for processing grid-like data. GCN, by performing convolution on graph-structured data, can retain topological structure information outside the nodes while learning node representations. Therefore, GCN is more suitable for solving graph-related problems.

[0068] Therefore, the graph convolutional network operation of this embodiment can be expressed as:

[0069]

[0070] In the formula, Add the identity matrix to the adjacency matrix A, yes The diagonal matrix of the degree matrix, H (l) is the node feature of the lth layer, W (l) is the weight matrix of the layer, and σ is the activation function. This formula is used in graph convolutional networks to calculate node features through the node features, adjacency matrix, weight matrix, and activation function of the previous layer.

[0071] The behavior recognition of pigs can be defined as a graph problem. Different postures of pigs will show different skeleton shapes, and this feature can be used to complete behavior recognition. The key points of the pig skeleton are connected to form an adjacency matrix, and the extracted key points of the pig skeleton are input into the graph convolutional network to extract feature vectors and predict their postures. The graph convolutional network structure is as follows: Figure 5 shown.

[0072] This embodiment also includes a convolution operation extended to a graph convolutional network. In order to adapt to the irregularity and variability of the connection structure between graph data nodes, a basic convolution block is introduced.

[23] The convolution block realizes the convolution operation on complex graph structures by aggregating neighborhood node features and combining adjacency matrix information, effectively capturing the local relationship between nodes.

[0073] The standard convolution operation of the basic convolution block is described as: given a convolution operator with a kernel size of K×K and an input feature map f with c channels in, the single-channel output value at a spatial position x is calculated by aggregating the information of the K×K neighborhood around x. Specifically, for each position x, the output value f out The calculation formula for (x) is:

[0074]

[0075] Where, sampling function s: Z 2 ×Z 2 →Z 2 The neighbors of position x are enumerated, that is, each neighbor position in the K×K region around position x in the input feature map is determined. The weight function θ: Z 2 →R 2 A weight vector in c-dimensional real space is provided for each convolution kernel position, which is used to calculate the inner product with the sampled c-dimensional input feature vector.

[0076] The standard convolution operation samples the direct neighbors of each node, that is, other nodes that are no more than one step away from the current node. The sampling function directly selects neighbor nodes as the sampling result. To avoid the problem that the number of neighbors of a node may be inconsistent, the neighbor nodes are marked by a fixed number of partitions to ensure the consistency of the number of samples each time, and the nodes are weighted according to the partitions they are in, so as to maintain the consistency of the weighted order. i The neighbor set of i )={v j |d(v j , v i )≤1}. Sampling function p: B(v i )→V can be written as p(v i , v j )=v j .

[0077] Furthermore, the residual structure proposed in step S2 introduces direct cross-layer connections to add the input to the output, reducing the complexity of the model and solving the gradient explosion and gradient vanishing problems during training. In graph convolutional networks, the main function of the convolution operation is to perform feature aggregation on the adjacency information of nodes. However, as the depth of the network increases, the model has an overfitting problem. Therefore, the introduction of a residual structure in the graph convolutional network can not only alleviate overfitting, but also accelerate the training and optimization of the model, and improve the performance and stability of the model.

[0078] In step S4, distance weighting is specifically as follows: In order to cope with the irregularity and variability of the connections between nodes in the graph convolutional network, distance weighting is used for feature aggregation. Distance weighting is a distance-based weighting method that determines the node weight according to the distance between the neighbor node and the root node. The closer the distance, the greater the weight of the node, and vice versa. In order to have a fixed number of samples and a fixed weighting order, this method uses a fixed number of partitions to mark the neighbor nodes around the root node, and weights the root node according to the partitions of the neighbor nodes. This weighting method can effectively emphasize the importance of neighbor nodes to the root node features, while retaining the information of distant neighbor nodes and enhancing the robustness of feature representation. The weight calculation of distance weighting is shown in formula (3). The specific method is as follows Figure 6 As shown, Figure 6 The size of the midpoint represents the weight of the node in the root node.

[0079] weight ij =exp(-||p i -p j ||) (3)

[0080] In the formula, weight ij is the weight between nodes i and j, ||p i -p j || is the Euclidean distance between nodes i and j. Substituting equation (3) into equation (2), it can be rewritten as:

[0081]

[0082] In the formula, Z i (v j ) is the normalization factor.

[0083] The pig behavior recognition method based on skeleton features proposed in this embodiment first collects pig pen images to construct a relevant data set, and trains a posture estimation model based on the data set to extract skeleton key points. Then, a graph convolutional network with a residual structure is designed to avoid gradient vanishing and performance degradation problems, and accelerate model training and optimization. A weighting strategy is then introduced to perform distance weighting on the skeleton feature information, while retaining the information of distant neighbor nodes, enhancing the feature expression of neighbor nodes and improving model performance. Finally, the skeleton features are input into the graph convolutional network to achieve accurate classification of the three behaviors of standing, sitting, and lying pigs.

[0084] Example 2

[0085] In order to verify the practical application effect of the pig behavior recognition method based on skeleton features in the above embodiment, a verification test is carried out in this embodiment, and corresponding results and analysis are obtained.

[0086] Specifically, it includes model training environment, evaluation indicators, analysis and comparison of posture estimation results, analysis and comparison of behavior recognition results, and the impact of graph convolutional network depth on classification accuracy.

[0087] (1) Model training environment.

[0088] The experimental environment operating system is Ubuntu 23.04, the CPU model is Intel i7-13700KF, the independent graphics card is NVIDIA GeForce RTX3090Ti graphics card 24G video memory, the Python version is 3.8.0, and the deep learning framework Pytorch version is 1.13.1. Yolov8-Pose sets batch_size to 32, the model input size is 640×640, and training is 200 epochs. The graph convolution model uses the RMSprop optimizer, the learning rate and L2 regularization term are 0.001, and training is 1000 epochs.

[0089] (2)Evaluation indicators

[0090] The performance of the pig behavior recognition model is evaluated using precision, and its calculation is shown in formula (5). Among them, TP represents the number of samples that are correct and the inference results are also correct, and FP represents the number of inference results that do not match the true results. The key point accuracy evaluation uses the OKS indicator, which is defined as shown in formula (6).

[0091]

[0092] Where, d i is the Euclidean distance between the predicted key point and the real key point, s represents the scale of the target (pig scale in this example), k i is a constant for key point i, used to control the importance of the key point, v i is the visibility label of key point i, δ is the indicator function, when v i When δ > 0, δ is 1, otherwise δ is 0. AP50, AP75, and AP90 are calculated based on the OKS score.

[0093] (3) In order to further evaluate the performance of the pose estimation model proposed in this embodiment, the performance of several classic pose estimation models is compared for the key point detection task, and the results are shown in Table 3. As can be seen from Table 3, Yolov8-Pose outperforms other models in overall performance, especially in terms of high precision threshold (AP90). Compared with other methods, Yolov8-Pose not only maintains excellent performance under low precision thresholds, but also significantly improves the performance under higher precision thresholds. The results of pose estimation based on Yolov8-Pose are shown in Table 3. Figure 7 shown.

[0094] Table 3: Comparison of pose estimation model performance.

[0095]

[0096] Depend on Figure 6 It can be seen that the postures of most pigs have been accurately recognized. However, due to the complex environment and poor lighting of the pig house, especially the occlusion of the feeding trough and railings in the venue, the extraction of key points is affected. Figure 8 As shown, the yellow box ( Figure 8 Although these situations have an impact on the key point extraction effect, since the possible missing of key points has been considered when training the graph convolutional network, a small number of key point errors or losses will not significantly affect the output results of the overall model.

[0097] (4) Analysis and comparison of behavior recognition results.

[0098] In order to further evaluate the classification effect of the method of this embodiment on the three behaviors of standing, sitting and lying of pigs, several commonly used posture classification models were compared, and the results are shown in Table 4. It can be seen from Table 4 that compared with other methods, the graph convolutional network method has better classification effect on the three behaviors of standing, lying and sitting. Among the three behaviors, the recognition accuracy of lying is the lowest, only 74.5%. This phenomenon is also reflected in the results of models such as Yolov8 and Yolov7. This may be attributed to the fact that lying postures are divided into two types and are difficult to distinguish from standing postures. In contrast, the effect of machine learning methods is generally worse than that of deep learning methods, but the XGBoost classifier has achieved better results, indicating that it is feasible to use skeleton data to analyze behavior. Fig. 9 The effect of pig behavior recognition from different perspectives is demonstrated.

[0099] Table 4: Behavior recognition results.

[0100]

[0101] (5) The impact of graph convolutional network depth on classification accuracy.

[0102] In order to explore the impact of the depth of graph convolutional networks on behavior classification, the performance of 2-, 3-, and 4-layer graph convolutional networks in classification tasks was verified respectively. The results are shown in Table 5, where the inference speed is the average time used for the classification model to infer 100 times. It can be seen from Table 5 that graph convolutional networks with different numbers of layers have different effects, and the two-layer network has the worst effect. This may be because the relatively simple structure is difficult to capture the complex features in the graph data, thereby limiting the expressive power and classification performance of the model. The three-layer network is the most balanced. Although the recognition accuracy of lying behavior is low, the average accuracy is the highest. The five-layer network has the best recognition effect on the lying behavior of live pigs. It may be due to the deeper network structure that helps to extract more abstract and detailed features, thereby better recognizing complex behaviors, but the recognition accuracy of relatively simple behaviors decreases. In summary, this embodiment selects a three-layer network with the best overall performance as the experimental model.

[0103] Table 5: Behavior classification results of graph convolution with different layers.

[0104]

[0105] This embodiment proposes a pig behavior recognition method based on skeleton features, and uses a graph convolutional network to achieve a relatively accurate classification of the three behaviors of pigs standing, lying and sitting. Compared with the pig behavior recognition method based on Yolo, Alphapose and traditional machine learning, the accuracy of the method in this embodiment is significantly improved.

[0106] First, by collecting, annotating and generalizing pig images, a dataset covering pig target detection, key point extraction and behavior recognition was constructed, containing a total of 2020 images. In addition, a graph convolutional network dataset for behavior classification was constructed, and a graph data enhancement method was designed for situations where key points may be misdetected or missed in actual scenes. Second, Yolov8-pose was used to estimate the posture of pigs. The trained Yolov8-Pose detector performed well, and its AP50 reached 83% when OKS was used as the evaluation indicator. In the target recognition and posture estimation tasks, the comprehensive performance of Yolov8-Pose is better than that of Deeplabcut, Alphapose and Yolov7-Pose. Third, the constructed behavior recognition graph convolutional network model achieved an average recognition accuracy of 87.3% for the three behaviors of standing, lying and sitting, among which the sitting posture recognition accuracy was as high as 94.1%. Compared with methods such as Yolov8, Yolov7, XGBoost and MLP, this model performs better in behavior recognition tasks.

[0107] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.

[0108] In addition, the operations of the process described in this embodiment may be performed in any suitable order, unless otherwise indicated in this embodiment or otherwise clearly contradicted by the context. The process described in this embodiment (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as a code (e.g., executable instructions, one or more computer programs, or one or more applications) executed jointly on one or more processors, by hardware or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0109] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or part thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described in this embodiment includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques described in the present invention, the present invention also includes the computer itself. The computer program can be applied to input data to perform the functions described in this embodiment, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on a display.

[0110] As used in this application, the terms "component", "module", "system", etc. are intended to refer to a computer-related entity, which can be hardware, firmware, a combination of hardware and software, software, or software in operation. For example, a component can be, but is not limited to: a process running on a processor, a processor, an object, an executable file, a thread in execution, a program, and / or a computer. As an example, an application running on a computing device and the computing device can both be components. One or more components can exist in a process and / or thread in execution, and a component can be located in a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures thereon. These components can communicate in a local and / or remote process manner, such as according to a signal having one or more data packets (e.g., data from a component that interacts with another component in a local system, a distributed system, and / or interacts with other systems in a signal manner through a network such as the Internet).

[0111] It should also be understood that the present invention is described through implementation methods, and the embodiments are only clear and complete descriptions of the technical solutions proposed in the claims of the present invention, that is, explanations of the claims. Therefore, when judging whether the technical solutions recorded in the specification of the present invention are sufficiently disclosed, full consideration should be given to the core essence of the solutions defined by the claims. In the specification, there must be other technical problems that are not related to the core technical problems proposed in this embodiment. The corresponding technical features and technical solutions do not belong to the essence of this embodiment and are non-essential technical features. Therefore, they can be referred to as implicit disclosures. Those skilled in the art can fully implement them in combination with existing technologies and common knowledge. Therefore, there is no need to describe them in detail.

[0112] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A pig behavior recognition method based on skeleton features, characterized by: The following steps are included: Collect and build datasets; Construct a graph convolutional network model for behavior recognition, introduce a residual structure into the graph convolutional network model, and accelerate the training and optimization process of the model; Using the Yolov8-Pose model to estimate the posture of the input video, detect the key points of the pig, extract the pig skeleton features, and input the pig skeleton features into the graph convolutional network model; A weighting strategy is introduced to weight the information of the pig skeleton features by distance, thereby enhancing the feature expression of the neighboring nodes while retaining the information of the distant neighboring nodes; The graph convolutional network model outputs recognition results for accurate classification of three behaviors of pigs: standing, sitting, and lying.

2. The pig behavior recognition method based on skeleton features according to claim 1 is characterized in that: The pig skeleton feature includes the following processing steps in the graph convolutional network model: The connection relationship of the skeleton key points is represented as an adjacency graph G = (V, E), where V represents the set of nodes and E is the set of connections or edges between nodes; Define pig behavior recognition as a graphical problem; Then connect the key points of the pig skeleton to form an adjacency matrix A. Each element of the matrix A ij Indicates whether there is a direct connection relationship between node i and node j. The adjacency matrix A is a square matrix of |V|×|V|, and its size is determined by the number of nodes in the graph; The extracted pig skeleton key points are input into the graph convolutional network model to extract feature vectors and predict their postures.

3. The pig behavior recognition method based on skeleton features according to claim 2 is characterized in that: The graph convolutional network model is expressed as: In the formula, Add the identity matrix to the adjacency matrix A, yes The diagonal matrix of the degree matrix, H (l) is the node feature of the lth layer, W (l) is the weight matrix of the layer and σ is the activation function.

4. The pig behavior recognition method based on skeleton features according to claim 3 is characterized in that: The graph convolutional network model includes introducing a basic convolution block, which captures the local relationship between nodes by aggregating neighborhood node features and combining adjacency matrix information.

5. The method for pig behavior recognition based on skeleton features according to claim 4 is characterized in that: The standard convolution operation of the basic convolution block includes the following steps: Given a convolution operator with a kernel size of K×K and an input feature map f with c channels in ; The single-channel output value at the spatial position x is calculated by aggregating the information of the K×K neighborhood around x; For each position x, output value f out The calculation formula for (x) is: Where, the sampling function s:Z 2 ×Z 2 →Z 2 Enumerates the neighbors of position x, weight function θ:Z 2 →R 2 A weight vector in c-dimensional real space is provided for each convolution kernel position, which is used to calculate the inner product with the sampled c-dimensional input feature vector.

6. The pig behavior recognition method based on skeleton features according to claim 4 is characterized in that: The standard convolution operation includes, Sample the direct neighbors of each node and directly select the neighboring nodes as the sampling results; Neighbor nodes are marked by a fixed number of partitions and weighted according to the partition in which the node is located. Node v i The neighbor set of i )={v j |d(v j , v i )≤1}, sampling function p:B(v i )→V is p(v i , v j )=v j .

7. The method for pig behavior recognition based on skeleton features according to claim 5 is characterized in that: The distance weighting includes the following steps: The weight calculation is as follows: weight ij =exp(-||p i -p j ||) In the formula, weight ij is the weight between nodes i and j, ||p i -p j || is the Euclidean distance between nodes i and j; Substitute the weight calculation into the output value f out (x), we get the following formula: In the formula, Z i (v j ) is the normalization factor.

8. The method for pig behavior recognition based on skeleton features according to claim 1, characterized in that: The data set also includes a posture estimation data set and a pig behavior data set, and the pig behavior data set is completed by performing behavior annotation on the posture estimation data set.

9. The method for pig behavior recognition based on skeleton features according to claim 8, characterized in that: The posture estimation dataset uses image annotation to divide the skeleton model into four parts: head, back, tail and limbs, with a total of 14 key points, and is divided into a training set and a test set in an 8:2 ratio.

10. The pig behavior recognition method based on skeleton features according to claim 8, characterized in that: The pig behavior dataset includes the following construction steps: Define the three behaviors of pigs: standing, sitting, and lying; The data of pigs with a visible body range of less than 60% were eliminated, and the skeleton data of each image was extracted and saved as a csv file after screening; The data is enhanced by randomly removing 1-3 key points from samples and randomly moving 1-3 key points by 0-8 pixels.

Images