A pig lameness grading method and system based on key point skeleton and motion trajectory

Abstract

The application relates to the technical field of image data acquisition and processing, and discloses a pig lameness grading method and system based on key point skeletons and motion trajectories, which comprises the following specific steps: S1, acquiring pig activity videos; S2, cutting the pig activity videos into sample videos; target detection and key point recognition are performed on each frame in the sample videos; S3, connecting the recognized key points into skeleton graphs; a space-time skeleton is formed; the center coordinates of a pig target frame obtained through target detection are taken as the position of the pig, and the motion trajectory of the pig is generated; S4, features of the space-time skeleton of the pig are extracted through an ST-GCN recognition model; features of the motion trajectory of the pig are captured through an LSTM recognition model; the features of the space-time skeleton and the features of the motion trajectory are fused, and the pig is classified according to the set lameness grade of the pig. The application solves the problem that the prior art cannot comprehensively consider the dynamic changes of the whole body of the pig during walking, and has the characteristics of strong search capability and high convergence speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data acquisition processing, and more particularly to a pig lameness grading method and system based on key point skeleton and motion trajectory. BACKGROUND

[0002] In modern pig farming, pig health is a problem that needs special attention, and lameness as a common health problem needs to be paid more attention to. When pigs have lameness, it will affect their normal movement and feeding, leading to slow weight gain, and if not treated in time, it may cause bacterial infection and further endanger the life of the pig. According to research, the proportion of dead sow due to lameness is more than 25%. In the piglet period, due to the incomplete development of the skeleton and muscle of piglets, their resistance is relatively weak, and they are prone to health problems such as lameness. In the fattening period, the weight of pigs grows rapidly, which is easy to cause the hoof or knee of the pig to be injured, and to cause lameness and other diseases. Therefore, regular inspection of the walking condition of pigs, timely detection of lameness pigs, and taking treatment measures can effectively reduce the loss of pigs. Different degrees of lameness pigs need to take different treatment methods, and the higher the grade of lameness, the higher the corresponding treatment cost. For pigs with slight lameness, the breeder will add appropriate nutrients to increase their physical strength and reduce joint fatigue. For pigs with moderate lameness, in addition to nutritional supplements, they also need drug treatment and improvement of the feeding environment. For pigs with severe lameness, surgical treatment is tried, and when the leg is severely deteriorated and the treatment cost is too high, the pig can be considered for slaughter or euthanasia.

[0003] At present, the detection of pig lameness mainly relies on artificial observation by the breeder, which determines whether the pig has lameness by observing the walking gait of the pig or whether the hoof or leg is swollen. But this method needs to consume a lot of manpower and material resources, and cannot cope with the current large-scale breeding. Therefore, it is necessary to find a more efficient and accurate method to identify pig lameness.

[0004] In recent years, with the rapid development of deep learning, target detection, semantic segmentation and instance segmentation algorithms are becoming more and more mature, and more and more researches apply these algorithms to pigs. However, there are few researches on detecting pig lameness using deep learning. Wu Yan et al. identified the lameness of pigs by constructing a star skeleton model to extract joint angle data related to gait, but this method only considers the back features and ignores the more obvious leg joint features; Zou Anqi et al. processed each image by Gaussian kernel convolution to obtain a corresponding heat image, then set detection sensors on each feature point when obtaining the training set, detected the position of each feature point by the detection sensor, and then converted it to the established three-dimensional coordinate system to obtain the coordinates of each feature point in the three-dimensional coordinate system. Finally, the knee angle in a gait cycle is calculated to determine whether the pig is lame, but the knee angle may be affected by factors such as stride and moving speed, and the knee angle alone cannot guarantee the accuracy of lameness recognition.

[0005] The existing mammal posture recognition method based on body contour and leg joint skeleton includes two parts: the first part, for wild mammal images in complex outdoor environment, edge extraction is performed by Outline Mask R-CNN to obtain the animal peripheral contour; based on the contour map, a Tiny VGG light convolutional neural network is constructed for posture classification of wild mammals. The second part, for wild mammal video sequences in complex outdoor environment, the LEAP is used to quickly track the position of the leg joint of the animal to form a skeleton map; the change of the leg joint angle in the animal skeleton map is taken as a feature, and the LSTM is taken as a classifier for posture classification of wild mammals.

[0006] However, the existing pig lameness recognition technology fails to fully consider the dynamic changes of the whole body during the walking process of the pig, and how to invent a pig lameness grading method and system based on key point skeleton and motion trajectory is a technical problem that needs to be solved in the technical field. SUMMARY

[0007] The present application provides a pig lameness grading method and system based on key point skeleton and motion trajectory, which has the characteristics of strong search ability and high convergence speed.

[0008] To achieve the above-mentioned purposes of the present application, the technical solutions adopted are as follows:

[0009] A pig lameness grading method based on key point skeleton and motion trajectory, comprising the following specific steps:

[0010] S1, monitor the pigs in the pig house and obtain pig activity video;

[0011] S2, cutting the pig activity video into a plurality of sample videos containing a single pig walking a complete step; target detection and key point recognition are performed on each frame in the sample video;

[0012] S3, connecting the recognized key points into a skeleton graph; combining the skeleton graph with all frames in the sample video to form a space-time skeleton; taking the center coordinates of the pig target box obtained by target detection as the position of the pig, and combining all frames in the sample video to generate the motion trajectory of the pig;

[0013] S4, setting the lameness grade of the pig; extracting the features of the space-time skeleton of the pig from the time and space dimensions through the ST-GCN recognition model; capturing the features of the motion trajectory of the pig through the LSTM recognition model; fusing the features of the space-time skeleton and the features of the motion trajectory, and classifying the lameness grade of the pig according to the set lameness grade of the pig.

[0014] Preferably, in step S1, pig data in the pig house is collected, specifically: the camera is placed on the side of the pig house passageway, and the passageway only allows one pig to pass at the same time, and a side-view video in the passageway is obtained.

[0015] Further, in step S2, target detection is performed on each frame in the sample video, specifically: an improved target detection model based on Yolov7 algorithm is used to perform target detection on each frame in the sample video; the improved target detection model based on Yolov7 algorithm adds a hole convolution module based on Yolov7;

[0016] When target detection is performed on the pig, the hole convolution module is used to arbitrarily expand the receptive field to capture multi-scale context information, and the Yolov7 algorithm is used to extract feature information at different depths of the pig side.

[0017] Further, in step S2, key point recognition is performed, specifically: an improved HrNet model is used to perform key point recognition on each frame in the sample video;

[0018] When key point recognition is performed, if a key point is missing in the i-th frame in the sample video, the improved HrNet model selects a frame a and a frame b before and after the current frame, if the corresponding key points are also missing in the frames a and b, it will continue to select adjacent frames forward or backward until a frame with complete key points is found.

[0019] Further, after key point recognition, the horizontal coordinate x i and the vertical coordinate y i are:

[0020] x i =w a ·xa +w b ·x b

[0021] y i =w a ·y a +w b ·y b

[0022] wherein,

[0023]

[0024]

[0025] Further, in the step S4, the lameness grade of the pig is set, specifically:

[0026] 41. The pig with one or several abnormal leg joints, slight staggering or unstable gait, and overall posture and movement trajectory similar to normal pigs is classified as a slight lameness grade;

[0027] 42. The pig with obvious abnormal leg joints, uncoordinated leg swing and reduced leg swing amplitude, slight arching of the back, and obvious differences in movement trajectory and posture from normal pigs is classified as a moderate lameness grade;

[0028] 43. The pig with severe abnormal leg joints, unable to bend the knee when lifting the leg, signs of falling when landing, and obvious arching of the back, which seriously affects the pig's walking, is classified as a severe lameness grade.

[0029] Further, in the step S4, the spatiotemporal skeleton features of the pig are extracted from the time and space dimensions through the ST-GCN recognition model, specifically: the spatiotemporal skeleton is input into the ST-GCN recognition model, and the spatiotemporal information of the spatiotemporal skeleton is aggregated through the GCN graph convolution network and the TCN time convolution network; after global average pooling, the feature vector of the spatiotemporal skeleton is output.

[0030] Further, in the step S4, the features of the pig's movement trajectory are captured through the LSTM recognition model, specifically:

[0031] The model for recognizing the motion trajectory is constructed through the LSTM, the motion trajectory of the slightly limping pig is that the target center of individual frames is downwardly moved, the motion trajectory of the moderately limping pig is that the downward movement degree of the target center is more obvious than that of the slightly limping pig, and the target center is downwardly moved in more frames, and the overall motion trajectory is more tortuous, and the motion trajectory of the severely limping pig is that the trajectory appears large irregularity and jitter, and the LSTM recognition model constructed is used for retaining the trajectory features through the gating mechanism, and the output of the last time step is used as the feature vector of the entire motion trajectory.

[0032] Further, in the step S4, the features of the space-time skeleton and the motion trajectory are fused, and the pig limping grade is classified according to the set pig limping grade, and specifically, the step S4 comprises the following steps of:

[0033] The features of the space-time skeleton and the motion trajectory are spliced and input into the full connection layer and the softmax function, and the probabilities of the normal, slightly limping, moderately limping and severely limping are output according to the set pig limping grade.

[0034] A pig limping grading system based on key point skeleton and motion trajectory comprises a video acquisition module, a video processing module and a pig limping recognition module.

[0035] The video acquisition module is used for monitoring the pigs in the pig house and acquiring pig activity videos.

[0036] The video processing module is used for editing the pig activity videos into a plurality of sample videos containing a single pig walking a complete step, performing target detection and key point recognition on each frame in the sample video, connecting the recognized key points into a skeleton graph, combining the skeleton graph with all the frames in the sample video to form a space-time skeleton, and taking the pig target frame center coordinates obtained through the target detection as the position of the pig, and combining all the frames in the sample video to generate the motion trajectory of the pig.

[0037] The pig limping recognition module is used for setting the pig limping grade, extracting the features of the space-time skeleton of the pig from the time and space dimensions through the ST-GCN recognition model, capturing the features of the motion trajectory of the pig through the LSTM recognition model, and fusing the features of the space-time skeleton and the motion trajectory, and classifying the pig limping grade according to the set pig limping grade.

[0038] The pig limping grading system has the following beneficial effects:

[0039] This invention discloses a method for classifying lameness in pigs based on keypoint skeletons and movement trajectories. The method involves editing pig activity videos into several sample videos, each containing a single pig completing a full step. Object detection and keypoint recognition are performed on each frame of the sample videos. This simultaneously reflects the degree of limb and back flexion during gait. Lame pigs exhibit abnormal limb and back flexion due to leg discomfort and pain, resulting in a gait and skeleton inconsistent with the overall trend of normal pigs. Furthermore, different degrees of lameness manifest different gait and skeletal changes. The proposed method effectively avoids getting trapped in local optima, possesses strong search capabilities and high convergence speed, and can effectively solve the problem of lameness classification in pigs based on keypoint skeletons and movement trajectories. Attached Figure Description

[0040] Figure 1 This is a flowchart of a method for classifying lameness in pigs based on key point skeletons and motion trajectories according to the present invention.

[0041] Figure 2 This is a schematic diagram of a camera deployment scheme for a method of classifying lameness in pigs based on key point skeletons and motion trajectories according to the present invention.

[0042] Figure 3 This is a flowchart of a method for classifying lameness in pigs based on key point skeletons and motion trajectories, as described in Example 2.

[0043] Figure 4 This is a schematic diagram of the movement trajectories of normal pigs and lame pigs.

[0044] Figure 5 This is a flowchart illustrating the acquisition of key point skeleton sequences and motion trajectories in a method for classifying lameness in pigs based on key point skeletons and motion trajectories according to the present invention.

[0045] Figure 6 This is a network structure diagram of a pig lameness classification method based on key point skeleton and motion trajectory according to the present invention. Detailed Implementation

[0046] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0047] Example 1

[0048] like Figure 1 As shown, a method for classifying lameness in pigs based on key point skeletons and motion trajectories includes the following specific steps:

[0049] S1. Monitor the pigs in the pigsty and obtain video footage of their activities;

[0050] S2, the pig activity video is cut into a plurality of sample videos containing a single pig completing a step; target detection and key point identification are performed on each frame in the sample video;

[0051] S3, the recognized key points are connected into a skeleton graph; the skeleton graph is combined with all frames in the sample video to form a space-time skeleton; the center coordinates of the pig target box obtained by target detection are taken as the position of the pig, and the motion trajectory of the pig is generated by combining all frames in the sample video;

[0052] S4, setting the lameness grade of the pig; extracting the features of the space-time skeleton of the pig from the time and space dimensions through the ST-GCN recognition model; capturing the features of the motion trajectory of the pig through the LSTM recognition model; fusing the features of the space-time skeleton and the features of the motion trajectory, and classifying the lameness grade of the pig according to the set lameness grade of the pig.

[0053] Embodiment 2

[0054] More specifically, as shown in Figure 2 In one specific embodiment, in step S1, pig data in the pig house is collected, specifically: the camera is placed on the side of the pig house passageway, the passageway allows only one pig to pass at the same time, and the side-view video in the passageway is obtained.

[0055] In this embodiment, a 2D color camera is erected on the side of the pig house passageway. The camera is erected at a position 1 m away from the ground on the side of the passageway, and the camera lens is parallel to the passageway rail, so that the side of the pig can be clearly and completely shot. The passageway is a one-way passageway, only one pig is allowed to pass through each time, ensuring that the number of pigs shot by the camera each time is 1, and it is suitable for pigs of any size.

[0056] In one specific embodiment, in step S2, target detection is performed on each frame in the sample video, specifically: a target detection model based on an improved Yolov7 algorithm is used to perform target detection on each frame in the sample video; the improved target detection model based on the Yolov7 algorithm adds a hole convolution module based on the Yolov7 algorithm;

[0057] When detecting the pig, the hole convolution module is used to arbitrarily expand the receptive field and capture multi-scale context information, and the Yolov7 algorithm is used to extract feature information of different depths of the pig side.

[0058] In this embodiment, the pig is identified from the side, and the head and hips of the pig are more prominent, while the abdomen and back are relatively smooth and not easy to detect. The use of a cavity convolution can better capture the feature information of different depths of the pig side and more accurately limit the range of subsequent key point recognition. The algorithm adds a cavity convolution module based on Yolov7. The cavity convolution can arbitrarily expand the receptive field, capture multi-scale context information, and improve the feature extraction capability. When detecting large-scale targets, a smaller receptive field may not be able to capture the overall appearance of the target, so the receptive field of the convolution layer needs to be increased. In this application, when detecting pigs from the side, the shape of the pig is relatively flat, and the cavity convolution can better capture the horizontal features.

[0059] In this embodiment, as shown in Figure 3 The camera captures an RGB video of the pig;

[0060] Before target detection, an improved target detection model based on the Yolov7 algorithm needs to be trained; in this embodiment, the pre-processed videos containing limp walking and normal walking are divided into a training set, a validation set and a test set according to a ratio of 7:2:1.

[0061] The improved target detection model based on the Yolov7 algorithm captures multi-scale context information from the training set, the validation set and the test set respectively, and extracts feature information of different depths of the pig side in combination with the Yolov7 algorithm; when extracting the feature information, the validation set is used to verify the convergence of the target detection model during the training process, and the hyperparameters of the target detection model are adjusted; finally, the generalization ability of the target detection model is evaluated through the test set, and a trained target detection model is obtained.

[0062] After obtaining the trained target detection model, the key points are identified according to the detection results of the target model, and a key point sequence is formed by recognizing the skeleton of the pig; the key point sequence is input into the ST-GTCN for feature extraction, and the center of the video frame target detection is formed into a motion trajectory, which is input into the LSTM model for feature extraction; and finally, the feature vectors of the ST-GTCN and the LSTM model are fused, and the fused feature vectors are classified.

[0063] In one specific embodiment, in step S2, the key points are identified by using an improved HrNet model to identify the key points of each frame in the sample video.

[0064] In this embodiment, the key points of the application include 16 points, i.e., the left front hoof, the left front knee, the left front shoulder, the right front hoof, the right front knee, the right front shoulder, the left rear hoof, the left rear knee, the left rear shoulder, the right rear hoof, the right rear knee, the right rear shoulder, the hip, the back, the neck and the head.

[0065] In this embodiment, the key point loss will affect the subsequent lameness detection, so the algorithm makes improvements on the basis of the original algorithm to complete the key point loss. When the key point loss appears in the i-th frame of the sample video, the improved HrNet model selects one frame a and b before and after the current frame. If the corresponding key points are also missing in frames a and b, it will continue to select adjacent frames forward or backward until a frame with complete key points is found.

[0066] In one specific embodiment, after key point recognition, the obtained horizontal coordinate x i and vertical coordinate y i are:

[0067] x i =w a ·x a +w b ·x b

[0068] y i =w a ·y a +w b ·y b

[0069] wherein,

[0070]

[0071]

[0072] In one specific embodiment, in step S4, the pig lameness level is set, specifically:

[0073] 41. The pig with one or more leg joint movement abnormalities, slight staggering or unstable gait, and overall posture and movement trajectory similar to normal pigs is classified as a mild lameness level;

[0074] 42. The pig with leg joint abnormalities, uncoordinated leg swing and reduced leg swing amplitude, and slight arching of the back, with a significant difference in movement trajectory and posture from normal pigs, is classified as a moderate lameness level;

[0075] 43. The pig with severe leg joint abnormalities, unable to bend the knee when lifting the leg, and falling down when landing due to leg pain, with a significant arching of the back, and severely affecting the pig's walking, is classified as a severe lameness level.

[0076] In one specific embodiment, as Figure 6As shown, in step S4, the ST-GCN recognition model extracts the spatio-temporal skeleton features of the pig from the time and space dimensions, specifically: inputting the spatio-temporal skeleton into the ST-GCN recognition model, aggregating the spatio-temporal information of the spatio-temporal skeleton through the GCN graph convolution network and the TCN time convolution network; after global average pooling, outputting the feature vector of the spatio-temporal skeleton.

[0077] In one specific embodiment, in step S4, the LSTM recognition model captures the features of the pig's motion trajectory, specifically:

[0078] As shown Figure 4 , the LSTM model is constructed to recognize the motion trajectory; the motion trajectory of the pig with slight lameness is that the target center of individual frames is downward; the motion trajectory of the pig with moderate lameness is that the downward degree of the target center is more obvious than that of the pig with slight lameness, and the target center appears downward in more frames, and the overall motion trajectory is more tortuous; the motion trajectory of the pig with severe lameness is that the trajectory appears large irregularity and jitter; the constructed LSTM recognition model retains the trajectory features through the gating mechanism, and takes the output of the last time step as the feature vector of the entire motion trajectory.

[0079] In one specific embodiment, in step S4, the features of the spatio-temporal skeleton and the motion trajectory are fused, and the pig's lameness level is classified according to the set pig's lameness level, specifically:

[0080] The features of the spatio-temporal skeleton and the motion trajectory are spliced and input into the full connection layer and the softmax function, and the probabilities of normal, slight lameness, moderate lameness and severe lameness are output according to the set pig's lameness level.

[0081] Embodiment 3

[0082] In this embodiment, as shown Figure 5 , if no pig is detected during target detection, the detection is discarded; if a pig is detected, a motion trajectory is made and the pig key points are recognized; for the recognized pig key points, if they are incomplete, the key points are completed, and after the key points are completed, a key point skeleton sequence, i.e. a spatio-temporal skeleton, is made, and if they are complete, a key point skeleton sequence is directly made.

[0083] Embodiment 4

[0084] A pig lameness grading system based on key point skeleton and motion trajectory, comprising a video acquisition module, a video processing module and a pig lameness recognition module.

[0085] The video acquisition module is used to monitor the pigs in the pig house and acquire pig activity videos.

[0086] The video processing module is used for cutting the pig activity video into a plurality of sample videos containing a single pig walking a complete step; target detection and key point identification are performed on each frame in the sample video; the identified key points are connected into a skeleton graph; the skeleton graph is combined with all frames in the sample video to form a space-time skeleton; the center coordinates of the pig target frame obtained by target detection are taken as the position of the pig, and the motion trajectory of the pig is generated by combining all frames in the sample video;

[0087] The pig limp identification module is used for setting a pig limp grade; a feature of a space-time skeleton of the pig is extracted from time and space dimensions through an ST-GCN recognition model; a feature of a motion trajectory of the pig is captured through an LSTM recognition model; the features of the space-time skeleton and the motion trajectory are fused, and the pig limp grade is classified according to the set pig limp grade.

[0088] The application discloses a pig limp grading method based on key point skeletons and motion trajectories. The scheme of the application cuts a pig activity video into a plurality of sample videos containing a single pig walking a complete step; target detection and key point identification are performed on each frame of the sample video; the application proposes 16 key points for pig modeling, which can reflect the bending degrees of limbs and the back of the pig when walking, and the bending degrees of the limbs and the back of the lame pig when walking will be abnormal due to leg discomfort and pain, so that the gait and skeleton of the pig are inconsistent with the overall trend of the normal pig, and the gait and skeleton change trends of pigs with different degrees of lameness will also be different. The application also proposes a method for improving a target detection model and calculating missing points of a current frame by using key points of front and rear frames. Since the body length of a pig is relatively long, a traditional target detection model is difficult to capture multi-scale context information, and in key point detection, key points may be missing in a single frame image due to changes in the posture of the pig or light factors. In view of these problems, an improved scheme is proposed by adding a hollow convolution and calculating missing points by using key points of front and rear frames, so as to improve the accuracy of pig target detection and key point detection, and make the subsequent pig limp recognition more accurate. The method proposed by the application can effectively avoid falling into local optimization, has the characteristics of strong search ability and high convergence speed, and can effectively solve the pig limp grading problem based on key point skeletons and motion trajectories. Obviously, the above embodiments of the application are only examples for clearly illustrating the application, and are not a limitation on the embodiments of the application. Any modification, equivalent replacement and improvement made within the spirit and principle of the application shall be included in the protection scope of the claims of the application.

Claims

1. A method for classifying a pig limp based on keypoint skeleton and motion trajectory, characterized in that: The method comprises the following specific steps: S1, monitoring pigs in a pig house to obtain pig activity video; S2, cutting the pig activity video into a plurality of sample videos containing a single pig completing a step; performing target detection and key point recognition on each frame in the sample video; wherein, the target detection on each frame in the sample video is specifically: performing target detection on each frame in the sample video by an improved target detection model based on Yolov7 algorithm; the improved target detection model based on Yolov7 algorithm adds a hole convolution module on the basis of Yolov7; when detecting the pig, the hole convolution module is used to arbitrarily expand the receptive field and capture multi-scale context information, and the Yolov7 algorithm is used to extract feature information of different depths of the pig side; S3, connecting the recognized key points into a skeleton graph; combining the skeleton graph with all frames in the sample video to form a space-time skeleton; taking the center coordinates of the pig target box obtained by target detection as the position of the pig, and combining all frames in the sample video to generate the movement trajectory of the pig; S4, setting the lameness grade of the pig; extracting the features of the space-time skeleton of the pig from the time and space dimensions by an ST-GCN recognition model; capturing the features of the movement trajectory of the pig by an LSTM recognition model; fusing the features of the space-time skeleton and the features of the movement trajectory, and classifying the lameness grade of the pig according to the set lameness grade of the pig.

2. The keypoint skeleton and motion trajectory based pig lameness grading method according to claim 1, characterized in that: In the step S1, pig data in the pig house is collected, specifically: the camera is placed on the side of the passageway in the pig house, and only one pig is allowed to pass through the passageway at the same time, and a side-view video in the passageway is obtained.

3. The keypoint skeleton and motion trajectory based pig lameness grading method according to claim 1, characterized in that: In the step S2, key point recognition is specifically: performing key point recognition on each frame in the sample video by an improved HrNet model; When key point recognition is performed, if a key point is missing in the i-th frame in the sample video, the improved HrNet model selects a frame a and a frame b before and after the current frame, if the corresponding key points are also missing in the frames a and b, the adjacent frames will be continuously selected until the frame with complete key points is found.

4. The keypoint skeleton and motion trajectory based pig lameness grading method according to claim 3, characterized in that: After the key point recognition, the obtained horizontal coordinate x i and the vertical coordinate y i are: x i = w a • x a + w b • x b y i = w a · y a + w b · y b In the step S4, the lameness grade of the pig is set, specifically:

5. The keypoint skeleton and motion trajectory based pig lameness grading method according to claim 1, wherein: 41, the pig with one or more abnormal leg joint movements, slight staggering or unstable gait, and overall posture and movement trajectory similar to normal pigs is classified as a pig with slight lameness; 42, the pig with obvious abnormal leg joints, uncoordinated leg swinging and reduced leg swinging amplitude, slight arching back, and obvious difference between the movement trajectory and the posture and the normal pig is classified as a pig with moderate lameness; 43, the pig with severe abnormal leg joints, unable to bend the knee when lifting the leg, falling sign when landing, and obvious arching back, which seriously affects the walking of the pig, is classified as a pig with severe lameness. ​ 6. The keypoint skeleton and motion trajectory based pig lameness grading method according to claim 5, characterized in that: In the step S4, the ST-GCN recognition model is used to extract the spatial-temporal skeleton features of the pig from the time and space dimensions, specifically: the spatial-temporal skeleton is input into the ST-GCN recognition model, and the GCN graph convolution network and the TCN time convolution network are used to aggregate the spatial-temporal information of the spatial-temporal skeleton; and after global average pooling, the feature vector of the spatial-temporal skeleton is output.

7. The keypoint skeleton and motion trajectory based pig lameness grading method according to claim 6, characterized in that: In the step S4, the LSTM recognition model is used to capture the features of the motion trajectory of the pig, specifically: The LSTM is used to build a model for recognizing the motion trajectory; the motion trajectory of the pig with slight lameness is that the target center of individual frames is downwardly moved; the motion trajectory of the pig with moderate lameness is that the downward movement of the target center is more obvious than that of the pig with slight lameness, and the target center is downwardly moved in more frames, and the overall motion trajectory is more tortuous; the motion trajectory of the pig with severe lameness is that the trajectory appears large irregularity and jitter; the built LSTM recognition model retains the trajectory features through the gating mechanism, and the output of the last time step is taken as the feature vector of the entire motion trajectory.

8. The keypoint skeleton and motion trajectory based pig lameness grading method according to claim 7, characterized in that: In the step S4, the features of the spatial-temporal skeleton and the motion trajectory are fused, and the pig lameness level is classified according to the set pig lameness level, specifically: The features of the spatial-temporal skeleton and the motion trajectory are spliced and input into the full connection layer and the softmax function, and the probabilities of normal, slight lameness, moderate lameness and severe lameness are output according to the set pig lameness level.

9. A keypoint skeleton and motion trajectory based pig lameness grading system, characterized in that: The method for realizing the method according to any one of claims 1-8, comprising a video acquisition module, a video processing module, and a pig lameness recognition module; The video acquisition module is used to monitor the pigs in the pig house and acquire pig activity videos; The video processing module is used to edit the pig activity videos into a plurality of sample videos containing a single pig completing a step; perform target detection and key point recognition on each frame in the sample video; connect the recognized key points into a skeleton graph; combine the skeleton graph with all frames in the sample video to form a spatial-temporal skeleton; and take the pig target box center coordinates obtained by target detection as the position of the pig, and combine all frames in the sample video to generate the motion trajectory of the pig; The pig lameness recognition module is used to set the pig lameness level; the ST-GCN recognition model is used to extract the features of the spatial-temporal skeleton of the pig from the time and space dimensions; the LSTM recognition model is used to capture the features of the motion trajectory of the pig; and the features of the spatial-temporal skeleton and the motion trajectory are fused, and the pig lameness level is classified according to the set pig lameness level.

Images