A multi-objective tracking and behavior statistics method for group-raised live pigs

Abstract

The application discloses a kind of group raising live pig multi-objective tracking and behavior statistics method, comprising the following steps: S1: establishing the video dataset of group raising live pig, constructs YOLOX-X model, the YOLOX-X model includes feature extraction main stem network, intermediate multi-scale feature fusion network, target detection positioning and behavior identification network three parts in turn;S2: construct YOLOX-X+ByteTrack model, the YOLOX-X+ByteTrack model includes YOLOX-X model, Kalman filtering, IoU matching algorithm, hungarian matching algorithm and the creation, deletion and merging of trajectory;S3: pig anchor frame and trajectory interpolation post-processing strategy are designed in YOLOX-X+ByteTrack model, obtain improved YOLOX-X+ByteTrack model;S4: improved YOLOX-X+ByteTrack model is used to group raising live pig target tracking and behavior statistics.The application can effectively improve ID frequent error transformation, improve the accuracy of tracker, keep excellent detection tracking speed, generate the behavior and health condition analysis table of each pig, realize the real-time monitoring and fast and accurate tracking of pig multi-class behavior of group raising pig farm.

Description

Technical Field

[0001] This invention relates to the field of pig behavior recognition in agricultural computer vision, and more particularly to a method for multi-target tracking and behavior statistics of group-raised pigs. Background Technology

[0002] Animal husbandry is a fundamental industry of China's national economy, with pig farming being a pillar industry that occupies a pivotal position in the national economy. The health of pigs determines the development and economic benefits of pig farming. Many swine diseases are often preceded by abnormal behavior, so monitoring pigs' movement, eating habits, and other behaviors is helpful in assessing their health. Currently, with the development of image processing technology, the fusion of manual observation and computer vision monitoring is the main management method for large-scale pig farms. This requires a certain amount of labor and utilizes intelligent ear tags (for pig identification) and computer-aided methods to achieve semi-automatic monitoring of pig behavior. In real pig farms, changes in lighting and dense occlusion by pigs can easily lead to missed and false detections, causing frequent jumps in pig target IDs during tracking and reducing tracking performance. Therefore, to achieve efficient and fully automated monitoring of pig health, developing stable and efficient multi-target tracking technology to complete automatic diagnosis and intelligent monitoring of pig health in pig farms is of great significance for realizing smart pig farming.

[0003] In the area of ​​multi-target tracking in group-housed pigs, traditional target detection and multi-target tracking methods are currently the main approaches. Publication CN113947617A discloses a multi-target tracking method for suckling piglets based on long short-term memory (LSTM). This method utilizes a neural network to match target IDs of suckling piglets in continuous images to obtain tracking results. It overcomes difficulties such as irregular piglet movement, significant body deformation, huddling, and difficulty in re-identification in fixed scenarios, automatically identifying and tracking suckling piglets and solving the problem of inaccurate judgment caused by fatigue of farm workers during long hours of patrols. Publication CN113763429A discloses a video-based pig behavior recognition system and method. First, based on video image information of pigs in group-housed pens, a detection model is used to detect the target positions of pigs in the video images, and a multi-target tracking algorithm is used to track the detected pig targets. Then, a keypoint detection model is used to predict the pig behavior category. Finally, the predicted pig behavior category is recorded and prompts are provided, greatly saving manual labor costs. Publication No. CN113486813A discloses a method for pig re-identification. This method establishes a pig identification model based on deep learning and creates a pig file table based on the identification results. Pigs are re-identified using real-time video footage, and the re-identification results are used to update the pig file table. This method solves the problems of high labor costs, large workload, low efficiency, easy tracking failure, and low accuracy in existing technologies. Currently, there is limited research on using multi-target tracking technology in video surveillance to achieve rapid and accurate detection and various behavioral recognition of individual pigs in group-housed environments. Therefore, designing and implementing a behavior recognition model for group-housed pigs based on multi-target tracking is of significant research importance for the automation of large-scale pig farming. Summary of the Invention

[0004] This invention aims to at least partially solve one of the problems in related technologies. Therefore, the purpose of this invention is to provide a method for multi-target tracking and behavior statistics of group-raised pigs, which can effectively improve the accuracy of trackers by reducing frequent ID errors, maintaining excellent detection and tracking speed, generating a behavior and health status analysis table for each pig, and realizing real-time monitoring and rapid, accurate tracking of various behaviors of pigs in group-raised pig farms.

[0005] To achieve the above objectives, this application adopts the following technical solution: a method for multi-target tracking and behavior statistics of group-raised pigs, comprising the following steps:

[0006] S1: Establish a video dataset of group-raised pigs and construct a YOLOX-X model. The YOLOX-X model consists of three parts: a feature extraction backbone network, an intermediate multi-scale feature fusion network, and a target detection and localization and behavior recognition network.

[0007] S2: Construct the YOLOX-X+ByteTrack model, which includes the YOLOX-X model, Kalman filter, IoU matching algorithm, Hungarian matching algorithm, and trajectory creation, deletion, and merging;

[0008] S3: Design pig anchor frames and trajectory interpolation post-processing strategies in the YOLOX-X+ByteTrack model to obtain an improved YOLOX-X+ByteTrack model;

[0009] S4: An improved YOLOX-X+ByteTrack model was used to perform target tracking and behavior statistics on group-raised pigs.

[0010] Furthermore, the video dataset for group-raised pigs established in step S1 includes:

[0011] S11: Label the pig behaviors in the video dataset into four categories: lying down, standing, eating, and others; divide the video dataset into a training set and a test set.

[0012] Furthermore, the construction of the YOLOX-X model in step S1 specifically includes:

[0013] S12: Construct the feature extraction backbone network of the YOLOX-X model, which is a fusion of the Focus network component and four Dark network blocks;

[0014] S13: Construct an intermediate multi-scale feature fusion network. For the feature extraction backbone network, use a top-down FPN layer and a bottom-up PAN layer structure to fuse semantic and positional information of feature maps of different sizes.

[0015] S14: Construct a target detection, localization, and behavior recognition network to achieve multi-target detection and classification.

[0016] Furthermore, step S2, which involves constructing the YOLOX-X+ByteTrack model, specifically includes:

[0017] S21: Divide the target detection results output by the YOLOX-X model into high-scoring detection boxes and low-scoring detection boxes;

[0018] S22: Construct a Kalman filter and use the Kalman filter to predict the position of the pig detection box in the next frame of each frame;

[0019] S23: Use the IoU matching algorithm and the Hungarian matching algorithm to match the pig detection boxes in each frame of the model with the existing trajectories and assign ID numbers;

[0020] S24: Create, delete, and merge trajectories in the model.

[0021] Furthermore, step S21, which divides the YOLOX-X target detection results into high-scoring and low-scoring detection boxes, specifically includes:

[0022] S211: Input the images in the training set into the YOLOX-X model to obtain detection boxes with different confidence levels, and set the high-scoring box threshold and low-scoring box threshold for the detection boxes.

[0023] S212: In the detection results, if the confidence value of the detection box is greater than the high-scoring box threshold, the detection box is added to the high-scoring detection box set; if the confidence value of the detection box is less than the high-scoring box threshold but greater than the low-scoring box threshold, the detection box is added to the low-scoring detection box set, and the background is filtered out.

[0024] Furthermore, step S22 specifically includes:

[0025] S221: Kalman filter prediction: First, calculate the mean error m between the detection box and the trajectory box, then calculate the covariance. N Mapped to the detection space; m and N The calculation formulas are as follows:

[0026] ,

[0027] ,

[0028] Where p is the mean value of the detection boxes detected in frame k+1, H is called the measurement matrix, y and S are the mean and covariance matrices of the trajectory box prediction stage in frame k, and R is the noise matrix of the detector; where the trajectory box refers to the detection boxes before the current frame, that is, the detection boxes predicted by the internal algorithm.

[0029] S222: Calculate the Kalman gain K: The formula for calculation is:

[0030] ,

[0031] S223: Update the mean and covariance based on the Kalman filter gain K: The calculation formula is as follows:

[0032] ,

[0033] ,

[0034] in, and The mean and covariance state matrix after updating the trajectory bounding box in frame (k+1). As a variable parameter, in this experiment = 1.

[0035] Furthermore, step S23 specifically includes:

[0036] S231: First IoU Matching: Calculate the high-scoring detection box set and the trajectory set. IoU Distance matrix, where IoU The calculation formula is:

[0037] ;

[0038] Where A and B represent the high-resolution bounding box and the predicted trajectory bounding box, respectively;

[0039] S232: Second time IoU Matching: For successfully matched trajectories, update their Kalman filter and add them to the current frame's trajectory set; for unmatched trajectories, add them to the set of trajectories that failed to match in the first association. In the process, high-scoring detection boxes that failed to match are added to the set of detection boxes that failed to match in the first association. middle;

[0040] S233: Hungarian Matching: Calculating Low-Scoring Detection Boxes and... Trajectory set IoU The distance matrix is ​​used to add the trajectories that failed to match to the lost trajectory set. In the process, low-scoring detection boxes that fail to match are directly deleted; for successfully matched trajectories, their Kalman filters are updated and added to the current frame trajectory set.

[0041] Furthermore, step S24 specifically includes:

[0042] S241: Track Creation: For If the confidence value of the detection box in the frame is greater than the tracking score threshold, a new trajectory is created for it and merged into the current frame trajectory set; otherwise, no processing is performed.

[0043] S242: Track deletion: For tracks retained in If the trajectory exceeds the preset number of frames, it is considered a lost trajectory and deleted.

[0044] S243: Trajectory Merging: Returns the set of all trajectories in the current frame, uses it as the existing trajectory set for the next frame image, performs Kalman filtering to predict the trajectory of the next frame, and finally outputs the trajectory set.

[0045] Furthermore, step S3 specifically includes:

[0046] S31: In the YOLOX-X+ByteTrack model, add the variable classes, and the variable classes=4, which represent the four categories of pigs: lying down, standing, eating, and others.

[0047] S32: Design the detection frame and its proportions according to the shape of the pig to ensure that it conforms to the length and width of the pig;

[0048] S33: Design and implement a trajectory interpolation post-processing strategy for BYTE data association:

[0049] Assuming trajectory exist Frames and Frames are lost due to occlusion between frames, if the current trajectory Exactly in the first frame( ), at this time the trajectory The interpolation frame can be calculated using the following formula:

[0050]

[0051] In the formula, Indicates the first The coordinates of the frame's bounding box. Indicates the first The coordinates of the frame's bounding box. Indicates the first The coordinates of the frame's trajectory box.

[0052] Furthermore, step S4 specifically includes:

[0053] S41: Calculate the behavioral time of each individual pig in the group pig farming video; specifically including:

[0054] S411: Construct a video sequence processing model, using a time interval of a minutes as the sampling frequency, and randomly select b minutes of surveillance video samples for multi-target tracking;

[0055] S412: Based on the video sequence processing model, the lying time, eating time, standing time, and other times of each pig in the group-raised pig videos are statistically analyzed; by using the daily sampled videos, the various behavioral times of each pig over 24 hours are statistically analyzed, and all video segments are processed.

[0056] S42: Generate daily behavior and health analysis for each pig based on the set threshold for each behavior.

[0057] Compared with the prior art, the above-mentioned technical solutions provided in this application have the following advantages: The present invention improves the basic ByteTrack algorithm, effectively improves the ID error transformation caused by occlusion, improves the accuracy of the tracker, maintains excellent detection and tracking speed, and realizes real-time monitoring and fast and accurate tracking of various behaviors of pigs in actual pig farms. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a schematic diagram of the YOLOX-X model.

[0061] Figure 2 This is a structural diagram of the ByteTrack model;

[0062] Figure 3 The process of establishing the YOLOX-X+ByteTrack model;

[0063] Figure 4 This is a directory structure diagram of the YOLOX-X dataset;

[0064] Figure 5 This is one of the recognition results for the YOLOX-X+ByteTrack model;

[0065] Figure 6 Another recognition result for the YOLOX-X+ByteTrack model;

[0066] Figure 7 One of the recognition results of the improved YOLOX-X+ByteTrack model;

[0067] Figure 8 Another recognition result for the improved YOLOX-X+ByteTrack model;

[0068] Figure 9 A flowchart for daily statistical analysis of the behavior of each pig. Detailed Implementation

[0069] To provide a clearer understanding of the technical features, objectives, and effects of this invention, specific embodiments are now described in detail with reference to the accompanying drawings. In the following description, it should be understood that the orientations or positional relationships indicated by terms such as "front," "rear," "upper," "lower," "left," "right," "longitudinal," "horizontal," "vertical," "horizontal," "top," "bottom," "inner," "outer," "head," and "tail" are based on the orientations or positional relationships shown in the accompanying drawings, and are constructed and operated in a specific orientation. They are only for the convenience of describing this technical solution and do not indicate that the referred mechanism or element must have a specific orientation; therefore, they should not be construed as limitations on this invention.

[0070] It should also be noted that, unless otherwise explicitly specified and limited, terms such as "installation," "connection," "linking," "fixing," and "setting" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. When an component is referred to as being "on" or "below" another component, the component can be located "directly" or "indirectly" on the other component, or there may be one or more intermediary components. The terms "first," "second," "third," etc., are only for the convenience of describing this technical solution and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. For those skilled in the art, the specific meaning of the above terms in this invention can be understood according to the specific circumstances.

[0071] In the following description, specific details such as particular system structures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, mechanisms, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0072] Example 1

[0073] As attached Figure 1 As shown, a method for multi-target tracking and behavior statistics of group-raised pigs includes the following steps:

[0074] S1: Based on a video dataset of group-raised pigs, a YOLOX-X model was constructed. The YOLOX-X model consists of three parts: a feature extraction backbone network, an intermediate multi-scale feature fusion network, and a target detection and localization network for behavior recognition. Pig behavior was categorized into four types: lying down, standing, eating, and others. The video dataset based on group-raised pigs specifically includes:

[0075] S11: Collect video data of group-raised pigs in natural scenes. The video data includes videos of group-raised pigs under three scenarios: daytime and nighttime, crowded and sparsely populated pigs, and frequent and infrequent pig activity. The images in the video dataset are then enhanced with Gaussian noise reduction, Gaussian blurring, brightness adjustment, saturation adjustment, and tone adjustment. Specifically, S11 includes:

[0076] S111: Collect video data of group-raised pigs in natural scenes. The video data includes group-raised pig videos in three scenarios: daytime and nighttime, crowded and sparsely populated pigs, and frequent and infrequent pig activity.

[0077] S112: The acquired videos of group-raised pigs are segmented into images, and the images are labeled with categories to form the video dataset;

[0078] S113: Divide the video dataset into two categories: training set and test set; the training set is used to build the model, and the test set is used to test the built model.

[0079] S114: Apply Gaussian noise, Gaussian blur, brightness adjustment, saturation adjustment, and hue adjustment to the images in the video dataset, and then merge the adjusted images into the video dataset.

[0080] S12: As Figure 1 As shown, the feature extraction backbone network of the YOLOX-X model is constructed, which is a fusion of the Focus network component and four Dark network blocks; S12 specifically includes:

[0081] S121: Input the images from the training set into the Focus network. The input images are RGB images with a size of 640×640 pixels and 3 channels. The Focus network obtains four complementary feature maps through a slice operation, and then concatenates them to obtain a new feature map. Specifically, during the feature extraction process of the Focus network, the image size is reduced and the input channels are expanded. That is, the original image size is 640×640 pixels with 3 channels, and after passing through the Focus network structure, it becomes a feature map with a size of 320×320 pixels and 32 channels.

[0082] S122: Input the feature maps into Dark2, Dark3, and Dark4 network structures, including CBS and CSP deep learning network structures. The CBS deep learning network structure consists of convolutional layers (Conv), batch normalization layers (BN), and activation function layers (SiLU). The CSP deep learning network structure includes CSP1_x and CSP2_x, where x in CSP1_x represents x residual structures, and x in CSP2_x represents x CBS deep learning network structures. (See attached...) Figure 1 As shown, ResUnit represents the residual structure, which allows the network to be built deeper, and add represents the addition of tensors; CSP1_x is composed of two CBS deep learning network structures fused with x residual structures (Res Units), and CSP2_x is composed of two CBS deep learning network structures fused with x CBS deep learning network structures.

[0083] S123: Input the feature map generated in S212 into the Dark5 block structure. The SPP deep learning network structure adopts the CBS deep learning network structure combined with max pooling to perform multi-scale feature fusion. Specifically, the 40×40×256 feature map is halved by a 1×1 convolution, and then goes through three max pooling layers of different sizes: 5×5, 9×9 and 13×13. Finally, a 1×1 convolution operation is used to obtain a 20×20×512 feature map.

[0084] S13: As Figure 1 As shown, an intermediate multi-scale feature fusion network is constructed. For the feature extraction backbone network, a top-down FPN layer and a bottom-up PAN layer structure (PAFPN) are used to fuse semantic and positional information from feature maps of different sizes. S13 specifically includes:

[0085] S131: Based on the feature mapping of the backbone network, FPN uses the CBS deep learning network structure combined with upsampling operation to fuse deep semantic features and shallow positional features, thereby enhancing the semantic power of multi-scale features.

[0086] S132: Based on multi-scale semantic features, PAN transmits shallow positioning information to deep layers, enhancing the positioning capability of multi-scale features. Through the PAFPN network, three feature maps of 20×20×512, 40×40×256, and 80×80×128 are generated.

[0087] S14: As Figure 1 As shown, a target detection, localization, and behavior recognition network is constructed to achieve multi-target detection and classification; S14 specifically includes:

[0088] S141: Construct three decoupled head branches. First, use the CBS deep learning network structure to reduce the dimensionality of Dark3, Dark4, and Dark5. Then, each branch uses two CBS deep learning network structures. One CBS deep learning network structure uses a convolutional layer and a sigmoid function, while the other CBS deep learning network structure uses two branches. One branch contains a convolutional layer and a sigmoid function, and the other branch contains a convolutional layer. Then, the feature maps are concat-fused in the three branches. The three decoupled branches form 20×20×9, 40×40×9, and 80×80×9 feature information, respectively.

[0089] S142: Each decoupling head includes multiple CBS deep learning network structures, convolutional layers, and a Sigmoid function. The three decoupling heads undergo Concat and Reshape operations respectively, and finally Concat fusion is performed. Label assignment is used to associate the predicted box and the target box.

[0090] S143: The head category loss and target loss are calculated using BCEWithLogitsLoss and GIoU_Loss, respectively. The formula for BCEWithLogitsLoss is as follows:

[0091] ;

[0092] Where, x i Represents the predicted value, y i w represents the actual value. i is the weight, and i is the label category.

[0093] The GIoU_Loss function is as follows:

[0094] ;

[0095] Where A and B are two bounding boxes, C represents the smallest volume (or area) that encloses A and B.

[0096] S2: Construct the ByteTrack model. The ByteTrack model includes Kalman filtering, IoU Matching algorithms, Hungarian matching algorithm, and the creation, deletion, and merging of trajectories; models such as Figure 2 As shown, the target detection results of the model are first divided into high-resolution detection boxes. and low-resolution detection boxes The Kalman filter is used to predict the position of the current object in the next video frame. Then, the Hungarian matching algorithm is used to match the predicted bounding box and the detection bounding box. Finally, after the matching is completed, a new set of trajectories is obtained, and the parameters of the Kalman filter are updated. More specifically, step S2 is as follows:

[0097] S21: Divide the YOLOX-X target detection results into high-resolution detection boxes. and low-resolution detection boxes S21 specifically includes:

[0098] S211: Input the image into the YOLOX-X model to obtain detection boxes with different confidence levels, and set the confidence thresholds for the detection boxes, namely the high-scoring box threshold and the low-scoring box threshold.

[0099] S212: In the detection results, if the confidence value of the detection box is greater than the high-scoring box threshold, the detection box is added to the high-scoring detection box set. If the confidence value of a detection box is less than the high-scoring threshold but greater than the low-scoring threshold, then the detection box is added to the low-scoring detection box set. In the middle, and filter out the background.

[0100] S22: Construct a Kalman filter and use it to predict the position of the pig detection box in the next frame of each frame; S22 specifically includes:

[0101] S221: Kalman filter prediction: First, calculate the mean error m between the detection box and the trajectory box, then calculate the covariance. N Mapped to the detection space; m and N The calculation formulas are as follows:

[0102] ,

[0103] ,

[0104] Where p is the mean value of the detection box detected in frame k+1, H is called the measurement matrix, y and S are the mean and covariance matrices of the trajectory box prediction stage in frame k, and R is the noise matrix of the detector.

[0105] S222: Calculate the Kalman gain K: The formula for calculation is:

[0106] ;

[0107] S223: Update the mean and covariance based on the Kalman filter gain K: The calculation formula is as follows:

[0108] ;

[0109] ;

[0110] in, and The mean and covariance state matrix after updating the trajectory bounding box in frame (k+1). As a variable parameter, in this experiment = 1.

[0111] S23: Utilize IoU The matching algorithm and the Hungarian matching algorithm match the pig detection boxes in each frame of the model with the existing trajectories and assign ID numbers; S23 specifically includes:

[0112] S231: First IoU Matching: Calculated High-resolution bounding box and trajectory set IoU Distance matrix, where IoU The calculation formula is:

[0113] ;

[0114] Where A and B represent the high-resolution bounding box and the predicted trajectory bounding box, respectively;

[0115] S232: Second time IoU Matching: For successfully matched trajectories, update their Kalman filter and add them to the current frame's trajectory set. Trajectories that fail to match are added to... (From the set of trajectories that failed to match in the first association), high-scoring detection boxes that could not be successfully matched are placed... (In the set of detection boxes that failed to match in the first association);

[0116] S233: Hungarian Matching: Calculation Low-resolution box and Trajectory set IoU The distance matrix is ​​used to store the trajectories that failed to match. In the (lost trajectory set), low-scoring detection boxes that failed to match are directly deleted; these boxes are considered background boxes. For successfully matched trajectories, their Kalman filters are updated, and they are added to the current frame's trajectory set.

[0117] S24: Create, delete, and merge trajectories in the model; S24 specifically includes:

[0118] S241: Track Creation: For If the confidence value of the detection box in the frame is greater than the tracking score threshold, a new trajectory is created for it and merged into the current frame trajectory set; otherwise, no processing is performed.

[0119] S242: Track deletion: For tracks retained in If the trajectory in the frame exceeds 30 frames, it is considered a lost trajectory and deleted.

[0120] S243: Trajectory Merging: Returns the set of all trajectories in the current frame, uses it as the existing trajectory set for the next frame image, performs Kalman filtering to predict the trajectory of the next frame, and finally outputs the trajectory set.

[0121] S3: Construct an improved YOLOX-X+ByteTrack model. The improved YOLOX-X+ByteTrack model includes designing suitable anchor frames for pigs and a post-processing strategy for trajectory interpolation; S3 specifically includes:

[0122] S31: In the YOLOX-X+ByteTrack model, add the variable classes, and the variable classes=4, which represent the four categories of pigs: lying down, standing, eating, and others.

[0123] S32: Design a detection frame suitable for group-raised pigs, taking into account their shape, and improve its proportions to match the length and width of the pigs.

[0124] S33: Design and implement a trajectory interpolation post-processing strategy for BYTE data association. Its working principle is as follows:

[0125] Assuming trajectory exist Frames and Frames are lost due to occlusion between frames, if the current trajectory Exactly in the first frame( ), at this time the trajectory The interpolation frame can be calculated using the following formula:

[0126] ;

[0127] In the formula, Indicates the first The frame's bounding box coordinates (containing four values: the coordinates of the top-left corner x1, y1 and the coordinates of the bottom-right corner x2, y2). Indicates the first The coordinates of the frame's bounding box. Indicates the first The frame's bounding box coordinates. At the same time, a hyperparameter needs to be set. , represents the maximum frame interval for performing interpolation, meaning that interpolation is only performed when Trajectory interpolation is performed only at certain times.

[0128] S4: Construct a statistical analysis module to calculate the time spent on four different behaviors per pig per day in each pen, and generate a behavior and health status analysis table for each pig based on set thresholds; S4 specifically includes:

[0129] S41: Calculate the behavioral time of each individual pig in the group-raising pig video; S41 specifically includes:

[0130] S411: Construct a video sequence processing model, using a sampling frequency of 10 minutes per time period, randomly select 1 minute of surveillance video samples for multi-target tracking, and sample for 6 minutes per hour;

[0131] S412: Based on the video sequence processing model, the time of each pig's four behaviors (lying down time, eating time, standing time, and other times) in the group-raised pig videos is statistically analyzed. By using daily sampled videos, the time of each behavior of each pig over 24 hours is statistically analyzed, and all video segments are processed.

[0132] S42: Generate daily behavior and health analysis for each pig based on the set threshold for each behavior.

[0133] S42 specifically includes:

[0134] S421: Establish time threshold ranges for each behavior of pigs. Based on different pig breeds and age groups, set the normal lying time for pigs to be [14-20] hours, the feeding time to be [2-5] hours, the standing time to be [2-3] hours, and other times to be [0-2] hours;

[0135] S422: Based on the daily activity time statistics of each pig, generate a table showing the health status of each pig every day.

[0136] Example 2

[0137] More specifically, based on Example 1, the video dataset contains videos of pigs of different ages, sizes, numbers, and environments. Each video segment has a resolution of 2688×1520 and a frame rate of 5 frames per second. All datasets consist of overhead video clips. Due to the influence of camera height and focal length, objects outside the pigpen are inevitably captured. Therefore, in the experiment, a video cropping method was used to fix the perspective to inside the pigpen to reduce the influence of the external environment.

[0138] After obtaining the video data, FFmpeg software was used to edit the video segments, extracting segments with dense and sparse pigs, and segments showing daytime and nighttime activity. Based on observation, videos with a large number of pigs and significant occlusion were classified as dense videos, while those with fewer pigs were classified as sparse videos. Ultimately, 15 short video segments, each 1 minute long and with a frame rate of 5 frames per second, were completed. The environments of these 15 videos are shown in Table 1. Five videos were filmed at night, 10 were filmed during the day, four had fewer than 10 pigs, and 11 had more than 10 pigs. These video segments all included pigs lying down, eating, standing, or engaging in other behaviors to meet the experimental requirements. Then, DarkLabel software was used to label the data. After all data labeling was completed, 11 video segments were assigned as the training set, and 4 video segments were assigned as the test set.

[0139] Table 1 Video Environment

[0140]

[0141] In the specific implementation process, an improved ByteTrack algorithm was used to achieve behavior recognition and multi-target tracking of group-raised pigs. The software environment for training the model included: a 64-bit Linux Ubuntu 18.04 operating system on the server, Python 3.7 programming language, PyTorch 1.9.1 deep learning framework, and CUDA version 11.1. The CPU was a 12th Gen Intel(R) i9-12900KF, the GPU was an RTX 3090, and the memory was 32GB.

[0142] In the specific implementation process, the establishment of a tracking model based on the improved ByteTrack algorithm mainly involves four steps: dataset creation, detection model construction, tracking model construction, and model optimization. Figure 3 As shown, firstly, 15 video files were labeled using DarkLabel to obtain labeled files. Then, a Python script was used to convert the labeled files into JSON format to complete the YOLOX-X labeled dataset. Subsequently, during the training of the detection network, the YOLOX-X dataset was fed into the training network to obtain the output results of the detection model. Finally, the detection model, the improved ByteTrack tracking model, and their combination were optimized.

[0143] The directory structure of the dataset is as follows Figure 4As shown, a Dataset folder is created in the root directory, which is divided into three folders: Images, Datapath, and Annotations. Datapath is a mapping file for the dataset, indicating which images are used for testing and which images are used for training. It includes the corresponding paths of the images. Images and Annotations respectively store the images and their corresponding JSON annotation files. These two folders are further divided into training set and test set.

[0144] In the experiment, the improved YOLOX-X+ByteTrack model was trained using 11 videos and tested using 4 videos. The test videos were 0102, 0402, 0602, and 1502. Video 0102 had fewer than 10 pigs and was set in a daytime environment. Videos 0402, 0602, and 1502 all had more than 10 pigs. Videos 0402 and 0602 were set in daytime environments, while video 1502 was set in a nighttime environment. The model was evaluated using a combination of metrics, including Identification F1 (IDF1), Mostly Tracked (MT), False Positives (FP), ID Switch (IDs), Multiple Object Tracking Accuracy (MOTA), and Multiple Object Tracking Precision (MOTP). Among them, IDF1, IDs, and MOTA are important indicators of tracker performance. The higher the IDF1 and MOTA values, the better the performance, and the lower the IDs value, the better the performance.

[0145] The results of the YOLOX-X detection model on the test set are shown in Table 2. The test set included 1200 images. The accuracy for each category exceeded 92%, recall exceeded 88%, mAP exceeded 95%, and F1 score exceeded 0.82. The average accuracy reached 94.2%, the average recall reached 89.7%, the average mAP reached 96.9%, and the average F1 score reached 0.88. It can be seen that the YOLOX-X detection model performs well in terms of metrics and achieves a high level of detection capability for pigs.

[0146] Table 2 Detection results of the YOLOX-X model

[0147]

[0148] In the specific implementation process, the trajectory score threshold was set to 0.76, the matching threshold was set to 0.98, and the input image size was 1344×506. The test results of the YOLOX-X+ByteTrack model are shown in Table 3. This method achieved 95.3% on MOTA and 93.9% on IDF1, indicating that the tracker has good performance for group-housed pigs in pigsty scenarios. The IDF1 value was 13, indicating that the model can continuously track and has stable performance. Considering other indicators, this tracker is suitable for multi-target tracking in group-housed pig environments.

[0149] Table 3 Test results of the YOLOX-X+ByteTrack model

[0150]

[0151] The test results of the improved ByteTrack model during implementation are shown in Table 4. It can be observed that the IDF1 values ​​of test videos 0102, 0402, and 0602 are between 92% and 93%, indicating good trajectory tracking time for these three videos. Test video 1502, on the other hand, achieves over 98% precision, recall, and IDF1 score, maintaining target tracking for an even longer time. The average IDF1 for all test videos reaches 94.5%, indicating that the overall trajectory tracking performance of the model algorithm is excellent. Regarding precision (Prcn) and recall (Rcll), the average values ​​for all test videos are 97.8% and 98.0%, respectively, indicating that the model can accurately identify the pig target. All videos achieve optimal performance on both manual and machine learning (MT) tests, indicating that all trajectories are correctly tracked more than 80% of the time, with no significant deviation in tracking persistence. Regarding the IDs metric, the total number of IDs for the four videos is only 9, while the total number of IDs for the 1502 video segments is 0. This indicates that the improved ByteTrack tracking model can continuously track targets in dense and complex occlusion environments without loss of tracking, demonstrating stable tracking performance. Meanwhile, the test videos all achieved a MOTA (Motion on Target Adaptation) score above 90%, averaging 95.8%, indicating that the model algorithm has excellent accuracy in target detection and trajectory stability. The MOTP (Motion on Target Adaptation) score averages 0.189, indicating a small positional error. In conclusion, the improved multi-target tracking model performs exceptionally well in terms of both target recognition accuracy and trajectory persistence and stability.

[0152] Table 4 Test results of the improved YOLOX-X+ByteTrack model

[0153]

[0154] In the specific implementation process, some results of the YOLOX-X+ByteTrack model are as follows: Figure 5-6 As shown; some results of the improved YOLOX-X+ByteTrack model are as follows. Figure 7-8 As shown. Among them, Figure 5 and Figure 7 This is the 264th frame of video segment 0102. Figure 6 and Figure 8 This is the 117th frame of video segment 0402. (Attached) Figure 5-8 In the experiment, during the testing phase, ByteTrack was optimized for multi-target tracking. The pig behaviors detected by the detector were labeled as lying, standing, eating, and other categories and added to the tracker to achieve multi-behavioral category tracking. Figure 5 and Figure 7 The comparison revealed that when a pig with ID 4 was occluded by a pig with ID 1, the tracking result still retained ID 4 after the improved YOLOX-X+ByteTrack trajectory interpolation process, and no ID switching occurred. Figure 6 and Figure 8 The comparison revealed that, before the improved YOLOX-X+ByteTrack trajectory interpolation process, the trajectory frame of pig with ID 11 was lost after being occluded by pig with ID 15, while the interpolation process did not cause any loss.

[0155] The statistical analysis module generates the daily behavior time for each pig. Based on set thresholds, the module generates an analysis of each pig's behavior and health status. The process is as follows: Figure 9 As shown. The specific steps are as follows:

[0156] (a) Using a 10-minute time interval as the sampling frequency, a 1-minute monitoring video sample was randomly selected for multi-target tracking. All samples were input into the video sequence processing model, and pig behaviors were divided into four categories: lying down, eating, standing, and other.

[0157] (b) After processing a single sampled video, the behavior time of each pig in each frame is obtained, and the behavior time of each individual pig in the sampled video is calculated.

[0158] (c) Based on the behavior time of each pig in the sampled video generated in the previous step, namely lying time, eating time, standing time, and other time, the behavior time of each pig in 24 hours is statistically determined by the sampled video of one day, and all video segments are processed.

[0159] (d) Take the time of various behaviors of each pig generated in the previous step over 24 hours, and set the thresholds for each behavior as follows: normal lying time for pigs is [14-20] hours, eating time is [2-5] hours, standing time is [2-3] hours, and other time is [0-2] hours. Generate a daily analysis of the behavior and health status of each pig.

[0160] It is understood that the above embodiments only illustrate preferred embodiments of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can freely combine the above technical features without departing from the concept of the present invention, and can also make several modifications and improvements, all of which fall within the protection scope of the present invention. Therefore, all equivalent transformations and modifications made with respect to the scope of the claims of the present invention should fall within the scope of the claims of the present invention.

Claims

1. A method for multi-target tracking and behavioral statistics of group-raised pigs, characterized in that, Includes the following steps: S1: Establish a video dataset of group-raised pigs and construct a YOLOX-X model. The YOLOX-X model consists of three parts: a feature extraction backbone network, an intermediate multi-scale feature fusion network, and a target detection and localization and behavior recognition network. S2: Construct the YOLOX-X+ByteTrack model, which includes the YOLOX-X model, Kalman filtering, IoU matching algorithm, Hungarian matching algorithm, and trajectory creation, deletion, and merging; specifically, constructing the YOLOX-X+ByteTrack model includes: S21: Divide the target detection results output by the YOLOX-X model into high-scoring detection boxes and low-scoring detection boxes; S22: Construct a Kalman filter and use the Kalman filter to predict the position of the pig detection box in the next frame of each frame; S23: Use the IoU matching algorithm and the Hungarian matching algorithm to match the pig detection boxes in each frame of the model with the existing trajectories and assign ID numbers; S24: Create, delete, and merge trajectories in the model; S3: Design pig anchor frames and trajectory interpolation post-processing strategies in the YOLOX-X+ByteTrack model to obtain an improved YOLOX-X+ByteTrack model; specifically including: S31: In the YOLOX-X+ByteTrack model, add the variable classes, and the variable classes=4, which represent the four categories of pigs: lying down, standing, eating, and others. S32: Design the detection frame and its proportions according to the shape of the pig to ensure that it conforms to the length and width of the pig; S33: Design and implement a trajectory interpolation post-processing strategy for BYTE data association: Assuming trajectory exist Frames and Frames are lost due to occlusion between frames, if the current trajectory Exactly in the first frame, Current trajectory The interpolation frame is calculated using the following formula: ； In the formula, Indicates the first The coordinates of the frame's bounding box. Indicates the first The coordinates of the frame's bounding box. Indicates the first The coordinates of the frame's bounding box; S4: An improved YOLOX-X+ByteTrack model was used to perform target tracking and behavior statistics on group-raised pigs.

2. The method for multi-target tracking and behavior statistics of group-raised pigs according to claim 1, characterized in that, Step S1 involves creating a video dataset of herd-raised pigs, including: S11: Label the pig behaviors in the video dataset into four categories: lying down, standing, eating, and others; divide the video dataset into a training set and a test set.

3. The method for multi-target tracking and behavior statistics of group-raised pigs according to claim 1, characterized in that, Step S1, which involves constructing the YOLOX-X model, specifically includes: S12: Construct the feature extraction backbone network of the YOLOX-X model, which is a fusion of the Focus network component and four Dark network blocks; S13: Construct an intermediate multi-scale feature fusion network. For the feature extraction backbone network, use a top-down FPN layer and a bottom-up PAN layer structure to fuse semantic and positional information of feature maps of different sizes. S14: Construct a target detection, localization, and behavior recognition network to achieve multi-target detection and classification.

4. The method for multi-target tracking and behavior statistics of group-raised pigs according to claim 1, characterized in that, Step S21, which divides the YOLOX-X target detection results into high-scoring and low-scoring detection boxes, specifically includes: S211: Input the images in the training set into the YOLOX-X model to obtain detection boxes with different confidence levels, and set the high-scoring box threshold and low-scoring box threshold for the detection boxes. S212: In the detection results, if the confidence value of the detection box is greater than the high-scoring box threshold, the detection box is added to the high-scoring detection box set; if the confidence value of the detection box is less than the high-scoring box threshold but greater than the low-scoring box threshold, the detection box is added to the low-scoring detection box set, and the background is filtered out.

5. The method for multi-target tracking and behavior statistics of group-raised pigs according to claim 1, characterized in that, Step S22 specifically includes: S221: Kalman filter prediction: First, calculate the mean error m between the detection box and the trajectory box, then calculate the covariance. N Mapped to the detection space; m and N The calculation formulas are as follows: ； ； Where p is the mean value of the detection box detected in frame k+1, H is called the measurement matrix, y and S are the mean and covariance matrices of the trajectory box prediction stage in frame k, and R is the noise matrix of the detector. S222: Calculate the Kalman gain K: The formula for calculation is: ； S223: Update the mean and covariance based on the Kalman filter gain K: The calculation formula is as follows: ， ， in, and The mean and covariance state matrix after updating the trajectory bounding box in frame (k+1). As a variable parameter, in this experiment = 1.

6. The method for multi-target tracking and behavior statistics of group-raised pigs according to claim 1, characterized in that, Step S23 specifically includes: S231: First IoU Matching: Calculate the high-scoring detection box set and the trajectory set. IoU Distance matrix, where IoU The calculation formula is: ， Where A and B represent the high-resolution bounding box and the predicted trajectory bounding box, respectively; S232: Second time IoU Matching: For successfully matched trajectories, update their Kalman filter and add them to the current frame trajectory set; for unmatched trajectories, add them to the first set of unmatched trajectories T. remain In the process, high-scoring detection boxes that failed to match are added to the set D of the first set of unmatched detection boxes. remain middle; S233: Hungarian Matching: Calculating Low-Scoring Detection Boxes with T remain Trajectory set IoU The distance matrix is ​​used to add the unmatched trajectories to the lost trajectory set T. lost In the process, low-scoring detection boxes that fail to match are directly deleted; for successfully matched trajectories, their Kalman filters are updated and added to the current frame trajectory set.

7. The method for multi-target tracking and behavior statistics of group-raised pigs according to claim 1, characterized in that, Step S24 specifically includes: S241: Trajectory Creation: For D remain If the confidence value of the detection box in the frame is greater than the tracking score threshold, a new trajectory is created for it and merged into the current frame trajectory set; otherwise, no processing is performed. S242: Track deletion: For tracks retained in T lost If the trajectory exceeds the preset number of frames, it is considered a lost trajectory and deleted. S243: Trajectory Merging: Returns the set of all trajectories in the current frame, uses it as the existing trajectory set for the next frame image, performs Kalman filtering to predict the trajectory of the next frame, and finally outputs the trajectory set.

8. The method for multi-target tracking and behavior statistics of group-raised pigs according to claim 1, characterized in that, Step S4 specifically includes: S41: Calculate the behavioral time of each individual pig in the group pig farming video; specifically including: S411: Construct a video sequence processing model, using a time interval of a minutes as the sampling frequency, and randomly select b minutes of surveillance video samples for multi-target tracking; S412: Based on the video sequence processing model, the lying time, eating time, standing time, and other times of each pig in the group-raised pig videos are statistically analyzed; by using the daily sampled videos, the various behavioral times of each pig over 24 hours are statistically analyzed, and all video segments are processed. S42: Generate daily behavior and health analysis for each pig based on the set threshold for each behavior.

Images