Pig multi-target tracking method and application

Abstract

The invention discloses a live pig multi-target tracking method and application, and the method comprises the following steps: constructing a diversified data set labeled by employing a directional bounding box; performing live pig target detection by using a YOLOv8n model; carrying out confidence classification on the input video sequence; outputting a predicted target trajectory by using an improved unscented Kalman filter, and carrying out primary correlation matching on the predicted target trajectory and the high-confidence target; performing secondary association matching on the track which is not successfully matched and the low-confidence target, and re-associating the detection target which is not successfully matched in the two times and the track which is considered to be lost in the previous record by utilizing a supplementary matching strategy; a new track is formed after successful matching in the supplementary matching strategy. The method has the beneficial effects that the high-order tracking precision is 96.10%, 83.10%, 76.50% and 84.00% respectively in four complex scenes of day heavy density, day extreme density, night heavy density and night extreme density, and the I DF1 scores are 95.7%, 78.2%, 70.1% and 77.6% respectively.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-target tracking of live pigs, and in particular to a multi-target tracking method of live pigs and applications thereof. Background Art

[0002] Pig farming has become a pillar industry of my country's animal husbandry, and monitoring the health status of pigs is a key link in pig production. The use of multi-target tracking technology can effectively record the daily activity trajectory of pigs and detect their physiological state, which helps to detect abnormalities in pigs early and take intervention measures quickly, thereby reducing the losses of breeding companies. Traditional pig farming management often relies on manual observation, which is not only inefficient and narrow in coverage, but also difficult to detect subtle changes and potential problems in pigs. With the rapid development of intelligent farming technology, multi-target tracking technology has gradually been applied to pig farming, which can automatically capture and analyze multi-dimensional information such as pigs' movement trajectories, posture changes, and social behaviors, providing farmers with comprehensive and accurate data support.

[0003] Most existing studies have adopted the TBD tracking algorithm. Although researchers have adopted a variety of different strategies to improve the performance of detectors and trackers, the tracking scenarios in these studies are relatively simple and do not fully reflect the various complex situations that may be encountered in real environments. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by the present invention is to provide a multi-target tracking method for pigs.

[0007] To solve the above technical problems, the present invention provides the following technical solutions: a method for tracking multiple targets of live pigs, comprising the following steps: constructing a diversified data set annotated with directional bounding boxes; using the YOLOv8n model to detect live pig targets, and outputting the bounding box coordinates, width, height, confidence and category of each target; performing confidence classification on the input video sequence, including high confidence and low confidence; using the improved unscented Kalman filter output to predict the target trajectory and perform an association match with the high-confidence target; updating the trajectory of the successfully matched targets and trajectories, and performing a secondary association match with the low-confidence target for the unmatched trajectories, and re-associating the detected targets that were not successfully matched in both times and the trajectories that were considered lost in the previous records using a supplementary matching strategy; in the supplementary matching strategy, a new trajectory is formed after a successful match, and the unmatched trajectory is marked as a lost trajectory, so as to maintain continuous tracking of multiple targets of live pigs.

[0008] Preferably, the improvement of the unscented Kalman filter is to expand the state vector, including adding angle and angular velocity parameters, and adding a residual function to process angle variables.

[0009] Preferably, the unscented Kalman filter approximates the probability distribution of the state vector through a set of samples called sigma points and propagates uncertainty. The construction formula of the sigma point is as follows:

[0010]

[0011] In the formula, represents the i-th sigma point, represents the mean estimate of the state vector at time K, Pk represents the covariance matrix of the state vector at time k, reflecting the uncertainty of the state estimate, n is the dimension of the state vector, and λ is an adjustment parameter.

[0012] Preferably, the adding of angle and angular velocity parameters comprises defining the Kalman filter state vector as In the formula, x and y represent the corresponding center point coordinates, a represents the aspect ratio, and h represents the height. and represents the x and y components of velocity, and Then they represent the rate of change of aspect ratio and height respectively; increase the rotation angle θ and angular velocity ω, and redefine them as

[0013] Preferably, the adding of the residual function to process the angle variable includes introducing a custom residual function to optimize the processing method of the angle variable and improve the accuracy and stability of the unscented Kalman filter in the application scenario containing the angle value state vector. The angle value residual is as follows:

[0014] y[i]=(a[i]-b[i]+π)%(2π)-π

[0015] In the formula, for i targets, the angle difference is first calculated, then π is added to the difference and modulo 2π to ensure that the result is within [0,2π], and finally π is subtracted to translate the result to [-π,π] to ensure the continuity and correctness of the residual angle.

[0016] Preferably, the unscented Kalman filter introduces additional constraints on the generation of trajectories, adds a time series for storing the number of recently detected targets, and avoids the ID growth caused by false detection in some frames by calculating the average value of the detected targets within a certain number of frames, as shown in the following formula:

[0017]

[0018] Where Ad represents the average number of detected targets, Td represents the total number of targets detected from the current frame to the previous t-th frame, and Tf represents the total number of frames from the current frame to the previous t-th frame.

[0019] Preferably, the input video frame rate is 16 frames per second, and Tf is set to 160, which means that in the past 10 seconds, only when the average number of detected targets Ad is greater than the number of currently accurately tracked trajectories.

[0020] Preferably, the YOLOv8n model performs pig target detection, including a backbone network, a feature fusion layer and an output head; the backbone network is responsible for feature extraction, and uses Darknet with a CSP structure to process image features, gradually reducing the resolution and increasing the depth of the feature map; in the feature fusion layer, feature fusion is achieved through PANet and features at different levels are integrated using bottom-up and top-down paths to ensure the effective combination of high- and low-resolution features; the output head generates a detection box and category probability prediction based on the fused feature map, and outputs the bounding box coordinates, width and height, confidence and category of each target.

[0021] The present invention also provides an application of a multi-target tracking method for live pigs, including the multi-target tracking method for live pigs as described above, which is applied to multi-target tracking of live pigs.

[0022] Preferably, in four complex scenarios of heavy density during the day, extremely dense during the day, heavy density at night, and extremely dense at night, the high-order tracking accuracies of the multi-target tracking method for live pigs are 96.10%, 83.10%, 76.50% and 84.00%, respectively, and the IDF1 scores are 95.7%, 78.2%, 70.1% and 77.6%, respectively.

[0023] The beneficial effects of the present invention are as follows: in four complex scenarios of heavy density during the day, extremely dense during the day, heavy density at night, and extremely dense at night, the high-order tracking accuracies are 96.10%, 83.10%, 76.50% and 84.00% respectively, and the IDF1 scores are 95.7%, 78.2%, 70.1% and 77.6% respectively, which are 21, 17.1, 5.5 and 5.5 percentage points higher than those of the ByteTrack algorithm and the UKFTrack algorithm, and 4.7, 9.1, 5.8 and 5.0 percentage points higher than those of the IDF1. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0025] Figure 1 This is a schematic diagram of the overall logical structure of the pig multi-target tracking method of the present invention;

[0026] Figure 2 Schematic diagram of video data examples in different scenarios according to the present invention;

[0027] Figure 3 A schematic diagram of an example of video data in another different scenario according to the present invention;

[0028] Figure 4 A schematic diagram of the horizontal frame annotation and the rotation frame annotation of the present invention;

[0029] Figure 5 A schematic diagram of the network structure of YOLOv8n according to the present invention;

[0030] Figure 6 Schematic diagram of the HBB of the present invention in the 864th frame, the 886th frame and the 901st frame in Video 06;

[0031] Figure 7 Schematic diagram of the OBB of the present invention in the 864th frame, the 886th frame and the 901st frame in Video 06;

[0032] Figure 8 A schematic diagram of an example of a target disappearing in a partial scene of Video 08 of the present invention;

[0033] Fig. 9 Schematic diagram of an example of eating behavior tracked in two different annotation methods in some scenes of Video 07 of the present invention;

[0034] Fig.10 This is a schematic diagram of visual comparison of different algorithms in frame 1664 of Video 06 of the present invention;

[0035] Fig.11 This is a schematic diagram of visual comparison of different algorithms in frame 1728 of Video 06 of the present invention;

[0036] Fig.12 This is a schematic diagram of visual comparison of different algorithms in frame 1744 of Video 06 of the present invention;

[0037] Fig.13 It is a schematic diagram of the 320th frame of the process of losing the target tracked at night in Video 08 of the present invention;

[0038] Fig.14 It is a schematic diagram of the 448th frame of the process of losing the target tracked at night in Video 08 of the present invention;

[0039] Fig.15 It is a schematic diagram of the 480th frame of the process of losing the target tracked at night in Video 08 of the present invention;

[0040] Fig.16 Schematic diagram of the tracking results of the last frame of our algorithm in four different scenarios described in the present invention;

[0041] Fig.17 Schematic diagram of the tracking results of the last frame of the OBB algorithm in the four different scenarios described in the present invention;

[0042] Fig.18 Schematic diagram of the tracking results of the last frame of the HBB algorithm in the four different scenarios described in the present invention. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0045] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0046] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0047] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0048] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0049] Example 1

[0050] Multi-target tracking technology is of great significance to the precision breeding of pigs. In view of the challenges of multi-target tracking caused by differences in breeding environments, rapid movement of pigs, and frequent occlusion between groups of pigs, this embodiment proposes a pig multi-target tracking algorithm (UKFTrack). Figure 1 , which is a schematic diagram of the overall idea of ​​the pig multi-target tracking method proposed in this embodiment, and specifically includes the following steps:

[0051] S1: Building a diverse dataset with oriented bounding box annotations;

[0052] S2: Use the YOLOv8n model to detect pig targets and output the bounding box coordinates, width, height, confidence and category of each target; In this step, the YOLOv8n model is used to detect pig targets, including a backbone network, a feature fusion layer and an output head; The backbone network is responsible for feature extraction, and uses Darknet with a CSP structure to process image features, gradually reducing the resolution and increasing the depth of the feature map; In the feature fusion layer, feature fusion is achieved through PANet and features at different levels are integrated using bottom-up and top-down paths to ensure the effective combination of high and low resolution features; The output head generates a detection box and category probability prediction based on the fused feature map, and outputs the bounding box coordinates, width, height, confidence and category of each target.

[0053] S3: classify the input video sequence into confidence levels, including high confidence and low confidence levels;

[0054] S4: Use the improved unscented Kalman filter output to predict the target trajectory and perform an association match with a high-confidence target. The improvement of the unscented Kalman filter in this step is to expand the state vector, including adding angle and angular velocity parameters, and adding a residual function to process the angle variable; the details are as follows:

[0055] The unscented Kalman filter of this embodiment approximates the probability distribution of the state vector through a set of samples called sigma points and propagates uncertainty. The construction formula of the sigma point is as follows:

[0056]

[0057] In the formula, represents the i-th sigma point, represents the mean estimate of the state vector at time K, P k Represents the covariance matrix of the state vector at time k, reflecting the uncertainty of state estimation, n is the dimension of the state vector, and λ is an adjustment parameter;

[0058] Added angle and angular velocity parameters include:

[0059] The Kalman filter state vector is defined as In the formula, x and y represent the corresponding center point coordinates, a represents the aspect ratio, and h represents the height. and represents the x and y components of velocity, and Then they represent the rate of change of aspect ratio and height respectively; increase the rotation angle θ and angular velocity ω, and redefine them as

[0060] Adding residual functions to process angle variables includes:

[0061] A custom residual function is introduced to optimize the processing method of angle variables and improve the accuracy and stability of the unscented Kalman filter in application scenarios with angle value state vectors. The angle value residual is as follows:

[0062] y[i]=(a[i]-b[i]+π)%(2π)-π

[0063] In the formula, for i targets, the angle difference is first calculated, then π is added to the difference and modulo 2π to ensure that the result is within [0,2π], and finally π is subtracted to translate the result to [-π,π] to ensure the continuity and correctness of the residual angle.

[0064] Furthermore, in this embodiment, the unscented Kalman filter introduces additional constraints on the generation of trajectories, adds a time series to store the number of recently detected targets, and calculates the average value of the detected targets within a certain number of frames to avoid the increase of IDs caused by false detections in some frames, as shown in the following formula:

[0065]

[0066] Where Ad represents the average number of detected targets, Td represents the total number of targets detected from the current frame to the previous t-th frame, and Tf represents the total number of frames from the current frame to the previous t-th frame. In this embodiment, the input video frame rate is 16 frames per second, and Tf is set to 160, which means that in the past 10 seconds, only when the average number of detected targets Ad is greater than the number of currently accurately tracked trajectories.

[0067] S5: The successfully matched targets and trajectories are updated, and the unmatched trajectories are re-associated with low-confidence targets. At the same time, the detected targets that were not successfully matched twice and the trajectories that were considered lost in the previous records are re-associated using the supplementary matching strategy;

[0068] S6: After a successful match in the supplementary matching strategy, a new trajectory is formed, and the unmatched trajectory is marked as a lost trajectory to maintain continuous tracking of multiple pig targets.

[0069] In this embodiment, a diversified data set annotated with an oriented bounding box (OBB) is first constructed, covering various movement modes of pigs, different feeding scenarios and pig population densities; secondly, an unscented Kalman filter is introduced to better adapt to the OBB annotation, and the traditional state vector is expanded, angle and angular velocity parameters are added, and a residual function is designed to process angle variables to avoid errors caused by direct subtraction. Finally, a multi-stage matching strategy is proposed. Through multiple trajectory associations and supplementary matching mechanisms, new trajectories are generated for high-confidence targets after successful re-matching, and trajectories are updated for low-confidence targets to ensure that the target can be continuously tracked in the case of severe occlusion or violent movement. The tracking algorithm of this embodiment can achieve accurate tracking of group pigs in complex environments, and shows strong robustness, which can provide reliable technical support for pig behavior and health monitoring in practical applications.

[0070] Example 2

[0071] This embodiment proposes a specific application description of a multi-target tracking method for live pigs, which is actually applied to the multi-target tracking scenario of live pigs, and a data set covering various movement modes of pigs, different feeding scenarios and pig population densities is built. Then a UKFTrack algorithm suitable for non-directional annotation boxes is proposed to achieve accurate tracking of pigs in complex environments. The details are as follows:

[0072] (1) Dataset construction.

[0073] ① Video data collection:

[0074] The data in this study comes from the public dataset of the University of Nebraska-Lincoln. The selected videos were shot from a top-down perspective to fully observe the movement of pigs in the fence. The data contains 8 videos with high activity levels. 4 of them were selected for model training and verification, and the video length was 22 minutes. The remaining 4 were used for tracking tests, each with a length of 8 minutes, containing about 31,000 frames of data, with a resolution of 1920×1080 and a frame rate of 16 frames / s. The video data is shown in Table 1.

[0075] Table 1: Video data.

[0076]

[0077]

[0078] The above data covers video samples during the day and night, in different fence environments and at different pig population densities, such as Figure 2-3 As shown in Figure 1, examples of video data in different scenarios are shown. Figure 2It can be seen that the activities of pigs in the sample showed significant changes. Figure 2 From left to right, the images show an extremely active scene of pigs moving at high speed (daytime), a situation where pigs eat at night and continue to move (nighttime), and a high-density and high-activity situation at night (nighttime). Figure 3 From left to right, the figure shows the situation of high density and high activity during the day (daytime), the challenge brought by mutual obstruction between pigs in a high-density environment (daytime), and the phenomenon of high density and accompanying obstruction at night (nighttime).

[0079] ②Video data collection:

[0080] 854 key frames were extracted from the training data set, and the location information of each pig was annotated using X-AnyLabeling software. For the same training set data, two different annotation methods were used: horizontal bounding box (HBB) and oriented bounding box (OBB). Taking video 02 as an example, the annotation results are as follows: Figure 4 As shown, Figure 4 The left side of the figure shows horizontal bounding box annotation, and the right side shows directional bounding box annotation. The annotated images were enhanced by adding noise, color enhancement, horizontal flipping, and mirror flipping. Finally, a pig target detection dataset containing 3416 images was obtained, and the ratio of training set to validation set was divided into 8:2.

[0081] (2)Multi-target tracking algorithm.

[0082] In order to solve the problem of decreased tracking performance due to rapid movement and mutual occlusion of pigs in high-density breeding environments, the filter and trajectory matching mechanism of the ByteTrack algorithm were improved, and then a pig multi-target tracking algorithm UKFTrack (i.e., the multi-target tracking algorithm of this embodiment) that can adapt to complex environments was proposed. The design process is again referred to Figure 1 As shown in the figure. First, the input video sequence is classified by confidence and then sent to the two-association process. In order to better meet the needs of oriented bounding box (OBB) tracking with angle values, the original Kalman filter is replaced by the improved unscented Kalman filter (UKF). For targets and trajectories that are still not matched after two associations, an additional supplementary matching mechanism is introduced. To further improve the stability of tracking, when the third additional matching fails, the restriction condition of activating the trajectory is set.

[0083] In this embodiment, in order to balance the detection effect and computational efficiency, the YOLOv8n model with the smallest number of parameters in YOLOv8 is selected for pig target detection. Its main structure consists of a backbone network (Backbone), a feature fusion layer (Neck) and an output head (Head). Figure 5 As shown. The backbone network is responsible for feature extraction, using Darknet with a CSP structure to process image features, gradually reducing the resolution and increasing the depth of the feature map. In the feature fusion layer, YOLOv8's neck uses PANet (Path Aggregation Network) to achieve feature fusion and integrates features at different levels using bottom-up and top-down paths to ensure the effective combination of high- and low-resolution features. The output head (Head) generates detection boxes and category probability predictions based on the fused feature map, and outputs the bounding box coordinates, width, height, confidence, and category of each target. Through the collaborative work of these three parts, the YOLOv8n model can accurately detect multiple targets in group pig farming while maintaining efficient inference speed.

[0084] The Kalman Filter (KF) in the ByteTrack algorithm is used to predict the position information of the previous frame trajectory. In the multi-target tracking scenario with OBB annotation, the linear assumption of KF is difficult to handle complex nonlinear motion, especially when the pigs rotate, turn sharply, and change speed. This will not only lead to a decrease in tracking accuracy, but also cause the loss of the target in complex scenes with frequent occlusion and interaction. At the same time, the rotation angle of the object in OBB tracking is a key factor, and KF lacks the ability to effectively model rotation and nonlinear states, making it difficult to cope with challenges in dynamic environments.

[0085] The improved unscented Kalman filter (UKF) used in this embodiment has significant advantages in processing nonlinear systems. UKF can effectively capture the nonlinear characteristics of the system through unscented transformation, especially in the case of target rotation and non-uniform motion, and can provide more accurate prediction and update. In addition, UKF can directly process state variables such as corner points, center points, width, height and rotation angle of OBB, and its performance is much better than KF in scenarios where the rotation of pigs changes significantly. UKF can also reduce the error propagation caused by linear approximation, ensure the accurate update of state and covariance matrices, and thus maintain robust tracking of targets in complex environments.

[0086] ①State vector:

[0087] In UKF, the probability distribution of the state vector is approximated by a set of samples called sigma points, and the uncertainty is propagated. The construction formula of the sigma point is:

[0088]

[0089] In the formula, represents the i-th sigma point, represents the mean estimate of the state vector at time K, P k represents the covariance matrix of the state vector at time k, reflecting the uncertainty of the state estimate, n is the dimension of the state vector, and λ is an adjustment parameter. Specifically, each sigma point is derived from the mean Starting from, in order to maintain the true reflection of the sigma point set on the mean of the state trajectory, It ensures that the sigma point can accurately reflect the covariance structure of the state vector, thereby effectively characterizing the Gaussian distribution of the state vector.

[0090] The state vector is used to describe the motion characteristics of the target at a given moment. Its accurate estimation is crucial to maintaining the continuity and integrity of the target. The Kalman filter state vector is defined as:

[0091]

[0092] In the formula, x and y represent the corresponding center point coordinates, a represents the aspect ratio, and h represents the height. and represents the x and y components of velocity, and They represent the rate of change of aspect ratio and height respectively.

[0093] Because the state vector cannot fully capture the rotation information of the target, the prediction accuracy will be reduced when rotation and sudden turns occur, especially in nonlinear motion. To solve this problem, the rotation angle θ and angular velocity ω are added to the above state vector and redefined as:

[0094]

[0095] The introduction of θ and ω can improve the estimation accuracy of UKF for the future position and direction of the target in the prediction stage, and make the filter more flexible in dealing with nonlinear motion.

[0096] ②Residual function:

[0097] Furthermore, this embodiment adds a residual function, and the residual function in the Kalman filter is used to calculate the difference between two vectors. For non-angle value variables, the residual can be obtained by direct subtraction. However, for angle value variables, since the angle usually varies between the interval [-π,π] or [0,2π], direct subtraction will ignore the periodicity of the angle, resulting in discontinuity of the result and causing deviation in the residual calculation. To this end, this embodiment introduces a custom residual function to optimize the processing method of the angle variable, thereby improving the accuracy and stability of UKF in application scenarios containing angle value state vectors. The angle value residual calculation formula is as follows:

[0098] y[i]=(a[i]-b[i]+π)%(2π)-π

[0099] In the formula, for i targets, the angle difference is first calculated, then π is added to the difference and modulo 2π to ensure that the result is within [0,2π], and finally π is subtracted to translate the result to [-π,π] to ensure the continuity and correctness of the residual angle.

[0100] ③ Constraints and supplementary matching strategies for tracking trajectories:

[0101] In order to solve the problem of frequent ID increments in the tracker due to false detection of the detector during target tracking, additional constraints are introduced to the generation of trajectories, and a time series is designed to store the number of recently detected targets. The ID increase caused by false detection in some frames is avoided by calculating the average value of the detected targets within a certain number of frames. The calculation formula is as follows:

[0102]

[0103] Where Ad represents the average number of detected targets, Td represents the total number of targets detected from the current frame to the previous t-th frame, and Tf represents the total number of frames from the current frame to the previous t-th frame. In this study, the video frame rate is 16 frames per second, so Tf is set to 160, which means that in the past 10 seconds, only when the average number of detected targets (Ad) is greater than the number of currently accurately tracked trajectories, new trajectories are allowed to be activated. This restriction strategy effectively reduces the increase in IDs caused by false detections by the detector.

[0104] In addition, based on the original tracking and matching framework, an innovative strategy is introduced to recover interrupted trajectories. In this strategy, when the detected target fails to find the corresponding trajectory in the initial matching and the subsequent low-confidence matching is also unsuccessful, a round of supplementary matching process is implemented. This process attempts to reassociate the unpaired target with the trajectory that was considered lost in the previous record through the Hungarian Algorithm, so as to effectively recover the trajectory of the target object that is temporarily out of sight due to long-term occlusion.

[0105] Example 3

[0106] In order to verify the actual effect of the pig multi-target tracking method and specific application proposed in the above embodiment, this embodiment conducts a verification experiment. The details are as follows:

[0107] (1) Experimental environment:

[0108] The processor used is Intel(R) Core(TM) i5-12490F CPU; 32.0GB memory; an NVIDIA GeForceRTX 4060Ti GPU; Python-3.8, PyTorch-2.1.0, CUDA-12.5. YOLO is used as the basic detection framework, and yolov8n is used to train the detection model. The training parameters of HBB and OBB are kept consistent, and a total of 200 training iterations are performed.

[0109] (2) Model evaluation indicators:

[0110] The current mainstream multi-target tracking evaluation indicators are adopted, including higher order tracking accuracy (HOTA), multiple object tracking accuracy (MOTA), IDF1 score (identification F1 score) and identification switch number (IDS). Among them, HOTA comprehensively measures the accuracy of detection, association tracking and positioning, and comprehensively reflects the accuracy of the multi-target tracker. MOTA evaluates the accuracy and stability of the tracker in target detection and trajectory tracking by considering missed detection, false detection and identity switch number. The IDF1 score is the harmonic mean of ID accuracy and ID recall, focusing on reflecting the duration and trajectory integrity of the tracked target. In addition to the multi-target tracking evaluation indicators, three auxiliary indicators are also used, namely the total number of false detections (FP), the total number of missed detections (FN) and the frame rate (FPS). Among these indicators, the higher the values ​​of HOTA, MOTA, IDF1 and FPS, and the lower the values ​​of FP, FN, ML and IDSW, the better the performance of the multi-target tracking algorithm.

[0111] (3) Comparative analysis of the impact of different annotation methods on tracking performance:

[0112] In order to verify the impact of different annotation methods on the tracking performance of ByteTrack in complex scenes, the tracking algorithms with OBB and HBB annotations are defined as OBB-ByteTrack and HBB-ByteTrack, respectively, and their performances are compared under the same parameter settings.

[0113] Performance comparison in daytime scenes:

[0114] Videos 05 and 06 with fast motion and occlusion were selected for tracking experiments to test the tracking performance of OBB-ByteTrack and HBB-ByteTrack in daytime and under different pig densities. The results are shown in Table 2. As shown in Table 2, compared with HBB-ByteTrack, the tracking effect of OBB-ByteTrack in video 05 decreased by 5.7% on HOTA and 7.9% on IDF1, and the processing speed was faster and the overall tracking performance was better. In video 06 with a higher density, OBB-ByteTrack reached 66.00% on HOTA, which was 9.3% higher than HBB-ByteTrack; MOTA was 94.00%, which was similar to HBB-ByteTrack; IDF1 increased by 8.6%; IDS and FP both decreased. Although the processing speed of OBB-ByteTrack is not as fast as HBB-ByteTrack, the OBB annotation method is more effective in scenarios with a large number of pigs and more complex situations.

[0115] Table 2: Performance comparison of OBB-ByteTrack and HBB-ByteTrack tracking algorithms in daytime scenarios.

[0116]

[0117] The above analysis shows that when there are fewer tracking targets, HBB-ByteTrack still has certain advantages, but when there are more tracking targets, OBB-ByteTrack performs better, especially when the targets are tilted or overlapped with each other. This is because the OBB annotation method can fit the target contour more closely, reduce the interference of the background area, and provide more accurate location information.

[0118] To further compare the difference in tracking effects between the two annotation methods, we take Video 06 as an example to visualize the two. Figure 6-7 shown. Figure 6 and Figure 7 From left to right in the figure are the 864th frame, the 886th frame and the 901th frame. In the 864th frame, id11 and id5 in HBB-ByteTrack correspond to id5 and id6 in OBB-ByteTrack respectively. In the 886th frame, due to the rapid approach of the two pigs, the detection frames overlapped, resulting in the wrong identification of the pigs. Therefore, when the pigs were re-detected in the 901th frame, id11 of HBB was considered as a new target, resulting in the original id11 not being accurately tracked. In OBB-ByteTrack, both pigs can be accurately tracked and no id switching occurs.

[0119] Performance comparison in night scenes:

[0120] like Figure 8-9 As shown in the figure, compared with the daytime scene, the nighttime scenes 07 and 08 not only need to deal with the challenges brought by occlusion and motion, but also face the problem that the pig's body parts exceed the video boundary. Figure 8 It can be seen that in video 08, as the pigs continued to move, the target finally disappeared completely. These factors further increase the complexity of target tracking in night scenes. Fig. 9 Video 07 shows an example of eating behavior tracked using two different annotation methods.

[0121] The tracking experiment results in night scenes shown in Table 3 show that OBB-ByteTrack is also effective, especially in video 07, where OBB-ByteTrack's HOTA increased by 23.6%, IDF1 increased by 18.5%, and IDS decreased by 55 compared to HBB-ByteTrack. By observing the tracking video, it is found that there are a lot of eating behaviors in video 07 ( Fig. 9 ), HBB-ByteTrack is more prone to misidentification and ID switching in extremely crowded scenes, while OBB annotation has a smaller circumscribed rectangular area, so the probability of misidentification and ID switching is significantly lower than that of HBB annotation. In Video 08, the HOTA, MOTA and IDF1 of OBB-ByteTrack are 78.50%, 97.40% and 72.60%, respectively, which are 15.8, 1.3 and 3.1 percentage points higher than those of HBB-ByteTrack, but the IDS and FP are reduced by 25 and 58, respectively. In summary, compared with HBB-ByteTrack, OBB-ByteTrack has better accuracy and stability when tracking high-density pigs, especially in the face of target overlap, occlusion and rapid movement, it can better maintain continuous tracking of the target and reduce the occurrence of ID switching and misidentification. In addition, the OBB annotation method effectively reduces the misidentification rate in complex scenes and improves the overall tracking performance by virtue of its tighter fit to the target contour, making it more advantageous in practical applications.

[0122] Table 3: Tracking results of HBB-ByteTrack and OBB-ByteTrack in night scenes.

[0123]

[0124] Performance comparison of the method in this embodiment and other MOT methods:

[0125] To verify the effectiveness of this method, it is compared with two traditional tracking algorithms BootSORT and DeepSORT. Both algorithms use YOLOv8n as the basic detection model and use OBB and HBB methods for tracking, respectively. The results are shown in Table 4. As shown in Table 4, compared with BootSORT and DeepSORT, the HOTA of this method is improved by 0.8% and 45.8% on the 05 video, 5.9% and 46.5% on the 06 video, 1.9% and 28.3% on the 07 video, and 3.1% and 28.3% on the 08 video; IDF1 is improved by 0.2%, 5.5%, 1.3%, and 3.1% respectively compared with BootSORT, and by 41.4%, 41.2%, 23.8%, and 25.2% respectively compared with DeepSORT; IDS, FP, FN, and FPS are also better than BootSORT and DeepSORT. Therefore, this method shows significant advantages in key indicators HOTA and IDF1, especially in complex scenes, where tracking accuracy and robustness are significantly improved. In addition, this method reduces the number of misidentifications, ID switching, and missed detections while maintaining a high processing speed, proving its effectiveness and practicality in high-density pig tracking tasks.

[0126] Table 4: Comparison of tracking results between this method and other MOTs in various scenarios.

[0127]

[0128] To further analyze the performance of this method in actual scenarios, taking Video 06 as an example, the tracking results of the algorithms in Table 4 in various complex scenarios are visualized. Figure 10-12 As shown, Figures 10 to 12 From left to right are our(Video06), BootSORT(OBBVideo 06), DeepSORT(HBBVideo 06), and Fig.10 For frame 1664, Fig.11 For frame 1728, Fig.12The 1744th frame. When DeepSORT was used for tracking, the pig with ID 7 was correctly identified and tracked at frame 1664. However, as other pigs moved and approached, interference gradually occurred at frame 1728, which eventually caused the pig with ID 7 to switch its identity at frame 1744. Similarly, the same situation occurred when the BootSORT algorithm was used for tracking. At frame 1744, the pig with ID 7 was mistakenly switched to 25. In contrast, the UKFTrack algorithm did not have the problem of ID switching at frame 1744, which shows that the multi-stage matching mechanism can achieve a more stable tracking effect.

[0129] The visual tracking effect of this method at night is as follows Figure 13-15 As shown, Figures 13 to 15 From left to right are our(Video 08), BootSORT(OBB Video 08), DeepSORT(HBB Video 08), and Fig.13 For frame 320, Fig.14 For frame 448, Fig.15 The 480th frame. In Video 08, the three tracking methods can achieve accurate tracking at the 320th frame, but at the 448th frame, due to the mutual occlusion between pigs, the detector cannot accurately detect the occluded individual pigs. Therefore, when the occluded pigs are re-detected at the 480th frame, the DeepSORT and BootSORT algorithms have ID switching, while the present method still maintains the correct ID. This result shows that even in low-light environments at night, the present method can still maintain high stability and robustness.

[0130] Figure 16-Figure 18 The tracking results of the three algorithms in the last frame in four different scenarios are given. Figures 16 to 18 From left to right, they are Video 05, Video 06, Video 07, Video 08, and Fig.16 For our Fig.17 For botsort (OBB), Fig.18is DeepSORT(HBB). The maximum tracking IDs of this method in each scene are 14, 41, 27 and 31 respectively; the maximum tracking IDs of BotSort(OBB) are 13, 48, 30 and 48 respectively; and the maximum tracking IDs of DeepSORT(HBB) are 55, 119, 55 and 119. Compared with BotSort(OBB), the difference of this method in low-density scenes is smaller, increasing the number of IDs by 1 and reducing the number of IDs by 3, respectively, but in high-density scenes, the number of IDs is reduced by 7 and 17, respectively. Compared with DeepSORT(HBB), the number of IDs is reduced by 41, 78, 28 and 88, respectively. This shows that in high-density scenes, this method can more effectively retrieve lost tracks, reduce ID switching, and achieve more accurate pig target tracking.

[0131] Aiming at the problem of low accuracy of multi-target tracking of pigs in complex breeding environments, this method introduces an improved unscented Kalman filter (UKF) and a multi-stage matching mechanism, and proposes a new multi-target tracking method to solve the tracking challenges caused by the rapid movement and frequent occlusion of pigs.

[0132] First, this method constructs a challenging OBB annotated dataset. This dataset covers complex scenes such as high-density farming, severe occlusion, high-speed movement, sudden turns, and nighttime feeding. It is not only advantageous in quantity, but also far exceeds existing public datasets in terms of quality, time length, and diversity. It can better meet the data needs of multi-target tracking of pigs in complex scenes and provide strong data support for subsequent research. Second, a multi-target tracking algorithm for pigs in complex scenes is proposed. By introducing an improved unscented Kalman filter algorithm, the OBB tracking model is better adapted, and the state vector and residual function are optimized to improve the prediction accuracy and robustness of the model. In addition, an additional matching mechanism is added to enable the algorithm to achieve continuous tracking of the target even in the case of severe occlusion.

[0133] The actual experimental results show that in four complex scenarios of pigs being heavily dense during the day, extremely dense during the day, heavily dense at night, and extremely dense at night, the HOTA of this method reaches 96.10%, 83.10%, 76.50%, and 84.00%, respectively, which are 0.6, 10.4, 7.7, 9.5 and 45.8, 46.6, 28.3, and 28.3 percentage points higher than BootSORT and DeepSORT, respectively; IDF1 reaches 95.70%, 78.20%, 70.10%, and 77.60%, respectively, which are 0.2, 5.5, 1.3, 3.1 and 41.4, 41.2, 23.8, and 25.2 percentage points higher than BootSORT and DeepSORT, respectively. Therefore, for multi-target tracking of pigs in dense breeding and diversified motion scenes, a reliable, stable, robust and efficient solution is provided. In the future, with the further expansion of the data set and the improvement of the annotation method, this method is expected to provide more comprehensive technical support for smart farming applications such as health monitoring and behavior analysis of pigs, and has high application prospects.

[0134] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.

[0135] In addition, the operations of the process described in this embodiment may be performed in any suitable order, unless otherwise indicated in this embodiment or otherwise clearly contradicted by the context. The process described in this embodiment (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as a code (e.g., executable instructions, one or more computer programs, or one or more applications) executed jointly on one or more processors, by hardware or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0136] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or part thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described in this embodiment includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques described in the present invention, the present invention also includes the computer itself. The computer program can be applied to input data to perform the functions described in this embodiment, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on a display.

[0137] As used in this application, the terms "component", "module", "system", etc. are intended to refer to a computer-related entity, which can be hardware, firmware, a combination of hardware and software, software, or software in operation. For example, a component can be, but is not limited to: a process running on a processor, a processor, an object, an executable file, a thread in execution, a program, and / or a computer. As an example, an application running on a computing device and the computing device can both be components. One or more components can exist in a process and / or thread in execution, and a component can be located in a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures thereon. These components can communicate in a local and / or remote process manner, such as according to a signal having one or more data packets (e.g., data from a component that interacts with another component in a local system, a distributed system, and / or interacts with other systems in a signal manner through a network such as the Internet).

[0138] It should also be understood that the present invention is described through implementation methods, and the embodiments are only clear and complete descriptions of the technical solutions proposed in the claims of the present invention, that is, explanations of the claims. Therefore, when judging whether the technical solutions recorded in the specification of the present invention are sufficiently disclosed, full consideration should be given to the core essence of the solutions defined by the claims. In the specification, there must be other technical problems that are not related to the core technical problems proposed in this embodiment. The corresponding technical features and technical solutions do not belong to the essence of this embodiment and are non-essential technical features. Therefore, they can be referred to as implicit disclosures. Those skilled in the art can fully implement them in combination with existing technologies and common knowledge. Therefore, there is no need to describe them in detail.

[0139] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for tracking multiple targets of live pigs, characterized by: The following steps are included: Building a diverse dataset with oriented bounding box annotations; Use the YOLOv8n model to detect pig targets and output the bounding box coordinates, width, height, confidence, and category of each target; Perform confidence classification on the input video sequence, including high confidence and low confidence; Using the improved unscented Kalman filter output to predict the target trajectory and perform an association match with the high-confidence target; The successfully matched targets and trajectories are updated, and the unmatched trajectories are matched with the low-confidence targets for a second time. Meanwhile, the detected targets that were not matched successfully twice and the trajectories that were considered lost in the previous records are re-associated using a supplementary matching strategy; In the supplementary matching strategy, a new track is formed after a successful match, and the track that has not been successfully matched is marked as a lost track, so as to keep tracking the multiple targets of the live pigs.

2. The pig multi-target tracking method according to claim 1, characterized in that: The improvement of the unscented Kalman filter is to expand the state vector, including adding angle and angular velocity parameters, and adding a residual function to process angle variables.

3. The pig multi-target tracking method according to claim 2 is characterized in that: The unscented Kalman filter approximates the probability distribution of the state vector through a set of samples called sigma points and propagates uncertainty. The construction formula of the sigma point is as follows: In the formula, represents the ith sigma point, represents the mean estimate of the state vector at time K, P k It represents the covariance matrix of the state vector at time k, reflecting the uncertainty of state estimation, n is the dimension of the state vector, and λ is an adjustment parameter.

4. The pig multi-target tracking method according to claim 2 is characterized in that: The added angle and angular velocity parameters include: The Kalman filter state vector is defined as In the formula, x and y represent the corresponding center point coordinates, a represents the aspect ratio, and h represents the height. and represents the x and y components of velocity, and They represent the rate of change of aspect ratio and height respectively; Increase the rotation angle θ and angular velocity ω and redefine it as 5. The pig multi-target tracking method according to claim 2 is characterized in that: The adding of the residual function to process the angle variable includes: A custom residual function is introduced to optimize the processing method of angle variables and improve the accuracy and stability of the unscented Kalman filter in application scenarios containing angle value state vectors. The angle value residual is as follows: y[i]=(a[i]-b[i]+π)%(2π)-π In the formula, for i targets, the angle difference is first calculated, then π is added to the difference and modulo 2π to ensure that the result is within [0,2π], and finally π is subtracted to translate the result to [-π,π] to ensure the continuity and correctness of the residual angle.

6. The pig multi-target tracking method according to claim 1, characterized in that: The unscented Kalman filter introduces additional constraints on the generation of trajectories, adds a time series to store the number of recently detected targets, and calculates the average value of the detected targets within a certain number of frames to avoid the increase of IDs caused by false detections in some frames, as shown in the following formula: Where Ad represents the average number of detected targets, Td represents the total number of targets detected from the current frame to the previous t-th frame, and Tf represents the total number of frames from the current frame to the previous t-th frame.

7. The method for tracking multiple live pig targets according to claim 6, characterized in that: The input video frame rate is 16 frames per second, and Tf is set to 160, which means that in the past 10 seconds, only when the average number of detected targets Ad is greater than the number of currently accurately tracked trajectories.

8. The pig multi-target tracking method according to claim 1, characterized in that: The YOLOv8n model performs pig target detection, including a backbone network, a feature fusion layer, and an output head; The backbone network is responsible for feature extraction, using Darknet with CSP structure to process image features, gradually reducing the resolution and increasing the depth of feature maps; In the feature fusion layer, feature fusion is achieved through PANet and features at different levels are integrated using bottom-up and top-down paths to ensure the effective combination of high-resolution and low-resolution features; The output head generates a detection box and category probability prediction based on the fused feature map, and outputs the bounding box coordinates, width and height, confidence level, and category of each target.

9. An application of a multi-target tracking method for pigs, characterized in that: It includes the multi-target tracking method for live pigs as described in claims 1-8, which is applied to multi-target tracking of live pigs.

10. Application of the pig multi-target tracking method according to claim 9, characterized in that: In four complex scenarios of heavy density during the day, extremely dense during the day, heavy density at night, and extremely dense at night, the high-order tracking accuracies of the multi-target tracking method for live pigs were 96.10%, 83.10%, 76.50% and 84.00% respectively, and the IDF1 scores were 95.7%, 78.2%, 70.1% and 77.6% respectively.

Images