A method and related equipment for judging the health status of pigs based on multi-target tracking

Abstract

This invention discloses a method and related equipment for judging the health status of pigs based on multi-target tracking. The method includes: splitting the collected video data of the pigs to be detected into video sequences of preset durations and performing preprocessing operations; inputting the preprocessed video sequences into a pre-trained, frame-by-frame target detection data model; tracking the pig ID number and the location information of the pigs to be detected based on the detected data input frame Deep; calculating five types of motion indicators for each pig to be detected to form five first data matrices; performing dimensionality reduction processing on the first data matrices and inputting the resulting second data matrices into an anomaly detection model to output the abnormal health status of the pigs to be detected, thereby using the five types of motion indicators and other pig motion information obtained by multi-target tracking technology to detect abnormal pig health.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and in particular to a method and related equipment for judging the health status of pigs based on multi-target tracking. Background Technology

[0002] In the past, large-scale pig farms relied heavily on manual observation of pig behavior and regular temperature measurements to assess pig health. This method suffers from poor real-time performance, high subjectivity, and a high false alarm rate. However, with the continuous development of IoT and machine learning technologies, more and more large-scale pig farms are adopting smart hardware to monitor various bodily data of pigs in real time. Compared to contact-based sensor monitoring methods like RFID (Radio Frequency Identification) ear tags, which use computer vision technology to process pig image data acquired by cameras to achieve real-time monitoring of pig activity and behavior, this non-contact method has less impact on pig habits and is also more cost-effective.

[0003] Currently, deep learning technology has achieved good results in multi-target tracking of individual pigs. However, the above work does not mention how to use the pig movement information obtained by multi-target tracking technology to detect the health status of pigs, or it simply selects a threshold for pig activity to judge the health status of pigs. That is, after deriving the movement distance of pigs, the threshold for pig movement distance is manually selected based on literature and the breeding experience of farmers, and the health of pigs is rated and scored. This method not only has huge errors when the pig breeds, ages, breeding areas, or breeding seasons are different, but also, since there are very few relevant datasets of sick pigs, supervised learning techniques cannot be applied to the field of pig health status detection.

[0004] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0005] The main objective of this invention is to provide a method and related equipment for judging the health status of pigs based on multi-target tracking, aiming to solve the problem that existing technologies cannot detect abnormal health conditions in pigs by obtaining pig movement information through multi-target tracking technology.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for judging the health status of pigs based on multi-objective tracking, the method comprising the following steps:

[0008] The collected video data of the pigs to be detected is divided into video sequences of preset duration, and the video sequences are preprocessed using median filtering algorithm and video mask operation;

[0009] The preprocessed video sequence is input into a pre-trained object detection model, which outputs detection data; wherein, the detection data includes: object type, object location, and confidence data;

[0010] The detection data is input into the DeepSORT tracking model. The Kalman filter, appearance feature extraction model and Hungarian matching algorithm in the DeepSORT tracking model are used to perform frame-by-frame data association operations. The frame number, the ID number of the pig to be detected and the location information of the pig to be detected with the corresponding ID number are output in each frame of detection data.

[0011] Based on the frame number, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number, five types of motion indicators are calculated for each pig to be detected, and five first data matrices are formed using the motion indicators; wherein, the motion indicators include: movement distance, maximum movement speed, maximum movement acceleration, average movement speed, and average movement acceleration;

[0012] Principal component analysis is used to reduce the dimensionality of the first data matrix to form a second data matrix. The second data matrix is ​​then input into the anomaly detection model to output the abnormal health status of the pigs to be detected.

[0013] In the method for judging the health status of pigs based on multi-target tracking, the step of splitting the collected video data of the pigs to be detected into video sequences of preset durations and preprocessing the video sequences using a median filtering algorithm and video masking operations includes:

[0014] Multiple RGB video data segments were collected from multiple pigs to be inspected in a single pen using a camera.

[0015] The RGB video data is split into video sequences of preset duration, and the video sequences are preprocessed using the median filtering algorithm and the video mask operation.

[0016] In the method for judging the health status of pigs based on multi-target tracking, the step of inputting the preprocessed video sequence into the pre-trained target detection model and outputting detection data specifically includes:

[0017] The pre-processed video sequence is input into the initial object detection model, and the model parameters are adjusted until the error value is lower than the set threshold to obtain the pre-trained object detection model.

[0018] The enhanced video sequence is input into a pre-trained target detection model, which outputs the detection data.

[0019] The method for judging the health status of pigs based on multi-target tracking includes the following steps: inputting the detection data into the DeepSORT tracking model, using the Kalman filter, appearance feature extraction model, and Hungarian matching algorithm in the DeepSORT tracking model to perform frame-by-frame data association, and outputting the frame number, the ID number of the pig to be detected, and the location information of the pig with the corresponding ID number in each frame of detection data.

[0020] Create a corresponding tracking box for the initial frame detection data, initialize the motion variables of the Kalman filter, predict the tracking box of the next frame detection data using the Kalman filter, and then calculate the Mahalanobis distance between the target detection box and the tracking box of the next frame detection data.

[0021] The RGB image in the target detection box is input into a pre-trained appearance feature extraction model to calculate the cosine distance. The cost matrix is ​​then calculated based on the cosine distance, the Mahalanobis distance, and a preset threshold matrix.

[0022] The cost matrix is ​​input into the Hungarian algorithm to match the tracking box predicted by the Kalman filter in the previous frame with the target detection box in the next frame, and to obtain the first tracking box that is successfully matched, the first detection box that is not matched, and the second tracking box that is not matched.

[0023] The position information of the first tracking box that is successfully matched is input into the Kalman filter. The IOU thresholds of the first detection box and the second tracking box that are not matched are calculated respectively. The Hungarian algorithm is used to match them to obtain the third tracking box that is successfully matched, the second detection box that is not matched, and the fourth tracking box that is not matched.

[0024] The third tracking box that matches successfully, the second detection box that fails to match, and the fourth tracking box that fails to match are processed accordingly.

[0025] The mean and covariance matrix are updated using a Kalman filter, and after determining whether the tracker's state is deterministic, the frame number, the ID number of the pig to be detected, and the location information of the pig with the corresponding ID number are output for each frame of detection data.

[0026] The method for judging the health status of pigs based on multi-target tracking, specifically includes the following steps: calculating five types of motion indicators for each pig to be detected based on the frame number, the ID number of the pig to be detected, and the location information of the pig with the corresponding ID number, and using the motion indicators to form five first data matrices.

[0027] Based on the frame number, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number, calculate five types of motion indicators for each pig to be detected;

[0028] After extracting the same class of motion indicators from N pigs to be tested to form five types of first motion indicator feature vectors, the five types of first motion indicator feature vectors are used to form five first data matrices; the length of each first data matrix is ​​N.

[0029] In the method for judging the health status of pigs based on multi-target tracking, the step of using principal component analysis to reduce the dimensionality of the first data matrix to form a second data matrix, and inputting the second data matrix into the anomaly detection model to output the abnormal health status of the pig to be detected specifically includes:

[0030] Calculate the mean of all first motion index feature vectors for each class, and calculate the difference between each first motion index feature vector in the same class and the mean of the first motion index feature vectors in the class.

[0031] After forming five types of second motion index feature vectors from the five types of differences, five third data matrices are formed using the five types of second motion index feature vectors; wherein, the length of each third data matrix is ​​N.

[0032] Solve for the covariance matrix of the third data matrix, and use the eigenvalue decomposition method to calculate the eigenvalues ​​and eigenvectors of the covariance matrix; wherein, the eigenvectors are the new coordinate axis direction, the rotation direction of the data, or the new principal component direction; the eigenvalues ​​are the variance of the projection of the detected data onto the corresponding new coordinate axis, or the amount of information contained in the eigenvector corresponding to the detected data;

[0033] The eigenvalues ​​are sorted from largest to smallest. The eigenvectors of the two largest eigenvalues ​​after sorting are used to form the fourth data matrix. The first data matrix and the fourth data matrix are then multiplied to obtain the second data matrix.

[0034] The second data matrix is ​​split to obtain a dataset. A dataset with a preset ratio is used as a training dataset and input into the trained anomaly detection model for prediction, and the abnormal health status of the pig to be detected is output.

[0035] The method for judging the health status of pigs based on multi-target tracking specifically includes the following steps: processing the successfully matched third tracking box, the unmatched second detection box, and the unmatched fourth tracking box accordingly.

[0036] The position information of the third tracking box that is successfully matched is input into the Kalman filter, and a new tracking box is added for the second detection box that fails to match.

[0037] If the fourth tracking box does not match the target detection box of any other frame detection data, the fourth tracking box enters an uncertain state;

[0038] If the fourth tracking box fails to match the current detection box in three consecutive frames of detection data and the maximum waiting time is exceeded, then the fourth tracking box is deleted.

[0039] If the fourth tracking box has successfully matched the target detection box of any other frame detection data, then the fourth tracking box enters the deterministic state.

[0040] In the method for judging the health status of pigs based on multi-target tracking, the steps of updating the mean and covariance matrix using a Kalman filter, determining whether the tracker's state is deterministic, and outputting the frame number, the ID number of the pig to be detected, and the location information of the pig with the corresponding ID number in each frame of detection data specifically include:

[0041] Update the mean and covariance matrix based on the location information of the first and third successfully matched tracking boxes;

[0042] If the fourth tracking box, which has entered an uncertain state, matches a newly added tracking box within three consecutive frames of detection data, then the fourth tracking box in the uncertain state becomes a definite state.

[0043] Once all detection data has been correlated, the frame number, the ID number of the pig to be detected, and the location information of the corresponding ID number of the pig to be detected in each frame of detection data are output and saved in a text format containing line-by-line organization.

[0044] A system for judging the health status of pigs based on multi-objective tracking, the system comprising:

[0045] The splitting and preprocessing module is used to split the collected video data about the pigs to be detected into video sequences of preset durations, and to preprocess the video sequences using a median filtering algorithm and video masking operations.

[0046] The detection data output module is used to input the preprocessed video sequence into the pre-trained target detection model and output detection data; wherein, the detection data includes: object type, object position and confidence data;

[0047] The data association module is used to input the detection data into the DeepSORT tracking model, and use the Kalman filter, appearance feature extraction model and Hungarian matching algorithm in the DeepSORT tracking model to perform frame-by-frame data association operation, and output the frame number, the ID number of the pig to be detected and the location information of the pig to be detected with the corresponding ID number in each frame of detection data.

[0048] The motion index calculation module is used to calculate five types of motion indexes for each pig to be detected based on the frame number, the pig ID number to be detected, and the location information of the pig to be detected with the corresponding ID number, and to use the motion indexes to form five first data matrices; wherein, the motion indexes include: movement distance, maximum movement speed, maximum movement acceleration, average movement speed, and average movement acceleration;

[0049] The abnormal health status output module is used to perform dimensionality reduction processing on the first data matrix using the principal component analysis method to form a second data matrix, and input the second data matrix into the abnormal detection model to output the abnormal health status of the pig to be detected.

[0050] A computer-readable storage medium stores a pig health status assessment program based on multi-target tracking, wherein when executed by a processor, the pig health status assessment program based on multi-target tracking implements the steps of the pig health status assessment method based on multi-target tracking as described above.

[0051] Compared to existing technologies, this invention provides a method and related equipment for judging the health status of pigs based on multi-target tracking. The method includes: splitting the collected video data of the pigs to be detected into video sequences of preset durations and performing preprocessing operations; inputting the preprocessed video sequences into a pre-trained target detection model; inputting the obtained detection data into a DeepSORT tracking model and performing frame-by-frame data association operations; calculating five types of motion indicators for each pig to be detected based on the frame number, the ID number of the pig to be detected, and the position information of the corresponding ID number in each frame of output detection data, and forming five first data matrices; performing dimensionality reduction processing on the first data matrices, and inputting the obtained second data matrices into an anomaly detection model to output the abnormal health status of the pigs to be detected, thereby using the five types of motion indicators and other pig motion information obtained by multi-target tracking technology to detect abnormal health status of pigs. Attached Figure Description

[0052] Figure 1 A flowchart illustrating a preferred embodiment of the method for judging the health status of pigs based on multi-target tracking provided by the present invention;

[0053] Figure 2 The flowchart shows the overall process of the method for judging the health status of pigs based on multi-target tracking provided by the present invention.

[0054] Figure 3 The effect diagram of video masking provided by the present invention;

[0055] Figure 4 The flowchart of step S100 is shown in a preferred embodiment of the method for judging the health status of pigs based on multi-target tracking provided by the present invention.

[0056] Figure 5 A schematic diagram illustrating the training of the target detection model provided by this invention;

[0057] Figure 6 This invention provides a display result of annotation on different 6 frames of data from a video of a pig to be detected.

[0058] Figure 7 The image shows the effect of a second annotation on the original image dataset for target detection after 90% data augmentation, as provided by this invention.

[0059] Figure 8 The flowchart of step S200 is provided in a preferred embodiment of the method for judging the health status of pigs based on multi-target tracking provided by the present invention.

[0060] Figure 9 This is a schematic diagram of the specific structure of the DeepSORT tracking model provided by the present invention;

[0061] Figure 10 The flowchart for the calculation of appearance feature model and data association provided by the present invention;

[0062] Figure 11 A graph showing the decrease in training loss of the trained appearance feature extraction model provided by this invention.

[0063] Figure 12 This is a flowchart of step S300 in a preferred embodiment of the pig health status judgment method based on multi-target tracking provided by the present invention.

[0064] Figure 13 This is a schematic diagram of the target tracking model training process provided by the present invention;

[0065] Figure 14 The flowchart of step S350 is provided in a preferred embodiment of the method for judging the health status of pigs based on multi-target tracking provided by the present invention.

[0066] Figure 15 The flowchart of step S360 is shown in a preferred embodiment of the pig health status judgment method based on multi-target tracking provided by the present invention.

[0067] Figure 16 The flowchart of step S400 is provided in a preferred embodiment of the method for judging the health status of pigs based on multi-target tracking provided by the present invention.

[0068] Figure 17 The present invention provides a motion trajectory diagram of 8 pigs to be tested;

[0069] Figure 18 The image shows the tracking effect of the movement trajectories of eight pigs to be detected, as provided by this invention.

[0070] Figure 19 A comparative histogram of the movement distances of eight pigs to be tested, provided by this invention;

[0071] Figure 20 The present invention provides a line graph of the movement speed of the pig to be tested, with ID number 7.

[0072] Figure 21 The present invention provides a line graph of the motion acceleration of the pig to be tested, with ID number 7.

[0073] Figure 22 A schematic diagram illustrating the average motion velocity of a single pig to be detected in a video frame, as provided by the present invention.

[0074] Figure 23 A schematic diagram illustrating the average acceleration of a single pig to be detected in a video frame, provided by the present invention.

[0075] Figure 24 A schematic diagram illustrating the maximum acceleration of a single pig to be detected in a video frame, as provided by the present invention.

[0076] Figure 25 A schematic diagram illustrating the maximum speed of a single pig to be detected in a video frame, as provided by the present invention.

[0077] Figure 26 A flowchart of the motion data extraction section provided by this invention;

[0078] Figure 27 The flowchart of step S500 is provided in a preferred embodiment of the method for judging the health status of pigs based on multi-target tracking provided by the present invention.

[0079] Figure 28 A two-dimensional scatter plot of motion features after dimensionality reduction and recombination using the PCA method, provided by this invention;

[0080] Figure 29A flowchart illustrating the training process of the PAC data dimensionality reduction and anomaly detection model provided by this invention;

[0081] Figure 30 This is a schematic diagram of the singularity detection results provided by the present invention;

[0082] Figure 31 A schematic diagram of the confusion matrix for the prediction results provided by this invention;

[0083] Figure 32 This is an architecture diagram of the pig health status assessment system based on multi-target tracking provided by the present invention. Detailed Implementation

[0084] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0085] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0086] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0087] When pigs are infected with infectious diseases such as African swine fever, they often exhibit reduced activity, huddling together, decreased appetite, and reddening of the skin. This invention proposes a method for judging the health status of pigs based on multi-target tracking. This method combines pig movement information and appearance features to determine abnormal pig health. First, an improved DeepSORT tracking algorithm is used to extract the center point coordinates of each pig in each frame. After removing abnormal points caused by DeepSORT tracking algorithm jitter, the motion between adjacent frames is approximated as uniform linear motion. The pig's movement distance, instantaneous velocity, instantaneous acceleration, average velocity, and average acceleration are calculated, and principal component analysis is used to reduce the data dimensionality. Simultaneously, the RGB image of each individual pig in each frame is extracted, and a 6-layer convolutional neural network is used to extract features, resulting in a 6-dimensional feature vector. Finally, the individual pig appearance feature vector and movement data are normalized and input into a trained abnormality network model to determine whether the pig's health is abnormal.

[0088] This invention provides a method and related equipment for judging the health status of pigs based on multi-target tracking. In this invention, video data of the pigs to be detected is first split into video sequences of preset durations for preprocessing, and then input into a DeepSORT tracking model for frame-by-frame data association. Then, based on the frame number, the ID number of the pig to be detected, and the location information of the corresponding ID number in each frame of the output detection data, five types of motion indicators are calculated for each pig to be detected. Finally, five first data matrices are formed using these motion indicators, and dimensionality reduction is performed. The resulting second data matrix is ​​then input into an anomaly detection model, which outputs the abnormal health status of the pig to be detected. This achieves the detection of abnormal pig health based on pig motion information obtained through multi-target tracking technology.

[0089] The following describes the design scheme of the method for judging the health status of pigs based on multi-target tracking through specific exemplary embodiments. It should be noted that the following embodiments are only used to explain the technical solution of the invention and are not intended to limit it specifically:

[0090] Please see Figure 1 This invention provides a method for judging the health status of pigs under test based on multi-target tracking. The method includes the following steps:

[0091] S100. The collected video data of the pigs to be detected is split into video sequences of preset duration, and the video sequences are preprocessed using median filtering algorithm and video mask operation.

[0092] Specifically, the entire method for judging the health status of pigs based on multi-target tracking in this invention can be divided into three main parts: first, target detection; second, target tracking; and third, anomaly detection. The overall flowchart of the method for judging the health status of pigs based on multi-target tracking is shown below. Figure 2 As shown.

[0093] The target detection part includes a target detection model training part and a target detection model prediction part. Before making predictions using the target detection model, it is necessary to obtain prediction input data and perform preprocessing operations on the input data, specifically:

[0094] When examining the physiological health of the pigs to be tested, it is first necessary to collect relevant video data of the pigs to be tested and split it into video sequences of preset duration. Then, preprocessing operations such as median filtering algorithm and video mask operation are performed on the video sequences to facilitate the processing of the video sequences.

[0095] The basic principle of median filtering is to replace the value of a point in a digital image or digital sequence with the median value of all points in its neighborhood, making the surrounding pixel values ​​closer to the true value, thereby eliminating isolated noise points. The purpose of the median filtering algorithm is to reduce the impact of image noise on the detection and tracking of the target pig. It replaces the value of a point in a digital image or digital sequence with the median value of all points in its neighborhood, making the surrounding pixel values ​​closer to the true value, thereby eliminating isolated noise points. It filters out random noise in the image while preserving the image edge features as much as possible. In this embodiment, the kernel size is selected as 5.

[0096] The video masking operation involves manually marking the area outside the pigpen to be monitored and replacing it entirely with black to isolate the area of ​​the pigpen to be monitored. This prevents pigs in other pens from interfering with multi-target tracking. Its main function is to preserve the pigpen to be observed while masking other pens, preventing non-monitored pigs in other pens from affecting the monitored pig. The effect of the video masking operation is shown in the image below. Figure 3 As shown in the figure, this embodiment uses 8 pigs to be tested as the target.

[0097] Furthermore, please refer to Figure 4 The step S100, which involves splitting the collected video data of the pigs to be detected into video sequences of preset durations and preprocessing the video sequences using a median filtering algorithm and video masking operations, includes:

[0098] S110. Use a camera to collect multiple RGB video data segments of multiple pigs to be inspected in a pen;

[0099] S120. The RGB video data is split into video sequences of preset duration, and the video sequences are preprocessed using the median filtering algorithm and the video mask operation.

[0100] Specifically, when examining the physiological health of the pigs to be tested, the first step is to collect video data about the pigs: using a USB monocular camera, multiple RGB video segments are captured from multiple pigs in one pen, and the videos are saved in AVI format with a resolution of 1280×720. Here, the USB camera is installed at a height of H meters above the ground, with a maximum field of view of w*L meters. The camera is almost perpendicular to the ground, and the captured images perfectly cover the entire pen of pigs to be tested.

[0101] Then, the RGB video data is split into video sequences of a preset duration (e.g., 5 minutes), and the video sequences are preprocessed using the median filtering algorithm and the video masking operation.

[0102] For further information, please refer to [link / reference]. Figure 1 S200, Input the preprocessed video sequence into the pre-trained target detection model and output detection data; wherein, the detection data includes: object type, object position and confidence data.

[0103] Specifically, in the object detection model training section of the object detection part, before pre-training the object detection model, it is necessary to obtain the dataset for the object detection model in order to perform pre-training. The steps are as follows, and the training diagram of the object detection model is shown in the figure. Figure 5 As shown:

[0104] After acquiring multiple RGB video data segments, DarkLabel software was used to perform the first annotation of the RGB video data. DarkLabel is a lightweight software for annotating multi-object tracking datasets. To ensure the accuracy of the tracking dataset, the pigs to be detected in each frame can be manually annotated, such as... Figure 6 This document presents the annotation results for a video of pig data to be detected, displayed at 30 frames, 60 frames, 160 frames, 220 frames, 300 frames, and 500 frames. The annotation file is then exported in MOT16 format. A total of 20 videos were annotated, each 5 minutes long and containing 1800 frames of image data, for use in evaluating the tracking model.

[0105] Then, before obtaining the pre-trained object detection model, it is necessary to create an object detection dataset: split the original RGB video data into image data frame by frame. Considering that the movement of the pig to be detected is relatively slow and the change in the movement of the pig to be detected between adjacent frames is very small, 30 frames can be selected as the extraction interval, and 60 images are extracted from a single video, for a total of 1440 images as the original image dataset for object detection.

[0106] Secondly, to improve the model's generalization ability and robustness, the open-source imgaug library was used to augment the original object detection image dataset through translation, rotation, and noise addition. The original object detection image dataset was augmented to 4320 images, resulting in a data-augmented dataset. 90% (3888 images) was used for model training, and 10% (432 images) was used for testing. Then, the 90% augmented object detection image dataset was further annotated using the labelimg software, such as... Figure 7 This is the second version of the annotated image.

[0107] Therefore, after the object detection model has been pre-trained, inputting the pre-processed video sequence into the pre-trained object detection model will directly output detection data (object type, object location, and confidence data), i.e., direct output. Figure 2 The prediction results in the target detection section.

[0108] The information of the detection bounding boxes in the images stored in the annotation file (the detected information file) is shown in part below, in MOT16 format:

[0109] 1,-1,1359.1,413.27,128.26,362.77,2.3092,-1,-1,-1

[0110] 1,-1,571.03,402.13,104.56,315.68,1.5828,-1,-1,-1

[0111] 1,-1,650.8,455.86,63.98,193.94,8.33276,-1,-1,-1

[0112] 1,-1,721.23,446.86,41.871,127.61,0.27401,-1,-1,-1

[0113] 2,-1,468.48,442.1,98.896,274.69,6.26749,-1,-1,-1

[0114] 2, -1, 643.66, 461.78, 59.629, 180.89, 0.18023, -1, -1, -1

[0115] 2,-1,1494.4,408.29,112.14,338.41,8.16075,-1,-1,-1

[0116] 2, -1, 572.83, 364.89, 128.96, 388.88, -0.011851, -1, -1, -1

[0117] 3,-1,1359.1,413.27,120.26,362.77,2.3387,-1,-1,-1

[0118] 3, -1, 571.03, 402.13, 104.56, 315.68, 0.79923, -1, -1, -1

[0119] 3, -1, 1482.5, 390.88, 128.96, 388.88, 0.35271, -1, -1, -1 ...

[0121] 600, -1, 788.37, 476.87, 20.435, 63.306, -0.1483, -1, -1, -1

[0122] 600, -1, 421.14, 446.86, 41.871, 127.61, -6.19971, -1, -1, -1

[0123] 600, -1, 613.25, 412.69, 51.78, 157.34, -0.38627, -1, -1, -1

[0124] 600,-1,375.27,454.66,48.246,146.74,-8.42444,-1,-1,-1.

[0125] Furthermore, please refer to Figure 8 The step S200, which involves inputting the preprocessed video sequence into a pre-trained target detection model and outputting detection data, specifically includes:

[0126] S210. Input the preprocessed video sequence into the initial target detection model and adjust the model parameters until the error value is lower than the set threshold to obtain the pre-trained target detection model.

[0127] S220. Input the enhanced video sequence into the pre-trained target detection model and output the detection data.

[0128] Specifically, after preprocessing the video sequence using median filtering and video masking, the preprocessed video sequence is input into the initial object detection model. The model parameters are continuously adjusted using backpropagation until the error value is reduced below a set threshold, thus obtaining the pre-trained object detection model. Here, the image input size can be set to 416x416, the optimizer can be SGDM (Stochastic Gradient Descent with Momentum, which selects a mini-batch instead of all samples and uses gradient descent to update model parameters; it solves the problem of random mini-batch samples), the maximum learning rate is set to 0.01, the momentum coefficient is set to 0.937, and the learning rate is adjusted using cosine annealing.

[0129] In deep learning backpropagation, the optimizer guides the parameters of the loss function (objective function) to update to appropriate values ​​in the correct direction, ensuring that the updated parameters continuously approach the global minimum of the loss function (objective function). SGDM (Stochastic Gradient Descent with Momentum) is a method that, compared to traditional stochastic gradient descent, calculates the current gradient by performing an exponentially weighted average of previous gradient values. It then sums these weighted averages with the currently calculated gradient, assigning different weights to each, to obtain the final gradient.

[0130] When using gradient descent to optimize the objective function, the learning rate should decrease as the model approaches the global minimum of the loss value to ensure the object detection model gets as close to that point as possible. Cosine annealing uses the cosine function to reduce the learning rate. In the cosine function, the cosine value initially decreases slowly as x increases, then accelerates, and then decreases slowly again. This descent pattern works well with the learning rate, producing excellent results in a highly efficient computational manner. The learning rate (also called the step size) is used to backpropagate the output error to the network parameters to fit the sample output. Essentially, it's an optimization process that gradually approaches the optimal solution. However, the amount of error used in each parameter update needs to be controlled by a parameter.

[0131] Secondly, to accelerate model training and prevent overfitting, training is performed based on a YOLOv4 pre-trained weight model (the initial object detection model). The backbone network in the YOLOv4 pre-trained weight model has already been trained on the ImageNet dataset. The entire model training process is divided into a freeze phase and an unfreeze phase. In the freeze phase, the feature extraction part of the backbone network is frozen, while other parameters are fine-tuned. Here, the training epochs are set to 50, and the batch size is set to 2. In the unfreeze phase, the entire network is trained. Here, the training epochs are set to 300, and the batch size is set to 2. See the training diagram of the object detection model for further details. Figure 5 .

[0132] Finally, the enhanced video sequence is input into the pre-trained object detection model, which outputs the detection data (object type, object location, and confidence data).

[0133] For further information, please refer to [link / reference]. Figure 1 S300. Input the detection data into the DeepSORT tracking model, and use the Kalman filter, appearance feature extraction model and Hungarian matching algorithm in the DeepSORT tracking model to perform frame-by-frame data association operation, and output the frame number, the ID number of the pig to be detected and the location information of the pig to be detected with the corresponding ID number in each frame of detection data.

[0134] Specifically, the target tracking part also includes a target tracking model training part and a target tracking model prediction part; in the target tracking model training part, before inputting the detection data into the DeepSORT tracking model, an appearance feature extraction model needs to be trained first, as follows:

[0135] The appearance feature extraction part borrows a network model from the pedestrian re-identification field. This part of the network needs to be pre-learned offline, and its function is to extract discriminative features of the target object. The DeepSORT tracking model uses a wide residual network (the initial appearance feature extraction model). The specific structure of the DeepSORT tracking model is as follows: Figure 9 As shown.

[0136] Then, using a WRN network, feature vectors are extracted from the object detection boxes and predicted tracking boxes after convolution, pooling, and residual processing. The cosine similarity between the feature vectors of the object detection boxes and tracking boxes is then calculated to measure appearance features. The final output of the WRN network is a 128-dimensional vector representing this part of the appearance features (generally, higher dimensions result in higher discriminative power but also higher computational cost). Finally, L2 normalization is used to map the features onto a unit hypersphere so that cosine similarity can be further used to measure similarity. Figure 10 This is a flowchart for calculating and associating data for the appearance feature model. L2 normalization is achieved by dividing each value of the vector by the square root of the sum of the squares of the vectors.

[0137] Here, the model for extracting the appearance features of pigs to be detected is trained. The input image size is set to 128x64, the training epochs are 40, the optimizer is SGD, and the learning rate is set to 0.001. The final trained appearance feature extraction model achieves a classification accuracy of 99.91% on the test set. Figure 11 The graph shows the training loss decrease of the trained appearance feature extraction model, where the horizontal axis represents the training epoch and the vertical axis represents the loss function. As can be seen from the graph, the trained appearance feature extraction model performs very well on both the training dataset and the training and test sets.

[0138] After obtaining the trained appearance feature extraction model, the detection data is input into the DeepSORT tracking model. The Kalman filter, the trained appearance feature extraction model, and the Hungarian matching algorithm in the DeepSORT tracking model are used to perform frame-by-frame data association operations, and the frame number, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number are output in each frame of detection data.

[0139] Furthermore, please refer to Figure 12 The steps in S300, namely, inputting the detection data into the DeepSORT tracking model, using the Kalman filter, appearance feature extraction model, and Hungarian matching algorithm in the DeepSORT tracking model to perform frame-by-frame data association operations, and outputting the frame number, the ID number of the pig to be detected, and the location information of the pig with the corresponding ID number in each frame of detection data, specifically include:

[0140] S310. Create a corresponding tracking box for the initial frame detection data, initialize the motion variables of the Kalman filter, predict the tracking box of the next frame detection data using the Kalman filter, and calculate the Mahalanobis distance between the target detection box and the tracking box of the next frame detection data.

[0141] S320. Input the RGB image in the target detection box into the pre-trained appearance feature extraction model, calculate the cosine distance, and calculate the cost matrix based on the cosine distance, the Mahalanobis distance, and the preset threshold matrix.

[0142] S330. Input the cost matrix into the Hungarian algorithm to match the tracking box predicted by the Kalman filter in the previous frame with the target detection box in the next frame, and obtain the first tracking box that is successfully matched, the first detection box that is not matched, and the second tracking box that is not matched.

[0143] S340. Input the position information of the first tracking box that is successfully matched into the Kalman filter, calculate the IOU threshold of the first detection box that fails to match and the second tracking box that fails to match respectively, and use the Hungarian algorithm to match to obtain the third tracking box that is successfully matched, the second detection box that fails to match and the fourth tracking box that fails to match.

[0144] S350. Process the third tracking box that matches successfully, the second detection box that fails to match, and the fourth tracking box that fails to match accordingly.

[0145] S360 updates the mean and covariance matrix using a Kalman filter, and after determining whether the tracker's state is deterministic, outputs the frame number, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number in each frame of detection data.

[0146] Specifically, the object position and confidence information (confidence data) from the pre-trained target detection model's predicted output are input into the DeepSORT tracking algorithm. The DeepSORT algorithm mainly consists of three parts: a Kalman filter, a cascaded matching network, and an appearance feature extraction network. Its specific process within the DeepSORT algorithm is as follows:

[0147] We will now move on to the target tracking model prediction section:

[0148] 1. Kalman filter initialization: Create corresponding tracking boxes for the initial frame detection data, initialize the motion variables of the Kalman filter, and predict the tracking boxes for the next frame detection data through the Kalman filter. At this time, the state of the tracking box is unconfirmed.

[0149] 2. Cascaded Matching: Calculate the Mahalanobis distance between the target detection box in the next frame and the tracking box predicted by the Kalman filter in the previous frame. Simultaneously, input the RGB image of the target detection box into the appearance feature extraction model to calculate the cosine distance. Calculate the cost matrix based on the cosine distance, the Mahalanobis distance, and the preset threshold matrix. Then, input the cost matrix into the Hungarian algorithm to match the tracking box predicted by the Kalman filter in the previous frame with the target detection box in the next frame, resulting in three states of tracking boxes: Matched Tracks (the first successfully matched tracking box), UnmatchedDetections (the first unmatched detection box), and Unmatched Tracks (the second unmatched tracking box).

[0150] III. For the tracking boxes in the three states, process them separately:

[0151] 1. For the first matched track, the position information of the first matched track is input into the Kalman filter;

[0152] 2. For the first unmatched detections and the second unmatched tracking tracks, calculate the IOU threshold and use the Hungarian algorithm to match them to obtain three new tracking tracks in three states after further processing: Matched Tracks (the third tracking track that is successfully matched), UnmatchedDetections (the second unmatched detections), and Unmatched Tracks (the fourth unmatched tracking tracks).

[0153] 3. For the new three states of the tracking box after IOU threshold processing, perform corresponding processing respectively.

[0154] Finally, the mean and covariance matrix are updated using the Kalman filter, and after determining whether the tracker's state is deterministic, the frame number, the ID number of the pig to be detected, and the location information of the corresponding ID number of the pig to be detected in each frame of detection data are output. The target tracking model training process is illustrated in the diagram below. Figure 13 As shown, the DeepSORT tracing model includes the ReID model.

[0155] Furthermore, please refer to Figure 14 The steps of S350, which involve processing the successfully matched third tracking box, the unmatched second detection box, and the unmatched fourth tracking box respectively, specifically include:

[0156] S351. Input the position information of the third tracking box that is successfully matched into the Kalman filter, and add a tracking box for the second detection box that fails to match.

[0157] S352. If the fourth tracking box does not match the target detection box of any other frame detection data, the fourth tracking box enters an uncertain state.

[0158] S353. If the fourth tracking box fails to match the current detection box in three consecutive frames of detection data and the maximum waiting time is exceeded, then the fourth tracking box is deleted.

[0159] S354. If the fourth tracking box has been successfully matched with the target detection box of any other frame detection data, then the fourth tracking box enters the deterministic state.

[0160] Specifically, after calculating the IOU threshold for the first unmatched detections and the second unmatched tracking tracks, and using the Hungarian algorithm to perform matching to obtain three new tracking boxes in three states after further processing, the following corresponding processing is performed on the new tracking boxes in the three states after IOU threshold processing:

[0161] (1) For the matched third tracking boxes, the position information of the matched third tracking boxes is input into the Kalman filter;

[0162] (2) For the second detection box that failed to match the tracking box, a new tracking box is added; because the newly entered detection data of the cascade matching only has detection box information and no tracking box, a new tracking box will be matched after it comes in. At this time, the new tracking box is in an uncertain state (unconfirmed state). Only when it matches the current detection box for three consecutive frames will the new tracking box turn into a confirmed state (confirmed state).

[0163] (3) For the fourth tracking box that fails to match the detection box (Unmatched Tracks), if it has not been successfully matched with any detection box before, it enters the unconfirmed state. This means that the fourth tracking box that failed to match the detection box is a newly entered tracking box for cascading matching. If the fourth tracking box fails to match the current detection box in three consecutive frames of detection data, and the maximum waiting time is exceeded, the fourth tracking box is deleted. If the fourth tracking box has been successfully matched with the target detection box of any other frame of detection data, it means that the pig to be detected has left the field of view, and the fourth tracking box enters the confirmed state. The difference between the confirmed state and the uncertain state is whether it has been matched with a detection box. If it has been matched, it enters the confirmed state; otherwise, it enters the uncertain state.

[0164] Furthermore, please refer to Figure 15 The steps of S360, which involve updating the mean and covariance matrix using a Kalman filter, determining whether the tracker's state is deterministic, and then outputting the frame number, the ID number of the pig to be detected, and the location information of the pig with the corresponding ID number in each frame of detection data, specifically include:

[0165] S361. Update the mean and covariance matrix based on the location information of the first and third successfully matched tracking boxes.

[0166] S362. If the fourth tracking box that has entered an uncertain state matches the newly added tracking box within three consecutive frames of detection data, then the fourth tracking box in the uncertain state becomes a definite state.

[0167] S363. After all the correlation operations on the detection data are completed, output the frame number, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number in each frame of detection data, and save it in the form of text organized by line.

[0168] Specifically, after processing the successfully matched third tracking box, the unmatched second detection box, and the unmatched fourth tracking box accordingly, the mean and covariance matrix are updated using a Kalman filter. After determining whether the tracker's state is deterministic, the frame number, the ID number of the pig to be detected, and the location information of the pig with the corresponding ID number in each frame of detection data are output. The specific steps are as follows:

[0169] First, the Kalman filter updates the mean and covariance matrices:

[0170] The mean and covariance matrix are updated together with the position information of the first successfully matched tracking box and the position information of the third successfully matched tracking box after processing the IOU threshold.

[0171] Secondly, determine whether the tracker status is confirmed:

[0172] If a fourth tracking box that is in an unconfirmed state matches a newly added tracking box within three consecutive frames of detection data, the fourth tracking box in an uncertain state can be transformed into a confirmed state.

[0173] Finally, repeat the above steps until all detection data have been correlated. Output the frame number of the successfully matched tracking box, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number in each frame of detection data, and save it in the form of a txt (line organization) file.

[0174] For further information, please refer to [link / reference]. Figure 1 S400. Based on the frame number, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number, calculate five types of motion indicators for each pig to be detected, and use the motion indicators to form five first data matrices; wherein, the motion indicators include: movement distance, maximum movement speed, maximum movement acceleration, average movement speed, and average movement acceleration.

[0175] Specifically, the target tracking model prediction section also includes a motion data extraction subsection to obtain the motion indicators. Specifically, after performing frame-by-frame data association using the Kalman filter, appearance feature extraction model, and Hungarian matching algorithm in the DeepSORT tracking model, and outputting the frame number, the pig ID number to be detected, and the location information of the corresponding pig ID number in each frame of detection data, the five types of motion indicators (movement distance, maximum speed, maximum acceleration, average speed, and average acceleration) for each pig to be detected are calculated based on the frame number, the pig ID number to be detected, and the location information of the corresponding pig ID number in each frame of detection data. Figure 2 Motion information in the target tracking section.

[0176] The anomaly detection part also includes an anomaly detection model training part and an anomaly detection model prediction part. Before training the anomaly detection model, the anomaly detection model input data (frame number, pig ID number to be detected, and location information of the corresponding pig ID number in each frame of detection data) needs to be processed by PAC data dimensionality reduction. The first step of the PAC data dimensionality reduction process is to use five types of motion indicators to form five first data matrices.

[0177] Furthermore, please refer to Figure 16 The step S400, which involves calculating five types of motion indicators for each pig to be detected based on the frame number, the pig ID number to be detected, and the location information of the corresponding pig ID number, and then using the motion indicators to form five first data matrices, specifically includes:

[0178] S410. Calculate five types of motion indicators for each pig to be detected based on the frame number, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number.

[0179] S420. Extract the same type of motion indicators from N pigs to be tested to form five types of first motion indicator feature vectors. Then, use the five types of first motion indicator feature vectors to form five first data matrices. The length of each first data matrix is ​​N.

[0180] Specifically, in the motion data extraction section, the specific steps for calculating the five types of motion indicators (movement distance, maximum speed, maximum acceleration, average speed, and average acceleration) for each pig under test based on the frame number, the pig ID number to be detected, and the location information of the corresponding pig ID number are as follows:

[0181] First, it is necessary to extract the movement trajectory of the pigs to be tested:

[0182] To avoid the impact of occasional tracker jitter on the calculation of the speed and acceleration of the pig being detected, the movement trajectory of the pig can be extracted every 6 frames. Figure 17 This image shows the movement trajectories of eight pigs to be detected. The eight pigs are labeled ID1, ID2, ID3, ID4, ID5, ID6, ID7, and ID8, where ID represents the identification information assigned to that detection frame. The movement trajectory points are marked with black dots on the original RGB background image. Figure 18 The tracking effect diagrams of the movement trajectories of 8 pigs to be tested are shown here. The tracking effect diagrams at time points of 1s, 10s, 30s, 50s, 120s, 180s, 250s, 300s, 350s, 400s, 450s and 500s are selected.

[0183] Secondly, calculate the five types of motor indicators for each pig to be tested:

[0184] First, the calculation of the movement distance:

[0185] Here, the movement distance of the pig to be detected is first calculated. Since the time interval between two adjacent frames of detection data is on the order of milliseconds, the movement of the pig between adjacent frames can be approximated as a straight line. By accumulating the movement distances of the pig between adjacent frames, the total movement distance of the pig can be obtained. Figure 19 This is a histogram comparing the movement distances of the 8 pigs to be tested. Here, since the actual movement distance of the pigs to be tested is calculated based on the camera's field of view and the size of the video frame, and the aspect ratio and size of the detection frame will change as the pigs to be tested rotate and move, only an approximate movement distance of the pigs to be tested can be calculated here.

[0186] Second, the calculation of average velocity and average acceleration:

[0187] First, calculate the instantaneous velocity and instantaneous acceleration. Using the time-distance formula between two points, calculate the instantaneous velocity of each pig at each black point. Then, using the time interval-velocity difference formula, calculate the instantaneous acceleration of each pig at each black point. Finally, take the absolute values ​​of the calculated instantaneous velocity and instantaneous acceleration. Figure 20 and Figure 21 The following are line graphs showing the movement speed and acceleration of the pig under test with ID number 7. Then, based on the total distance traveled and the total time, the average speed of the pig under test is calculated. The average acceleration of each pig under test is calculated based on its instantaneous speed and instantaneous acceleration at each black point. The average acceleration is then calculated based on the average acceleration at each black point. The schematic diagrams of the average speed and average acceleration of a single pig under test in one video frame are shown below. Figure 22 and Figure 23 As shown.

[0188] Third, extraction of maximum acceleration and maximum velocity:

[0189] Here, the bubble sort algorithm is used to extract the maximum acceleration and maximum velocity sequences of a single pig under test in a video frame. The schematic diagrams of the maximum acceleration and maximum velocity of a single pig under test in a video frame are shown below. Figure 24 and Figure 25 As shown.

[0190] After calculating the five types of movement indicators for each pig to be tested, the movement indicators of the same type for N pigs to be tested are extracted to form five types of first movement indicator feature vectors. The five types of first movement indicator feature vectors are then used to form five first data matrices. Specifically, the movement distance feature vector, maximum movement speed, maximum movement acceleration, average movement speed, and average movement acceleration of the N pigs are extracted to form five feature vectors (first movement indicator feature vectors). The five feature vectors of length N are then combined to form an N x 5 data matrix (first data matrix), where each first data matrix has a length of N.

[0191] The target detection model prediction part, the target tracking model prediction part, and the motion data extraction section together constitute the motion data extraction part. The flowchart of the motion data extraction part is as follows: Figure 26 As shown.

[0192] For further information, please refer to [link / reference]. Figure 1 S500. The first data matrix is ​​reduced in dimensionality using principal component analysis to form a second data matrix. The second data matrix is ​​then input into the anomaly detection model to output the abnormal health status of the pigs to be detected.

[0193] Specifically, the PAC data dimensionality reduction process continues: after forming five first data matrices using the motion indicators, principal component analysis is used to reduce the dimensionality of the first data matrices to form second data matrices. These second data matrices are then input into the anomaly detection model to begin the training phase, in order to output the abnormal health status of the pigs to be detected, i.e., the output... Figure 2 The prediction results in the anomaly detection section enable the detection of abnormal health conditions in pigs based on the pig movement information obtained from multi-target tracking technology.

[0194] This embodiment primarily utilizes PCA (Principal Component Analysis) for data dimensionality reduction. By extracting the largest individual differences revealed by the principal components, it identifies features more easily understood by humans. It can also be used to reduce the number of variables in regression and cluster analyses. The main idea of ​​PCA is to map n-dimensional features onto k dimensions. These k dimensions are entirely new orthogonal features, also known as principal components, which are reconstructed from the original n-dimensional features. In this embodiment, the original five-dimensional feature vector is used to form a new two-dimensional feature vector using the PCA method.

[0195] Furthermore, please refer to Figure 27The steps of S500, which use principal component analysis to reduce the dimensionality of the first data matrix to form a second data matrix and input the second data matrix into the anomaly detection model to output the abnormal health status of the pig to be detected, specifically include:

[0196] S510. Calculate the mean of all first motion index feature vectors for each class, and calculate the difference between each first motion index feature vector in the same class and the mean of the first motion index feature vectors in the class.

[0197] S520. After forming five types of second motion index feature vectors from the five types of differences, five third data matrices are formed using the five types of second motion index feature vectors; wherein, the length of each third data matrix is ​​N.

[0198] S530. Solve for the covariance matrix of the third data matrix, and use the eigenvalue decomposition method to calculate the eigenvalues ​​and eigenvectors of the covariance matrix; wherein, the eigenvectors are the new coordinate axis direction, the rotation direction of the data, or the new principal component direction; the eigenvalues ​​are the variance of the projection of the detected data onto the corresponding new coordinate axis, or the amount of information contained in the eigenvectors corresponding to the detected data;

[0199] S540. Sort the eigenvalues ​​from largest to smallest, take the eigenvectors of the two largest eigenvalues ​​after sorting to form a fourth data matrix, and perform matrix multiplication between the first data matrix and the fourth data matrix to obtain the second data matrix.

[0200] S550. The second data matrix is ​​split to obtain a dataset. A dataset with a preset ratio is used as a training dataset and input into the trained anomaly detection model for prediction. The abnormal health status of the pig to be detected is output.

[0201] Specifically, after forming five first data matrices using the aforementioned motion indicators, the data centralization operation is first performed:

[0202] First, calculate the mean of all first motion index feature vectors for each class, and then calculate the difference between each first motion index feature vector in the same class and the mean of the first motion index feature vectors in that class, forming new custom vectors, namely, five types of second motion index feature vectors. Second, use the five types of second motion index feature vectors to form five third data matrices; that is, the new feature vectors constitute new data matrices. Each third data matrix has a length of N.

[0203] Next, we solve for the covariance matrix and its eigenvalues ​​and eigenvectors:

[0204] Solve for the covariance matrix of the third data matrix. Use the eigenvalue decomposition method to calculate the eigenvalues ​​and eigenvectors of the covariance matrix. Here, the eigenvectors obtained from the covariance matrix are the new coordinate axis directions, the rotation direction of the data, or the new principal component directions. The eigenvalues ​​obtained from the covariance matrix are the variance of the data projected onto the corresponding new coordinate axis, or the amount of information contained in the corresponding eigenvector.

[0205] Furthermore, the eigenvalues ​​in the obtained covariance matrix are sorted from largest to smallest. The eigenvectors of the two largest eigenvalues ​​after sorting are taken as the first and second principal components. These two principal components are used to form the fourth data matrix. The fourth data matrix is ​​then multiplied by the first data matrix to obtain a new N x 2 data matrix (the second data matrix), where each second data matrix also has a length of N. For example... Figure 28 This is a two-dimensional scatter plot of the motion features after dimensionality reduction and recombination using the PCA method.

[0206] Finally, the second data matrix is ​​split to obtain a dataset. A predetermined proportion of this dataset is used as the training dataset and input into the trained anomaly detection model for prediction. The model then outputs the abnormal health status of the pigs to be detected, i.e.:

[0207] The second data matrix after PCA dimensionality reduction is split to obtain the dataset. 80% of the dataset (a preset proportion of the dataset) is taken as the training dataset and input into the anomaly detection model. The model is trained on the input training dataset and outputs the abnormal health status of the pigs to be detected. 20% of the dataset is used as the test dataset.

[0208] After generating the second data matrix, the second data matrix is ​​input into the anomaly detection model, and the anomaly detection model is trained. The algorithm principle for training the anomaly detection model is as follows: here, a single-class support vector machine (OCSVM) is used to implement the anomaly detection function. SVM, also known as support vector machine, is a binary classification model. Its main idea is to find a hyperplane in the space that can separate all data samples, and minimize the distance from all data in the dataset to this hyperplane.

[0209] The training process for the anomaly detection model is as follows: using zero points as negative sample points and other data as positive sample points, a support vector machine is trained to obtain the hyperplane equation. The training strategy is to use a kernel function to map the data to the feature space corresponding to the kernel, construct a hyperplane between the data and the zero points, and solve the hyperplane equation using the KKT conditions to maximize the distance between the hyperplane and the zero points.

[0210] The parameters for training the anomaly detection model are selected as follows: the kernel function is the RBF kernel function, and the best parameter C (central axis of the Gaussian function) and gamma (variance of the Gaussian function) values ​​are selected using cross-validation on the training dataset; where the RBF kernel function is also called the Gaussian function, C refers to the expectation, which determines the central axis of symmetry of the Gaussian function; gamma determines the variance of the Gaussian function.

[0211] The main technical metric for evaluating anomaly detection models is the F1 score, which is defined as follows:

[0212]

[0213]

[0214]

[0215] The cross-validation method involves changing the values ​​of the central axis C and the variance gamma of the Gaussian function within a certain range, and then calculating the F1 score. The model is considered to have the best performance when the F1 score is the largest. The values ​​of C and gamma at this point are selected.

[0216] The anomaly detection model outputs the following: After training, the anomaly detection model yields model parameters (i.e., coefficients of the classification decision function) and support vectors. The trained model is then saved to an XML file. The flowchart for the PAC data dimensionality reduction and anomaly detection model training process is as follows: Figure 29 As shown, the anomaly detection model is a single-class support vector set model.

[0217] Model evaluation (model prediction) of the anomaly detection model: Here, the trained anomaly detection model is used to evaluate the health status of the pigs to be detected, and the resulting singular value detection results are as follows. Figure 30 As shown, white dots represent the data used for training, black dots represent the test set data in the dataset, and gray dots represent the collected abnormal pig movement data. Figure 30 The data shows that 25 randomized data points of pig movement were generated, all exhibiting significant abnormalities in movement speed, distance, and acceleration. (The rest of the text appears to be unrelated and possibly machine-generated.) Figure 31 The image shows the confusion matrix of the prediction results, where the vertical axis represents the true value, the horizontal axis represents the predicted value, -1 represents anomaly (no ID information, which is the address information assigned to the target detection box), and 1 represents normal. Figure 31 As can be seen, almost all anomalies have been detected.

[0218] Furthermore, please refer to Figure 32A system for judging the health status of pigs under test based on multi-target tracking, the system comprising:

[0219] The splitting and preprocessing module 10 is used to split the collected video data about the pigs to be detected into video sequences of preset duration, and to preprocess the video sequences using a median filtering algorithm and video masking operations.

[0220] The detection data output module 20 is used to input the preprocessed video sequence into the pre-trained target detection model and output detection data; wherein, the detection data includes: object type, object position and confidence data;

[0221] The data association module 30 is used to input the detection data into the DeepSORT tracking model, and use the Kalman filter, appearance feature extraction model and Hungarian matching algorithm in the DeepSORT tracking model to perform frame-by-frame data association operation, and output the frame number, the ID number of the pig to be detected and the location information of the pig to be detected with the corresponding ID number in each frame of detection data.

[0222] The motion index calculation module 40 is used to calculate five types of motion indexes for each pig to be detected based on the frame number, the pig ID number to be detected, and the location information of the pig to be detected with the corresponding ID number, and to use the motion indexes to form five first data matrices; wherein, the motion indexes include: movement distance, maximum movement speed, maximum movement acceleration, average movement speed, and average movement acceleration;

[0223] The abnormal health status output module 50 is used to perform dimensionality reduction processing on the first data matrix using the principal component analysis method to form a second data matrix, and input the second data matrix into the abnormal detection model to output the abnormal health status of the pig to be detected.

[0224] Specifically, in this embodiment, the specific functions of each module of the pig health status judgment system based on multi-target tracking based on the above-mentioned pig health status judgment method can be referred to the corresponding description in the above-mentioned pig health status judgment method based on multi-target tracking, and will not be repeated here.

[0225] Furthermore, a computer-readable storage medium stores a program for judging the health status of a pig under test based on multi-target tracking. When executed by a processor, the program implements the steps of the method for judging the health status of a pig under test based on multi-target tracking as described above. Since the steps of the method for judging the health status of a pig under test based on multi-target tracking have been described in detail above, they will not be repeated here.

[0226] In summary, this invention provides a method and related equipment for judging the health status of pigs based on multi-target tracking. The method includes: splitting the collected video data of the pigs to be detected into video sequences of preset durations and performing preprocessing operations; inputting the preprocessed video sequences into a pre-trained target detection model; inputting the obtained detection data into a DeepSORT tracking model and performing frame-by-frame data association operations; calculating five types of motion indicators for each pig to be detected based on the frame number, the ID number of the pig to be detected, and the position information of the corresponding ID number in each frame of output detection data, and forming five first data matrices; performing dimensionality reduction processing on the first data matrices, and inputting the obtained second data matrices into an anomaly detection model to output the abnormal health status of the pigs to be detected, thereby using the five types of motion indicators and other pig motion information obtained by multi-target tracking technology to detect abnormal health status of pigs.

[0227] It is understood that those skilled in the art can make equivalent substitutions or modifications to the technical solution and inventive concept of the present invention, and all such substitutions or modifications should fall within the protection scope of the appended claims.

Claims

1. A method for judging the health status of pigs based on multi-target tracking, characterized in that, The method for judging the health status of pigs based on multi-target tracking includes the following steps: The collected video data of the pigs to be detected is divided into video sequences of preset duration, and the video sequences are preprocessed using median filtering algorithm and video mask operation; The preprocessed video sequence is input into a pre-trained object detection model, which outputs detection data; wherein, the detection data includes: object type, object location, and confidence data; The detection data is input into the DeepSORT tracking model. The Kalman filter, appearance feature extraction model and Hungarian matching algorithm in the DeepSORT tracking model are used to perform frame-by-frame data association operations. The frame number, the ID number of the pig to be detected and the location information of the pig to be detected with the corresponding ID number are output in each frame of detection data. Based on the frame number, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number, five types of motion indicators are calculated for each pig to be detected, and five first data matrices are formed using the motion indicators; wherein, the motion indicators include: movement distance, maximum movement speed, maximum movement acceleration, average movement speed, and average movement acceleration; The first data matrix is ​​reduced in dimensionality using principal component analysis to form a second data matrix. The second data matrix is ​​then input into the anomaly detection model to output the abnormal health status of the pigs to be detected. The step of inputting the detection data into the DeepSORT tracking model, using the Kalman filter, appearance feature extraction model, and Hungarian matching algorithm in the DeepSORT tracking model to perform frame-by-frame data association operations, and outputting the frame number, the ID number of the pig to be detected, and the location information of the pig with the corresponding ID number in each frame of detection data specifically includes: Create a corresponding tracking box for the initial frame detection data, initialize the motion variables of the Kalman filter, predict the tracking box of the next frame detection data using the Kalman filter, and then calculate the Mahalanobis distance between the target detection box and the tracking box of the next frame detection data. The RGB image in the target detection box is input into a pre-trained appearance feature extraction model to calculate the cosine distance. The cost matrix is ​​then calculated based on the cosine distance, the Mahalanobis distance, and a preset threshold matrix. The cost matrix is ​​input into the Hungarian algorithm to match the tracking box predicted by the Kalman filter in the previous frame with the target detection box in the next frame, and to obtain the first tracking box that is successfully matched, the first detection box that is not matched, and the second tracking box that is not matched. The position information of the first tracking box that is successfully matched is input into the Kalman filter. The IOU thresholds of the first detection box and the second tracking box that are not matched are calculated respectively. The Hungarian algorithm is used to match them to obtain the third tracking box that is successfully matched, the second detection box that is not matched, and the fourth tracking box that is not matched. The third tracking box that matches successfully, the second detection box that fails to match, and the fourth tracking box that fails to match are processed accordingly. The mean and covariance matrix are updated using a Kalman filter, and after determining whether the tracker's state is deterministic, the frame number, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number are output for each frame of detection data. The steps of processing the successfully matched third tracking box, the unmatched second detection box, and the unmatched fourth tracking box respectively include: The position information of the third tracking box that is successfully matched is input into the Kalman filter, and a new tracking box is added for the second detection box that fails to match. If the fourth tracking box does not match the target detection box of any other frame detection data, the fourth tracking box enters an uncertain state; If the fourth tracking box fails to match the current detection box in three consecutive frames of detection data and the maximum waiting time is exceeded, then the fourth tracking box is deleted. If the fourth tracking box has successfully matched the target detection box of any other frame detection data, then the fourth tracking box enters the deterministic state; The steps of updating the mean and covariance matrix using a Kalman filter, determining whether the tracker's state is deterministic, and outputting the frame number, the ID number of the pig to be detected, and the location information of the pig with the corresponding ID number in each frame of detection data specifically include: Update the mean and covariance matrix based on the location information of the first and third successfully matched tracking boxes; If the fourth tracking box, which has entered an uncertain state, matches a newly added tracking box within three consecutive frames of detection data, then the fourth tracking box in the uncertain state becomes a definite state. Once all detection data has been correlated, the frame number, the ID number of the pig to be detected, and the location information of the corresponding ID number of the pig to be detected in each frame of detection data are output and saved in a text format containing line-by-line organization.

2. The method for judging the health status of pigs based on multi-target tracking according to claim 1, characterized in that, The step of splitting the collected video data of the pigs to be detected into video sequences of preset durations, and preprocessing the video sequences using a median filtering algorithm and video masking operations includes: Multiple RGB video data segments were collected from multiple pigs to be inspected in a single pen using a camera. The RGB video data is split into video sequences of preset duration, and the video sequences are preprocessed using the median filtering algorithm and the video mask operation.

3. The method for judging the health status of pigs based on multi-target tracking according to claim 1, characterized in that, The step of inputting the preprocessed video sequence into the pre-trained target detection model and outputting detection data specifically includes: The pre-processed video sequence is input into the initial object detection model, and the model parameters are adjusted until the error value is lower than the set threshold to obtain the pre-trained object detection model. The enhanced video sequence is input into a pre-trained target detection model, which outputs the detection data.

4. The method for judging the health status of pigs based on multi-target tracking according to claim 1, characterized in that, The step of calculating five types of motion indicators for each pig to be detected based on the frame number, the pig ID number to be detected, and the location information of the corresponding pig ID number, and then using the motion indicators to form five first data matrices, specifically includes: Based on the frame number, the ID number of the pig to be detected, and the location information of the pig to be detected with the corresponding ID number, calculate five types of motion indicators for each pig to be detected; After extracting the same class of motion indicators from N pigs to be tested to form five types of first motion indicator feature vectors, the five types of first motion indicator feature vectors are used to form five first data matrices; the length of each first data matrix is ​​N.

5. The method for judging the health status of pigs based on multi-target tracking according to claim 4, characterized in that, The steps of using principal component analysis to reduce the dimensionality of the first data matrix to form a second data matrix, and then inputting the second data matrix into the anomaly detection model to output the abnormal health status of the pigs to be detected specifically include: Calculate the mean of all first motion index feature vectors for each class, and calculate the difference between each first motion index feature vector in the same class and the mean of the first motion index feature vectors in the class. After forming five types of second motion index feature vectors from the five types of differences, five third data matrices are formed using the five types of second motion index feature vectors; wherein, the length of each third data matrix is ​​N. Solve for the covariance matrix of the third data matrix, and use the eigenvalue decomposition method to calculate the eigenvalues ​​and eigenvectors of the covariance matrix; wherein, the eigenvectors are the new coordinate axis direction, the rotation direction of the data, or the new principal component direction; the eigenvalues ​​are the variance of the projection of the detected data onto the corresponding new coordinate axis, or the amount of information contained in the eigenvector corresponding to the detected data; The eigenvalues ​​are sorted from largest to smallest. The eigenvectors of the two largest eigenvalues ​​after sorting are used to form the fourth data matrix. The first data matrix and the fourth data matrix are then multiplied to obtain the second data matrix. The second data matrix is ​​split to obtain a dataset. A dataset with a preset ratio is used as a training dataset and input into the trained anomaly detection model for prediction, and the abnormal health status of the pig to be detected is output.

6. A system for judging the health status of pigs based on multi-target tracking, wherein the system is applied to the method for judging the health status of pigs based on multi-target tracking as described in any one of claims 1-5, characterized in that, The pig health status assessment system based on multi-target tracking includes: The splitting and preprocessing module is used to split the collected video data about the pigs to be detected into video sequences of preset durations, and to preprocess the video sequences using a median filtering algorithm and video masking operations. The detection data output module is used to input the preprocessed video sequence into the pre-trained target detection model and output detection data; wherein, the detection data includes: object type, object position and confidence data; The data association module is used to input the detection data into the DeepSORT tracking model, and use the Kalman filter, appearance feature extraction model and Hungarian matching algorithm in the DeepSORT tracking model to perform frame-by-frame data association operation, and output the frame number, the ID number of the pig to be detected and the location information of the pig to be detected with the corresponding ID number in each frame of detection data. The motion index calculation module is used to calculate five types of motion indexes for each pig to be detected based on the frame number, the pig ID number to be detected, and the location information of the pig to be detected with the corresponding ID number, and to use the motion indexes to form five first data matrices; wherein, the motion indexes include: movement distance, maximum movement speed, maximum movement acceleration, average movement speed, and average movement acceleration; The abnormal health status output module is used to perform dimensionality reduction processing on the first data matrix using the principal component analysis method to form a second data matrix, and input the second data matrix into the abnormal detection model to output the abnormal health status of the pig to be detected.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a pig health status judgment program based on multi-target tracking, which, when executed by a processor, implements the steps of the pig health status judgment method based on multi-target tracking as described in any one of claims 1-5.

Images