Multi-target pig tracking method, device, equipment and medium in group-raising environment

Abstract

The present invention discloses a method, device, equipment and medium for tracking multiple targets of pigs in a group-raising environment, the method comprising: obtaining a video sequence of pigs; performing target detection according to the video sequence of pigs in combination with a rotating frame detection model to obtain a pig detection result; performing target tracking according to the pig detection result in combination with a pre-constructed target tracking algorithm to obtain a pig tracking result; the pre-constructed target tracking algorithm further comprises: introducing a Hungarian algorithm and a Kalman filter algorithm on the basis of an existing Byte algorithm, the Hungarian algorithm being used for the third association in the Byte algorithm, and the Kalman filter algorithm being used to predict the possible position of the tracking frame in the next frame before association. The present invention realizes accurate tracking of pigs through a multi-target tracking algorithm based on deep learning, realizes real-time and accurate identity recognition of individual pigs, is conducive to the automated management of group pigs, and has important research significance for intelligent breeding.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer multi-target tracking algorithms, and in particular to long-term automatic tracking of group-raised pigs based on a rotating frame detection model and a Byte algorithm, and specifically to a multi-target pig tracking method, device, equipment and medium in a group-raised environment. Background Art

[0002] As living standards improve, consumers and the livestock industry are paying more and more attention to the sustainability of livestock systems, that is, the full monitoring of the growth of individual animals from birth to death in large-scale farming scenarios. Therefore, it is necessary to use smart farming systems to detect health and welfare issues affecting pigs as early as possible, so as to intervene in time, reduce losses and improve pig welfare, thereby achieving disease prevention and control to achieve the purpose of reducing losses. Intelligent farming is based on individual pig tracking and identification, and it is necessary to establish individual files for pigs, record their daily behavior, and monitor their health status.

[0003] In the field of pig monitoring, the problem of pig identification was mainly solved by radio frequency identification (RFI) technology in the early days. However, this method can cause stress reactions in pigs, and piercing may also cause inflammation of the pigs' skin, thus affecting the healthy growth of pigs. At the same time, identification based on ear tags requires pigs to be in a specific area to read their identity information. With the development of computer vision technology based on deep learning, the identity of pigs can be automatically identified by remote video monitoring equipment, which facilitates the subsequent recording of the daily behavior of each pig and reduces the workload of manual monitoring. This technology only requires some low-cost equipment to implement the deployment of remote video monitoring systems. Staff do not need to have direct contact with pigs, and there will be no stressful situations for pigs, which reduces the risk of zoonotic diseases.

[0004] The multi-target tracking technology in computer vision combines target detection and multi-target tracking algorithms. By using the target detection model, the pig target can be located. Then the multi-target tracking algorithm determines whether it is the same target based on the similarity of the same target at different times, so as to achieve the tracking effect and thus achieve the pig identification effect. Compared with the limitations of RFID technology on the detection location of pigs and the identification function that can only be used for identity information, the pig multi-target tracking technology can not only monitor the identity information of pigs anytime and anywhere, but also be used for subsequent analysis of pig behavior.

[0005] At present, the target detection model based on undirected frame is mainly used for target detection of pigs, but its accuracy is not high enough. The mainstream of current multi-target tracking algorithm is to use the results of target detection to perform data association to achieve multi-target tracking, but in complex scenarios such as a large number of targets and overlapping occlusion, the current multi-target tracking algorithm cannot achieve excellent tracking effect. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a multi-target pig tracking method, device, equipment and medium in a group farming environment, which can realize accurate tracking of pigs through a multi-target tracking algorithm based on deep learning, and realize real-time and accurate identity recognition of individual pigs, which is beneficial to the automated management of group pigs and has important research significance for intelligent farming.

[0007] According to one aspect of the present invention, a method for tracking multiple targets of pigs in a group-raising environment is provided, comprising:

[0008] Obtaining video sequences of pigs;

[0009] According to the pig video sequence, combined with the rotating frame detection model, target detection is performed to obtain a pig detection result;

[0010] According to the pig detection results, combined with a pre-built target tracking algorithm, target tracking is performed to obtain pig tracking results;

[0011] The pre-built target tracking algorithm also includes: based on the existing Byte algorithm, introducing the Hungarian algorithm and the Kalman filter algorithm, the Hungarian algorithm is used for the third association in the Byte algorithm, and the Kalman filter algorithm is used to predict the possible position of the tracking box in the next frame before association.

[0012] The above technical solution uses the rotating box detection model Oriented RepPoints to accurately locate pigs. It can output the pig's detection box more accurately while receiving the video sequence, which is used to locate the pig in any direction; after obtaining the pig's position in real time through target detection, a pre-built target tracking algorithm is used to achieve data association. The target tracking algorithm applies the Hungarian algorithm in the third association process, which can connect two boxes that have no intersection but belong to the same target, thereby solving the ID switching problem caused by the pig's irregular movement and body deformation during the tracking process; and before association, the Kalman filter algorithm is applied to predict the possible position in the next frame, so that the tracking result is more stable and accurate.

[0013] As a further technical solution, the target tracking algorithm further includes:

[0014] Get a video sequence, an object detector, a detection score threshold, and a tracking score threshold;

[0015] Use the object detector to predict the object detection box and corresponding confidence score in each frame of the video;

[0016] Traverse each tracking box in the tracking list and check whether it is lost. If a tracking box is not lost, apply the Kalman filter algorithm to it to predict its possible position in the next frame;

[0017] For detection boxes and tracking boxes with scores higher than the threshold, IoU is used to calculate similarity for association;

[0018] Update the remaining unmatched detection boxes, update the remaining unmatched tracking boxes, and use IoU to calculate the similarity and associate the remaining unmatched detection boxes and the remaining unmatched tracking boxes;

[0019] Update the remaining unmatched tracking frames, and associate the remaining unmatched detection frames and the remaining unmatched tracking frames by calculating the similarity using the center distance;

[0020] Update the unmatched tracking box and update the unmatched detection box.

[0021] As a further technical solution, before using the target detector to predict the target detection box and the corresponding confidence score in the frame, it also includes: initializing the tracking list to be empty.

[0022] As a further technical solution, for the tracking frame that is still not matched after three associations, it is deleted from the tracking list; for the detection frame that is still not matched after three associations, if its score is greater than the tracking score threshold, a new tracking frame is initialized and the detection frame is added to the tracking list.

[0023] As a further technical solution, when performing the third association, the association process assigns an ID number based on an affinity matrix between the tracking box and the detection box, where the affinity matrix is ​​calculated based on the Euclidean distance between the box centers.

[0024] As a further technical solution, the following constraints are also included: Kalman filter prediction is not performed on the tracking bounding box in the lost state, and the remaining unmatched detection boxes are not initialized before all tracking bounding boxes are matched.

[0025] According to one aspect of the present invention, a multi-target pig tracking device in a group-raising environment is provided, comprising:

[0026] An acquisition module, used for acquiring a video sequence of pigs;

[0027] A target detection module, used to perform target detection based on the pig video sequence in combination with a rotating frame detection model to obtain a pig detection result;

[0028] A target tracking module is used to perform target tracking based on the pig detection results in combination with a pre-built target tracking algorithm to obtain pig tracking results;

[0029] Among them, the pre-constructed target tracking algorithm also includes: based on the existing Byte algorithm, introducing the Hungarian algorithm and the Kalman filter algorithm, the Hungarian algorithm is used for the third association in the Byte algorithm, and the Kalman filter algorithm is used to predict the possible position of the tracking box in the next frame before association.

[0030] According to one aspect of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for tracking multiple targets of pigs in a group-raising environment when executing the program.

[0031] According to one aspect of the present invention, there is provided a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for tracking multiple targets of pigs in a group-raising environment are implemented.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention uses a rotating frame detection model Oriented RepPoints to accurately locate pigs. Compared with the problem that the horizontal frame obtained by the existing detection model cannot well represent the shape of the pig photographed at any angle, resulting in a part of the detection frame not framing the pig body, causing redundancy of the detection frame part that does not contain the pig body, the rotating frame used in the present invention is more suitable for images taken from a bird's-eye view, can better align with ground objects, reduce background interference, and enable the model to extract more accurate object features.

[0034] The present invention improves the tracking algorithm Byte in multi-target tracking, introduces the Hungarian algorithm and Kalman filter, and before association, predicts the possible position of the tracking frame in the next frame by applying the Kalman filter algorithm, thereby making the tracking result more stable and accurate.

[0035] The present invention solves the problem of ID switching caused by the irregular movement and body deformation of pigs during tracking. Compared with the problem in the prior art that the detection frames of the same target do not overlap due to various deformations and irregular movements of pigs during the association process, which in turn causes the Byte algorithm relying on IoU to be unable to correctly associate the pig ID, the present invention applies the Hungarian algorithm in the third association process in Byte and adopts a center distance matching mechanism to connect two frames that have no intersection but belong to the same target, thereby solving the problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of the flow of a method for tracking multiple targets of pigs in a group-raising environment according to an embodiment of the present invention.

[0037] Figure 2 Schematic diagram of the ORP-Byte structure according to an embodiment of the present invention.

[0038] Figure 3 (a)-(b) are schematic diagrams for visually comparing the tracking results of ORP-Byte and original Byte according to an embodiment of the present invention.

[0039] Figure 4 Schematic diagram of the comparison of visualization results of video tracking between ORP-Byte and Faster RCNN+ improved Byte according to an embodiment of the present invention.

[0040] Figure 5 It is a schematic diagram of tracking scatter points of pigs with IDs 1, 5, and 8 during 1 hour during the day and at night based on ORP-Byte according to an embodiment of the present invention.

[0041] Figure 6 (a)-(b) are schematic diagrams comparing the tracking results of Oriented RepPoints-SORT and ORP-Byte according to an embodiment of the present invention.

[0042] Figure 7 (a)-(b) are schematic diagrams of tracking results of Faster RCNN+SORT and ORP-Byte according to an embodiment of the present invention.

[0043] Figure 8 Schematic diagram of visual tracking results of the ORP-Byte method on a public video according to an embodiment of the present invention.

[0044] Fig. 9 This is a schematic diagram of various shapes of overlaps between two frames caused by frames being rotated at different angles according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] It should be noted that the multi-target tracking technology in computer vision mainly combines target detection and multi-target tracking algorithms. The present invention improves both target detection and multi-target tracking algorithms.

[0046] Most of the target detection is based on the horizontal box detection model to output the position, motion and appearance features of the pig, and then integrate these features as a reference for the association algorithm to complete the pig tracking task. However, the horizontal box obtained by the detection model cannot well represent the shape of the pig shot at any angle, which will result in a part of the detection box not framing the pig's body, resulting in redundancy of the detection box part that does not contain the pig's body. For images taken from a bird's-eye view, rotating boxes are more suitable. They can better align with ground objects, reduce background interference, and enable the model to extract more accurate object features. Consistent with the above scenario, the video used to monitor farm pigs is also shot from an overhead angle. The pig's body has multiple directions, so the present invention selects a rotating box detection model Oriented RepPoints to accurately locate the pig. Oriented RepPoints can more accurately output the pig's detection box while receiving the video sequence, which is used to locate pigs in any direction.

[0047] After obtaining the position of the pig in real time through target detection, the present invention uses the tracking algorithm as the second part of the multi-target tracking technology to achieve data association. The simplicity and efficiency of the Byte algorithm prompted us to apply it to the multi-target tracking task of pigs. Before association, the Kalman filter algorithm is applied to predict the possible position in the next frame, making the tracking result more stable and accurate.

[0048] In the association process, Byte only relies on the IoU metric between target frames. The present invention prioritizes matching high-scoring detection frames obtained by the detection model, and secondarily matches low-scoring detection frames. However, various deformations and irregular movements of pigs cause the detection frames of the same target to not overlap. This results in the inability of the Byte algorithm that relies on IoU to correctly associate pig IDs. Therefore, it is proposed to apply the Hungarian algorithm in the third association process in Byte, which adopts a center distance matching mechanism. This mechanism can connect two frames that do not have any intersection but belong to the same target. It can effectively solve the ID switching problem caused by the irregular movement and body deformation of pigs during tracking.

[0049] The detection-based tracking method proposed in this invention is named ORP-Byte. The ORP-Byte method includes target detection and multi-target tracking of pigs. It adopts a rotating box detection model called Oriented RepPoints to output the detection box of the pig when receiving the video sequence. Then, it applies the improved Byte algorithm, which correlates the detection results to confirm the ID number of the pig.

[0050] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. The embodiments of the present invention and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0051] Example 1

[0052] This embodiment introduces a multi-target pig tracking method in a group-raising environment. Figure 1 As shown, including:

[0053] Step 1, obtaining a video sequence of pigs;

[0054] Step 2, performing target detection based on the pig video sequence and a rotating frame detection model to obtain a pig detection result;

[0055] Step 3, according to the pig detection result, combined with a pre-built target tracking algorithm, target tracking is performed to obtain a pig tracking result;

[0056] The pre-built target tracking algorithm also includes: based on the existing Byte algorithm, introducing the Hungarian algorithm and the Kalman filter algorithm, the Hungarian algorithm is used for the third association in the Byte algorithm, and the Kalman filter algorithm is used to predict the possible position of the tracking box in the next frame before association.

[0057] This embodiment proposes an ORP-Byte structure, which uses an improved Byte algorithm (as shown in Table 1) combined with the rotating frame detection model Oriented RepPoints to perform long-term automatic tracking of group pigs. The ORP-Byte structure is an association algorithm that processes model outputs and mainly consists of two parts: target detection and multi-target tracking. Figure 2 As shown, the rotation box detection model using OrientedRepPoints outputs the detection box of the pig when receiving the video sequence, and then applies the improved Byte algorithm, which associates the detection results to confirm the ID number of the pig.

[0058] Table 1 Improved Byte pseudo code

[0059]

[0060]

[0061]

[0062]

[0063] The tracking detection method in the multi-target tracking algorithm consists of two parts: target detection and tracking. For images taken from a bird's-eye view, rotating boxes are more suitable. They can better align with ground objects, reduce background interference, and enable the model to extract more accurate object features. Consistent with the above scenario, the video used to monitor farm pigs is also shot from an overhead angle. The pig's body has multiple directions, so we chose a rotating box detection model OrientedRepPoints to accurately locate the pig. For tracking, in order to solve the problem of ID switching caused by the irregular movement and body deformation of the pig during tracking, it is proposed to apply the Hungarian algorithm in the third association process in Byte, which adopts a center distance matching mechanism. This mechanism can connect two boxes that have no intersection but belong to the same target; it is proposed to predict the possible position of the tracking box in the next frame by applying the Kalman filter algorithm before association, so as to make the tracking result more stable and accurate.

[0064] like Figure 3 As shown in (a)-(b), the tracking results of the improved Byte and the original Byte are visually compared. The ORP-Byte of the present invention successfully tracked one of the pigs (ID 4), but the original Byte tracking failed. Figure 3 (a) is the tracking result of the original Byte algorithm based on Oriented RepPoints. Figure 3 (b) is the tracking result of the improved Byte algorithm of the present invention. The dotted line in the image represents the target that is not tracked, and the number in the box represents the pig's ID.

[0065] like Figure 4 As shown, frames are sampled from the video with the tracking box, and the tracking results of the improved Byte based on Faster RCNN and ORP-Byte are compared in the video of 1h during the day and night. The numbers in the boxes represent the IDs of the pigs. Since the improved Byte of the present invention uses the center distance matching mechanism to connect the tracking bounding box of the untracked pig with the detection bounding box, ORP-Byte rarely switches the ID or loses the track of the pig, which reduces the tracking failure caused by the high-speed movement and body deformation of the pig.

[0066] The whole algorithm process implements an improved MOT algorithm Byte for tracking multiple pigs in a video. The input includes a video sequence v, an object detector Det, detection score thresholds t_high and t_low, and a tracking score threshold ε, and the output is the tracking result T of the video. The algorithm follows the following steps:

[0067] Initialize the tracking list T to be empty.

[0068] For each frame f_k in the video, the object detector Det is used to predict the object detection box D_k and the corresponding confidence score in the frame.

[0069] The detection boxes with detection scores higher than the threshold t_high are added to the set D_high, and the detection boxes with detection scores between t_high and t_low are added to the set D_low.

[0070] Traverse each tracking box t in the tracking list T and check whether it is lost. If a tracking box t is not lost, then apply the Kalman filter algorithm to it. Specifically, the Kalman filter algorithm can predict the position of the object in the next frame based on the current mean and variance of the object. Therefore, in this code, for the tracking box t that is not lost, the Kalman filter algorithm is applied to predict its possible position in the next frame, making the tracking result more stable and accurate.

[0071] For the detection box D_high and the tracking box T whose scores are higher than the threshold t_high, the similarity is calculated using IoU to associate them, completing the first association.

[0072] Update the remaining unmatched detection box to D_remain, and update the remaining unmatched tracking box to T_remain.

[0073] For the remaining unmatched detection boxes D_remain and the remaining unmatched tracking boxes T_remain, the IoU is used to calculate the similarity for association, completing the second association.

[0074] Update the remaining unmatched tracking boxes to T_(re-remain).

[0075] For the remaining unmatched detection boxes D_remain and the remaining unmatched tracking boxes T_(re-remain), the center distance is used to calculate the similarity for association, completing the third association.

[0076] Update the unmatched tracking box to T_delete, and update the unmatched detection box to D_(re-remain).

[0077] Delete the item in T_delete from the tracking list T.

[0078] If the remaining tracking list T_lost is empty, it means that there is no lost target being tracked.

[0079] Furthermore, for each detection box d in D_(re-remain) updated after the third association:

[0080] If the score of d is greater than the tracking score threshold ε, a new tracking frame is initialized and d is added to the tracking list T. The tracking list T is returned.

[0081] The improved MOT algorithm Byte can better handle situations such as pig deformation and high-speed movement, improving the accuracy and efficiency of tracking.

[0082] The Kalman filter is an algorithm that generates an estimate of the unknown variable at the next moment in time by fitting the joint probability distribution of a series of measurements investigated over time (including statistical noise). In our proposed method, the Kalman filter algorithm has two functions: (1) It predicts the center position of the tracking box in the next frame based on the mean and variance of the current distribution. This improves the alignment between the tracking box in the next frame and the detection box output by the detection model. (2) Based on the center position of the matched detection bounding box, the center position of the tracking box and the mean and variance of the current distribution are updated. This adjusts the real-time position and the real-time joint distribution of the object. Considering that the deformation of the pig causes significant fluctuations in the aspect ratio of the detection bounding box, we do not use a linear system based Kalman filter to predict and update the size of the tracking box. Instead, we only use the center point of the tracking box for prediction and update.

[0083] The Hungarian algorithm is a combinatorial optimization algorithm that solves the assignment problem in polynomial time. We use it in this study to obtain the maximum similarity matching pair between the detection box and the tracking box. Its role in this paper is in the third association process. The entire three-way association process is:

[0084] First association: We perform association between the tracked box and the high-scoring detection boxes that are larger than the detection score threshold.

[0085] Second association: We complete the association between the remaining unmatched tracking boxes and the low-scoring detection boxes.

[0086] To address the problem of mismatched high-score detection boxes caused by pig deformation or high-speed motion, we implemented a third association process between the remaining high-score detection boxes and the remaining tracking bounding boxes based on the original Byte algorithm.

[0087] Third association: The Hungarian algorithm used in the association process assigns ID numbers based on the affinity matrix between the tracking box and the detection box. Note that the affinity matrix of the first two association processes is calculated based on the loU between the boxes, and the affinity matrix of the third association process is calculated based on the Euclidean distance between the box centers.

[0088] In addition, we impose two constraints: do not perform Kalman filter prediction on the tracked bounding boxes in the lost state, and do not initialize the remaining unmatched detection boxes before all tracked bounding boxes are matched.

[0089] Since we are using rotated boxes, we will use a different method for calculating the intersection over union. The intersection over union (IoU) is a widely used evaluation metric in object detection. It calculates the ratio of the intersection area between two bounding boxes to the union area, which provides a measure of the degree of overlap between the predicted box and the true box. In the detection of horizontal bounding boxes, when two intersecting bounding boxes overlap, the overlapping area remains as a horizontal rectangle. Therefore, it is easy to calculate the IoU using the center coordinates, width, and height of the two boxes. However, unlike horizontal box detection, boxes with different rotation angles can cause overlaps of various shapes between the two boxes, such as triangles and pentagons ( Fig. 9 ). IoU (Intersection over Union) represents the ratio of the intersection area to the union area between (a) two horizontal boxes or (b) two rotated boxes with different angles. The letters in the figure represent the intersection point of the two boxes.

[0090] The calculation process of IoU is more complicated: (1) We use the OpenCV library function to obtain all the intersection points (x1, y1), (x2, y2), ... (2) Use the Gaussian shoelace formula to calculate the area of ​​the polygon formed by the intersection points; (3) Determine the ratio of the polygon area to the joint area of ​​the two boxes to obtain IoU. The relevant formula is as follows:

[0091]

[0092]

[0093] The first formula represents the Gauss shoelace formula, which uses the coordinates of the polygon intersection to calculate the polygon area, denoted by S p In the formula, S b1 ,S b2 Represents the area of ​​the two boxes after rotation. Note that in the subsequent loss calculation of the training detection model, the calculation method of IoU depends on whether the object is a horizontal box or a rotated box. This calculation method is also used for non-maximum suppression (NMS) post-processing in the model inference process and data association of the tracking algorithm.

[0094] like Figure 5 As shown, it shows the tracking scatter plot of pigs with IDs 1, 5, and 8 based on ORP-Byte during daytime and nighttime for 1 hour, where the horizontal and vertical coordinates represent the resolution of the image. Figure 5 Based on the proposed ORP-Byte method, the real-time locations of pigs with ID 1, ID 5, and ID 8 are shown in the form of a scatter plot, including their tracking trajectories for 1 hour during the day and night. The tracking trajectories of the proposed ORP-Byte method are consistent with the actual situation, which shows that the ORP-Byte method of the present invention has high accuracy in locating and tracking pigs.

[0095] like Figure 6 As shown in (a)-(b), the tracking results of Oriented RepPoints-SORT (a) and ORP-Byte (b) are compared, and the dotted line in the image represents the untracked target. In the figure, the pig (ID 4) tracked by the ORP-Byte of the present invention cannot be tracked by OrientedRepPoints-SORT. Compared with the Oriented RepPoints-SORT algorithm, the number of pig IDs tracked by the ORP-Byte algorithm of the present invention is within a more stable range. Since the ORP-Byte algorithm of the present invention takes into account the low-confidence detection frame generated by the detection model in complex scenarios such as pig overlap and occlusion, there are fewer switched IDs and untracked targets in pig video tracking. This helps to reduce lost tracking and ID switching.

[0096] like Figure 7 (a)-(b) show the tracking results of Faster RCNN+SORT and ORP-Byte. The dotted lines in the images represent untracked targets. In the figure, the pig (ID 8) tracked by ORP-Byte failed to be tracked using Faster RCNN.

[0097] like Figure 8 The figure shows the visual tracking results of the ORP-Byte method of the present invention on a public video. These results in the figure once again verify the effectiveness of the present invention's improvement on the Byte algorithm and the applicability of the rotating box detection model to the tracking algorithm. ORP-Byte still works well in the case of fast movement and high density of pigs. The experimental results show that ORP-Byte has strong generalization ability and can be applied to pig tracking in natural breeding scenarios. Even in low frame rate videos, our ORP-Byte still shows good tracking performance.

[0098] Example 2

[0099] Based on the same inventive concept as Example 1, this example introduces a multi-target pig tracking device in a group-raising environment, comprising:

[0100] An acquisition module, used for acquiring a video sequence of pigs;

[0101] A target detection module, used to perform target detection based on the pig video sequence in combination with a rotating frame detection model to obtain a pig detection result;

[0102] A target tracking module is used to perform target tracking based on the pig detection results in combination with a pre-built target tracking algorithm to obtain pig tracking results;

[0103] Among them, the pre-constructed target tracking algorithm also includes: based on the existing Byte algorithm, introducing the Hungarian algorithm and the Kalman filter algorithm, the Hungarian algorithm is used for the third association in the Byte algorithm, and the Kalman filter algorithm is used to predict the possible position of the tracking box in the next frame before association.

[0104] The specific functions of the above modules are implemented by referring to the relevant contents of the method in Example 1, which will not be described in detail. It is particularly pointed out that:

[0105] The target tracking algorithm also includes:

[0106] Get a video sequence, an object detector, a detection score threshold, and a tracking score threshold;

[0107] Use the object detector to predict the object detection box and corresponding confidence score in each frame of the video;

[0108] Traverse each tracking box in the tracking list and check whether it is lost. If a tracking box is not lost, apply the Kalman filter algorithm to it to predict its possible position in the next frame;

[0109] For detection boxes and tracking boxes with scores higher than the threshold, IoU is used to calculate similarity for association;

[0110] Update the remaining unmatched detection boxes, update the remaining unmatched tracking boxes, and use IoU to calculate the similarity and associate the remaining unmatched detection boxes and the remaining unmatched tracking boxes;

[0111] Update the remaining unmatched tracking frames, and associate the remaining unmatched detection frames and the remaining unmatched tracking frames by calculating the similarity using the center distance;

[0112] Update the unmatched tracking box and update the unmatched detection box.

[0113] Example 3

[0114] Based on the same inventive concept as other embodiments, this embodiment introduces an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the multi-target pig tracking method in a group-raising environment when executing the program.

[0115] Example 4

[0116] Based on the same inventive concept as other embodiments, this embodiment introduces a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the multi-target pig tracking method in a group-raising environment are implemented.

[0117] In summary, the present invention uses a rotating frame detection model to accurately locate pigs. Compared with the problem that the horizontal frame obtained by the existing detection model cannot well represent the shape of the pig photographed at any angle, resulting in a part of the detection frame not framing the pig body, causing redundancy of the detection frame part that does not contain the pig body, the rotating frame used in the present invention is more suitable for images taken from a bird's-eye view, can better align with ground objects, reduce background interference, and enable the model to extract more accurate object features.

[0118] The present invention improves the tracking algorithm Byte in multi-target tracking, introduces the Hungarian algorithm and Kalman filter, and before association, predicts the possible position of the tracking frame in the next frame by applying the Kalman filter algorithm, thereby making the tracking result more stable and accurate.

[0119] The present invention solves the problem of ID switching caused by the irregular movement and body deformation of pigs during tracking. Compared with the problem in the prior art that the detection frames of the same target do not overlap due to various deformations and irregular movements of pigs during the association process, which in turn causes the Byte algorithm relying on IoU to be unable to correctly associate the pig ID, the present invention applies the Hungarian algorithm in the third association process in Byte and adopts a center distance matching mechanism to connect two frames that have no intersection but belong to the same target, thereby solving the problem.

[0120] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0124] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A multi-target pig tracking method in a group-raising environment, characterized in that: include: Obtaining pig video sequences; According to the pig video sequence, combined with the rotating frame detection model, target detection is performed to obtain a pig detection result; According to the pig detection results, combined with a pre-built target tracking algorithm, target tracking is performed to obtain pig tracking results; The pre-built target tracking algorithm also includes: based on the existing Byte algorithm, the Hungarian algorithm and the Kalman filter algorithm are introduced, the Hungarian algorithm is used for the third association in the Byte algorithm, and the Kalman filter algorithm is used to predict the possible position of the tracking box in the next frame before association; the target tracking algorithm also includes: Get a video sequence, an object detector, a detection score threshold, and a tracking score threshold; Use the object detector to predict the object detection box and corresponding confidence score in each frame of the video; Traverse each tracking box in the tracking list and check whether it is lost. If a tracking box is not lost, apply the Kalman filter algorithm to it to predict its possible position in the next frame; For detection boxes and tracking boxes with scores higher than the threshold, IoU is used to calculate similarity for association; Update the remaining unmatched detection boxes, update the remaining unmatched tracking boxes, and use IoU to calculate the similarity and associate the remaining unmatched detection boxes and the remaining unmatched tracking boxes; Update the remaining unmatched tracking frames, and associate the remaining unmatched detection frames and the remaining unmatched tracking frames by calculating the similarity using the center distance; Update the unmatched tracking box and update the unmatched detection box.

2. The multi-target pig tracking method in a group-raising environment according to claim 1 is characterized in that: Before using the target detector to predict the target detection box and the corresponding confidence score in the frame, it also includes: initializing the tracking list to be empty.

3. The multi-target pig tracking method in a group-raising environment according to claim 2 is characterized in that: For the tracking frame that is still not matched after three associations, it is deleted from the tracking list; for the detection frame that is still not matched after three associations, if its score is greater than the tracking score threshold, a new tracking frame is initialized and the detection frame is added to the tracking list.

4. The multi-target pig tracking method in a group-raising environment according to claim 1 is characterized in that: During the third association, the association process assigns ID numbers based on an affinity matrix between the tracking box and the detection box, which is calculated based on the Euclidean distance between the box centers.

5. The multi-target pig tracking method in a group-raising environment according to claim 4 is characterized in that: The following constraints are also included: Kalman filter prediction is not performed on the tracking bounding box in the lost state, and the remaining unmatched detection boxes are not initialized before all tracking bounding boxes are matched.

6. A multi-target pig tracking device in a group-raising environment, characterized in that: include: An acquisition module, used for acquiring video sequences of pigs; A target detection module, used to perform target detection based on the pig video sequence in combination with a rotating frame detection model to obtain a pig detection result; A target tracking module is used to perform target tracking based on the pig detection results in combination with a pre-built target tracking algorithm to obtain pig tracking results; The pre-built target tracking algorithm further includes: based on the existing Byte algorithm, introducing the Hungarian algorithm and the Kalman filter algorithm, wherein the Hungarian algorithm is used for the third association in the Byte algorithm, and the Kalman filter algorithm is used for predicting the possible position of the tracking box in the next frame before association; The target tracking algorithm also includes: Get a video sequence, an object detector, a detection score threshold, and a tracking score threshold; Use the object detector to predict the object detection box and corresponding confidence score in each frame of the video; Traverse each tracking box in the tracking list and check whether it is lost. If a tracking box is not lost, apply the Kalman filter algorithm to it to predict its possible position in the next frame; For detection boxes and tracking boxes with scores higher than the threshold, IoU is used to calculate similarity for association; Update the remaining unmatched detection boxes, update the remaining unmatched tracking boxes, and use IoU to calculate the similarity and associate the remaining unmatched detection boxes and the remaining unmatched tracking boxes; Update the remaining unmatched tracking frames, and associate the remaining unmatched detection frames and the remaining unmatched tracking frames by calculating the similarity using the center distance; Update the unmatched tracking box and update the unmatched detection box.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the multi-target pig tracking method in a group-raising environment as described in any one of claims 1-5 are implemented.

8. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for tracking multiple targets of pigs in a group-raising environment as described in any one of claims 1 to 5 are implemented.

Images