ViT-YOLOX-Dynamic Head-based cattle face image dynamic detection and tracking method

Abstract

The invention relates to a cattle face image dynamic detection and tracking method based on ViT-YOLOX-Dynamic Head, and the method comprises the steps: obtaining a cattle herd face video, and carrying out the data preprocessing; constructing a cattle face image dynamic detection and tracking model based on a YOLOX model; performing training to obtain a trained cattle face image dynamic detection and tracking model; and obtaining a to-be-detected cattle face image, performing preprocessing, inputting the preprocessed to-be-detected cattle face image into the trained cattle face image dynamic detection and tracking model, and obtaining a cattle face image detection result. According to the method, the capability of capturing context information of the model is enhanced, the parameter efficiency of the model is fully utilized, global information in the image can be better captured, and the method has higher recognition capability for cow face targets with complex backgrounds and different scales; a dynamic head method is adopted to perform multi-scale feature fusion and adaptively adjust feature levels by associating spatio-temporal features and facial features, so that the cow face detection and tracking capability of the model in different scales and complex backgrounds is improved.

Description

Technical Field

[0001] The invention relates to the technical field of animal husbandry image recognition, in particular to a method for dynamic detection and tracking of cattle face images based on ViT-YOLOX-Dynamic Head. Background Art

[0002] With the development of the breeding industry, especially the popularization of large-scale breeding, the detection and identification technology of individual livestock has gradually become an important research direction of modern agricultural management. Traditional identification methods include manual identification, ear tags and RFID radio frequency identification technology.

[0003] Among them, manual identification requires a lot of time and manpower, especially in large-scale farming. It is not only inefficient but also prone to errors. Manual identification cannot achieve real-time tracking and is sensitive to the environment and animal behavior, making it impossible to carry out refined management. Ear tags have certain limitations in animal behavior. They need to be manually marked throughout the process, which consumes a lot of manpower and material resources. They may even cause certain physiological damage to the animals and cause them to overreact. In severe cases, they may cause violence in livestock and affect their growth and health. In actual applications, it is found that the use of ear tags will increase the susceptibility of livestock to certain diseases, and ear tags may also be damaged and fall off, affecting their readability. Radio frequency technology uses radio waves to identify and track targets. In some environments, it may be subject to electromagnetic interference, resulting in increased errors in the accurate identification of livestock. In addition, RFID tags are expensive. Different types of RFID tags, such as high-frequency and ultra-high-frequency tags, may be incompatible with existing reading and writing equipment, resulting in additional equipment updates or conversion costs.

[0004] Therefore, how to use image recognition technology to accurately identify and track cow faces has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to solve the defect that the existing technology is difficult to accurately identify and track cow face images, the purpose of the present invention is to provide a cow face image dynamic detection and tracking method based on ViT-YOLOX-Dynamic Head, which can quickly and effectively detect and track the cow's face in complex scenes.

[0006] To achieve the above object, the present invention adopts the following technical solution: a method for dynamic detection and tracking of cow face images based on ViT-YOLOX-DynamicHead, the method comprising the following steps in order:

[0007] (1) Acquisition and preprocessing of cattle face images: Use a video acquisition system to acquire cattle face videos in the pasture under different conditions of cattle numbers, lighting changes, cattle posture changes, occlusion, and multiple angles, and perform data preprocessing to obtain preprocessed cattle face images. The preprocessed cattle face images constitute the format data set required for target detection;

[0008] (2) Construct a dynamic detection and tracking model for cow face images based on the YOLOX model;

[0009] (3) inputting the data set in the format required for target detection into the cow face image dynamic detection and tracking model for training, thereby obtaining a trained cow face image dynamic detection and tracking model;

[0010] (4) Obtaining a cow face image to be detected and preprocessing it, inputting the preprocessed cow face image to be detected into the trained cow face image dynamic detection and tracking model to obtain a cow face image detection result.

[0011] In step (1), the video acquisition system is a multi-channel monitoring system, which includes a camera, a video recorder, a switch and a router; the preprocessing refers to extracting a picture every 20 frames of the acquired video, and using the labelme tool to mark the cow heads to produce a data set in the format required for target detection.

[0012] The step (2) specifically means that: the YOLOX model includes a first layer, a second layer, a third layer and a fourth layer, the first layer is an input layer, and the size of the input image received by the input layer is 640×640; the second layer is a backbone network, and the backbone network includes a Darknet53 structure and an SPP architecture; the third layer is a neck network, and the neck network adopts a PAFPN structure; the fourth layer is a head network for processing feature maps and generating final target detection results;

[0013] The Vision Transformer module is integrated into the CSP module in the Darknet53 structure of the second-layer backbone network. The Vision Transformer module is used as the backbone of the CSP module. The CSP module and the Vision Transformer module together form the ViT-CSP module.

[0014] The Vision Transformer module includes Embedding, encoder and multi-layer perceptron. The input of the ViT-CSP module first passes through the convolution layer for preliminary feature extraction and processing, and then passes through the linear layer of Embedding to linearly transform the convolutional features. Then, the multi-head attention mechanism of the encoder focuses on the information at different positions of the convolutional features, captures the long-distance dependencies and important features in the input sequence, and merges the convolutional features to obtain a richer feature representation. After the fully connected layer of the multi-layer perceptron and the nonlinear transformation of the activation function, a feature output graph containing information at different levels is finally obtained.

[0015] In the third-layer neck network, a feature expression enhancement operation, an upsampling operation, and a concatenation operation of a bidirectional gated recurrent unit are added;

[0016] In the fourth layer head network, a unified object detection head method with multiple attention mechanisms is adopted.

[0017] The step (3) specifically comprises the following steps in order:

[0018] (3a) Build a Pytorch neural network training environment with Python=3.9.0 and CUDA=10.2;

[0019] (3b) Set the image input size to 640×640, the batch size to 8, the maximum number of iterations to 50, use SGD as the optimizer, the weight decay coefficient to 0.0005, the momentum to 0.937 as the default value, the initial learning rate to 0.01, and the cosine annealing algorithm for update;

[0020] (3c) Input the data set in the format required for target detection into the cow face image dynamic detection and tracking model, train it and generate a weight file.

[0021] The object detection head method adopts a dynamic head method, specifically, setting the object detection head to adopt an attention mechanism to enhance scale, spatial position and multi-task perception capabilities, and integrating these three attentions into the unified attention mechanism of the target detection head to fully extract cow facial related features;

[0022] Set the object detection head to convert the attention function into three consecutive attentions, each of which focuses on only one dimension:

[0023] W(F)=π C (π S (π L (F)·F)·F)·F

[0024] Where F represents the original attention function, π L(·) is the scale-aware attention function, π S (·) is the spatial perception attention function π S (·), π C (·) is the task-aware attention function.

[0025] It can be seen from the above technical scheme that the beneficial effects of the present invention are: first, the present invention enhances the ability of the model to capture contextual information by integrating the Vision Transformer module into the backbone network of the YOLOX model, and at the same time fully utilizes the parameter efficiency of the model, can better capture the global information in the image, and has a stronger recognition ability for cow face targets with complex backgrounds and different scales; second, the present invention adopts a dynamic head method to associate spatiotemporal features with facial features, perform multi-scale feature fusion, and adaptively adjust the feature hierarchy, thereby improving the model's ability to detect and track cow faces in different scales and complex backgrounds; third, the cow face image dynamic detection and tracking model constructed by the present invention shows high accuracy and real-time performance in cow face detection and tracking tasks. Experimental results show that the cow face image dynamic detection and tracking model shows significant efficiency improvement when processing cow face targets in complex backgrounds, and at the same time has good robustness, and can cope with challenges such as multiple targets, small targets and complex backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of the method of the present invention;

[0027] Figure 2 A schematic diagram of a cow face image sample in a data set in a format required for target detection in the present invention;

[0028] Figure 3 The test results before and after the improvement of the model of the present invention;

[0029] Figure 4 A schematic diagram of the network structure of the cow face image dynamic detection and tracking model in the present invention;

[0030] Figure 5 It is a schematic diagram of the structure of the ViT-CSP module of the present invention;

[0031] Figure 6 It is a structural schematic diagram of the dynamic head in the dynamic head method of the present invention. DETAILED DESCRIPTION

[0032] like Figure 1 As shown, a method for dynamic detection and tracking of cow face images based on ViT-YOLOX-Dynamic Head, the method comprises the following steps in sequence:

[0033] (1) Acquisition and preprocessing of cattle face images: Use a video acquisition system to acquire cattle face videos in the pasture under different conditions of cattle numbers, lighting changes, cattle posture changes, occlusion, and multiple angles, and perform data preprocessing to obtain preprocessed cattle face images. The preprocessed cattle face images constitute the format data set required for target detection;

[0034] (2) Construct a dynamic detection and tracking model for cow face images based on the YOLOX model;

[0035] (3) inputting the data set in the format required for target detection into the cow face image dynamic detection and tracking model for training, thereby obtaining a trained cow face image dynamic detection and tracking model;

[0036] (4) Obtaining a cow face image to be detected and preprocessing it, inputting the preprocessed cow face image to be detected into the trained cow face image dynamic detection and tracking model to obtain a cow face image detection result.

[0037] In step (1), the video acquisition system is a multi-channel monitoring system, which includes a camera, a video recorder, a switch and a router; the preprocessing refers to extracting a picture every 20 frames of the acquired video, and using the labelme tool to mark the cow heads to produce a data set in the format required for target detection.

[0038] like Figure 4 As shown, the step (2) specifically refers to: the YOLOX model includes a first layer, a second layer, a third layer and a fourth layer, the first layer is an input layer, and the size of the input image received by the input layer is 640×640; the second layer is a backbone network, and the backbone network includes a Darknet53 structure and an SPP architecture; the third layer is a neck network, and the neck network adopts a PAFPN structure; the fourth layer is a head network for processing feature maps and generating final target detection results; the Darknet53 structure is used for target detection and classification tasks, and the effect of feature extraction is improved by stacking multiple convolutional and residual connection layers; the SPP architecture is three consecutive maximum poolings with a residual structure, and finally the results before and after pooling are spliced;

[0039] like Figure 5 As shown in the figure, the VisionTransformer module is integrated into the CSP module in the Darknet53 structure of the second-layer backbone network, and the Vision Transformer module is used as the backbone of the CSP module. The CSP module and the VisionTransformer module together form the ViT-CSP module;

[0040] The Vision Transformer module includes Embedding, encoder and multi-layer perceptron. The input of the ViT-CSP module first passes through the convolution layer for preliminary feature extraction and processing, and then passes through the linear layer of Embedding to linearly transform the convolutional features. Then, the multi-head attention mechanism of the encoder focuses on the information at different positions of the convolutional features, captures the long-distance dependencies and important features in the input sequence, and merges the convolutional features to obtain a richer feature representation. After the fully connected layer of the multi-layer perceptron and the nonlinear transformation of the activation function, a feature output graph containing information at different levels is finally obtained.

[0041] In the third-layer neck network, a bidirectional gated recurrent unit feature expression enhancement operation, upsampling operation and splicing operation are added to extract shallower features, enhance the detection of medium and small objects, improve the model's feature extraction ability for medium and small objects, and make the model focus on facial features;

[0042] The fourth-layer head network adopts a unified object detection head method with multiple attention mechanisms to further associate spatiotemporal characteristics with facial features to improve the performance and robustness of the target detection and recognition method.

[0043] The step (3) specifically comprises the following steps in order:

[0044] (3a) Build a Pytorch neural network training environment with Python=3.9.0 and CUDA=10.2;

[0045] (3b) Set the image input size to 640×640, the batch size to 8, the maximum number of iterations to 50, use SGD as the optimizer, the weight decay coefficient to 0.0005, the momentum to 0.937 as the default value, the initial learning rate to 0.01, and the cosine annealing algorithm for update;

[0046] (3c) Input the data set in the format required for target detection into the cow face image dynamic detection and tracking model, train it and generate a weight file.

[0047] like Figure 6 As shown, the object detection head method adopts a dynamic head method, namely, Dynamic Head, which specifically means: setting the object detection head to adopt an attention mechanism to enhance scale, spatial position and multi-task perception capabilities, and integrating these three attentions into the unified attention mechanism of the target detection head to fully extract cow facial related features;

[0048] Set the object detection head to convert the attention function into three consecutive attentions, each of which focuses on only one dimension:

[0049] W(F)=πC (π S (π L (F)·F)·F)·F

[0050] Where F represents the original attention function, π L (·) is the scale-aware attention function, π S (·) is the spatial perception attention function π S (·), π C (·) is the task-aware attention function.

[0051] Existing detection models mainly use high-quality facial datasets to achieve face detection, but this does not improve the detection performance of the model in more complex and changeable real-world environments. Figure 2 As shown, there are occlusions and multiple angles in the cow face image samples, which increase the complexity of the samples and the difficulty of recognition; the cattle breeding environment is relatively complex, and occlusions, overlaps, light interference, etc. will cause missed detections and false detections. Under the premise of ensuring the detection accuracy of the constructed model, the real-time requirements of the ranch should also be met. In addition, how to improve the model to have efficient generalization and accuracy is also a difficulty in model construction. Based on the above situation, the model constructed by the present invention can well realize the dynamic detection and tracking of cow face images in complex scenes.

[0052] The present invention uses IDF1, MOTA, MOTP, IDs and Avg Time to evaluate the tracking performance of each model. Among them, IDs represents the number of times the label is changed in a tracking trajectory. The smaller the value, the better the tracking performance. The multi-target tracking accuracy MOTA takes into account the matching errors in the tracking process. The larger the value, the better the tracking performance. The calculation formula is:

[0053]

[0054] In the formula, FP is the number of false positive examples, FN is the number of false negative examples, and GT represents the number of true targets.

[0055] Multi-target tracking accuracy MOTP takes into account the positioning accuracy of the detection frame. The larger the value, the better the tracking performance. The calculation formula is:

[0056]

[0057] In the formula, c t is the number of successful matches in the current frame, d i,t is the intersection-and-union ratio of the detection frame and the real frame of the current frame; t represents the frame number, which is used to traverse different frames in the video sequence; i represents the index of the target;

[0058] The calculation formula of IDF1 is:

[0059]

[0060] Where TP is the number of true positive cases.

[0061] Avg Time is the time required for the tracker to process one frame.

[0062] The present invention compares the ViT-YOLOX-Dynamic Head, i.e., the dynamic detection and tracking model of cow face images, with YOLOv3, YOLOv4, and YOLOv5 that introduce the Dynamic Head method, and compares the impact of ViT on the model on this basis. The comparison of each evaluation index of each model is shown in Table 1. YOLOv5 and YOLOX both improve the detection effect and thus improve the tracking effect. IDF1 is improved by 1.3% and 1.0% respectively, and MOTA is improved by 1.6% and 2.5% respectively. The number of ID switching is not improved, and the single-frame tracking processing time is similar. Based on the two methods, the Vision Transformer is integrated, and there is no improvement on YOLOv5, but for YOLOX, IDF1 is increased by an additional 0.9%, MOTA is increased by 0.3%, MOTP is slightly reduced by 0.1%, the number of ID switching is reduced by half, and the processing time is reduced by 20%. The experiment also shows that the tracking effect is reduced when the ViT network is used alone. The above experimental results show that YOLOX can better solve the detection problem and has better detection effects on occlusion and interference. Under this premise, the fusion of the feature vector extracted by ViT reduces the number of ID switching times, better copes with the matching problem that reappears after a long disappearance, and the tracking processing time is also reduced.

[0063] Table 1 Comparison of evaluation indicators of each model

[0064]

[0065] like Figure 3 As shown, Figure 3 a1 to a4 are the cow faces identified by the YOLOX-Dynamic model at the 5th, 150th, 250th, and 500th frames and the IDs of the cows corresponding to the cow faces. Figure 3In the figure, b1 to b4 are the cow faces and the cow IDs corresponding to the cow faces identified by the cow face dynamic detection and tracking model after integrating Vision Transformer at the 5th, 150th, 250th, and 500th frames. The red frame tracks the recognition and tracking effect of the cow face with ID 7, and the yellow frame tracks the recognition and tracking effect of the cow face with ID 3. By comparing the tracking effects of the two models, it can be found that before integrating ViT, the cow faces with IDs 7 and 4 were occluded by each other, and the cow face label with ID 7 was changed to 4. The cow face with ID 3 was occluded by the railing and the distance was increased. After a short detection failure, the ID was switched to 13. ViT solves these two problems very well.

[0066] In summary, the present invention integrates the Vision Transformer module into the backbone network of the YOLOX model, thereby enhancing the model's ability to capture contextual information and making full use of the model's parameter efficiency. The Vision Transformer module can better capture the global information in the image and has a stronger recognition ability for cow face targets with complex backgrounds and different scales. The present invention adopts a dynamic head method to associate spatiotemporal features with facial features, perform multi-scale feature fusion, and adaptively adjust the feature hierarchy, thereby improving the model's ability to detect and track cow faces in different scales and complex backgrounds. The cow face image dynamic detection and tracking model constructed by the present invention shows high accuracy and real-time performance in cow face detection and tracking tasks. Experimental results show that the cow face image dynamic detection and tracking model shows significant efficiency improvement when processing cow face targets in complex backgrounds, and has good robustness, and can cope with challenges such as multiple targets, small targets and complex backgrounds.

[0067] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for dynamic detection and tracking of cow face images based on ViT-YOLOX-Dynamic Head, characterized by: The method comprises the following steps in order: (1) Acquisition and preprocessing of cattle face images: Use a video acquisition system to acquire cattle face videos in the pasture under different conditions of cattle numbers, lighting changes, cattle posture changes, occlusion, and multiple angles, and perform data preprocessing to obtain preprocessed cattle face images. The preprocessed cattle face images constitute the format data set required for target detection; (2) Construct a dynamic detection and tracking model for cow face images based on the YOLOX model; (3) inputting the data set in the format required for target detection into the cow face image dynamic detection and tracking model for training, thereby obtaining a trained cow face image dynamic detection and tracking model; (4) Obtaining a cow face image to be detected and preprocessing it, inputting the preprocessed cow face image to be detected into the trained cow face image dynamic detection and tracking model to obtain a cow face image detection result.

2. The method for dynamic detection and tracking of cow face images based on ViT-YOLOX-Dynamic Head according to claim 1, characterized in that: In step (1), the video acquisition system is a multi-channel monitoring system, which includes a camera, a video recorder, a switch and a router; the preprocessing refers to extracting a picture every 20 frames of the acquired video, and using the labelme tool to mark the cow heads to produce a data set in the format required for target detection.

3. The method for dynamic detection and tracking of cow face images based on ViT-YOLOX-Dynamic Head according to claim 1, characterized in that: The step (2) specifically means that: the YOLOX model includes a first layer, a second layer, a third layer and a fourth layer, the first layer is an input layer, and the size of the input image received by the input layer is 640×640; the second layer is a backbone network, and the backbone network includes a Darknet53 structure and an SPP architecture; the third layer is a neck network, and the neck network adopts a PAFPN structure; the fourth layer is a head network for processing feature maps and generating final target detection results; The Vision Transformer module is integrated into the CSP module in the Darknet53 structure of the second-layer backbone network. The Vision Transformer module is used as the backbone of the CSP module. The CSP module and the Vision Transformer module together form the ViT-CSP module. The Vision Transformer module includes Embedding, encoder and multi-layer perceptron. The input of the ViT-CSP module first passes through the convolution layer for preliminary feature extraction and processing, and then passes through the linear layer of Embedding to linearly transform the convolutional features. Then, the multi-head attention mechanism of the encoder focuses on the information at different positions of the convolutional features, captures the long-distance dependencies and important features in the input sequence, and merges the convolutional features to obtain a richer feature representation. After the fully connected layer of the multi-layer perceptron and the nonlinear transformation of the activation function, a feature output graph containing information at different levels is finally obtained. In the third-layer neck network, a feature expression enhancement operation, an upsampling operation, and a concatenation operation of a bidirectional gated recurrent unit are added; The fourth-layer head network adopts a unified object detection head method with multiple attention mechanisms.

4. The method for dynamic detection and tracking of cow face images based on ViT-YOLOX-Dynamic Head according to claim 1, characterized in that: The step (3) specifically comprises the following steps in order: (3a) Build a Pytorch neural network training environment with Python=3.9.0 and CUDA=10.2; (3b) Set the image input size to 640×640, the batch size to 8, the maximum number of iterations to 50, use SGD as the optimizer, the weight decay coefficient to 0.0005, the momentum to 0.937 as the default value, the initial learning rate to 0.01, and the cosine annealing algorithm for update; (3c) Input the data set in the format required for target detection into the cow face image dynamic detection and tracking model, train it and generate a weight file.

5. The method for dynamic detection and tracking of cow face images based on ViT-YOLOX-Dynamic Head according to claim 3 is characterized in that: The object detection head method adopts a dynamic head method, specifically, setting the object detection head to adopt an attention mechanism to enhance scale, spatial position and multi-task perception capabilities, and integrating these three attentions into the unified attention mechanism of the target detection head to fully extract cow facial related features; Set the object detection head to convert the attention function into three consecutive attentions, each of which focuses on only one dimension: W(F)=π C (p S (p L (F)·F)·F)·F Where F represents the original attention function, π L (·) is the scale-aware attention function, π S (·) is the spatial perception attention function π S (·), π C (·) is the task-aware attention function.

Images