Self-Attention Spatiotemporal Feature Fusion for Piglet Counting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for analyzing the number of piglets with suckling behavior are labor-intensive, prone to manual errors, and difficult to implement in large-scale breeding environments due to challenges in detecting individuals in crowded and complex lighting conditions.
Innovation Solution
A method utilizing self-attention spatiotemporal feature fusion, which involves collecting video data, detecting key frames and optical flow using CenterNet and FlowNet networks, extracting spatial and temporal features, and fusing them through transformers to generate a self-attention spatiotemporal feature map for accurate counting of suckling piglets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual observation is used to count suckling piglets, then the nutrition intake and health status can be judged, but it is time-consuming and labor-intensive
Solution Approach 1:
The patent replaces manual mechanical observation with an automated computer vision system. The system uses video采集 to capture piglet behavior, deep learning models (CenterNet for detection, ResNet for feature extraction) to automatically analyze suckling behavior, and algorithms to count piglets. This substitution eliminates manual labor while maintaining accurate judgment of nutrition intake and health status through continuous automated monitoring of suckling behaviors.
2Measurement precision
If individual piglet detection is performed in crowded conditions, then accurate counting can be achieved, but it is difficult due to complex illumination and blocking
Solution Approach 1:
The patent extracts and focuses on specific detectable features of suckling piglets that are visible even in crowded conditions. The system extracts key visual features such as piglet position relative to the sow, movement patterns characteristic of suckling behavior, and temporal activity patterns. By focusing on these extracted features rather than requiring complete individual identification, the system achieves accurate counting despite complex illumination and blocking challenges.
Solution Approach 2:
The patent transitions from spatial-only detection to spatiotemporal detection by incorporating temporal dimension. The system analyzes video sequences to detect suckling behaviors through temporal patterns of movement and position changes. This spatiotemporal approach allows the system to distinguish suckling piglets from non-suckling ones even when spatially crowded, by detecting characteristic temporal patterns of suckling activity that are unique to each event.
3Productivity
If deep learning network density estimation is used to predict the number of piglets, then direct quantity prediction can be achieved, but individual detection remains challenging
Solution Approach 1:
The patent merges two complementary approaches: individual detection methods (CenterNet for detecting piglet positions and ResNet for extracting features) with density estimation methods. The system first uses CenterNet to detect individual piglet positions and behaviors in the video, then applies density estimation algorithms to aggregate these detections and predict the total number of suckling piglets. This combination leverages the strengths of both approaches - individual detection provides accurate behavioral identification while density estimation efficiently handles the counting task in crowded scenarios.
Data Source
AI summary
A method for counting suckling piglets based on self-attention spatiotemporal feature fusion is disclosed, which includes: detecting a side-lying sow in a video frame by using CenterNet to acquire a key frame of suckling piglets and a region of interest of the video frame, and overcome the interference of the movement of non-suckling piglets on the spatiotemporal feature extraction for the region of interest; transforming spatiotemporal features extracted by a spatiotemporal two-stream convolutional network from a key frame video clip into a spatiotemporal feature vector, inputting the obtained spatiotemporal feature vector into a temporal, a spatial and a fusion transformer to obtain a self-attention matrix; performing element-wise product for the self-attention matrix and the fused spatiotemporal features to obtain a self-attention spatiotemporal feature map; and inputting the self-attention spatiotemporal feature map into a regression branch of the number of suckling piglets to complete the counting of the suckling piglets.