3D Tracking and Machine Learning for Social Behavior Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated systems for behavioral scoring in animals are limited to single animal assays, are labor-intensive, and lack accuracy, hindering the understanding of social behaviors and their disorders such as autism, due to reliance on manual scoring and limited tracking capabilities.
Innovation Solution
A system combining 3D tracking and machine learning using video cameras and depth sensors to automatically detect and quantify social behaviors in multiple subjects, enabling high-throughput analysis and classification of behaviors like attack, grooming, and social feeding, and allowing for the study of behavioral phenotypes and genotypes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual scoring of social behaviors is used, then behavioral analysis can be performed, but it is slow, labor-intensive, and subjective
Solution Approach 1:
The patent replaces manual mechanical observation and scoring with an automated computer vision system that uses video cameras, depth sensors, and machine learning algorithms to detect and classify social behaviors, eliminating human labor while maintaining or improving measurement precision
Solution Approach 2:
The system enables self-service behavioral analysis by automatically capturing video data, processing it through pose estimation algorithms, and generating behavioral classifications without requiring human observers, thereby dramatically increasing throughput while preserving accuracy
2Productivity
If automated systems for behavioral scoring are used, then productivity increases, but they are limited to single animal assays and simple tracking
Solution Approach 1:
The patent creates a universal behavioral analysis system that can detect multiple types of social behaviors (grooming, attacking, following, etc.) across different animal species and experimental conditions using the same hardware and software platform, enabling high-throughput analysis without sacrificing versatility
Solution Approach 2:
The system transitions from 2D video tracking to 3D pose estimation by incorporating depth sensor data, enabling accurate detection of social behaviors that occur in three-dimensional space and improving the system's ability to handle complex multi-animal interactions
3Measurement precision
If 3D tracking with depth sensors is implemented, then behavioral detection accuracy improves, but device complexity increases
Solution Approach 1:
The patent merges video camera and depth sensor data into a unified processing pipeline, combining RGB and depth information to achieve accurate 3D pose estimation while managing system complexity through integrated hardware and software architecture
Data Source
AI summary
Systems and methods for performing behavioral detection using three-dimensional tracking and machine learning in accordance with various embodiments of the invention are disclosed. One embodiment of the invention involves a the classification application that directs a microprocessor to: identify at least a primary subject interacting with a secondary subject within a sequence of frames of image data including depth information; determine poses of the subjects; extract a set of parameters describing the poses and movement of at least the primary and secondary subjects; and detect a social behavior performed by at least the primary subject and involving at least the second subject using a classifier trained to discriminate between a plurality of social behaviors based upon the set of parameters describing poses and movement.