ToF and Radar Sensor Fusion for 3D Human Tracking Near Robots
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated manufacturing systems struggle with transforming 2D sensor data into real 3D information about objects or obstacles, leading to unnecessary machine stops and reduced productivity, while also being costly.
Innovation Solution
A fusion system combining data from multiple sensors, including time-of-flight (ToF) sensors and radar sensors, to provide a high-resolution 3D perception of the environment, enabling the detection and tracking of human activities and obstacles in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 2D sensor data is used for obstacle detection, then the system complexity is reduced, but the measurement precision and reliability of 3D obstacle detection deteriorates
Solution Approach 1:
The patent transforms 2D sensor data into 3D spatial information by introducing depth estimation through stereo vision or time-of-flight measurements. This allows the system to achieve accurate 3D obstacle detection without requiring complex 3D sensor arrays, effectively adding a depth dimension to the detection capability while maintaining system simplicity.
Solution Approach 2:
The patent introduces machine learning algorithms as an intermediary that processes 2D sensor data and infers 3D obstacle characteristics. The ML model acts as a mediator between the simple 2D sensors and the complex 3D detection requirement, extracting depth and spatial information from 2D images through learned patterns.
2Device complexity
If conventional 2D sensor systems are used, then the device cost is reduced, but the productivity deteriorates due to unnecessary machine stops
Solution Approach 1:
The patent replaces complex mechanical 3D scanning systems with computationally intensive 2D sensor processing enhanced by machine learning. Instead of using expensive 3D laser scanners or multiple cameras, the system uses standard 2D sensors with ML-based depth estimation, substituting hardware complexity with software intelligence to maintain productivity.
Solution Approach 2:
The patent changes the processing parameters from direct 3D spatial measurement to 2D image feature analysis with inferred depth. By transforming the detection paradigm from measuring 3D coordinates directly to estimating depth from 2D patterns, the system achieves accurate obstacle detection at lower cost while avoiding false stops that reduce productivity.
3Speed
If direct sensor data triggering is used, then the response speed is improved, but the reliability of collision avoidance deteriorates due to false positives
Solution Approach 1:
The patent performs preliminary classification and filtering of sensor data using machine learning before triggering safety responses. The ML model pre-processes the raw sensor input to distinguish between actual obstacles and false positives (such as reflective surfaces or transient objects), ensuring that only genuine threats trigger machine stops and maintaining both fast response and high reliability.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system effectively reduces unnecessary machine stops by accurately detecting and classifying obstacles in 3D space, improving productivity and safety while being more cost-effective than traditional 3D scanner systems.
Implementation Method 1
combining one or more two-dimensional images obtained from a time-of-flight (ToF) sensor
Implementation Method 2
combining one or more two-dimensional images obtained from a time-of-flight (ToF) sensor with contemporaneously obtained data from a radar sensor
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A method of detecting and tracking human activities in the vicinity of a robot comprises the step of combining one or more two-dimensional images obtained from a time-of-flight (ToF) sensor with contemporaneously obtained data from a radar sensor, to obtain fused sensor data, and further comprises detecting the presence of a human in the vicinity of the robot, based on the fused sensor data, and estimating direction of motion and speed of motion of the human, based on the fused sensor data. In some embodiments, the detecting and estimating are performed using a machine-learning model, the machine-learning model having been trained using two-dimensional ToF images and radar sensor data representative of an environment for the robot.