Human Action Recognition in Interactive Assembly via Dual-Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing human action recognition methods in human-machine interactive assembly scenes suffer from low accuracy and slow recognition speeds, leading to inefficiencies in assembly processes.
Innovation Solution
A method involving two motion sensing devices to capture joint coordinate streams, which are then processed to normalize and smooth the data. This data is used to classify the scene as left-hand or right-hand and input into a neural network for action recognition, with outputs fused using a Softmax layer for final recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single motion sensing device is used in a simple assembly environment, then the device complexity is reduced, but the action recognition accuracy and speed deteriorate
Solution Approach 1:
The patent divides the action recognition task into multiple independent modules: motion sensing device segmentation (using two devices instead of one), data processing segmentation (joint coordinate stream screening, resampling, normalization, smoothing), scene classification segmentation (left-hand scene vs right-hand scene), and neural network segmentation (separate training for different scenes). This segmentation allows each module to be optimized independently, improving overall accuracy without requiring a single overly complex device
Solution Approach 2:
The patent transitions from two-dimensional planar recognition to three-dimensional spatial recognition by using two motion sensing devices positioned at different angles. This dimensional change enables the system to capture more comprehensive motion information and distinguish between left-hand and right-hand scenes, thereby improving recognition accuracy without proportionally increasing device complexity
2Ease of manufacture
If existing human action recognition methods are applied in simple assembly environments, then the implementation is straightforward, but the recognition speed is slow and accuracy is low
Solution Approach 1:
The patent performs preliminary actions on the joint coordinate streams before feeding them into the neural network: screening for complete skeleton joints, resampling at adjusted frequencies, normalizing coordinates, and smoothing the data. These preliminary processing steps prepare the data in advance, reducing the computational burden during real-time recognition and thereby increasing recognition speed while maintaining implementation simplicity
Solution Approach 2:
The patent dynamically adjusts key parameters including the resampling frequency based on motion characteristics, the threshold values for action detection, and the neural network input dimensions. By optimizing these parameters, the system achieves faster recognition speeds without complicating the overall implementation structure
3Device complexity
If existing methods are used without scene classification, then the processing flow is simpler, but the recognition accuracy deteriorates due to inability to distinguish left-hand and right-hand scenes
Solution Approach 1:
The patent segments the action recognition process into separate handling for left-hand scenes and right-hand scenes. By classifying scenes first and then processing them independently through dedicated neural network models, the system achieves higher accuracy without creating an overly complex unified processing flow
Solution Approach 2:
The patent applies different processing characteristics to different scenes (left-hand vs right-hand). Each scene type receives customized processing parameters and neural network inference, allowing optimization for specific local conditions rather than using a one-size-fits-all approach, thereby improving overall accuracy without proportionally increasing complexity
Data Source
AI summary
A method for human action recognition in a human-machine interactive assembly scene is disclosed in this application, joint coordinate streams of skeleton joints are obtained under a human action from motion sensing devices; a starting position and an ending position of the action are positioned according to data change based on a threshold value to obtain information of joints; resampling of angle change is made on the information of joints to obtain coordinates of joints; the coordinates of joints are normalized, to obtain a sequence of skeletons forming an action; obtaining a vector direction of the upper limb, and the scene is classified to be a left-hand scene or a right-hand scene; training is made for human action recognition in the left-hand scene and the right-hand scene respectively; human action outputs of the left-hand scene and the right-hand scene are fused to realize action recognition in a human-machine interaction scene.