Multi-scale Graph Convolutional Network for Human-Robot Collaboration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current human-robot interaction methods rely heavily on pre-programming, limiting flexibility and ability to perform complex tasks in dynamic environments, necessitating a more adaptive approach for robots to collaborate effectively with humans.
Innovation Solution
A human-robot collaboration method utilizing a multi-scale graph convolutional neural network for human behavior recognition, which involves data acquisition, model training, and human-robot interaction, enabling robots to adapt to changing situations by predicting human behaviors and making corresponding action plans.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If robots rely on pre-programming to perform tasks, then task execution reliability is improved, but flexibility and adaptability to complex environments deteriorate
Solution Approach 1:
The system continuously captures human skeleton data through RGB cameras and D435i depth cameras, processes this visual feedback through a multi-scale graph convolutional neural network to recognize human behaviors, and adjusts robot actions in real-time based on recognized human intentions, creating a closed-loop feedback system that enables adaptive collaboration
Solution Approach 2:
The patent replaces traditional mechanical pre-programming control with an intelligent vision-based recognition system using deep learning models. The multi-scale graph convolutional neural network substitutes rigid if-then programming logic with adaptive pattern recognition, allowing the robot to understand human behaviors through visual data rather than following predetermined instructions
2Device complexity
If robots use pre-programming for task execution, then system complexity is reduced, but ability to perform complex human-like actions deteriorates
Solution Approach 1:
The system segments human behavior recognition into multiple specialized modules: RGB-D data acquisition module, skeleton extraction module, multi-scale graph convolutional neural network for spatial feature extraction, temporal feature extraction module, and behavior recognition module. Each module handles a specific aspect of the complex recognition task, distributing complexity across modular components
Solution Approach 2:
The patent introduces an intermediary multi-scale graph convolutional neural network that mediates between raw visual data and robot control decisions. This intermediary layer translates complex human skeletal movements into interpretable behavior patterns, bridging the gap between visual input and robotic action without requiring direct complex programming
3Measurement precision
If multi-scale graph convolutional neural network is used for human behavior recognition, then recognition accuracy is improved, but computational complexity and processing time increase
Solution Approach 1:
The neural network is segmented into distinct functional layers: multi-scale spatial graph convolutional layers for extracting spatial features from skeleton data, temporal convolutional layers for extracting time sequence features, and a behavior recognition output layer. This segmentation allows parallel processing of spatial and temporal features, reducing overall computational complexity while maintaining high accuracy
Solution Approach 2:
The system performs preliminary skeleton extraction from RGB-D data before feeding it to the complex neural network. By pre-processing visual data into standardized skeleton representations with defined joint coordinates and relationships, the system reduces the complexity of input data to the graph convolutional network, enabling more efficient processing
Data Source
AI summary
The present invention discloses a human-robot collaboration method based on a multi-scale graph convolutional neural network. The method includes the following steps: S1, data acquisition: acquiring a dataset of a human skeleton in human-robot collaboration scenes, and performing pre-processing to obtain pre-processed data; S2, model training: loading the pre-processed data, and obtaining a human behavior recognition network model by training a multi-scale graph convolutional neural network; S3, human behavior recognition: predicting human behaviors through a trained deep learning network model; and S4, human-robot interaction: sending predicted information to a robot system through a communication algorithm, and enabling a robot to make action plans based on the human behaviors. By the human-robot collaboration method based on a multi-scale graph convolutional neural network disclosed by the present invention, a robot can predict human behaviors and intents in real scenes and make correct interaction.