Neural Network Device for Mobile Skeleton Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for estimating 2D joint coordinates and skeletons in images using deep learning are inefficient on mobile devices due to the need for additional object detection and post-processing by the CPU, leading to slow performance.
Innovation Solution
A method involving a neural network device that extracts feature data from images, uses pre-trained models to generate two-dimensional vector information through probability-based operations, and applies these models in a way that offloads CPU-intensive tasks to the GPU, improving processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the top-down method is used to detect human objects and infer joint information, then joint information accuracy is improved, but processing time increases when the number of human objects increases
Solution Approach 1:
The patent segments the image into multiple regions of interest (ROIs) based on detected human objects, and processes each ROI independently through the deep learning network. This segmentation allows parallel processing of multiple objects, reducing overall processing time while maintaining accurate joint information extraction for each object.
Solution Approach 2:
The patent performs preliminary object detection to identify human objects and their regions of interest before feeding the data to the deep learning network. This preliminary action filters out non-human areas, allowing the network to focus computational resources only on relevant regions, thereby improving efficiency without sacrificing accuracy.
2Ease of operation
If the conventional method uses CPU for post-processing skeletal information, then flexibility is maintained, but processing speed decreases significantly
Solution Approach 1:
The patent replaces the CPU-based mechanical processing system with a GPU-based parallel processing system. The GPU's architecture is specifically designed for parallel computations, allowing it to handle the post-processing of skeletal information much faster than the CPU while maintaining the same operational flexibility through software abstraction layers.
Solution Approach 2:
The patent changes the processing architecture parameter from CPU to GPU, fundamentally altering how the post-processing is performed. This parameter change enables utilizing parallel computing capabilities, transforming the processing speed from sequential CPU operations to parallel GPU operations, thereby significantly improving processing speed.
3Extent of automation
If the deep learning network performs all processing tasks, then integration is improved, but computational resource consumption increases
Solution Approach 1:
The patent segments processing tasks into two distinct phases: a deep learning network phase for extracting high-level features and a separate post-processing phase for computing skeletal information. This segmentation allows each phase to be optimized independently, reducing the computational burden on the deep learning network while maintaining high integration through automated data flow between phases.
Solution Approach 2:
The patent extracts the post-processing function from the deep learning network, separating the feature extraction task from the skeletal computation task. This extraction allows the deep learning network to focus on its core strength of feature extraction, reducing its computational resource consumption, while the extracted post-processing can be performed efficiently using GPU parallel computing.
Data Source
AI summary
A method of operating a neural network device for extracting result information using machine learning according to an embodiment of the present disclosure includes: extracting feature data from an image frame; storing a pre-trained first training model generated by performing machine learning on the feature data and including first training data; storing a pre-trained second training model that is generated by performing machine learning on the first training data and includes second training data generated according to the four arithmetic operations based on the first training data; and generating two-dimensional vector information on the result information by performing a probability-based operation on the second training data in the image frame based on the second training model.