3D Work Task Evaluation Using Point Clouds and Action Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing manufacturing processes rely on paper specifications that are difficult to follow intuitively, leading to unskilled operations, reduced efficiency, and lower product quality due to the complexity of tasks and high skill requirements.
Innovation Solution
A method and system that utilizes point cloud data and three-dimensional image frames to recognize workpiece attributes and worker actions, comparing them to standard operating procedures for comprehensive evaluation, with features like alarm and reminder messages for deviations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If paper specifications are used to instruct workers, then the operation process can be documented, but the specifications cannot intuitively instruct workers and are difficult to follow
Solution Approach 1:
The patent creates a visual copy of the standard operating procedure in the form of a three-dimensional avatar that replicates the worker's standard actions. This visual copy allows workers to intuitively understand and follow procedures without relying on text-based paper specifications, directly resolving the contradiction between documentation and intuitiveness.
Solution Approach 2:
The system uses color changes in the avatar's visual feedback to indicate compliance status. Different colors represent different levels of compliance with standard procedures, providing intuitive visual information that replaces complex textual instructions and improves both ease of operation and information transmission.
2Measurement precision
If comprehensive evaluation of work tasks is implemented, then evaluation accuracy is improved, but system complexity increases
Solution Approach 1:
The patent employs a multi-functional avatar system that simultaneously performs multiple functions: providing visual work instructions, monitoring worker actions, evaluating compliance, and giving real-time feedback. This consolidation of multiple evaluation functions into a single integrated system achieves comprehensive evaluation accuracy without proportionally increasing system complexity.
Solution Approach 2:
The system automatically performs comprehensive evaluation by comparing captured worker actions against standard procedures stored in the avatar, eliminating the need for manual inspection. The avatar self-compares and self-evaluates, achieving high measurement precision through automated multi-dimensional assessment while keeping operational complexity manageable.
3Reliability
If real-time monitoring of worker actions is implemented, then compliance with standard procedures is improved, but loss of time increases
Solution Approach 1:
The avatar system operates continuously in the background, constantly capturing and comparing worker actions against standard procedures without interrupting the work process. This continuous automated monitoring ensures compliance while minimizing time loss, as the evaluation occurs concurrently with the work rather than as a separate step.
Solution Approach 2:
The system provides real-time feedback to workers through the avatar's visual responses, enabling immediate correction of non-compliant actions. This feedback mechanism maintains high reliability by ensuring standards are met while reducing time loss through immediate rather than delayed correction, allowing workers to adjust their actions on the fly without stopping the workflow.
Data Source
AI summary
Various embodiments of the teachings herein include a method for evaluating a work task. An example includes: acquiring point cloud data of a workpiece in a work task and a three-dimensional image frame of a worker in the task; recognizing attribute information of the workpiece in a machine vision manner based on the data; recognizing action information of the worker in a deep learning manner based on the image frame; comparing combination information of the attribute information and the action information with a standard operating procedure of the work task, wherein the standard operating procedure comprising a standard attribute of the workpiece and a standard action of the worker in the work task; and evaluating the work task based on a comparing result.