AR Manual Feedback Using Operation-to-Image Correspondence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) manuals often contain deficiencies such as omissions or mistakes in work procedures, leading to failed operations due to users being unable to grasp these deficiencies when only notified of failure, making it difficult to correct the manuals.
Innovation Solution
An information processing system that acquires operation records from a target device and images user operations, displaying correspondence information on a display unit to highlight discrepancies between the manual's instructions and actual user actions, suggesting corrections and additional operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If only a notification that the work has failed is given from the target device, then the device can indicate error occurrence, but the user cannot grasp the deficiency of the AR manual and it is difficult to correct the manual
Solution Approach 1:
The system implements a feedback mechanism where the target device transmits operation records to the information processing device. This feedback loop enables the user to see the discrepancy between the AR manual instructions and actual device operations, allowing identification of manual deficiencies. The feedback includes detailed operation records that show what operations were actually performed versus what the manual described.
Solution Approach 2:
An information processing device acts as an intermediary between the target device and the user. This intermediary receives operation records from the target device, processes the information to identify non-correspondence portions, and presents this information to the user in a comprehensible format. The intermediary translates raw operation data into meaningful feedback about AR manual deficiencies.
2Ease of operation
If correspondence information between operation records and user operations is displayed, then users can easily grasp AR manual deficiencies, but the system requires acquiring and processing multiple types of data
Solution Approach 1:
The information processing device performs multiple functions: acquiring operation records from the target device, acquiring images of user operations, comparing these data sources to identify non-correspondence portions, and displaying the correspondence information. This multi-functional approach consolidates complex data processing capabilities into a single device that can handle various data types and processing tasks.
Solution Approach 2:
The system automatically acquires operation records and images, performs the comparison analysis, and generates the correspondence information without requiring manual intervention. The target device self-records its operations, and the information processing device automatically processes and compares the data, reducing the burden on the user while providing comprehensive feedback.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An information processing system includes one or plural processors configured to: acquire, for work performed by a user according to an AR manual indicating a work procedure for a target device, a record of an operation received by the target device in a period in which the work is performed; acquire image data in which a state of an operation performed by the user on the target device is imaged in the period in which the work is performed; and display correspondence information related to a correspondence between the operation received by the target device in the record and the operation performed by the user on the target device in the image data on a display unit.