Adaptive Gaze-Enabled Mockup Generation via Eye Tracking Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The current user experience (UX) development process for adding new features to complex enterprise-level computer systems is time-consuming and resource-intensive, requiring multiple stages and iterations before a prototype can be generated and validated, which can consume many man-hours and resources.
Innovation Solution
An adaptive gaze-enabled intelligent system (AGEIUXM) uses a generative adversarial network (GAN) for automated user experience mockup generation, where a user experience theme description and image set are input to generate multiple designs, with eye gaze tracking to identify and compare user interface elements, providing feedback to enhance the design process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated mockup generation using GAN is implemented, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent introduces an intermediary system comprising a GAN-based automated mockup generation module and an eye-tracking integration layer that mediates between the design requirements and the final mockup output. This intermediary architecture enables rapid automated generation while managing system complexity through modular design, where the GAN model serves as a specialized component handling the complex generative tasks separately from the main UX design workflow.
2Measurement precision
If eye gaze tracking is integrated to identify UI elements, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The system employs self-service mechanisms where the eye-tracking technology automatically captures and processes user gaze data without requiring manual intervention. The gaze tracking module autonomously identifies which UI elements users are looking at, and the system automatically compares these elements across different mockup designs, eliminating the need for manual element identification and reducing operational complexity despite the advanced tracking capabilities.
Solution Approach 2:
The patent implements a feedback loop where eye-tracking data from user interactions with generated mockups is fed back into the GAN system. This feedback mechanism allows the system to learn from actual user behavior patterns and continuously improve its mockup generation accuracy. The feedback process automates the refinement of design preferences, enhancing measurement precision while managing complexity through iterative learning rather than manual adjustment.
Data Source
AI summary
A user experience theme description is obtained, along with a new user experience feature image set. The theme description and new user experience feature image set are input into a generative adversarial network (GAN). The GAN is trained to output multiple user experience designs based on the new feature image set. The multiple designs are displayed on an electronic display device that includes an eye gaze tracking system. User interface elements and their corresponding positions within a user interface are identified based on eye gaze of a user towards the electronic display device. The position and type of user interface elements are compared between a desired user interface design and a generated user interface design. Errors between the desired user interface design and the generated user interface design are input as feedback into the GAN to further enhance the results.