Mixed Reality Learning Interface with Computer Vision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for tangible interfaces in learning are limited by high costs and lack pedagogical depth, often relying on pure exploration rather than providing a structured learning experience, while also failing to combine effectively the benefits of digital technologies and physical environments.
Innovation Solution
A system and method that integrates physical objects with sensors and a computer to create a mixed reality environment, allowing users to manipulate physical objects and receive guided feedback through virtual representations, promoting hands-on learning and collaboration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If embedded sensors are placed in physical objects to create tangible interfaces, then the system can detect and respond to physical manipulations, but the cost and device complexity increase significantly
Solution Approach 1:
The patent uses computer vision to create a virtual copy of the physical block tower, eliminating the need for embedded sensors in each block. The system captures images of the physical blocks and generates a digital representation that can be manipulated and observed in virtual space, resolving the contradiction between tangible interaction and system complexity
Solution Approach 2:
The patent introduces a camera and computer vision algorithm as an intermediary between the physical blocks and the digital system. This mediator captures visual information about the physical tower and translates it into virtual representations, enabling tangible interaction without requiring sensors embedded in each block
2Ease of operation
If physical objects are manipulated in pre-defined spaces with embedded sensors, then tangible interaction is enabled, but the adaptability and ease of setup are reduced
Solution Approach 1:
The patent employs dynamic computer vision algorithms that can adapt to different block configurations and table positions in real-time. The system continuously captures images and updates the virtual representation accordingly, allowing flexible setup without pre-defined fixed spaces or calibrated sensor positions
3Adaptability or versatility
If pure exploration-based systems are used for tangible learning, then children can freely manipulate objects, but pedagogical depth and structured learning are insufficient
Solution Approach 1:
The patent implements a feedback loop where the system captures images of the physical block tower, generates a virtual representation, and displays it on a screen. This provides children with immediate visual feedback about their physical manipulations, enabling both free exploration and structured learning by showing the consequences of their actions in a controllable virtual environment
Solution Approach 2:
The patent adds a virtual digital dimension to the physical manipulation activity. By projecting the physical block tower into a 2D virtual representation on a screen, the system enables children to observe and learn from their physical actions in an enhanced visual format that supports both exploration and structured pedagogy
Data Source
AI summary
The present invention relates generally to a system and method that bring together the advantages of computer games and the physical world to increase engagement, collaboration and learning. The system and method can be used with a myriad of physical setups and can be used for many different content areas in education. In one embodiment, a mixed reality interaction is facilitated with an EarthShake™ game presented on a display. The game is synchronized with a tangible interface comprising a physical object and a sensor capable of detecting a change in the condition of the physical object. The system and method help kids discover scientific and other learning principles while experimenting with real objects in a physical environment supported with audio and visual feedback. Students interactively make predictions, see results, grapple with disconfirming evidence and formulate explanations in forms of general principles.