Weapon-to-Goggle Video Registration with Automatic IMU Recalibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges with relative orientation errors due to bias drift in low-cost IMU sensors, requiring frequent recalibration that disrupts system operation and limiting the types of information provided by heads-up displays.
Innovation Solution
A system that uses two video sources, one mounted on a rifle and one in goggles, with sensors providing spatial orientation data to a computer, which determines and confirms the relative location of the images using sensor data and image comparisons, allowing for automatic calibration and overlaying of images without manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If low-cost IMU sensors are used for determining relative orientation, then device cost is reduced, but measurement precision deteriorates due to bias drift over time
Solution Approach 1:
The system continuously compares the IMU-based location with the image-comparison-based location and uses this feedback to adjust and recalibrate the IMU sensor readings automatically. This closed-loop feedback mechanism compensates for bias drift without requiring manual recalibration, maintaining measurement precision while using low-cost sensors.
Solution Approach 2:
The system performs automatic recalibration by comparing video images from both sources and using image comparison algorithms to determine accurate relative orientation. This self-service mechanism eliminates the need for manual user intervention to recalibrate the IMU sensors, allowing the system to maintain precision autonomously.
2Measurement precision
If manual recalibration is performed frequently to correct IMU bias drift, then measurement precision is maintained, but productivity deteriorates due to system operation disruption
Solution Approach 1:
The system automatically performs recalibration by comparing video images and computing corrections without requiring manual user intervention. This self-service approach maintains measurement precision while eliminating the need to stop system operation for recalibration, thus preserving productivity.
Solution Approach 2:
The system continuously compares images from both video sources and continuously adjusts IMU readings based on this comparison. This continuous operation ensures that recalibration happens seamlessly in the background without interrupting the useful action of providing accurate orientation data.
3Measurement precision
If image comparison methods are used to verify sensor-based location, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The system merges two different methods of determining location - IMU sensor data and image comparison - into a unified approach. By combining these methods, the system achieves higher measurement precision through mutual verification while managing complexity through integrated processing.
Solution Approach 2:
The computer acts as an intermediary that receives data from both the IMU sensors and the video sources, processes this information, and produces the final calibrated location data. This intermediary processing role manages the complexity by centralizing the integration logic.
4Ease of operation
If traditional HUD displays are used, then ease of operation is maintained, but loss of information increases due to limited types of data displayed
Solution Approach 1:
The HUD system is enhanced to display multiple types of information simultaneously - not only basic navigational data but also tactical information, object icons, and augmented reality overlays. This multi-functionality approach provides comprehensive information while maintaining the simple heads-up display interface that does not require user attention diversion.
Data Source
AI summary
Video sources and inertial sensors are attached to a weapon and to goggles. A computer receives video images from the weapon- and goggles-mounted sources and inertial data from the sensors. The computer calculates a location for an image from the weapon-mounted source within an image from the goggles-mounted source using the inertial sensor data. The sensor-based location is checked (and possibly adjusted) based on a comparison of the images. A database contains information about real-world objects in a field of view of the goggles-mounted source, and is used to generate icons or other graphics concerning such objects.