AR Spatial Localization via Ultrasound Mesh and Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current aircraft maintenance processes face challenges in accurately locating components within complex aircraft structures, particularly indoors, due to limitations of GPS systems and the need for prior knowledge about the aircraft, leading to increased time and costs.
Innovation Solution
A combination of ultrasound sensors and Augmented Reality (AR) technology, enhanced by Deep Learning, is used to provide precise location coordinates and guide technicians to specific components, utilizing a network of ultrasonic sensors and computer vision for accurate spatial reconstruction and navigation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If GPS systems are used for location, then positioning accuracy is improved, but the system fails indoors or beneath obstructions such as in an airplane cargo bay or under an aircraft fuselage
Solution Approach 1:
The system divides the positioning function into two segments: GPS for outdoor positioning and ultrasound sensors for indoor/obstructed positioning. This segmentation allows each system to operate in its optimal environment, with the ultrasound system specifically designed to work where GPS fails (indoors, under aircraft fuselage, in cargo bays).
Solution Approach 2:
The ultrasound sensor system acts as an intermediary positioning system that bridges the gap where GPS cannot operate. The system uses ultrasound waves as a mediator to transmit positioning information through environments that block electromagnetic signals, enabling continuous positioning coverage across both outdoor and indoor/obstructed spaces.
2Measurement precision
If troubleshooting procedures are followed to locate components, then component identification is improved, but the time required for maintenance tasks increases
Solution Approach 1:
The system performs preliminary action by pre-mapping the aircraft interior using ultrasound sensors to create a detailed spatial model of component locations. This preliminary spatial reconstruction allows technicians to immediately access accurate component positions without performing time-consuming troubleshooting procedures during actual maintenance tasks.
Solution Approach 2:
The system creates a digital copy or replica of the aircraft's internal spatial structure through ultrasound scanning and 3D reconstruction. This virtual model serves as a reference that technicians can query to instantly locate components, replacing the need to physically search through the aircraft structure during maintenance.
3Measurement precision
If prior knowledge about the aircraft is required to locate components, then component location accuracy is improved, but the ease of operation deteriorates
Solution Approach 1:
The system implements self-service by automatically acquiring and processing spatial data about the aircraft interior without requiring technician input. The ultrasound sensors autonomously scan the environment, reconstruct the 3D space, and identify component locations, eliminating the need for technicians to have extensive prior knowledge of aircraft structures.
Solution Approach 2:
The system replaces the mechanical/knowledge-based approach (technicians using prior knowledge to locate components) with an acoustic field-based approach (ultrasound sensors automatically detecting and mapping spatial positions). This substitution transforms the task from requiring human expertise to an automated sensing and processing system.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables rapid and precise location of maintenance components, reducing the time required for maintenance tasks and improving accuracy, even in complex indoor environments, while being tolerant to obstacles and not requiring extensive prior knowledge of the aircraft.
Implementation Method 1
The non-transitory computer-readable medium stores executable instructions that, when executed by the processor, cause the processor to: receive a set of raw acoustic signals from an array of ultrasound sensors... determine a time of flight for each acoustic signal... calculate a set of three-dimensional coordinates of a user in the environment based on the set of time of flights
Data Source
AI summary
Methods and system for locating an event on an aircraft using augmented-reality content, an array of ultrasonic devices configured in mesh topology, and deep learning.