A Virtual Maintenance Simulation Method Based on Depth Sensing Gesture Recognition

Abstract

The invention relates to a virtual maintenance simulation method based on depth perception gesture recognition. The invention includes: 1) hardware environment construction; 2) data preparation; 3) function layer realization; 4) interaction layer realization. This method adopts a non-wearable depth-sensing gesture recognition interaction device, so that users can perform natural, intuitive, and high-precision gesture interactions that conform to the natural usage habits of the human body, and can be used by more people without distinction. In the aircraft design stage, the present invention is used to perform maintenance simulation of each cabin of the aircraft, and the design digital model can be directly simulated in a human-in-the-loop mode in a virtual reality environment, which greatly simplifies the maintenance simulation process and improves efficiency; At the same time, by defining the operation steps and operation gestures, the problems of large interaction randomness and low interaction precision in the human-in-the-loop real-time maintainability simulation are solved, and the practicability and accuracy of the real-time human-in-the-loop maintainability simulation are improved.

Description

technical field [0001] The invention belongs to the technical field of aircraft maintenance and relates to a virtual maintainability simulation method based on depth perception gesture recognition. Background technique [0002] In the field of aircraft design, the application of aircraft digital design technology and VR virtual reality technology is becoming more and more mature, and it is being applied and developed in a deeper and wider range in the field of virtual maintainability simulation in the design stage. Virtual maintainability simulation is the simulation verification and analysis process of the maintenance process and maintenance action performed by the designer on the digital prototype model in the design process, and the visibility, accessibility, maintenance access, etc. of the maintenance object are verified, and the aircraft is optimized through continuous iteration. The overall design and maintenance plan of the product. In the human-in-the-loop simulatio...

AI Technical Summary

Problems solved by technology

At present, the typical interactive device in the multi-channel immersive virtual reality environment is the interactive handle Flystick of the ART optical tracking system. Although the immersive reality is better, the software and hardware are expensive, and the interactive function of the Flystick handle is relatively simple; the desktop interactive device is usually Keyboard / mouse, it is difficult to achieve intuitive three-dimensional interaction; the helmet-style interactive device is usually a handle used with the helmet, and a wearable motion capture kit can also be bound to a natural person, but the intervention of external devices makes human gesture recognition The accuracy and stability of the device are improved, but it conceals the natural expression of the gesture. At the same time, the wearing position of the wearable device is also random and does not fit all body types, resulting in poor matching accuracy between the natural human hand and the virtual human hand, and the movements are out of shape and deformed. shortcoming

At the same time, human biological characteristics determine that the trajectory of the hand in the interactive space cannot be standardized, and the gestures and movements of the hand are generated in real time during virtual maintenance simulation or training, and may collide with the maintenance model at any position and angle. Interaction, many existing human-computer interaction technologies are difficult to effectively understand gesture input intentions, and determine whether gestures meet maintenance operation requirements to complete the interaction

Images