Virtual maintainability simulation method based on depth perception gesture recognition

Abstract

The invention relates to a virtual maintainability simulation method based on depth perception gesture recognition. The method comprises the following steps: 1) building of a hardware environment; 2)data preparation; 3) realization of a functional layer; and 4) realization of an interactive layer. The method adopts non-wearable deep perception gesture recognition interaction equipment, so that auser can perform natural, visual and high-precision gesture interaction according with natural use habits of a human body, and the method can be used for more people without difference. In the aircraft design stage, maintainability simulation is conducted on all shipping spaces of an aircraft through the method, simulation operation can be directly conducted on a design digital model in a human-in-loop mode in the virtual reality environment, the maintainability simulation process is greatly simplified, and efficiency is improved; and by defining operation steps and operation gestures, the problems of high interaction randomness and low interaction precision of real-time maintainability simulation of the human in the loop are solved, and the practicability and accuracy of the real-time maintainability simulation of the human in the loop are improved.

Description

technical field [0001] The invention belongs to the technical field of aircraft maintenance and relates to a virtual maintainability simulation method based on depth perception gesture recognition. Background technique [0002] In the field of aircraft design, the application of aircraft digital design technology and VR virtual reality technology is becoming more and more mature, and it is being applied and developed in a deeper and wider range in the field of virtual maintainability simulation in the design stage. Virtual maintainability simulation is the simulation verification and analysis process of the maintenance process and maintenance action performed by the designer on the digital prototype model in the design process, and the visibility, accessibility, maintenance access, etc. of the maintenance object are verified, and the aircraft is optimized through continuous iteration. The overall design and maintenance plan of the product. In the human-in-the-loop simulatio...

AI Technical Summary

Problems solved by technology

At present, the typical interactive device in the multi-channel immersive virtual reality environment is the interactive handle Flystick of the ART optical tracking system. Although the immersive reality is better, the software and hardware are expensive, and the interactive function of the Flystick handle is relatively simple; the desktop interactive device is usually Keyboard / mouse, it is difficult to achieve intuitive three-dimensional interaction; the helmet-style interactive device is usually a handle used with the helmet, and a wearable motion capture kit can also be bound to a natural person, but the intervention of external devices makes human gesture recognition The accuracy and stability of the device are improved, but it conceals the natural expression of the gesture. At the same time, the wearing position of the wearable device is also random and does not fit all body types, resulting in poor matching accuracy between the natural human hand and the virtual human hand, and the movements are out of shape and deformed. shortcoming

At the same time, human biological characteristics determine that the trajectory of the hand in the interactive space cannot be standardized, and the gestures and movements of the hand are generated in real time during virtual maintenance simulation or training, and may collide with the maintenance model at any position and angle. Interaction, many existing human-computer interaction technologies are difficult to effectively understand gesture input intentions, and determine whether gestures meet maintenance operation requirements to complete the interaction

Images