Aircraft pipeline AR three-dimensional registration method and device

Abstract

The invention discloses an aircraft pipeline AR three-dimensional registration method based on deep learning and image processing. The aircraft pipeline AR three-dimensional registration method comprises the steps of performing multi-view image acquisition on a 3D model of an aircraft pipeline part; preprocessing the acquired multi-view images to form a contour image data set under multiple views; training the contour image data set through a deep learning network; preprocessing the real object scene image of the aircraft pipeline part; sending the preprocessed real object scene image into a trained deep learning network to detect an object and a frame thereof in the real object scene image; judging whether a registered object exists in the real object scene image or not; when a registered object exists in the real object scene image, determining a six-degree-of-freedom pose of the registered object under a camera coordinate system to complete registration; otherwise, outputting the information that no registered object exists in the real object scene image. The method can support augmented reality auxiliary operation guidance for aircraft pipelines, improves the registration recognition rate, and improves the visualization effect of guidance information.

Description

technical field [0001] The present invention relates to the technical field of aircraft digital assembly, and more specifically relates to an AR three-dimensional registration method for aircraft pipelines based on deep learning and image processing. Background technique [0002] The aircraft pipeline system is complex, and its assembly quality directly affects the quality of the aircraft system. The assembly cycle accounts for a large proportion of the aircraft final assembly cycle, and is a key technology in the aircraft final assembly process. In order to improve the efficiency and quality of assembly, using augmented reality (AR) technology to assist assembly workers is an effective new method. Augmented reality can superimpose virtual information on the real scene to achieve a unified visual display. Its characteristics of virtual and real fusion, good visualization effects and real-time guidance can effectively help operators understand assembly information, thereby im...

AI Technical Summary

Problems solved by technology

There are two main methods of 3D registration. One is based on deep learning. Through 3D object detection, the relative pose of the 3D object relative to the device is determined, and the coordinate relationship between the virtual space and the real space is established. The advantage is robustness. High performance, but need to prepare a large amount of data for training; the other is based on image processing method, by comparing the texture, outline and other characteristics of the 3D object in the scene under the current perspective and the prior model, to find the most similar object pose , this method is fast, but it is greatly affected by the environment and has low robustness

[0004] In summary, the existing augmented reality registration technology based on plane natural feature points has problems such as jitter, low recognition rate, and small recognition angle range when applied to AR-assisted assembly of aircraft pipelines; the AR registration technology based on deep learning has problems The training data is difficult to obtain, and the AR registration technology based on image processing has low robustness and is greatly affected by the environment.

Images