Augmented reality image projection system

Abstract

The present disclosure relates to an augmented reality (AR) image projection system capable of simultaneously capturing a real image and projecting an AR image, and capable of applying same to a subject requiring precise work or in an environment having spatial restrictions. The present disclosure includes a real image capture device for capturing a real image illuminated by invisible rays without interference with visible rays, and an AR image projection device for projecting an AR image, wherein the real image capture device captures a real image and the AR image projection device projects an AR image through the same optical axis. Further, the present disclosure includes a real image capture device, an AR image projection device, as well as additional imaging devices such as a fluorescent camera configured integrally along the same optical axis, and configured to avoid wavelength interference between a light source and an excitation light and the like.

Description

CROSS-REFERENCE TO RELATED APPLICATION[0001]The present application is the US national phase of International Patent Application No. PCT / KR2016 / 001049, filed Feb. 1, 2016, which claims priority to Korean Patent Application No. 10-2015-0032371, filed on Mar. 9, 2015. The disclosures of the above-listed applications are hereby incorporated by reference herein in their entirety.TECHNICAL FIELD[0002]The present disclosure in some embodiments relates to a technology for projecting an augmented reality (AR) image. More particularly, the present disclosure relates to an augmented reality projection system which is capable of simultaneously capturing the real image and projecting the augmented reality image, and is applicable to geometrically restricted spaces and / or to an object subject to a precision operation.BACKGROUND[0003]In general, augmented reality is a technology for showing computer generated graphics (virtual images) overlapped with a real image to make the viewer feel as if the...

AI Technical Summary

Benefits of technology

[0032]The augmented reality image projection system according to some embodiments can provide a simple and compact system by making both the augmented reality image projection device and the real image capturing device to share the same optical axis. It eliminates or minimizes the use of an additional alignment device for aligning the augmented reality image to the subject or alignment device for aligning the computer-generated virtual image to the real image.

[0033]Further, multiple real-time imaging can be added as the projection device and the imaging device are located on the same optical axis, making it free from spatial restrictions.

[0034]Further, as the system takes a real image with the invisible light source which has no interference with visible light, the projected image on the subject may have high contrast. It further enables integration of a plurality of fluorescent imaging devices in one system so that a variety of augmented reality images can be provided.

[0035]Further, the augmented reality image projection system according to at least one embodiment can be built in a compact form so that it minimizes blocked line of sight or intervened movement of the operator, providing easier manipulation in various surgical operations.

[0036]Further, the augmented reality image projection system according to at least one embodiment can be built in a compact form so that it can be used in a narrow and confined operational site, or in a tight space such as an abdominal cavity when integrated directly into an eyepiece of a laparoscope, cystoscope, or thoracoscope.

Problems solved by technology

However, an HMD or a semi-translucent screen is in between the operation site and the operator's eyes so it is prone to be contaminated and may cause a sense of visual discord by interfering with the line of sight partially.

However, a conventional augmented reality image projection system, which is a side-by-side arrangement of two dedicated units of a projection device and a real image capture device, has following issues.

First, it involves a complicated process of aligning the projection onto the subject.

The lack of unity between the projection device for projecting the AR image and the real image capture device which serves as an alignment reference results in their discrepancies in direction and distance.

Such projection system may still have discomfort for the viewer as there may exist a time delay in the projected image.

In addition, as the subject is three-dimensional having a certain thickness, the distance of the subject from the projection system differs portion by portion, which requires an additional calculation time for alignment even if a stereoscopic camera is in use.

Second, a size of the system is hard to be reduced in compact form if each device is configured independently.

As the separately installed camera and projection device may occupy a large space, 1) it may block the line of sight of the observer who needs to see an AR image projected onto the subject, which limits the usability of the system.

Size reduction of the AR image projection system is not feasible in typical projection system having separate devices for projecting and imaging including positional information.

Third, it is technically difficult to implement additional real-time imaging device such as a fluorescence imaging device.

Additional imaging device requires additional trackers or complicated calculation for image alignment.

Physical realization is hard as additional devices are necessary according to the additional number of images.

It becomes very complicated system occupying a lot of space.

Fourth, interference between the projection image and the illumination of the subject diminishes the contrast of projected AR image.

Additional fluorescence image further makes it difficult to configure the system as it interferes with the excitation light source.

In this case, increasing the intensity of the illumination to improve the captured image quality 1) makes the projected image less visible and 2) results in an adverse effect on alignment as the projection image is taken together with the real image.

Such device includes a complicated method using time-division illumination and toggling between illumination and image projection to prevent such interference.

However, this approach makes the system complex and also brings serious inconvenience to the user's view due to the flashing illumination.

3) In case a fluorescent image is appended, an additional time division is required as the white light illumination interferes with the excitation illumination which prevents simultaneous acquisition and projection of the images.

Previous AR projection system is unable to solve the interference issue between illumination, projection image, and special imaging.

Such interference is a problem needed to be solved to build a practical working system.

Images