Three-dimensional image display method using augmented reality and apparatus therefor

Abstract

The present invention relates to a three-dimensional image display method using augmented reality and an apparatus therefor. A three-dimensional image display method according to one embodiment of the present invention comprises: an eye position information collection step for collecting actual eye position information of a user; a virtual display panel preparation step of preparing a virtual display panel with an augmented reality display specification; a virtual barrier calculation step of calculating a virtual barrier based on the actual eye position information and the virtual display panel; a virtual display panel pixel allocation step of allocating pixels of the virtual display panel on the basis of the actual eye position information and the virtual barrier so that the pixels correspond to the actual viewpoint of the user; a rendering step of rendering the stereoscopic content corresponding to the actual viewpoint of the user; and a display step of displaying a three-dimensional image on the basis of the allocated pixels and the stereoscopic content.

Description

Technical Field

[0001] The present invention relates to a three-dimensional image display method and device utilizing augmented reality. Background Art

[0002] Augmented reality (AR) displays utilize optical see-through properties to blend virtual images with real scenes. To achieve a sufficient field of view (FoV) and virtual image distance (VID), the display is magnified by combining the display device with a projection optical system. Furthermore, the magnified image must possess three-dimensional features to better match the real object.

[0003] A three-dimensional display with a lenticular lens or a parallax barrier can be used as a source image for three-dimensional augmented reality.

[0004] Furthermore, rendering and pixel allocation for such displays requires methods for rendering and pixel allocation for specific viewpoints. Conventional techniques require searching for virtual images for both the actual and virtual viewpoints for each frame. However, this method requires additional computation to convert the actual viewpoint to the virtual viewpoint, leading to latency issues. Summary of the Invention

[0005] Technical issues

[0006] In order to solve the problems of the prior art as described above, an object of the present invention is to perform a method of searching for a virtual image with respect to an actual viewpoint by using a virtual barrier parameter, thereby enabling searching for a virtual image with a shorter waiting time.

[0007] Technical Solution

[0008] One aspect of the present invention for solving the technical problems described above can provide a three-dimensional image display method, which may include the following steps: an eye position information collection step for collecting actual eye position information of a user; a virtual display panel preparation step for preparing a virtual display panel having an augmented reality display specification; a virtual barrier calculation step for calculating a virtual barrier based on the actual eye position information and the virtual display panel; a virtual display panel pixel allocation step for allocating pixels of the virtual display panel based on the actual eye position information and the virtual barrier so that the pixels correspond to the actual viewpoint of the user; a rendering step for rendering stereoscopic content corresponding to the actual viewpoint of the user; and a display step for displaying a three-dimensional image based on the allocated pixels and the stereoscopic content.

[0009] According to an embodiment, it may be characterized in that the actual eye position information is collected by an eye tracking camera.

[0010] According to an embodiment, it may be characterized in that the actual viewpoints include a left-eye viewpoint and a right-eye viewpoint, and in the pixel allocation step, pixels corresponding to the left-eye viewpoint and pixels corresponding to the right-eye viewpoint are allocated separately.

[0011] According to an embodiment, it may be characterized in that the actual viewpoint includes a left-eye viewpoint and a right-eye viewpoint, and the stereoscopic content is paired data of content corresponding to the left-eye viewpoint and content corresponding to the right-eye viewpoint.

[0012] According to an embodiment, it may be characterized in that the augmented reality display specification includes a virtual image distance d and a viewing angle.

[0013] According to an embodiment, it may be characterized in that the virtual display panel has first parameters, and in the virtual barrier calculation step, second parameters of the virtual barrier are calculated from the augmented reality display specifications, the first parameters and the actual eye position information.

[0014] According to an embodiment, it may be characterized in that the first parameter includes the magnification M of the virtual display panel, the pitch P' of the pixels of the virtual display panel, P and quantity N.

[0015] According to an embodiment, it may be characterized in that the second parameter includes a virtual barrier gap and a virtual barrier pitch.

[0016] According to another aspect of the present invention, there may be provided a computer program stored in a computer-readable recording medium for executing one of the methods of the above-described embodiments in conjunction with a computer as hardware.

[0017] According to another aspect of the present invention, a three-dimensional image display method may be provided, the method comprising the following steps: an information collection step of collecting actual eye position information of a user; a virtual display panel setting step of setting a virtual display panel, the virtual display panel being located a predetermined virtual image distance d from the actual eye position information and having a first parameter; a virtual barrier calculation step of calculating a second parameter of a virtual barrier based on augmented reality display specifications, the first parameter, and the actual eye position information; a pixel allocation step of allocating pixels of the virtual display panel based on the virtual barrier for which the second parameter is calculated, actual left eye position information, and actual right eye position information, wherein the actual left eye position information and the actual right eye position information are included in the actual eye position information; a rendering step of rendering paired contents corresponding to the actual left eye position information and the actual right eye position information, respectively; and a display step of displaying a three-dimensional image based on the allocated pixels and the paired contents.

[0018] According to another aspect of the present invention, a three-dimensional image display device may be provided, which may include: a processor; and a memory containing computer-readable instructions, wherein, if the instructions are executed in the processor, the processor may calculate a virtual barrier based on a virtual display panel having collected actual eye position information of a user and augmented reality display specifications, may allocate pixels of the virtual display panel based on the actual eye position information and the virtual barrier so that the pixels correspond to the actual viewpoint of the user, may render stereoscopic content corresponding to the actual viewpoint of the user, and may display a three-dimensional image based on the allocated pixels and the stereoscopic content.

[0019] According to an embodiment, it may be characterized in that the augmented reality display specification includes a virtual image distance d and a viewing angle.

[0020] According to an embodiment, it may be characterized in that the virtual display panel has first parameters, and in the virtual barrier calculation step, second parameters of the virtual barrier are calculated from augmented reality display specifications, the first parameters and the actual eye position information.

[0021] According to an embodiment, it may be characterized in that the first parameter includes the magnification M of the virtual display panel, the pitch P' of the pixels of the virtual display panel, P and quantity N.

[0022] According to an embodiment, it may be characterized in that the second parameter includes a virtual barrier gap and a virtual barrier pitch.

[0023] Furthermore, according to another aspect of the present invention, a method for designing a three-dimensional image display device may be provided, the method comprising the following steps: an eye position information collection step for collecting actual eye position information of a user; a virtual display panel preparation step for preparing a virtual display panel having an augmented reality display specification; a virtual barrier calculation step for calculating a virtual barrier based on the actual eye position information and the virtual display panel; and an actual barrier design step for designing an actual barrier based on the virtual barrier.

[0024] According to an embodiment, it may be characterized in that the augmented reality display specification includes a virtual image distance d and a viewing angle.

[0025] According to an embodiment, it may be characterized in that the virtual display panel has first parameters, and in the virtual barrier calculation step, second parameters of the virtual barrier are calculated from augmented reality display specifications, the first parameters and the actual eye position information.

[0026] According to an embodiment, it may be characterized in that the first parameter includes the magnification M of the virtual display panel, the pitch P' of the pixels of the virtual display panel, P and quantity N.

[0027] According to an embodiment, it may be characterized in that the second parameter includes a virtual barrier gap and a virtual barrier pitch.

[0028] According to an embodiment, it may be characterized in that, in the actual barrier designing step, the second parameters of the virtual barrier are used to calculate and design the third parameters of the actual barrier.

[0029] According to an embodiment, it may be characterized in that the second parameter may include a virtual barrier gap and a virtual barrier pitch, and the third parameter includes an actual barrier gap and an actual barrier pitch.

[0030] According to an embodiment, it may be characterized in that, in the actual barrier designing step, it is achieved by calculating a distance from a principal plane of an actual optical system to an actual display panel and a distance from the principal plane of the actual optical system to the actual barrier.

[0031] According to another aspect of the present invention, a method for designing a three-dimensional image display device can be provided, the method including the following steps: an eye position information collection step of collecting actual eye position information of a user; a virtual display panel preparation step of preparing a virtual display panel having an augmented reality display specification; a virtual barrier calculation step of calculating a virtual barrier based on the actual eye position information and the virtual display panel; a virtual display panel pixel allocation step of allocating pixels of the virtual display panel based on the actual eye position information and the virtual barrier so that the pixels correspond to the actual viewpoint of the user; a rendering step of rendering stereoscopic content corresponding to the actual viewpoint of the user; a display step of displaying a three-dimensional image based on the allocated pixels and the stereoscopic content; and an actual barrier design step of designing an actual barrier based on the virtual barrier.

[0032] Effects of the Invention

[0033] According to the present invention, since a virtual image of a view point is not searched in each frame, the rendering and pixel allocation processes are simplified.

[0034] Furthermore, according to the present invention, a method for searching a virtual image for a "real viewpoint" using "virtual barrier parameters" allows the virtual barrier parameters used for rendering and pixel allocation to be calculated only once and remain similar throughout, resulting in faster computation speed.

[0035] Furthermore, according to the present invention, it is possible to improve the rendering process of an augmented reality display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a diagram illustrating a rendering process for a virtual screen and actual eyes according to the prior art.

[0037] Figure 2ais a block diagram illustrating a close method of searching for a virtual image for each frame according to the related art.

[0038] Figure 2b is a block diagram illustrating a method of searching for a virtual image for each frame according to an embodiment of the present invention.

[0039] Figure 3 FIG. 1 is a diagram illustrating a virtual barrier calculation method according to an embodiment of the present invention.

[0040] Figure 4 FIG. 1 is a diagram illustrating a pixel allocation method based on a short distance rule according to an embodiment of the present invention.

[0041] Figure 5 is a diagram showing a system layout for three-dimensional rendering of a stereo content scene with a single path from a virtual display to actual eyes according to an embodiment of the present invention.

[0042] Figure 6 is a diagram showing a system layout for distribution and rendering with a single path from the virtual display to the real eye according to an embodiment of the present invention.

[0043] Figure 7 FIG. 1 is a diagram illustrating a process of calculating an actual barrier from a virtual barrier in consideration of the configuration of an optical system according to an embodiment of the present invention.

[0044] Figure 8a FIG. 1 is a diagram illustrating an actual barrier as a parallax barrier having slits according to an embodiment of the present invention.

[0045] Figure 8b FIG. 1 is a diagram illustrating an actual barrier as a lenticular lens having an air gap according to an embodiment of the present invention.

[0046] Figure 8c FIG. 1 is a diagram illustrating an actual barrier as a solid cylindrical lens according to an embodiment of the present invention.

[0047] Description of Reference Numerals

[0048] 100: Virtual Barrier

[0049] 101: Virtual Display Panel

[0050] 110: Optical system

[0051] 111: Main plane

[0052] 120: Actual Barrier

[0053] 130: Actual display panel DETAILED DESCRIPTION

[0054] The advantages, features and methods for achieving the advantages of the present invention can be clearly understood by referring to the embodiments described in detail in conjunction with the accompanying drawings. However, the present invention can be implemented in a variety of forms that are identical to each other, is not limited to the embodiments proposed below, and should be understood to include all transformations, equivalents and substitutes within the scope of the concept and technology of the present invention. The embodiments proposed below are provided only to complete the disclosure of the present invention and to fully inform the scope of the present invention to those with ordinary knowledge in the technical field to which the present invention belongs. In the process of explaining the present invention, when it is determined that the specific description of the relevant known technology may confuse the gist of the present invention, its detailed description will be omitted.

[0055] The terms used in this application are only used to describe specific embodiments and are not intended to limit the present invention. Unless there is an obvious different meaning in the context, the singular form also includes the plural form.

[0056] In this application, terms such as "including" or "having" are used to specify the presence of features, numbers, steps, operations, constituent elements, components, or combinations thereof described in the specification, and should not be understood as excluding the presence or addition of one or more other features, numbers, steps, operations, constituent elements, components, or combinations thereof. Terms such as "first" and "second" can be used to describe multiple constituent elements, but the constituent elements should not be limited by these terms. These terms are only used to distinguish one constituent element from another.

[0057] Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, identical or corresponding components are given identical reference numerals, and repeated description thereof will be omitted.

[0058] Figure 1 is a diagram showing a rendering process for a virtual screen and an actual eye according to the prior art, Figure 2a is a block diagram illustrating a method of searching for a virtual image for each frame according to the prior art, Figure 2b is a block diagram illustrating a method of searching for a virtual image for each frame according to an embodiment of the present invention.

[0059] Reference Figure 1, using a 3D display device 10 and a catadioptric optical system 20 (or another optical system in front of the 3D display) according to a 3D rendering method of the prior art, an actual viewpoint position 40 is converted into a virtual image 50 of the viewpoint position.

[0060] In this case, pixel allocation may be performed for the virtual image 50 at the viewpoint position, and as Figure 1 As shown, content rendering may be performed for an actual viewpoint location 40 .

[0061] A pixel can be composed of multiple sub-pixels, and a pixel value can be assigned to each sub-pixel. In the present invention, the expression of pixel can refer to sub-pixels depending on the situation. Moreover, pixel allocation can refer to allocating sub-pixels.

[0062] Reference Figure 1 、 Figure 2a and Figure 2b Comparing the prior art with the present invention, the three-dimensional image display method according to the prior art requires a step of defining a virtual eye position 50 and virtual eye position data, but the three-dimensional image display method for realizing a three-dimensional display image according to an embodiment of the present invention does not require virtual eye position data.

[0063] Specifically, conventional technology forms an image on a display by optically transforming real eye position data to obtain virtual eye position data, and allocating pixel values ​​corresponding to the virtual eye position data to form a pixel allocation map. This method requires a step of defining the virtual eye position and the virtual eye position data.

[0064] However, the 3D image display method according to an embodiment of the present invention forms an image on a display by directly forming a pixel allocation map by allocating pixel values ​​in actual eye position data, and therefore does not require virtual eye position data.

[0065] According to an embodiment of the present invention, a method for allocating and rendering pixels at an actual viewpoint position without converting the pixels into a virtual image at the viewpoint position is proposed.

[0066] In the following, a three-dimensional display system that implements the pixel allocation and rendering method according to an embodiment of the present invention is first described.

[0067] An embodiment of a 3D display system for realizing 3D display may include a 3D image control unit, a 3D display device, and an optical system. Here, the optical system may include a catadioptric system.

[0068] According to an embodiment, the optical system may be located in front of the 3D display device. According to an embodiment, the optical system may be located between the user and the 3D display device.

[0069] Here, the 3D display device may include a display panel and an optical layer. According to an embodiment, the optical layer may be a parallax barrier or a lenticular lens. The display panel may include a plurality of pixels, and the 3D image control unit may assign pixel values ​​corresponding to a predetermined number of viewpoints to the plurality of pixels based on an input image. For example, the 3D image control unit may assign a pixel value corresponding to a first viewpoint to a first pixel, and may assign a pixel value corresponding to a second viewpoint to a second pixel.

[0070] According to an embodiment, the first viewpoint and the second viewpoint may include a left-eye viewpoint and a right-eye viewpoint for the user. The display panel may display a panel image based on pixel values ​​assigned by the three-dimensional image control unit.

[0071] Light from a light source can be provided to the pixels of the display panel. For example, the light source can be a backlight unit located on the rear surface of the display panel. As light is provided to the pixels, the light corresponding to the pixel value of the pixel can be visible to the user. At this time, the optical layer can limit the direction in which the user sees the light. Specifically, the parallax barrier can output light in a restricted direction through a slit at a predetermined interval, and the cylindrical lens can output light in a restricted direction through the curvature of the lens.

[0072] According to an embodiment, pixels assigned a pixel value based on a first viewpoint can be provided with light from the first viewpoint, while pixels assigned a pixel value based on a second viewpoint can be provided with light from the second viewpoint. A user in an eye space can view an image corresponding to the first viewpoint through the first viewpoint and an image corresponding to the second viewpoint through the second viewpoint. When the first viewpoint corresponds to the user's left eye and the second viewpoint corresponds to the user's right eye, the user can experience a sense of stereoscopic perception by viewing different images with both eyes.

[0073] The image output by the three-dimensional display device can be provided to the user through an optical system. For example, the optical system may include a mirror corresponding to a reflective optical system or a lens corresponding to a refractive optical system.

[0074] According to embodiments, a 3D display device can be implemented using a screen, and an optical system can amplify the image output to the 3D display device. For example, the 3D display device and the optical system can be implemented as part of a head-up display (HUD) provided in a vehicle. In this case, according to embodiments, the optical system can include a semi-transparent concave mirror.

[0075] According to an embodiment, the display panel may include a predetermined number of pixels, and the optical layer may include a periodic optical element covering the designated number of pixels. In this case, the number of pixels of the display panel may be smaller than a pitch of the optical element.

[0076] The optical system may be configured to transmit the image of the optical layer at a magnification smaller than that of the display panel to change the number of pixels on the lower side of each optical component from small to large.

[0077] To assign pixel values ​​to pixels of the display panel, the 3D image control unit may determine the path of light passing through the corresponding pixel. Light passing through pixels of the display panel may be referred to as light. For example, if light passing through a first pixel is provided to a first viewpoint, the 3D image control unit may assign the pixel value corresponding to the first viewpoint to the first pixel.

[0078] A pixel of a display may include a sub-pixel structure, which may be referred to as a pixel structure in this application.

[0079] The display panel according to one embodiment of the present invention may include an LCD panel with LED lighting, a self-luminous LED / OLED panel, a reproduction site of a holographic spatial light modulator (SLM), or may include one or more of all projected results using or not using a DLP engine or relay optics of a micro-electro-mechanical system (MEMS).

[0080] The optical layer according to an embodiment of the present invention may include one of a parallax barrier having opaque areas and transparent slits, a solid lenticular lens array, and a lenticular lens array having air gaps.

[0081] An optical system according to an embodiment of the present invention may be constructed using a single or multiple refractive or reflective components, and at least one of these components may function as a combiner to fuse the displayed virtual image with the actual scene.

[0082] Hereinafter, a three-dimensional image display method according to an embodiment of the present invention implemented by the above-mentioned three-dimensional display system will be described in detail.

[0083] According to an embodiment of the present invention, a three-dimensional display method includes the following steps: an eye position information collection step of collecting actual eye position information of a user; a virtual display panel preparation step of preparing a virtual display panel having an augmented reality display specification; a virtual barrier calculation step of calculating a virtual barrier based on the actual eye position information and the virtual display panel; a virtual display panel pixel allocation step of allocating pixels of the virtual display panel based on the actual eye position information and the virtual barrier so that the pixels correspond to the actual viewpoint of the user; a rendering step of rendering stereoscopic content corresponding to the actual viewpoint of the user; and a display step of displaying a three-dimensional image based on the allocated pixels and the stereoscopic content.

[0084] The 3D image display method according to this embodiment can use actual eye position data obtained from eye-tracking to perform pixel allocation and rendering. For example, the user's actual eye position information can be collected by an eye-tracking camera.

[0085] Figure 3 is a diagram illustrating a process of preparing a virtual display panel and a method of calculating a virtual barrier according to an embodiment of the present invention.

[0086] Reference Figure 3 In order to perform the three-dimensional image display method according to this embodiment, a virtual display panel may be prepared. Here, for example, the virtual display panel may be set to have augmented reality display specifications. According to an embodiment, the augmented reality display specifications may include a virtual image distance d and a viewing angle θ. Here, the virtual image distance d may refer to the distance d at which the virtual image is located when determining the distance from the actual eye position to which the virtual image is located.

[0087] According to an embodiment, the field of view angle θ may refer to an angle formed by two virtual lines when two virtual lines starting from both ends of the viewing width of a viewing zone including the actual eye position are set to intersect each other at positions passing through slits of a virtual barrier.

[0088] Reference Figure 3 , the virtual barrier 100 can be calculated for the virtual display panel. According to an embodiment, in this case, magnification based on the optical system can be considered. Specifically, the virtual barrier 100 can be calculated based on the parameters of the optical system and the position of the virtual display panel.

[0089] Reference Figure 3 , the virtual display panel can be arranged in a specific way to provide a virtual image in an initial plane that is a virtual image distance d away from the actual eye position.

[0090] In this case, the optical system can form a three-dimensional display image with a specific magnification.

[0091] In order to estimate the virtual barrier 100 according to this embodiment, virtual barrier parameters may be calculated, which may include a virtual barrier gap t' and a virtual barrier pitch P'.

[0092] First, the virtual barrier gap t' can be calculated by the following mathematical formula 1.

[0093] [Mathematical formula 1]

[0094]

[0095] Here, P P is the pixel pitch of the display panel, M is the magnification of the virtual display panel 101, N is the preferred number of pixels in the virtual display panel 101 for a single barrier pitch, and θ is the viewing angle covering the entire viewing zone for the left eye and the right eye.

[0096] According to an embodiment, P P and M can be replaced by the pitch P' of the virtual display pixel through the following mathematical formula 2 P .

[0097] [Mathematical formula 2]

[0098] P′ p =P p *M

[0099] Then, the virtual barrier pitch P′ can be calculated by the following Math Formula 3.

[0100] [Mathematical formula 3]

[0101]

[0102] Here, d is the virtual image distance.

[0103] In existing technologies, the process of converting the actual eye position to the virtual eye position requires continuous calculations based on the actual eye position, which can slow down the system. Specifically, the virtual eye position corresponding to the actual eye position must be converted, and this conversion is then used to transform the virtual screen. However, in the present invention, a fixed virtual screen is calculated once for the initial designed actual eye position, eliminating the need for additional calculations. Therefore, since continuous operations are not required, the system speed is improved.

[0104] After the virtual barrier calculation step, a pixel allocation step of the virtual display panel may be performed: pixels of the virtual display panel are allocated to the left eye viewpoint or the right eye viewpoint based on the actual eye position information and the parameters of the virtual barrier to correspond to the actual viewpoint of the user.

[0105] In the pixel allocation step of the virtual display panel according to the present embodiment, in order to allocate signals from pixels to the left eye or the right eye, the concept of a lenticular lens or a parallax barrier may be used.

[0106] Figure 4 FIG is a diagram illustrating a pixel allocation method based on a short distance rule according to an embodiment of the present invention. Figure 4 , the eye position can be tracked to pixels through the virtual barrier plane.

[0107] In a 3D image display method according to an embodiment of the present invention, actual eye position data acquired through eye tracking can be secured for pixel allocation and rendering. All pixels of the virtual display panel can then be allocated to the left or right eye based on the actual eye position. Subsequently, content rendering can be performed based on the actual eye position, and the image can be displayed on the display. This rendering step will be described later.

[0108] Reference Figure 4 ,First, the pixel allocation steps of the virtual display panel are described.

[0109] In the pixel allocation step of the virtual display panel, for determining whether to allocate pixels to the left eye or to the right eye, intersections having short distances to the virtual barrier slits may be considered.

[0110] like Figure 4 As shown, the actual eyes (ie, the left eye and the right eye) are set in a manner corresponding to the viewpoint. Here, accurate eye position data can be obtained through eye tracking.

[0111] Here, a virtual line corresponding to the left eye viewpoint and a virtual line corresponding to the right eye viewpoint are set to pass through the virtual barrier and converge to a pixel selected in the virtual display panel, respectively.

[0112] The distance (Δ R , Δ L ). That is, the distance Δ from the intersection of the virtual lines corresponding to the left eye viewpoint to the nearest virtual barrier slit can be L and the distance Δ from the intersection of the virtual line corresponding to the right eye viewpoint to the nearest virtual barrier slit R The minimum distance result value is selected, and the pixel can be allocated to the left eye or the right eye based on the minimum distance result value.

[0113] Reference Figure 4 , since the distance Δ from the intersection of the virtual line corresponding to the right eye viewpoint to the slit R is the minimum distance result value, so pixel k can be assigned to the right eye.

[0114] Specifically, the pixels of the virtual display panel may be allocated to the left eye or the right eye according to the following method.

[0115] First, the above-mentioned virtual line will be described.

[0116] (1) In the first step, each ray can be translated into an assigned pixel. Here, a real ray is extended into a virtual ray, and vice versa. Here, the ray can be formed into a line that intersects the virtual barrier plane at the actual eye position and also intersects the virtual display panel 101. Here, the pixel in the display panel that the ray intersects can be the assigned pixel.

[0117] (2) In the second step, the intersection point between the light ray and the barrier plane is calculated.

[0118] (3) In the third step, the distance (Δ R , Δ L ). At this time, the distance (Δ R , Δ L ) are compared with the left eye and the right eye respectively, and the minimum distance thereof becomes the allocation condition. That is, the distance (Δ R , Δ L )The pixels belonging to the minimum distance can be considered as the assigned pixels.

[0119] In particular, if the eye position data is (x R ,y R , d) and (x L ,y L , d), and the corresponding pixel is (x P ,y P ,0), then the center of the corresponding barrier is (x b ,y b , 0), therefore, Δ R and Δ L It can be calculated by the following mathematical formula 4.

[0120] [Formula 4]

[0121]

[0122] Here, the pixels can be based on the minimum distance (ie, Δ R and Δ L The minimum value between 0 and 1 is assigned to the left or right eye.

[0123] The following describes the 3D rendering steps of the 3D image display method according to an embodiment of the present invention.

[0124] Figure 5 FIG2 is a diagram showing a system layout for three-dimensional rendering of a stereo content scene with a single path from a virtual display to real eyes according to an embodiment of the present invention. Figure 5 In the optical system, the optical system is simplified to a single-mirror.

[0125] Reference Figure 5 , 3D scene stereo content can be used to generate eye position data (x R ,y R , z R )(xL ,y L , z R ) and the distance between the two eyes from the virtual image distance d to the actual eyes for rendering.

[0126] Here, the eye position data may include left eye position data (x L ,y L , z L ) and right eye position data (x R ,y R , z R ).

[0127] In the initial stage, z L / z R =d, but may vary depending on the user's movement.

[0128] The rendered stereo image is then distributed to pixels of the display panel.

[0129] Figure 6 FIG is a diagram showing a system layout for distribution and rendering with a single path from a virtual display to real eyes according to an embodiment of the present invention. Figure 6 , shows a state where the same layout as that rendered using virtual objects is allocated on a more complex optical system layout.

[0130] In addition, in the virtual barrier calculation step, after calculating the virtual barrier parameters, pixel allocation and rendering can be performed directly as described above, but physical equipment design is still required. In order to design the physical equipment (i.e., the actual 3D image display device), the actual barrier can be calculated from the virtual barrier.

[0131] Figure 7 1 is a diagram showing a process of calculating an actual barrier from a virtual barrier 100 in consideration of the configuration of the optical system 110 according to an embodiment of the present invention. Figure 7 In , the barrier and the screen are located at different distances from the principal plane of the optical system, so the magnifications for them are different. Figure 7 In the optical system, the optical system is simplified to a single-mirror.

[0132] According to an embodiment of the present invention, a method for designing a three-dimensional image display device may include the following steps: an eye position information collection step of collecting a user's actual eye position information (actual eye position data); a virtual display panel preparation step of preparing a virtual display panel having an augmented reality display specification (AR display specification); a virtual barrier calculation step of calculating a virtual barrier based on the actual eye position information and the virtual display panel; and an actual barrier design step of designing an actual barrier based on the virtual barrier.

[0133] The 3D image display device designed herein may include two main components (ie, a display panel and an optical layer) for splitting light signals from pixels to the viewpoints of the left eye or the right eye.

[0134] In the actual barrier design step according to the present embodiment, in order to realize the optical layer according to the parameters of the virtual barrier 100 calculated in the above-mentioned virtual barrier calculation step, the optical characteristics of the optical system 110 may be used.

[0135] The actual barrier 120 and the actual display panel 130 have different distances relative to the optical system 110 or the principal plane of the optical system 110. Figure 7 Shown are different magnifications.

[0136] In order to design the actual barrier 120 according to this embodiment, the parameters of the actual barrier can be calculated. Here, the parameters of the actual barrier can be calculated as the distance a from the main plane 111 of the optical system 110 to the actual barrier 120. b , actual barrier gap t and actual barrier pitch P.

[0137] Reference Figure 7 , a d is the distance from the main plane 111 to the actual display panel 130, a' d is the distance from the principal plane 111 to the virtual display panel 101, and f is the focal length of the optical system.

[0138] First, the distance a from the main plane 111 to the actual barrier 120 can be calculated by the following mathematical formula 5: b .

[0139] [Formula 5]

[0140]

[0141] Here, a' bis the distance from the principal plane 111 of the optical system 110 to the virtual barrier 100. The distance a′ from the principal plane 111 of the optical system 110 to the virtual barrier 100 is b It can be calculated using the following mathematical formula 6.

[0142] [Formula 6]

[0143] a′ b =a′ d -t′

[0144] Then, the actual barrier gap t can be calculated by the following mathematical formula 7.

[0145] [Formula 7]

[0146] t=a d -a b

[0147] Furthermore, the magnification M of the virtual barrier 100 is b It can be calculated using the following mathematical formula 8.

[0148] [Formula 8]

[0149]

[0150] Here, the magnification M of the virtual barrier 100 is b is smaller than the magnification M of the virtual display panel 101 .

[0151] The actual barrier pitch P can be calculated by the following mathematical formula 9.

[0152] [Formula 9]

[0153]

[0154] The actual barrier pitch P is slightly larger than N*P' P , and the virtual barrier pitch P' is slightly smaller than N*P' P This corresponds to the position of each viewpoint in front of and behind the display panel.

[0155] Figure 8a FIG. 1 is a diagram showing an actual barrier 120 as a parallax barrier having slits according to an embodiment of the present invention. Figure 8b FIG. 1 is a diagram illustrating an actual barrier 120 as a lenticular lens having an air gap according to an embodiment of the present invention. Figure 8c FIG. 1 is a diagram illustrating an actual barrier 120 as a solid cylindrical lens according to an embodiment of the present invention.

[0156] 8 , depending on the embodiment, the actual barrier can be designed as a parallax barrier, a lenticular lens with an air gap, or a solid lenticular lens. The barrier gap varies depending on the medium between the pixel and the barrier. In this embodiment, calculations are performed for the case where the medium is air.

[0157] exist Figure 8a In this case, the actual parallax barrier is designed as the actual barrier. In this case, the actual parallax barrier can be designed using the actual barrier gap t and the actual barrier pitch P as the parameters of the actual barrier.

[0158] That is, the actual barrier gap t as a parameter of the above-mentioned actual barrier can correspond to the thickness t of the actual parallax barrier, and the actual barrier pitch P as a parameter of the actual barrier can correspond to the distance P between the slits of the actual parallax barrier, so the actual barrier pitch P can be used to design the actual parallax barrier.

[0159] exist Figure 8b In this case, a real lenticular lens with an air gap is designed as the real barrier. In this case, the real barrier gap t, which is a parameter of the real barrier, may correspond to the distance t between the real lenticular lens to be designed and the real display panel, and the real barrier pitch P may correspond to the distance P between the centers of the convex portions of the real lenticular lens to be designed. Therefore, the real lenticular lens with an air gap can be designed by using the real barrier pitch P.

[0160] According to an embodiment, when a lenticular lens is used as the actual barrier, the radius of the lenticular lens may be calculated such that the actual barrier gap t corresponds to the focal length of the lenticular lens.

[0161] exist Figure 8c In this case, a real solid cylindrical lens is designed as the real barrier. In this case, the real barrier gap t, which is a parameter of the real barrier, can correspond to the thickness t of the real solid cylindrical lens, and the real barrier pitch P can correspond to the distance P between the centers of the convex parts of the real solid cylindrical lens. Therefore, the real solid cylindrical lens can be designed using this distance P.

[0162] Typically, a cylindrical lens is made of a solid material, and thus its actual thickness is t*n, where n is the refractive index of the cylindrical lens material.

[0163] The embodiments of the present disclosure described above can be implemented in the form of a computer program that can be executed on a computer through various components, and the computer program can be recorded in a computer-readable medium. In this case, the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as read-only optical disks (CD-ROMs) and DVDs, magneto-optical media such as floppy disks, and hardware devices specifically configured to store and execute program commands, such as read-only memories (ROMs) and random access memories (RAMs).

[0164] In addition, the computer program may be a computer program specifically designed and constructed for the present disclosure, or may be a computer program known and available to those skilled in the art of computer software. Examples of computer programs include not only machine language code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

[0165] In the specification of the present disclosure (especially in the claims), the use of the term "said" and similar indicative terms may apply to both the singular and the plural. Furthermore, when a range is described in the present disclosure, the invention is inclusive of the individual values ​​falling within the range (unless otherwise stated), as if the detailed description of the invention had described the individual values ​​constituting the range.

[0166] For the steps constituting the method of the present disclosure, if there is no clear record order or contrary record, the steps can be performed in an appropriate order. The present disclosure is not limited to the description order of the above steps. All examples or exemplary terms (such as, etc.) used in the present disclosure are only intended to describe the present disclosure in detail, and unless limited by the claims, the scope of the present disclosure is not limited by the examples or exemplary terms. In addition, it is obvious to those skilled in the art that various modifications, combinations and changes can be made according to design conditions and factors without departing from the scope of the appended claims or their equivalents.

[0167] Therefore, the concept of the present disclosure should not be limited to the above-described embodiments, and the appended claims and all scopes equivalent to or changed equivalently from the claims will fall within the scope of the concept of the present disclosure.

[0168] The above description is merely an illustrative description of the technical concept of the present invention. Anyone skilled in the art in the technical field to which the present invention pertains can make various modifications and variations without departing from the essential characteristics of the present invention.

[0169] The embodiments disclosed in the present invention are not intended to limit the technical concept of the present invention but to explain the technical concept of the present invention. The scope of the technical concept of the present invention is not limited to such embodiments.

Claims

1. A three-dimensional image display method, comprising: an eye position information collecting step, collecting the user's actual eye position information; a virtual display panel preparation step of preparing a virtual display panel having augmented reality display specifications; a virtual barrier calculation step of calculating a virtual barrier based on the actual eye position information and the virtual display panel; a pixel allocation step of a virtual display panel, allocating pixels of the virtual display panel based on the actual eye position information and the virtual barrier so that the pixels correspond to the actual viewpoint of the user; A rendering step of rendering stereoscopic content corresponding to the actual viewpoint of the user; as well as A display step of displaying a three-dimensional image based on the allocated pixels and the stereoscopic content.

2. The three-dimensional image display method according to claim 1, wherein: The actual eye position information is collected by an eye tracking camera.

3. The three-dimensional image display method according to claim 1, wherein: The actual viewpoints include a left-eye viewpoint and a right-eye viewpoint, In the pixel allocation step, pixels corresponding to the left-eye viewpoint and pixels corresponding to the right-eye viewpoint are allocated separately.

4. The three-dimensional image display method according to claim 1, wherein: The actual viewpoints include a left-eye viewpoint and a right-eye viewpoint, The stereoscopic content is paired data of content corresponding to the left-eye viewpoint and content corresponding to the right-eye viewpoint.

5. The three-dimensional image display method according to claim 1, wherein: The augmented reality display specifications include virtual image distance (d) and field of view angle (θ).

6. The three-dimensional image display method according to claim 1, wherein: The virtual display panel has a first parameter, In the virtual barrier calculation step, a second parameter of the virtual barrier is calculated from the augmented reality display specification, the first parameter, and the actual eye position information.

7. The three-dimensional image display method according to claim 6, wherein: The first parameter includes the magnification (M) of the virtual display panel, the pitch (P') of the pixels of the virtual display panel, P ) and quantity (N).

8. The three-dimensional image display method according to claim 6, wherein: The second parameters include a virtual barrier gap (t') and a virtual barrier pitch (P'). 9 . A computer program stored in a computer-readable recording medium for executing the method according to claim 1 in combination with a computer as hardware.

10. A three-dimensional image display method, comprising: An information collection step, collecting the user's actual eye position information; a virtual display panel setting step of setting a virtual display panel, wherein the virtual display panel is located at a predetermined virtual image distance (d) from the actual eye position information and has a first parameter; a virtual barrier calculation step of calculating a second parameter of the virtual barrier based on the augmented reality display specification, the first parameter, and the actual eye position information; a pixel allocating step of allocating pixels of the virtual display panel based on the virtual barrier calculated from the second parameter, actual left eye position information, and actual right eye position information, wherein the actual left eye position information and the actual right eye position information are included in the actual eye position information; a rendering step of rendering paired content corresponding to the actual left eye position information and the actual right eye position information respectively; and A display step of displaying a three-dimensional image based on the allocated pixels and the paired content.

11. A three-dimensional image display device, comprising: processor; as well as a memory containing computer-readable instructions, Wherein, if the instruction is executed in the processor, the processor calculates a virtual barrier based on a virtual display panel having collected actual eye position information of the user and augmented reality display specifications, allocating pixels of the virtual display panel based on the actual eye position information and the virtual barrier so that the pixels correspond to the actual viewpoint of the user, Rendering stereoscopic content corresponding to the actual viewpoint of the user, A three-dimensional image is displayed based on the allocated pixels and the stereoscopic content.

12. The three-dimensional image display device according to claim 11, wherein: The augmented reality display specifications include virtual image distance (d) and field of view angle (θ).

13. The three-dimensional image display device according to claim 11, wherein: The virtual display panel has a first parameter, In the virtual barrier calculation step, a second parameter of the virtual barrier is calculated from augmented reality display specifications, the first parameter, and the actual eye position information.

14. The three-dimensional image display device according to claim 13, wherein: The first parameter includes the magnification (M) of the virtual display panel, the pitch (P') of the pixels of the virtual display panel, P ) and quantity (N).

15. The three-dimensional image display device according to claim 13, wherein: The second parameters include a virtual barrier gap (t') and a virtual barrier pitch (P').

16. A method for designing a three-dimensional image display device, comprising: an eye position information collecting step, collecting the user's actual eye position information; a virtual display panel preparation step of preparing a virtual display panel having augmented reality display specifications; a virtual barrier calculation step of calculating a virtual barrier based on the actual eye position information and the virtual display panel; as well as The actual barrier design step is to design an actual barrier based on the virtual barrier.

17. The method for designing a three-dimensional image display device according to claim 16, wherein: The augmented reality display specifications include virtual image distance (d) and field of view angle (θ).

18. The method for designing a three-dimensional image display device according to claim 16, wherein: The virtual display panel has a first parameter, In the virtual barrier calculation step, a second parameter of the virtual barrier is calculated from augmented reality display specifications, the first parameter, and the actual eye position information.

19. The method for designing a three-dimensional image display device according to claim 18, wherein: The first parameter includes the magnification (M) of the virtual display panel, the pitch (P') of the pixels of the virtual display panel, P ) and quantity (N).

20. The method for designing a three-dimensional image display device according to claim 18, wherein: The second parameters include a virtual barrier gap (t') and a virtual barrier pitch (P').

21. The method for designing a three-dimensional image display device according to claim 16, wherein: In the actual barrier design step, third parameters of the actual barrier are calculated and designed using the second parameters of the virtual barrier.

22. The method for designing a three-dimensional image display device according to claim 21, wherein: The second parameters include a virtual barrier gap (t') and a virtual barrier pitch (P'), and the third parameters include an actual barrier gap (t) and an actual barrier pitch (P).

23. The method for designing a three-dimensional image display device according to claim 16, wherein: In the actual barrier design step, the distance (a b ) and the distance from the principal plane of the actual optical system to the actual barrier (a d ) to achieve it.

Images