Device and method of displaying augmented reality

Abstract

An apparatus and method for displaying augmented reality are disclosed. The disclosed apparatus for displaying augmented reality comprises: an optical engine; a waveguide through which light of a virtual image is output and light of a real scene is transmitted; a first lens unit and a second lens unit disposed with respect to the waveguide; and a processor. The first lens unit performs focus adjustment of the virtual image and correction of the user's vision and includes a first variable focus lens having a first refractive power and a fixed refractive lens having a fixed refractive power. The second lens unit compensates for distortion of the real scene caused by the first lens unit and includes a second variable focus lens having a second refractive power. The processor determines the first refractive power of the first variable focus lens based on the user's vision information, attribute depth information of the virtual image, and fixed refractive power information of the fixed refractive lens.

Description

Technology Field

[0001] The present disclosure relates to an apparatus and method for displaying augmented reality, and more specifically, to an apparatus and method for displaying augmented reality equipped with a variable focus lens. Background Technology

[0002] An augmented reality device is a device capable of viewing augmented reality (AR), such as augmented reality glasses. The image optical system of an augmented reality device includes an image generator that generates an image and a waveguide for sending the generated image to the eye.

[0003] Images emitted from an image generating device (such as a projector) are projected onto the eye through a waveguide, thereby allowing the user to observe the image. In displays utilizing such waveguides, the focal length of a virtual image can be, for instance, infinite; therefore, to create an immersive augmented reality environment, a means is required to position the focal length of the virtual image at an arbitrary distance where real-world objects exist. Meanwhile, among users of augmented reality devices, those who rely on glasses for vision correction need to use additional means such as prescription clips. However, due to the inconvenience of prescription clips, research is being conducted on augmented reality devices equipped with vision correction functions for people with low vision using variable focus lenses. The problem to be solved

[0004] The goal is to provide an augmented reality device capable of self-correcting vision.

[0005] The goal is to provide an immersive augmented reality environment.

[0006] The goal is to provide an augmented reality environment with improved virtual image quality.

[0007] The technical challenges to be solved are not limited to those mentioned above, and other technical challenges may exist. means of solving the problem

[0008] A device for displaying augmented reality according to one aspect comprises: an optical engine configured to output light of a virtual image; a waveguide through which light of a virtual image is output and light of a real scene is transmitted; a first lens unit and a second lens unit positioned with respect to the waveguide; and a processor. The first lens unit performs focus adjustment of a virtual image and correction of a user's vision and includes a first variable focus lens having a first refractive power adjustable by a processor and a fixed refractive lens having a fixed refractive power. The second lens unit compensates for distortion of a real scene caused by the first lens unit and includes a second variable focus lens having a second refractive power adjustable by a processor. The processor can determine the first refractive power of the first variable focus lens based on the user's vision information, the attribute depth information of the virtual image, and the fixed refractive power information of the fixed refractive lens.

[0009] In one embodiment, the first refractive power of the first variable focus lens is given by the mathematical formula It is determined by. Here, D1 is the first refractive power of the first variable focus lens, D 고정 is the fixed refractive power of a fixed-refractive lens, D 교정 represents the refractive power required to correct the user's refractive error, and f represents the focal length of the virtual image.

[0010] In one embodiment, the device for displaying augmented reality has a fixed refractive power D of a fixed refractive lens. 고정 , User's refractive power requiring correction D 교정 It may further include memory where the focal length f of the virtual image is stored.

[0011] In one embodiment, after the processor reads information from memory regarding the fixed refractive power of a fixed refractive lens, the refractive power requiring correction for the user, and the focal length of a virtual image, the first refractive power D1 of a first variable focal lens is It can be calculated as.

[0012] In one embodiment, a device for displaying augmented reality has focal length information f of a virtual image and a refractive power D requiring correction that reflects the refractive power of a fixed refractive lens. 수정 This further includes the stored memory, and the refractive power D requiring correction 수정 silver It is given as, and the processor reads the refractive power requiring correction and the focal length information of the virtual image from memory, and then the first refractive power D1 of the first variable focus lens It can be calculated as.

[0013] In one embodiment, the second refractive power D2 of the second variable focus lens is given by the mathematical formula It can be determined by.

[0014] In one embodiment, the fixed refractive lens may be a concave lens having a negative (-) refractive power.

[0015] In one embodiment, the first and second variable focus lenses may be liquid crystal lenses.

[0016] In one embodiment, the second variable focus lens is positioned between the waveguide and the fixed refractive lens, and the first variable focus lens, the waveguide, and the second variable focus lens may have a stacked structure.

[0017] In one embodiment, the device for displaying augmented reality may further include a user input unit into which at least one of the user's vision information and the focal length of a virtual image is input.

[0018] In one embodiment, the first lens portion may further include a polarizing plate disposed on the incident or exit surface of the fixed refractive lens.

[0019] In one embodiment, the second lens unit may further include a second fixed refractive lens that compensates for distortion of the real scene caused by the first lens unit together with the second variable focus lens.

[0020] In one embodiment, the second fixed refractive lens may be a convex lens having positive (+) refractive power.

[0021] In one embodiment, the second refractive power D2 of the second variable focus lens is given by the mathematical formula It is calculated by, where D 고정2 may be the fixed refractive power of the second fixed refractive lens (543).

[0022] In one embodiment, the device for displaying augmented reality may further include an eye-tracking sensor configured to acquire user's gaze information.

[0023] The processor can calculate the gaze point from the user's gaze information acquired from the eye tracking sensor and determine the focal length of the virtual image based on the calculated gaze point.

[0024] In one embodiment, the processor controls an optical engine to output at least one first character of a preset size; acquires at least one first input from a user for at least one first character, compares at least one first character and at least one first input, determines a first refractive power of a first variable focus lens based on the comparison result, and determines the user's required refractive power for correction based on the determined first refractive power of the first variable focus lens.

[0025] In one embodiment, at least one first character and at least one second character may have a size corresponding to a preset corrected visual acuity and may be displayed at a preset depth for measuring the user's visual acuity.

[0026] In one embodiment, the device may be a glasses-type device.

[0027] According to another aspect, a method for displaying augmented reality of an augmented reality device comprising an optical engine configured to output light of a virtual image and a waveguide through which light of a real scene is output and light of a real scene is transmitted, may include the steps of: arranging a first lens unit comprising a fixed refractive lens and a first variable focus lens and a second lens unit comprising a first variable focus lens with the waveguide in between; calculating a first refractive power of the first variable focus lens based on user's visual acuity information, focal length information of a virtual image, and a fixed refractive power of the fixed refractive lens; and calculating a second refractive power of the second variable focus lens to compensate for distortion of the real scene caused by the first lens unit.

[0028] According to another aspect, a computer-readable recording medium can be provided for executing a method of displaying augmented reality of an augmented reality device on a computer. Effects of the invention

[0029] According to the present disclosure, an apparatus and method for displaying augmented reality can provide its own vision correction function.

[0030] According to the present disclosure, an apparatus and method for displaying augmented reality can provide an immersive augmented reality environment by moving the focal length of a virtual image to any location where a real object is located.

[0031] According to the present disclosure, an apparatus and method for displaying augmented reality can improve the image quality of virtual images and real scenes by reducing the required level of refractive power of a variable focus lens. Brief explanation of the drawing

[0032] FIG. 1 is a drawing illustrating the appearance of an augmented reality device according to one embodiment. Figure 2 is a plan view of the augmented reality device of Figure 1. FIG. 3 is a diagram illustrating the arrangement of an optical engine and optical components according to one embodiment. Figure 4 is a block diagram of the augmented reality device of Figure 1. FIG. 5 illustrates a first variable focus lens according to one embodiment. FIG. 6 is the phase profile of a first variable focus lens when the control signal is a voltage profile corresponding to a concave lens with a predetermined refractive power (e.g. -2D). FIG. 7 is the phase profile of a first variable focus lens when the control signal is a voltage profile corresponding to a concave lens with a predetermined refractive power (e.g. -3D). FIG. 8 is a flowchart illustrating the operation of an augmented reality device according to one embodiment. FIG. 9 is a flowchart illustrating the operation of an augmented reality device according to one embodiment. FIG. 10 is a diagram illustrating the operation of an augmented reality device according to one embodiment. FIG. 11 is a diagram illustrating the arrangement of optical components of an augmented reality device according to one embodiment. FIG. 12 is a diagram illustrating the arrangement of optical components of an augmented reality device according to one embodiment. FIG. 13 is a block diagram of an augmented reality device according to one embodiment. FIG. 14 illustrates an eye-tracking sensor according to one embodiment. Figure 15 is a diagram illustrating a three-dimensional eye model of the user's gaze direction. Figure 16 is a diagram illustrating the relationship between the gaze angle and the gaze point in the left and right eyes. Figure 17 is a diagram illustrating the relationship between the gaze angle and the gaze point in the upward gaze direction. FIG. 18 is a flowchart illustrating the operation of an augmented reality device according to one embodiment. FIG. 19 is a diagram illustrating the operation of an augmented reality device according to one embodiment. FIG. 20 is a block diagram of an augmented reality device according to one embodiment. FIG. 21 is a diagram showing an example of an augmented reality device according to one embodiment performing an operation to obtain the user's correction required refractive power when the accuracy rate of the user's voice input is low. FIG. 22 is a diagram showing an example of an augmented reality device according to one embodiment performing an operation to obtain the user's correction required refractive power when the accuracy rate of the user's voice input is normal. FIG. 23 is a diagram showing an example of an augmented reality device according to one embodiment performing an operation to obtain the user's correction required refractive power when the accuracy rate of the user's voice input is high. FIG. 24 is a flowchart illustrating the operation of an augmented reality device according to one embodiment. Specific details for implementing the invention

[0033] Embodiments of the present disclosure are described below in detail with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present disclosure in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0034] The terms used in the embodiments of this specification have been selected to be as widely used as possible, taking into account the functions of the present disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description section of the relevant embodiments. Therefore, terms used in this specification should be defined not merely by their names, but based on their meanings and the overall content of the present disclosure.

[0035] In the present disclosure, "Augmented Reality (AR)" refers to overlaying a virtual image onto a physical environment space or a real object in the real world and displaying them together.

[0036] In the present disclosure, the term "Augmented Reality Device" refers to a device capable of expressing augmented reality, and includes not only augmented reality glasses in the form of glasses worn on the face by a user, but also head-mounted displays (HMDs) or augmented reality helmets worn on the head.

[0037] In the present disclosure, a ‘real scene’ is a scene of the real world viewed by an observer or user through an augmented reality device, and may include real world object(s).

[0038] A 'virtual image' is an image generated through an optical engine. A virtual image may include both static and dynamic images. Such a virtual image is observed alongside a real-world scene and may be an image that displays information about real-world objects within the scene, information about the operation of an augmented reality device, or control menus. A 'virtual object' may be represented as a portion of a virtual image. A virtual object may represent information related to a real-world object. A virtual object may include, for example, at least one of characters, numbers, symbols, icons, images, and animations.

[0039] In the present disclosure, the term "focus tunable lens" refers to a lens whose focal length can be adjusted. Liquid crystal (LC) lenses, fluid lenses, and other known focus-tunable optical systems may be used as such a focus tunable lens. As described below, when a user views a virtual image, the distance of the virtual image can be adjusted through the focus tunable lens.

[0040] In the present disclosure, 'focus' refers to the point where a straight line extending a ray parallel to the optical axis of a lens intersects the optical axis after the ray passes through the lens (or optical system). The distance in air from the principal plane of the lens (or optical system) to the focus is called the focal length.

[0041] In the present disclosure, 'refractive index' refers to the rate at which the speed of light decreases in a medium compared to a vacuum.

[0042] In this disclosure, 'refractive power' refers to the power that changes the direction of a ray or optical path by the curvature of a lens. The unit of refractive power is m -1 Alternatively, it is a diopter (D), and the diopter value is expressed as the reciprocal of the focal length. Additionally, the diopter is also referred to as the power of a lens with a corresponding refractive strength. The sign of the refractive power is positive (+) for a convex lens and negative (-) for a concave lens.

[0043] In the present disclosure, 'visual acuity (VA)' refers to the spatial resolution of the eye, that is, the ability to distinguish fine details when viewing a stationary object with the eye. Refractive errors that are too high or too low cause myopia or hyperopia, which can be corrected by means such as eyeglasses, contact lenses, or refractive surgery. Corrected visual acuity may be visual acuity measured for a user wearing lenses with a specified refractive power. The refractive power required for correction refers to the refractive power required to achieve corrected visual acuity.

[0044] In the present disclosure, the term "depth of a virtual image" refers to a distance or location in space where a virtual image is perceived to exist when a user views the virtual image. A 3D image utilizing binocular parallax generates a left-eye virtual image and a right-eye virtual image with different viewing directions, wherein the different viewing directions may be the viewing direction from the user's left eye and the viewing direction from the user's right eye. Accordingly, in a 3D image utilizing binocular parallax, the depth of the virtual image may be a distance calculated from the parallax (i.e., binocular parallax) caused by the viewing direction from the left eye and the viewing direction from the right eye.

[0045] In the present disclosure, 'gaze direction' refers to the direction in which a user gazes, and 'gaze' refers to an imaginary line extending from the user's pupil toward the gaze direction. The gaze direction is primarily calculated and the gaze is estimated from information obtained from an eye tracking sensor.

[0046] In the present disclosure, the term "gaze point" refers to a point where the user gazes, and can be calculated as the point where the user's two eyes intersect. When viewing a 3D image using binocular parallax, the user perceives the 3D image through binocular parallax; therefore, the gaze point obtained through the convergence angle of the user's two eyes may be the point where the user perceives that there is a virtual object (i.e., the depth of the virtual image).

[0047] The present disclosure will be described in detail below with reference to the attached drawings.

[0048] FIG. 1 is a drawing showing the exterior of an augmented reality device (100) according to one embodiment, and FIG. 2 is a plan view of the augmented reality device (100) of FIG. 1.

[0049] Referring to FIGS. 1 and 2, the augmented reality device (100) of the present embodiment is an augmented reality device (Augmented Reality Glasses) configured to be worn by a user, and includes a glasses-type body (101).

[0050] The eyeglass body (101) may, for example, include a frame (102) and temples (103). The frame (102) is on which the lenses (104L, 104R) are located and may, for example, have two rim shapes connected by a bridge. The lenses (104L, 104R) are exemplary and may or may not have refractive power (degree). Alternatively, the lenses (104L, 104R) may be formed as a single unit, in which case the rim and bridge of the frame (102) may not be distinguishable. The lenses (104L, 104R) may be omitted.

[0051] The temples (103) are connected to each end of the frame (102) and extend in one direction. The frame (102) and the temples (103) may be connected by a hinge (105). The hinge (105) is exemplary, and a known member connecting the frame (102) and the temples (103) may be employed. As another example, the frame (102) and the temples (103) may be connected integrally (or continuously).

[0052] An optical engine (110), a waveguide (120), a first lens section (130), a second lens section (140), and electronic components (190) are arranged in the eyeglass-shaped body (101).

[0053] The optical engine (110) is configured to generate light for a virtual image and may be an optical engine of a projector including an image panel, an illumination optical system, a projection optical system, etc. This optical engine (110) may include a left eye optical engine (110L) and a right eye optical engine (110R). The left eye optical engine (110L) and the right eye optical engine (110R) may be located at both ends of the frame (102). As another example, the left eye optical engine (110L) and the right eye optical engine (110R) may be located on the left temple (103L) and the right temple (103R), respectively.

[0054] Optical components are configured to transmit light from a virtual image generated by an optical engine (110) and light from a real scene to the user's pupil, and may include a waveguide (120), a first lens part (130), and a second lens part (140). The optical components may be placed on the left and right sides of the eyeglass body (101), respectively. Left eye optical components and right eye optical components may be placed or attached to the left lens (104L) and the right lens (104R), respectively. Alternatively, the left eye optical components and right eye optical components may be mounted on the frame (102) separately from the lenses (104L, 104R). As another example, the left eye optical components and right eye optical components may be integrally formed and mounted on the frame (102). As yet another example, the optical components may be placed on only one side of the left or right side of the eyeglass body (101).

[0055] The electronic components (190) include a processor (170 in FIG. 4), a user input unit (150 in FIG. 4), and a memory (160 in FIG. 4), and may be located at one location on the frame (102) or temples (104) of the eyeglass body (101) or distributed at multiple locations, and may be mounted on a PCB board, an FPCB board, etc. Although not illustrated, a first lens driving driver circuit for driving a first variable focus lens (131) may be placed adjacent to the first variable focus lens (131). A second lens driving driver circuit for driving a second variable focus lens (141) may be placed adjacent to the second variable focus lens (141). As another example, all or part of the first and second lens driving driver circuits may be located, for example, on a main board.

[0056] With reference to FIGS. 3 and FIGS. 4, the optical configuration and operation of an augmented reality device (100) according to one embodiment of the present disclosure will be described in more detail.

[0057] FIG. 3 schematically illustrates an augmented reality device (100) according to one embodiment of the present disclosure, and FIG. 4 is a block diagram illustrating the components of an augmented reality device (100) according to one embodiment of the present disclosure.

[0058] Referring to FIG. 3, an augmented reality device (100) according to one embodiment of the present disclosure is an optical system capable of displaying both a virtual image and a real scene, and includes an optical engine (110), a waveguide (120), a first lens unit (130), and a second lens unit (140).

[0059] The optical engine (110) is the light (L) of the virtual image. V It is configured to output ).

[0060] In one embodiment, the optical engine (110) comprises a light source that outputs light, an image panel that forms a two-dimensional virtual image using light output from the light source, and light (L) of the virtual image formed in the image panel. V It includes a projection optical system that projects an image and has the same function as a small projector. The light source can be implemented, for example, as an LED or an LD. The image panel can be implemented, for example, as a liquid crystal panel, an LCoS (Liquid Crystal on Silicon) panel, or a DMD (Digital Micromirror Device) panel. The projection optical system may include at least one projection lens.

[0061] In another embodiment, the optical engine (110) may include a light source that outputs light and a two-axis scanner that scans the light output from the light source in two dimensions.

[0062] In another embodiment, the optical engine (110) may include a light source that outputs light, a linear image panel that forms a linear image (i.e., a one-dimensional image) using the light output from the light source, and a one-axis scanner that scans the light of the linear image formed in the linear image panel.

[0063] The waveguide (120) is the light (L) of the virtual image. V ) is output and the light of the real scene (L R It can be formed as a single-layer or multi-layer structure of a transparent material through which light can be reflected and propagated internally. Here, "transparent material" means a material through which light in the visible light band can pass, and the transparency may not be 100% and may have a specific color. The waveguide (120) may have a flat plate shape or a curved plate shape.

[0064] The waveguide (120) is the light (L) of a virtual image projected facing the optical engine (120). VAn input region receiving ) and the light of the incident virtual image (L V A transmission area where ) is transmitted, and light of the virtual image transmitted in the transmission area (L V It includes an output region that outputs ). The input region and the output region are spaced apart. The transmission region may be located between the input region and the output region, or may overlap with at least some of the input region and the output region.

[0065] An input diffraction grating, a transfer diffraction grating, and an output diffraction grating are provided in the input region, the transfer region, and the output region, respectively. If the waveguide (120) is single-layer, the input diffraction grating, the transfer diffraction grating, and the output diffraction grating may be formed on the surface facing the optical engine (110) of the waveguide (120) and / or on the opposite surface. If the waveguide (120) is multi-layer, the input diffraction grating, the transfer diffraction grating, and the output diffraction grating may be formed on each layer of the waveguide (120) or on only some layers. The input diffraction grating is formed from light (L) output from the optical engine (110). V ) can be configured to couple to the waveguide (120). The transfer diffraction grating is configured to receive light (L) input from the input diffraction grating. V It can be configured to transmit ) to an output diffraction grating. For example, the transmission diffraction grating is the input light (L V It can be an expansion grating that allows ) to be replicated into multiple copies. The expansion grating is, incident light (L V When the light propagates through total reflection within the waveguide (120), it may be configured to be divided into multiple beamlets and propagated across the entire output region. The output diffraction grating is configured such that the light (L) propagating within the waveguide (120) V) is configured to output out of the waveguide (120) and may also perform the role of a transmission diffraction grating (e.g., an expansion grating). One of the projection optical systems of the optical engine (110) may be a collimating lens, and the light (L) emitted by the collimating lens V ) becomes parallel light, and accordingly, the light (L) transmitted to the final eye through the waveguide V ) can also be practically considered as parallel light flux. In other words, the light of the virtual image output through the output diffraction grating (L V ) can be regarded as light emitted from practically infinity. Here, the meaning of 'practically' is that, in terms of the visual perspective perceived by humans, the virtual image appears to be at a sufficient distance that is practically close to infinity.

[0066] The waveguide (120) is mounted on a frame so that when the user wears the augmented reality device (100), the output area is positioned in front of the user's pupil. Since the waveguide (120) is formed of a transparent material, the user can not only see virtual images through the augmented reality device (100) but also see real scenes, so the augmented reality device (100) can implement augmented reality.

[0067] The first lens unit (130) performs focus adjustment of the virtual image and correction of the user's vision, and can be positioned on the side where the virtual image is output from the waveguide (120). In other words, when the user wears the augmented reality device (100), the first lens unit (130) is positioned between the waveguide (120) and the user's eye.

[0068] The first lens unit (130) includes a first variable focus lens (131) and a fixed refractive lens (133).

[0069] The first variable focus lens (131) is a lens having a first refractive power that can be varied by a control signal of the processor (170 in FIG. 4). In other words, the first variable focus lens (131) is a lens whose focal length is varied by a control signal of the processor (170).

[0070] The first variable focus lens (131) according to one embodiment may be a liquid crystal lens (LC lens). For example, the first variable focus lens (131) may have liquid crystal positioned between upper and lower transparent substrates, and a common electrode and lens electrodes having a predetermined pattern may be disposed on the side where the upper and lower transparent substrates and the liquid crystal come into contact. The common electrode and the lens electrodes having a predetermined pattern may themselves be transparent electrodes. The liquid crystal lens mimics a Fresnel lens in that the birefringence distribution of the liquid crystal generated as voltage is applied between the common electrode and the lens electrodes.

[0071] FIG. 5 is a drawing illustrating a first variable focus lens (131) according to one embodiment. Referring to FIG. 5, the first variable focus lens (131) has a structure in which a liquid crystal layer (1314) is interposed between opposing first and second substrates (1311, 1318). A plurality of first electrodes (1312) having a predetermined pattern are provided on the first substrate (1311). The first electrodes (lens electrodes) (1312) may be arranged in two dimensions on the first substrate (1311). In one embodiment, the first electrodes (1312) may be formed in a concentric annular pattern. In another embodiment, the first electrodes (1312) may be formed in a two-dimensional pixel array pattern. Voltage may be applied to the first electrodes (1312) individually (i.e. independently) or in predetermined group units. A common second electrode (common electrode) (1317) is provided on the second substrate (1318). The second electrode (1317) can serve as a reference electrode for the first electrodes (1312). In some cases, the second electrode (1317) may also have an electrode array form. The positions of the first electrodes (1312) and the second electrode (1317) may be reversed. Reference numerals 1313 and 1316 represent an orientation layer that orients liquid crystal molecules (1315) within the liquid crystal layer (1314) in a predetermined direction. The original orientation of the liquid crystal molecules (1315) is determined by the direction of the force applied in the orientation layer (1313, 1316), but when an appropriate voltage is applied, the liquid crystal molecules (1315) rotate. Accordingly, when a voltage is applied to the liquid crystal layer (1314), the refractive index of the liquid crystal layer (1314) changes due to the reorientation of the liquid crystal molecules. By spatially applying a voltage profile to the first electrodes (1312) formed in a predetermined pattern to locally adjust the effective refractive index, the liquid crystal layer (1314) can provide a phase modulation profile having a desired focal length.

[0072] FIG. 6 is a phase profile of a first variable focus lens when the control signal is a voltage profile corresponding to a concave lens with a predetermined refractive power (e.g., -2D), and FIG. 7 is a phase profile of a first variable focus lens when the control signal is a voltage profile corresponding to a concave lens with a predetermined refractive power (e.g., -3D).

[0073] Referring to FIG. 6, when a voltage profile corresponding to a -2D concave lens is applied between the first electrodes (1312) and the second electrode (1317), the birefringence distribution of the liquid crystal layer (1314) generated accordingly mimics a Fresnel lens having a -2D refractive power.

[0074] Similarly, referring to FIG. 7, when a voltage profile corresponding to a -3D concave lens is applied between the first electrodes (1312) and the second electrode (1317), the resulting birefringence distribution of the liquid crystal layer (1314) mimics a Fresnel lens having a -3D refractive power.

[0075] This embodiment is described with the example that the first variable focus lens (131) is a liquid crystal lens, but it is not limited thereto. For example, electro-optical materials whose refractive index can change depending on the applied electric field, such as electroactive polymers, liquid crystalline polymers, or polymer dispersed liquid crystals, may be used instead of liquid crystals. As another example, the first variable focus lens (131) may be a fluid lens that collects or disperses light using the interface between two immiscible liquids.

[0076] In the case of the liquid crystal lens or other known variable focus lenses as described above, there may be limitations on the variable range of refractive power, the time required for variable power, resolution, etc., depending on the limitations of the manufacturing process, the characteristics of the liquid crystal material, or the driving method. Additionally, considering that the augmented reality device (100) is used while being worn by a user, there may be limitations on mechanical size or power. Therefore, as described below, there may be some limitations in resolving the user's abnormal refraction using only the first variable focus lens (131).

[0077] The fixed refractive lens (133) is an optical member having a fixed refractive power. In one embodiment, the fixed refractive lens (133) may be a concave lens having a negative (-) refractive power. This embodiment describes the case where the fixed refractive lens (133) is a concave lens, but it is not limited thereto. For instance, as another example, the fixed refractive lens (133) may be a Fresnel lens having a negative (-) refractive power, a GRIN (Graded Refractive Index) lens, a metalen, etc. As yet another example, the fixed refractive lens (133) may be a convex lens having a positive (+) refractive power. The refractive power information of the fixed refractive lens (133) may be stored in the memory (160).

[0078] The second lens unit (140) compensates for the distortion of the real scene caused by the first lens unit (130) and can be positioned on the opposite side of the surface where the first lens unit (130) is located, with the waveguide (120) in between. In other words, when a user wears the augmented reality device (100), the second lens unit (140) is positioned on the outer side of the waveguide (120) (the side where the real scene is located).

[0079] The second lens unit (140) includes a second variable focus lens (141).

[0080] The second variable focus lens (141) is a lens having a second refractive power that is variable by a control signal of the processor (170). This second variable focus lens (141) may have substantially the same configuration as the first variable focus lens (131) described above. In one embodiment, the second variable focus lens (141) may be a liquid crystal lens.

[0081] The first variable focus lens (131) and the second variable focus lens (141) may be attached to the waveguide (120) to form a stack structure. As another example, the first variable focus lens (131) and the second variable focus lens (141) may be spaced apart from the waveguide (120) at a predetermined distance. The fixed refractive lens (133) may also be attached to the first variable focus lens (131) or spaced apart from it at a predetermined distance. This embodiment describes the case where the first variable focus lens (131) is positioned between the waveguide (120) and the fixed refractive lens (133), but it is not limited thereto. As another example, the fixed refractive lens (133) may be positioned between the waveguide (120) and the first variable focus lens (131).

[0082] Referring to FIG. 4, the augmented reality device (100) may include a user input unit (150), a memory (160), and a processor (170) together with an optical system having a first variable focus lens (131) and a second variable focus lens (141).

[0083] Among the components shown in FIG. 4, the components having the same reference number as the components shown in FIG. 3 are identical to the components shown in FIG. 3. Therefore, redundant descriptions are omitted.

[0084] The user input unit (150) refers to a means for a user to input data for controlling the augmented reality device (100). For example, the user input unit (150) may include at least one of a key pad, a dome switch, a touch pad (contact capacitive method, pressure resistive method, infrared detection method, surface ultrasonic conduction method, integral tension measurement method, piezo effect method, etc.), a jog wheel, or a jog switch, but is not limited thereto. The user input unit (150) may receive user input related to at least one of the user's vision information and the focal length of a virtual image.

[0085] The memory (160) can store various data, programs, or applications that drive and control the augmented reality device (100) under the control of the processor (170), and data of input / output signals or virtual images. As examples of various data that drive and control the augmented reality device (100), the memory (160) may have user vision information, refractive power information of a fixed refractive lens, refractive power variable range of the first and second variable focus lenses (131, 141), etc., stored in advance. Alternatively, the voltage profile itself that enables the first and second variable focus lenses (131, 141) to operate with the corresponding refractive power may be stored in advance. The data of the virtual image may include attribute distance information of a virtual object within the virtual image.

[0086] The memory (160) may include at least one type of hardware device among, for example, a flash memory type, RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

[0087] The processor (170) may be composed of at least one of hardware such as, for example, a Central Processing Unit, a microprocessor, a Graphic Processing Unit, ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), and FPGAs (Field Programmable Gate Arrays), but is not limited thereto.

[0088] The processor (170) can control the overall operation of the augmented reality device (100), including the optical engine (120) and the first and second variable focus lenses (131, 141), by running an operating system or application, and can perform various data processing and calculations, including image data. For example, the processor (170) can determine the first refractive power of the first variable focus lens (131) based on the user's vision information, the focal length information of the virtual image, and the fixed refractive power information of the fixed refractive lens (133), and control the first variable focus lens (131) with a control signal corresponding to the first refractive power. As an example, the control signal may be the voltage profile itself applied to the first variable focus lens (131). As another example, the control signal may be a control command signal corresponding to preset voltage profiles. Additionally, the processor (170) can determine the second refractive power of the second variable focus lens (141) based on the focal length information of the virtual image and control the second variable focus lens (141) with a control signal corresponding to the second refractive power.

[0089] FIG. 8 is a flowchart illustrating the operation of an augmented reality device (100) according to the present embodiment.

[0090] Referring to FIG. 8, in order to determine the first and second refractive powers of the first and second variable focus lenses (131, 141) by reflecting the user's vision, the processor (170) loads the focal length of a virtual image, the user's vision information, and the fixed refractive power information from the memory (160) (S210).

[0091] As described above, the virtual image output from the waveguide (120) can be considered to be at a position of substantial infinity. Since the user views the virtual image output from the waveguide (120) through the first lens unit (130), the focal position of the virtual image is moved by the first lens unit (130).

[0092] Meanwhile, if the user has a refractive error, the movement of the focal position of the virtual image is restricted due to the user's refractive error, and a corrective refractive power is required to correct this refractive error. For example, if the user has a myopic refractive error, the corrective lens to correct it may require a corrective refractive power of minus diopters (e.g., -3D). According to one embodiment, the user's visual acuity information is a value of the corrective refractive power required, and the user's corrective refractive power may be stored in memory (150). If there are multiple users, the user's visual acuity information may be user identification information and the corresponding user's corrective refractive power required. This user's visual acuity information is stored in advance in memory (160). In one embodiment, the user's visual acuity information may be entered directly by the user, for example, through the user input unit (150). In one embodiment, the user's visual acuity information may be stored in another electronic device and may be received from the other electronic device via a wired or wireless method and stored in memory (160).

[0093] Virtual objects within a virtual image may include, for example, at least one of characters, numbers, symbols, icons, images, and animations. Virtual objects may be 3D objects as well as 2D objects.

[0094] In one embodiment, a virtual object may be natural to the user when it is perceived as being located at a certain distance due to its attributes. For example, when a user looks at a desk or table while sitting on a chair or sofa, the virtual image (virtual object) may contain an image of an item virtually placed on the desk or table or information about the item placed on the desk or table, and the attribute distance of such a virtual image (virtual object) may be approximately 0.5 to 0.7 meters. As another example, when a user is shopping at a store, the virtual image (virtual object) may display information about products in the store, and the attribute distance of such a virtual image (virtual object) may be approximately 1 to 2 meters. Accordingly, representative distance information of the virtual image (virtual object) or focal distance information corresponding to the attributes of each virtual image (virtual object) may be stored in memory (150) together with the virtual image data.

[0095] Next, the processor (170) determines the first refractive power of the first variable focus lens (131) based on the focal length of the virtual image, the user's vision information, and the fixed refractive power information of the fixed refractive lens (133) (S220).

[0096] In the case where the user has a refractive error as described above, the first refractive power of the first variable focus lens (131) of the first lens part (130) can be determined as follows.

[0097]

[0098] Here, D1 represents the first refractive power of the first variable focus lens (131), and D 고정 represents the fixed refractive power of the fixed refractive lens (133), and D 교정 represents the refractive power required to correct the user's refractive error. f represents the focal length of the virtual image.

[0099] The processor (170) can enable the user to naturally view the virtual image with corrected vision by adjusting the first refractive power of the first variable focus lens (131) so that the focal length f of the virtual image becomes the attribute distance of the virtual image.

[0100] This embodiment is described as an example where the virtual image (virtual object) has an attribute distance, but the virtual image (virtual object) may not have an attribute distance. In another embodiment, the virtual image focal length f is a fixed value independent of the attributes of the virtual image (virtual object), and can be set to, for example, 0.5 m or 0.7 m, taking into account the sun where the augmented reality device is used. In yet another embodiment, the virtual image focal length f may be a value adjustable by user input independent of the attributes of the virtual image (virtual object).

[0101] Next, the processor (170) determines the second refractive power of the second variable focus lens (141) based on the focal length information of the virtual image (S230).

[0102] Light originating from a real object passes through the second lens unit (140), the waveguide (120), and the first lens unit (130) and enters the user's pupil. However, light originating from the real scene may be refracted by the first refractive power of the first variable focus lens (131) of the first lens unit (130) and the fixed refractive power of the fixed refractive lens (133), thereby causing distortion in the real scene. Accordingly, the second variable focus lens (141) of the second lens unit (140) may have a predetermined refractive power to compensate for the distortion of the real scene caused by the first lens unit (130). That is, the second refractive power of the second variable focus lens (141) can be determined as follows.

[0103]

[0104] Here, D2 represents the second refractive power of the second variable focus lens (131).

[0105] Table 1 below exemplarily shows the first refractive power of the first variable focus lens (131), the second refractive power of the second variable focus lens (141), and the fixed refractive power of the fixed refractive lens (133) (concave lens) according to the user's vision.

[0106] division User's vision (D 교정 ) virtual focus (f) Concave lens (D 고정 )(-2D) 1st variable focus lens (D1) (+3D~-3D) 2nd variable focus lens (D2) (+3D~-3D) 1 3D myopia (-3D) Virtual @ 0.5m -2D -3D +2D 2 2D myopia (-2D) -2D -2D +2D 3 1.5D myopia (-1.5D) -2D -1.5D +2D 4 1D myopia (-1D) -2D -1D +2D 5 3D myopia (-3D) Virtual @ 0.7m -2D -2.5D +1.5D 6 2D myopia (-2D) -2D -1.5D +1.5D 7 1.5D myopia (-1.5D) -2D -1D +1.5D 8 1D myopia (-1D) -2D -0.5D +1.5D

[0107] Classification 1 of Table 1 is such that in the case where the user's required refractive power for correction is -3D, the focal length of the virtual image (virtual focus) is 0.5m, and the fixed refractive power of the concave lens is -2D, the first refractive power of the first variable focus lens (131) can be -3D and the second refractive power of the second variable focus lens (141) can be +2D.

[0108] As another example, in the case where the user's required refractive power for correction is -3D, the focal length of the virtual image (virtual focus) is 0.7m, and the fixed refractive power of the concave lens is -2D, the first refractive power of the first variable focus lens (131) can be -2.5D, and the second refractive power of the second variable focus lens (141) can be +1.5D. When the focal length of the virtual image is 0.7m, It can typically be calculated by considering it as 1.5D.

[0109] Overall, the sum of the first refractive power of the first variable focus lens (131) of the first lens section (130) and the fixed refractive power of the fixed refractive lens (133) and the second refractive power of the second variable focus lens (141) of the second lens section (140) are asymmetric. In other words, the absolute value of the sum of the refractive powers of the first lens section (130) and the absolute value of the refractive power of the second lens section (140) are not equal.

[0110] As described above, the first and second variable focus lenses (131, 141) may have limitations regarding the variable range of refractive power, the time required for variable power, resolution, etc., depending on the limitations of the manufacturing process, the characteristics of the material, or the driving method. For example, the effective refractive power variable range of the first and second variable focus lenses (131, 141) may be +3D to -3D. As shown in Table 1 above, the augmented reality device (100) of the present embodiment can determine the first and second refractive powers of the first and second variable focus lenses (131, 141) within the effective refractive power variable range.

[0111] Meanwhile, the first and second variable focus lenses (131, 141) may have limitations in having a large refractive power due to limitations in pattern miniaturization of the lens electrode (e.g., 1312 in FIG. 5) or instability of liquid crystal alignment at points where a rapid change in optical phase is required. For instance, if a user has a refractive error, it may be difficult for the first variable focus lens (131) alone to handle the required refractive power. In this embodiment, by placing a fixed refractive lens (concave lens) (133) in the first lens part (130), the burden of the refractive power of the first variable focus lens (131) is reduced, and accordingly, high resolution of the virtual image can be achieved.

[0112] The above-described embodiment is explained by exemplifying a case where the processor (170) loads the focal length of a virtual image, the user's vision information, and the fixed refractive power information from the memory (160), respectively, and determines the first refractive power of the first variable focus lens (131) from therein, but is not limited thereto. Since the user's vision information and the fixed refractive power information for a specific user are already fixed values, the user's vision information and the fixed refractive power information can be pre-calculated as shown in the following mathematical formula and pre-stored in the memory (160).

[0113]

[0114] Here, D수정 It can be understood as the refractive power requiring correction, which reflects the refractive power of the fixed refractive lens.

[0115] FIG. 9 is a flowchart illustrating the operation of an augmented reality device according to one embodiment. This embodiment describes a refractive power D requiring correction. 수정 This is a case where it is stored in advance in memory (160). Referring to FIG. 9, in order to determine the first and second refractive powers of the first and second variable focus lenses (131, 141) by reflecting the user's eyesight, the processor (170) determines the refractive power D requiring correction. 수정 The focal length information f of the virtual image is loaded from memory (160) (S310).

[0116] Next, the first refractive power of the first variable focus lens (131) of the first lens unit (130) is the refractive power D requiring correction in the memory (160). 수정 After reading the focal length of the virtual image, it can be determined as shown in the following mathematical formula (S320).

[0117]

[0118] In this way, by using the refractive power requiring correction that has been pre-calculated from the user's vision information and fixed refractive power information, the number of pieces of information loaded from memory (160) can be reduced, and the calculation for determining the first refractive power can be made more simplified.

[0119] Next, the processor (170) determines the second refractive power of the second variable focus lens (141) based on the focal length information of the virtual image as in Equation 2 (S330).

[0120] FIG. 10 is a diagram illustrating the operation of an augmented reality device (100) according to one embodiment.

[0121] A user with abnormal refraction requires refractive power that needs correction due to abnormal refraction. Meanwhile, a virtual image (virtual object) (O) displayed on an augmented reality device (100)V ) has a focal length that matches its attributes. Or a virtual image (virtual object) (O V ) can use a representative distance pre-entered into the augmented reality device (100) as the focal length. When the user wears the augmented reality device (100) and a virtual image (virtual object) (O V In order to view ), the processor (170) determines the first refractive power of the first variable focus lens (131) and the second refractive power of the second variable focus lens (141) as described with reference to FIG. 8 or FIG. 9, and controls the first and second variable focus lenses (131, 141) in correspondence with the determined first and second refractive power values. As a result, even if the user has abnormal refraction, vision correction can be performed by the first lens unit (130), and furthermore, a virtual image (virtual object) (O) can be performed by the first lens unit (130). V ) becomes visible from infinity distance at focal length f, so virtual image (virtual object) (O V The user can see the image clearly and naturally. In addition, due to the second refractive power of the second lens unit (140), the user can see the real scene without distortion caused by the first lens unit (130).

[0122] FIG. 11 is a diagram illustrating the arrangement of optical components of an augmented reality device (400) according to one embodiment.

[0123] Referring to FIG. 11, the augmented reality device (400) of the present embodiment includes an optical engine (110), a waveguide (120), a first lens unit (430), and a second lens unit (140). Since the augmented reality device (400) of the present embodiment is identical to the previously described embodiments except that the first lens unit (430) further includes a polarizer (432), the explanation will focus on the differences.

[0124] The first lens section (430) includes a first variable focus lens (131), a polarizer (432), and a fixed refractive lens (133). In one embodiment, the polarizer (432) may be placed between the first variable focus lens (131) and the fixed refractive lens (133). The polarizer (432) may be configured to allow light of the first polarization to pass through and to block light of the second polarization that is orthogonal to it. In this case, the first polarization may be linear polarization (e.g., p-polarization). The first variable focus lens (131) may be a liquid crystal lens. Due to its birefringence characteristics, the liquid crystal lens may have a different refractive index for light of the first polarization (e.g., p-polarization) and light of the second polarization (e.g., s-polarization) that is orthogonal to the first polarization. Therefore, by placing a polarizer (432) between the first variable focus lens (131) and the fixed refractive lens (133), light of a different refractive magnitude (i.e., noise) among the light passing through the first variable focus lens (131) can be removed.

[0125] FIG. 11 illustrates a case where a polarizing plate (432) is positioned between a first variable focus lens (131) and a fixed refractive lens (133), but is not limited thereto. In one embodiment, the polarizing plate (432) may be positioned between a waveguide (110) and a first variable focus lens (131). That is, they may be positioned in the order of a waveguide (110), a polarizing plate (432), a first variable focus lens (131), and a fixed refractive lens (133). As another example, they may be positioned in the order of a waveguide (110), a fixed refractive lens (133), a polarizing plate (432), and a first variable focus lens (131), or in the order of a waveguide (110), a polarizing plate (432), a fixed refractive lens (133), and a first variable focus lens (131).

[0126] FIG. 12 is a diagram illustrating the arrangement of optical components of an augmented reality device according to one embodiment.

[0127] Referring to FIG. 12, the augmented reality device (500) of the present embodiment includes an optical engine (110), a waveguide (120), a first lens unit (130), and a second lens unit (540). The first lens unit (130) includes a first variable focus lens (131) and a first fixed refractive lens (133), and the second lens unit (540) includes a second variable focus lens (141) and a second fixed refractive lens (543). In one embodiment, the second fixed refractive lens (543) may be a convex lens having positive (+) refractive power. The present embodiment is described with the case where the second fixed refractive lens (543) is a convex lens as an example, but is not limited thereto. For example, as another example, the second fixed refractive lens (543) may be a Fresnel lens, GRIN lens, meta lens, etc., having positive refractive power. The augmented reality device (500) of this embodiment is identical to the previously described embodiments except that the second lens part (540) further includes the second fixed refractive lens (543).

[0128] The refractive power D'2 that the second lens part (540) must have to compensate for the distortion of the real scene caused by the first lens part (130) is given as in the aforementioned mathematical formula 5.

[0129]

[0130] The refractive power D'2 of the second lens part (540) is the second refractive power D2 of the second variable focus lens (141) and the fixed refractive power D of the second fixed refractive lens (543). 고정2 Since it is given as the sum, the refractive power D2 of the second variable focus lens (141) is given by the following mathematical formula 6.

[0131]

[0132] As described above, the second lens unit (540) compensates for the distortion of the real scene caused by the first lens unit (130), but depending on the user's eyesight, the second refractive power that the second variable focus lens (141) of the second lens unit (540) must handle may be excessive. Accordingly, the present embodiment reduces the burden on the second variable focus lens (141) by distributing the second refractive power required for the second variable focus lens (141) with the second fixed refractive lens (543), thereby enabling high resolution of the virtual image.

[0133] Table 2 below shows the first refractive power D1 of the first variable focus lens (131), the second refractive power D2 of the second variable focus lens (141), and the fixed refractive power D of the first fixed refractive lens (133) (concave lens) according to the user's eyesight. 고정1 , and the fixed refractive power D of the second fixed refractive lens (543) (convex lens) 고정2 It shows an example of.

[0134] division User's vision (D 교정 ) virtual focus (f) First fixed refractive lens (D 고정1 )(-2D) 1st variable focus lens (D1) (+3D~-3D) Second fixed refractive lens (D 고정2 )(+1D) 2nd variable focus lens (D2) (+3D~-3D) 1 3D myopia (-3D) Virtual @ 0.5m -2D -3D +1D +1D 2 2D myopia (-2D) -2D -2D +1D +1D 3 1.5D myopia (-1.5D) -2D -1.5D +1D +1D 4 1D myopia (-1D) -2D -1D +1D +1D 5 3D myopia (-3D) Virtual @ 0.7m -2D -2.5D +1D +0.5D 6 2D myopia (-2D) -2D -1.5D +1D +0.5D 7 1.5D myopia (-1.5D) -2D -1D +1D +0.5D 8 1D myopia (-1D) -2D -0.5D +1D +0.5D

[0135] Category 1 in Table 2 refers to the user's required refractive power D for correction. 교정 is -3D, the focal length (virtual focus) f of the virtual image is 0.5m, and the fixed refractive power D of the first fixed refractive lens (133) (concave lens) 고정1 is -2D, and the fixed refractive power D of the second fixed refractive lens (543) (convex lens) 고정2 In the case of this +1D, the first refractive power D1 of the first variable focus lens (131) can be -3D, and the second refractive power D2 of the second variable focus lens (141) can be +1D.

[0136] As another example, Category 5 is the user's refractive power requiring correction D 교정 is -3D, the focal length (virtual focus) f of the virtual image is 0.7m, and the fixed refractive power D of the first fixed refractive lens (133) (concave lens) 고정1is -2D, and the fixed refractive power D of the second fixed refractive lens (543) (convex lens) 고정2 In the case of this +1D, the first refractive power D1 of the first variable focus lens (131) can be -2.5D, and the second refractive power D2 of the second variable focus lens (141) can be +0.5D. When the focal length of the virtual image is 0.7m, It can typically be calculated by considering it as 1.5D.

[0137] Overall, the sum of the first refractive power of the first variable focus lens (131) of the first lens section (130) and the fixed refractive power of the first fixed refractive lens (133) is asymmetrical with the sum of the second refractive power of the second variable focus lens (141) of the second lens section (540) and the fixed refractive power of the second fixed refractive lens (543). In other words, the absolute value of the sum of the refractive powers of the first lens section (130) and the absolute value of the sum of the refractive powers of the second lens section (540) are not equal.

[0138] FIG. 13 is a block diagram of an augmented reality device (600) according to one embodiment.

[0139] Referring to FIG. 13, the augmented reality device (600) of the present embodiment includes an optical engine (110), a first variable focus lens (131), and a second variable focus lens (141), along with a user input unit (150), a memory (160), a processor (170), and an eye tracking sensor (680). Since the augmented reality device (600) of the present embodiment is substantially the same as the augmented reality device (100) of the embodiment described with reference to FIG. 4 except that it has an additional eye tracking sensor (680), the description will focus on the differences arising from the additional eye tracking sensor (680).

[0140] The eye tracking sensor (680) is a device for tracking the direction of a user's gaze, and can detect the direction of the user's gaze by detecting an image of a person's pupil or iris, or by detecting the direction or amount of light reflected from the cornea by near-infrared light. This eye tracking sensor (680) includes a left eye eye tracking sensor and a right eye eye tracking sensor, and can detect the direction of the user's gaze of the left eye and the direction of the user's gaze of the right eye, respectively. Detecting the direction of the user's gaze may include the operation of acquiring gaze information related to the user's gaze.

[0141] FIG. 14 illustrates an eye-tracking sensor (680) according to one embodiment. Referring to FIG. 14, the eye-tracking sensor (680) may include an infrared irradiation unit (681) and a plurality of infrared detection units (685a to 685f). Although the plurality of infrared detection units (685a to 685f) are shown as six in FIG. 14, this is for convenience of explanation and the number of the plurality of infrared detection units (685a to 685f) is not limited to that shown.

[0142] The infrared irradiation unit (681) irradiates infrared light onto the corneal portion where the lens of the eye (E) is located, and a plurality of infrared detection units (685a to 685f) can detect infrared light reflected from the cornea. In one embodiment, the gaze tracking sensor (680) acquires information regarding the amount of infrared light detected by each of the plurality of infrared detection units (685a to 685f), and can acquire information regarding the direction of gaze of the user's eye (E) based on the acquired amount of infrared light. The gaze tracking sensor (680) can provide the acquired information regarding the direction of gaze to the processor (170). For example, the information regarding the direction of gaze acquired by the gaze tracking sensor (680) may be gaze angle information in the horizontal and vertical directions of the left eye and gaze angle information in the horizontal and vertical directions of the right eye.

[0143] The eye tracking sensor (680) of the present embodiment is described using an IR scanner method that utilizes infrared illumination, but is not limited thereto. As another example, the eye tracking sensor (680) may include an image sensor that captures an image of a person's eyeball or pupil. Based on the captured image of the user's eye, information on the gaze angle in the horizontal and vertical directions of the left eye and information on the gaze angle in the horizontal and vertical directions of the right eye may be detected.

[0144] Referring again to FIG. 13, according to one embodiment, the eye tracking sensor (680) can sense the eyes of a user wearing an augmented reality device (600) at predetermined time intervals. The processor (170) can calculate the user's gaze point by calculating information regarding the gaze direction of each of the left and right eyes detected by the eye tracking sensor (680). For example, when the user views an object in a real scene along with a virtual image displayed by the augmented reality device (100), the processor (170) can determine the depth (i.e., focal length) of the virtual image based on the calculated gaze point.

[0145] Next, with reference to FIGS. 15 to 17, a method for calculating a gaze point from information regarding the gaze direction measured by the gaze tracking sensor (680) will be explained.

[0146] Figure 15 is a diagram illustrating a three-dimensional eye model of the user's gaze direction.

[0147] Referring to FIG. 15, tracking of the gaze direction according to one embodiment can be based on a three-dimensional eye model of the gaze. The three-dimensional eye model of the gaze can mathematically model the gaze as shown in the following mathematical formula, provided that the eyeball is assumed to be a perfect sphere and that the eyeball undergoes ideal spatial rotational motion according to the gaze.

[0148]

[0149]

[0150]

[0151]

[0152] In Equation 7, d represents the distance between the center of the user's eye (Eo) and the virtual screen (S); α represents the angle of rotation of the user's eye in the x-axis (horizontal axis) direction relative to the case where the user's eye gazes directly at the virtual screen (S); and β represents the angle of rotation of the user's eye in the y-axis (vertical axis) direction relative to the case where the user's eye gazes directly at the virtual screen (S). Additionally, in Equation 8, r represents the radius of the sphere when the user's eye is assumed to be a sphere.

[0153] A gaze tracking sensor (680) according to one embodiment can measure the degree of rotation (e.g., α and β) of the user's eye (E), and an augmented reality device (500) can calculate two-dimensional position coordinate values ​​(x, y) of the direction of gaze of the user's eye (E) on a virtual screen (S) using the degree of rotation (α and β) of the user's eye (E). The degree of rotation (α and β) of the eye (E) can be understood as gaze angle information in the horizontal and vertical directions of the eye (E).

[0154] Actual eye movements do not involve ideal three-dimensional rotational motion, and especially in left and right gaze, the relaxation / contraction of eye muscles plays a significant role, so errors may occur when estimating the gaze in the up and down direction for left and right gaze based on an ideal three-dimensional rotational eye model. Such errors can be improved by having the augmented reality device (600) have the user look at a random point and statistically processing the estimated gaze direction value through the gaze tracking sensor (680) by comparing it with the actual gaze direction value for that point.

[0155] FIG. 16 is a diagram illustrating the relationship between the gaze angle and the gaze point in the left and right eyes, and FIG. 17 is a diagram illustrating the relationship between the gaze angle and the gaze point in the upward gaze direction.

[0156] Referring to FIGS. 16 and 17, the focal distance can be estimated through the difference in the gaze direction (or gaze coordinates) of both eyes obtained through the gaze tracking sensor (680). When calculating the focal distance to the gaze point, the gaze axes of both eyes may not meet; in such cases, it is assumed that the two eyes are at the same height, and the vertical axis (y-axis) coordinate can be calculated as the average of the vertical axis (y-axis) coordinates of the two eyes. The distance a between the two eyes can be assumed to be, for example, 7 cm. Using a proportional equation based on the above geometric assumptions, the following equation can be obtained.

[0157]

[0158] In the above mathematical formula 9, the distance d to the virtual screen and the distance a between the eyes are required. The distance d can be calculated by measuring the rotation angle of the eyeball using a gaze image of the user looking straight ahead. As a result, the distance D to the fixation point is given by the following mathematical formula.

[0159]

[0160] Here, △x is the horizontal interval of the gaze coordinates on the virtual screen (S) of both eyes, which can be obtained through the gaze angles of the user's left and right eyes, respectively, as can be seen in Equations 7 and 8 above.

[0161] FIG. 18 is a flowchart illustrating the operation of an augmented reality device (600) according to one embodiment.

[0162] Referring to FIG. 18, in order to determine the first and second refractive powers of the first and second variable focus lenses (131, 141) by reflecting the user's vision, the augmented reality device (600) obtains the focal length of a virtual image.

[0163] First, the eye tracking sensor (680) of the augmented reality device (600) obtains information about the direction of the user's left eye's gaze and information about the direction of the user's right eye's gaze (S710).

[0164] Next, as in the example described in FIGS. 15 to 17, the processor (170) of the augmented reality device (600) calculates information about the gaze point from information about the direction of the user's left eye's gaze and information about the direction of the user's right eye's gaze (S720).

[0165] Next, the processor (170) determines the focal length of the virtual image based on the obtained gaze point (S730).

[0166] In one embodiment, when a user views a real scene along with a virtual image displayed by an augmented reality device (100), the user may gaze at a real object that is the subject of their interest in the real scene, and it may be natural for the virtual image (virtual object) to be placed at the same depth as this real object. Thus, the focal length of the virtual image can be set to a depth similar to the depth to the user's gaze point (i.e., the distance between the user's eye and the gaze point). Here, "similar depth" means not only the case where the focal length of the virtual image is the same as the depth to the user's gaze point, but also encompasses a depth within a range that the user can naturally perceive. For example, the focal length of the virtual image may be changed within the approximate size range of the real object.

[0167] As described below, by setting the focal length of the virtual image to a depth corresponding to the user's gaze point detected by the eye tracking sensor (680), the user can naturally view the real scene along with the virtual image displayed by the augmented reality device (100).

[0168] Although this embodiment describes steps S720 and S730 separately, step S730 can be substantially omitted by considering the distance to the calculated gaze point as the focal length of the virtual image.

[0169] Meanwhile, the processor (170) loads the user's vision information and fixed refractive power information from memory (160) (S740). Step S740 may be reversed from or performed simultaneously with steps S710 through S730.

[0170] Next, the processor (170) determines the first refractive power of the first variable focus lens (131) based on the focal length of the virtual image, the user's vision information, and the fixed refractive power information of the fixed refractive lens (133) (S720). For example, the first refractive power of the first variable focus lens (131) of the first lens part (130) can be determined as in the aforementioned mathematical formula 1, allowing the user to naturally view the virtual image with corrected vision.

[0171] This embodiment is described with the example that the user's vision information and fixed refractive power information are stored in advance in the memory (160), but it is not limited thereto. As in the example described with reference to FIG. 9, the user's vision information and fixed refractive power information are calculated in advance and stored in the memory (160) with a value such as the refractive power requiring correction, and the processor (170) may determine the first refractive power of the first variable focus lens (131) based on the focal length of the virtual image and the refractive power requiring correction information.

[0172] Next, the processor (170) determines the second refractive power of the second variable focus lens (141) of the second lens unit (140) based on the focal length information of the virtual image (S760). The second refractive power of the second variable focus lens (141) can be determined as in the aforementioned mathematical formula 2 and compensates for the distortion of the real scene caused by the first lens unit (130).

[0173] In this embodiment, the virtual image (virtual object) can be 3D as well as 2D. For example, the virtual image can provide a sense of depth through binocular parallax. A virtual image utilizing binocular parallax generates a left-eye virtual image and a right-eye virtual image at different viewpoints, where the different viewpoints may be the viewpoint from the user's left eye and the viewpoint from the user's right eye. Therefore, by making the virtual image have binocular parallax corresponding to the focal length determined in step S730, the user may be able to view the virtual image more naturally.

[0174] FIG. 19 is a diagram illustrating the operation of an augmented reality device according to one embodiment.

[0175] While the user is wearing an augmented reality device (100), they can see products (real objects) (O) in a store R When viewing ), the augmented reality device (100) provides information about the product as a virtual image (virtual object) (O V It can be provided (i.e., displayed) as ). In other words, the user can provide (i.e., display) the product (real-world object) (O R When a user gazes at ), the eye tracking sensor (680) of the augmented reality device (100) tracks the user's gaze, and the processor (170) calculates the gaze point from the information regarding the tracked direction of the user's gaze and from this, the product (real object) (O R The distance to ) virtual image (virtual object) (O V The focal length f of ) can be determined. The processor (170) is a virtual image (virtual object) (O V The first refractive power of the first variable focus lens (131) is determined based on the focal length f of ), the user's vision information, and the fixed refractive power information of the fixed refractive lens (133), and a virtual image (virtual object) (O VBased on the focal length f of the second lens unit (140), the second refractive power of the second variable focus lens (141) is determined, and the first and second variable focus lenses (131, 141) are controlled in correspondence with the determined first and second refractive power values. As a result, even if the user has abnormal refraction, vision correction can be performed by the first lens unit (130), and furthermore, a virtual image (virtual object) (O) can be created by the first lens unit (130). V ) at infinite distance product(real object)(O R By pulling the focal length f to the vicinity of the location where ) is placed, the virtual image (virtual object) (O V ) can be seen clearly and naturally. In addition, due to the second refractive power of the second lens part (140), the user can see the product (real object) (O R ) can be viewed without distortion caused by the first lens part (130).

[0176] FIG. 20 is a block diagram of an augmented reality device (800) according to one embodiment.

[0177] Referring to FIG. 20, the augmented reality device (800) may include a user input unit (150), a memory (160), a processor (170), and a microphone (890), along with an optical system having an optical engine (110), a first variable focus lens (131), and a second variable focus lens (141). Since components shown in FIG. 20 that have the same reference number as components shown in FIG. 4 are identical to components shown in FIG. 4, redundant descriptions will be omitted.

[0178] The microphone (890) receives an external acoustic signal and processes it into electrical voice data. For example, the microphone (890) can receive an acoustic signal from an external device or a speaker. The microphone (890) can use various noise removal algorithms to remove noise generated during the process of receiving the external acoustic signal. The microphone (890) can receive voice input from a user to control the augmented reality device (800). The microphone (890) can receive voice input from a user reading a character (602 in FIG. 13) displayed through the augmented reality device (800).

[0179] Referring to FIGS. 21 to 23, an example of a specific operation for obtaining the user's correction required refractive power will be described.

[0180] FIG. 21 is a diagram showing an example in which an augmented reality device (800) according to one embodiment performs an operation to obtain the user's correction required refractive power when the accuracy rate of the user's voice input is low.

[0181] Referring to FIG. 14, the augmented reality device (800) can sequentially display characters of a certain size at a focal distance for visual acuity measurement and receive voice input from a user regarding the displayed characters. For example, the augmented reality device (800) can sequentially display the characters 'B' (812), '0' (814), and 'E' (816) at different locations on a virtual visual acuity test board (801) displayed at a focal distance for visual acuity measurement. In this case, the characters 'B' (812), 'E' (814), and 'O' (816) displayed on the visual acuity test board (800) may appear severely blurry to a user with poor eyesight, as shown in FIG. 14. Accordingly, after displaying the character 'B' (812), the augmented reality device (800) can receive voice input from a user saying, "I can't see it." Additionally, thereafter, the augmented reality device (800) can display the letter 'E' (814) and receive voice input from the user saying "It is 8". Additionally, thereafter, the augmented reality device (800) can display the letter 'O' (816) and receive voice input from the user saying "It is 6".

[0182] The augmented reality device (800) can identify the voice input “I can’t see it,” compare the letter ‘O’ with the letter ‘8’, and compare the letter ‘E’ with the letter ‘6’. Additionally, based on the comparison results, the augmented reality device (800) can identify that the accuracy rate of the user’s voice input is 0% and change the refractive power of the first variable focus lens (131) from ‘0D’ to ‘-2D’.

[0183] FIG. 22 is a diagram showing an example in which an augmented reality device (800) according to one embodiment performs an operation to obtain the user's correction required refractive power when the accuracy rate of the user's voice input is normal.

[0184] Referring to FIG. 22, after the refractive power of the first variable focus lens (131) is changed to '-2D', the augmented reality device (800) can sequentially display characters of a certain size at a focal length for visual acuity measurement and receive voice input from the user regarding the displayed characters. For example, the augmented reality device (800) can sequentially display the characters 'B' (822), 'E' (824), and 'O' (826) at different locations on a virtual visual acuity test plate (802) displayed at a focal length for visual acuity measurement. In this case, the visual acuity test plate (802) may be identical to the visual acuity test plate (801). Additionally, the characters 'B' (822), 'E' (824), and '0' (826) displayed on the visual acuity test plate (802) may appear moderately blurry to the user, as in FIG. 15. Accordingly, the augmented reality device (800) can receive voice input from a user saying “It is 8” after displaying the character ‘B’ (822). Additionally, after that, the augmented reality device (800) can display the character ‘E’ (824) and receive voice input from a user saying “It is 6”. Additionally, after that, the augmented reality device (800) can display the character ‘0’ (826) and receive voice input from a user saying “It is O”.

[0185] The augmented reality device (800) can compare the character 'B' with the voice input '8', compare the character 'E' with the voice input '6', and compare the character 'O' with the voice input '0'. Additionally, based on the comparison results, the augmented reality device (800) can identify that the accuracy rate of the user's voice input is 33.3% and change the refractive power of the first variable focus lens (131) from "-2D" to "-3D".

[0186] FIG. 23 is a diagram showing an example in which an augmented reality device (800) according to one embodiment of the present disclosure performs an operation to obtain the user's correction required refractive power when the accuracy rate of the user's voice input is high.

[0187] Referring to FIG. 23, the augmented reality device (800) can sequentially display characters of a certain size at a focal length for visual acuity measurement and receive voice input from the user regarding the displayed characters after the refractive power of the first variable focal lens (131) is changed to '-3D' as in FIG. 22. For example, the augmented reality device (800) can sequentially display the characters 'B' (832), 'O' (834), and 'E' (836) at different locations on a virtual visual acuity measurement board (803) displayed at a focal length for visual acuity measurement. In this case, the visual acuity measurement board (803) may be identical to the visual acuity measurement board (801). Additionally, the characters 'B' (832), 'O' (834), and 'E' (836) displayed on the visual acuity measurement board (803) can be clearly seen by the user as in FIG. 23. After the augmented reality device (800) displays the character 'B' (832), it can receive voice input from the user saying "It is B." Additionally, after that, the augmented reality device (800) can display the character 'O' (834) and receive voice input from the user saying "It is O." Additionally, after that, the augmented reality device (800) can display the character 'E' (836) and receive voice input from the user saying "It is E."

[0188] The augmented reality device (800) can compare the character 'B' with the voice input 'B', compare the character 'O' with the voice input 'O', and compare the character 'E' with the voice input 'E'. Additionally, based on the comparison results, the augmented reality device (800) identifies that the accuracy rate of the user's voice input is 100% and determines the user's refractive power D requiring correction or the refractive power D requiring modification correction from the current refractive power (i.e., the first refractive power) of the first variable focus lens (131). 수정 It can be converted as shown in the following mathematical formula.

[0189]

[0190]

[0191] Here, D 1C represents the current refractive power of the first variable focus lens (131), and f is the focal length of the virtual image, which in this embodiment represents the distance from the user's eye to the visual acuity test plate (801, 802, 803).

[0192] The user's refractive power requiring correction D determined as above 교정 Refractive power D requiring correction or modification 수정 The first and second refractive powers of the first and second variable focus lenses (131, 141) can be stored in memory (160) to calculate them.

[0193] Although FIGS. 21 to 23 describe the sequential display of three characters and the additional change of the refractive power of the first variable focus lens (131) after receiving voice inputs from the user, the number of characters displayed is not limited thereto. For example, the augmented reality device (800) may display one character and, after receiving voice inputs from the user, determine whether the user's voice input is correct. Additionally, the augmented reality device (800) may change the refractive power of the first variable focus lens (131) when the user inputs an incorrect answer.

[0194] Additionally, the refractive power of the first variable focus lens (131) may be changed differently from the degree of change in refractive power in FIGS. 21 and FIGS. 22. In this case, the degree to which the refractive power of the first variable focus lens (131) is changed may be varied based on the user's accuracy rate. For example, when the user's accuracy rate is low, the augmented reality device (800) may change the refractive power of the first variable focus lens (131) significantly, thereby reducing the number of changes in refractive power for the user's vision correction. For example, when the user's accuracy rate is high, the augmented reality device (800) may change the refractive power of the first variable focus lens (131) less significantly, thereby allowing for finer correction of the user's vision.

[0195] FIG. 24 is a flowchart illustrating the operation of an augmented reality device (800) according to one embodiment.

[0196] Referring to FIG. 24, for the operation to obtain the user's refractive power required for correction, at least one first character of a preset size is first output through an optical engine (110) to a virtual visual acuity measurement plate (801) marked at the focal length for visual acuity measurement, and at least one first voice input of the user for at least one first character is obtained (S910). Next, at least one first character and at least one first voice input are compared (S920). Next, based on the comparison result, the first refractive power of the first variable focus lens (131) is determined (S930). Next, based on the determined first refractive power of the first variable focus lens (131), the user's refractive power required for correction D as in the above-described mathematical formula 11 교정 Determines (S940). Of course, based on the first refractive power of the determined first variable focus lens (131), the refractive power D requiring correction as in the mathematical formula 12 described above is determined. 수정 It may also be decided.

[0197] In the embodiment with reference to FIGS. 20 to 24, the case of receiving voice input from a user through a microphone (890) is described as an example, but is not limited thereto. For example, the augmented reality device (800) may also receive information regarding the user's character reading through touch input from a user input unit (150).

[0198] One embodiment of the present disclosure may also be implemented in the form of a recording medium comprising computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include computer storage media and communication media. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data. A communication medium may typically include other data of modulated data signals, such as computer-readable instructions, data structures, or program modules.

[0199] Additionally, computer-readable storage media may be provided in the form of non-transitory storage media. Here, 'non-transitory storage media' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, 'non-transitory storage media' may include a buffer in which data is stored temporarily.

[0200] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0201] Additionally, in this specification, “part” may be a hardware component, such as a processor or circuit, and / or a software component executed by a hardware component, such as a processor.

[0202] Additionally, in this specification, “comprising at least one of a, b, or c” may mean “comprising only a, comprising only b, comprising only c, comprising a and b, comprising b and c, comprising a and c, or comprising all of a, b, and c.”

[0203] The foregoing description of the present disclosure is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present disclosure. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0204] The apparatus and method for displaying augmented reality according to the present invention described above have been explained with reference to embodiments illustrated in the drawings for the sake of understanding, but this is merely illustrative and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be determined by the appended claims. Explanation of the symbols

[0205] 100, 400, 500, 600: Augmented reality display device 101: Glasses-type body 102: Frame 110: Optical engine 120: Waveguide 130, 430: First lens section 131: First variable focus lens 133: (1st) Fixed refractive lens 140, 540: 2nd lens part 141: Second variable focus lens 150: User input section 160: Memory 170: Processor 190: Electronic Components 432: Polarizer 543: Second fixed refractive lens 680: Eye tracking sensor 801, 802, 803: Visual acuity charts 890: Microphone E: eye f: focal length of the virtual image LV : Light L of the virtual image R : Light in a realistic scene O V : Virtual object O R : Reality object

Claims

Claim 1 An optical engine configured to output light of a virtual image; a waveguide through which light of the virtual image is output and light of a real scene is transmitted; a first lens unit and a second lens unit positioned with respect to the waveguide; and a processor; wherein the first lens unit performs focus adjustment of the virtual image and correction of the user's vision, and includes a first variable focus lens having a first refractive power adjustable by the processor and a fixed refractive lens having a fixed refractive power, and the second lens unit compensates for distortion of the real scene caused by the first lens unit, and includes a second variable focus lens having a second refractive power adjustable by the processor, and the first refractive power of the first variable focus lens is [formula]. Determined by, where D1 is the first refractive power of the first variable focus lens, D 고정 is the fixed refractive power of the above fixed refractive lens, D 교정 A device for displaying augmented reality, wherein is a refractive power required for correction of a user's refractive error, f represents the focal length of a virtual image, and the processor adjusts the first refractive power of the first variable focus lens so that the focal length of the virtual image becomes the attribute distance of the virtual image, and the processor controls the optical engine to output at least one first character of a preset size; obtains at least one first input of a user for the at least one first character, compares the at least one first character and the at least one first input, determines the first refractive power of the first variable focus lens based on the comparison result, and determines the user's refractive power required for correction based on the determined first refractive power of the first variable focus lens. Claim 2 delete Claim 3 In claim 1, the fixed refractive power D of the fixed refractive lens 고정 , User's refractive power requiring correction D 교정 , and further includes a memory storing the focal length f of a virtual image, and after the processor reads information on the fixed refractive power of a fixed refractive lens, the refractive power requiring correction of the user, and the focal length of a virtual image from the memory, the first refractive power D1 of the first variable focus lens is A device that displays augmented reality, which performs calculations. Claim 4 In claim 1, focal length information f of a virtual image and refractive power D requiring correction reflecting the refractive power of a fixed refractive lens 수정 This further includes the stored memory, and the above-mentioned refractive power D requiring correction 수정 silver It is given as, and the processor reads the refractive power requiring correction and the focal length information of the virtual image from the memory, and then the first refractive power D1 of the first variable focus lens A device that displays augmented reality, which performs calculations. Claim 5 In claim 1, the second refractive power D2 of the second variable focus lens is a mathematical formula A device that displays augmented reality, determined by Claim 6 A device for displaying augmented reality according to claim 1, wherein the fixed refractive lens is a concave lens having negative (-) refractive power. Claim 7 A device for displaying augmented reality, wherein the first and second variable focus lenses are liquid crystal lenses. Claim 8 A device for displaying augmented reality according to claim 1, wherein the second variable focus lens is positioned between the waveguide and the fixed refractive lens, and the first variable focus lens, the waveguide, and the second variable focus lens are in a stacked structure. Claim 9 A device for displaying augmented reality according to claim 1, further comprising a user input unit into which at least one of the user's vision information and the focal length of a virtual image is input. Claim 10 A device for displaying augmented reality according to claim 1, wherein the first lens part further comprises a polarizing plate disposed on the incident surface or exit surface of the fixed refractive lens. Claim 11 A device for displaying augmented reality according to claim 1, wherein the second lens part further includes a second fixed refractive lens that compensates for distortion of the real scene caused by the first lens part together with the second variable focus lens. Claim 12 A device for displaying augmented reality according to claim 11, wherein the second fixed refractive lens is a convex lens having positive (+) refractive power. Claim 13 In claim 11, the second refractive power D2 of the second variable focus lens is the mathematical formula It is calculated by, where D 고정2 A device for displaying augmented reality, which is the fixed refractive power of the second fixed refractive lens (543). Claim 14 A device for displaying augmented reality according to claim 1, further comprising an eye-tracking sensor configured to acquire user's eye-tracking information. Claim 15 In claim 14, the device for displaying augmented reality, wherein the processor calculates a gaze point from user gaze information obtained from the gaze tracking sensor and determines the focal length of a virtual image based on the calculated gaze point. Claim 16 delete Claim 17 A device for displaying augmented reality according to claim 1, wherein at least one first character has a size corresponding to a preset corrected visual acuity and is displayed at a preset depth for measuring the user's visual acuity. Claim 18 In claim 1, the device is a glasses-type device, a device for displaying augmented reality. Claim 19 A method for displaying augmented reality in an augmented reality device comprising an optical engine configured to output light of a virtual image and a waveguide through which light of the virtual image is output and light of a real scene is transmitted, the method comprising: a first lens part including a fixed refractive lens and a first variable focus lens and a second lens part including a second variable focus lens, positioned with respect to the waveguide; and a fixed refractive power D of the fixed refractive lens 고정 , User's refractive power requiring correction D 교정 , and the step of reading the focal length f of the virtual image from memory; the first refractive power D1 of the first variable focus lens A method for displaying augmented reality, comprising: a step of calculating through a calculation such that the focal length of the virtual image becomes the attribute distance of the virtual image; and a step of calculating the second refractive power of the second variable focal lens to compensate for distortion of the real scene caused by the first lens part. Claim 20 delete Claim 21 delete Claim 22 delete Claim 23 delete Claim 24 delete Claim 25 delete Claim 26 delete Claim 27 delete

Images