Head mounted display device including eye tracking sensor and operating method for head mounted display device

Abstract

The present disclosure provides a head mounted display (HMD) device and an operating method for the same, where the device includes: a display; an optical lens disposed adjacent to the first side of the display; at least one light source emitting light; and at least one eye movement tracking sensor disposed adjacent to the second side of the display and receiving light that has been output from the at least one light source and reflected by an eye of the user to acquire gaze information of the user, the display includes a second display area corresponding to a position where the at least one eye movement tracking sensor is disposed, and a first display area other than the second display area. And the number of pixels per unit area included in the first display area is greater than the number of pixels per unit area included in the second display area.

Description

Technical Field

[0001] The present disclosure relates to a head mounted display (HMD) device and an operating method of the HMD device. Background Art

[0002] As technology advances, head mounted display (HMD) devices that provide images to a user via a display located close to the user's eyes while being worn on the user's head have been developed.

[0003] The display included in the HMD device may include an optical see-through display for overlaying a virtual image on a real-world physical environment space or a real-world object to display them together, or a video see-through display for displaying an image that is an image of the real-world physical environment space captured by using a camera or the like and presenting the image to a user.

[0004] Recently, a technology has been developed that senses a line of sight direction of a user wearing an HMD device, corrects an image displayed on a display based on the sensed line of sight direction, and presents the image to the user. Summary of the invention

[0005] Technical Solution

[0006] In an embodiment of the present disclosure, there is provided a head mounted display (HMD) device including a display and an optical lens positioned adjacent to a first side of the display. The HMD device may include at least one light source. The HMD device may include at least one eye tracking sensor, wherein the at least one eye tracking sensor is disposed adjacent to a second side of the display and is configured to obtain line of sight information about a user by receiving reflected light, wherein the reflected light is obtained when light emitted from the at least one light source is reflected from the user's eyes. The display may include a second display area corresponding to a position where the at least one eye tracking sensor is disposed and a first display area other than the second display area. The number of pixels per unit area of ​​the first display area may be greater than the number of pixels per unit area of ​​the second display area.

[0007] In an embodiment of the present disclosure, there is also provided an HMD device including a display and an optical lens disposed adjacent to a first side of the display. The HMD device may include at least one light source. The HMD device may include at least one eye tracking sensor, wherein the at least one eye tracking sensor is disposed adjacent to a second side of the display and is configured to receive reflected light, wherein the reflected light is obtained when light emitted from the at least one light source is reflected from the user's eye. The HMD device may include a memory storing at least one instruction and at least one processor configured to execute the at least one instruction stored in the memory. The at least one processor may be configured to obtain line of sight information about the user based on the reflected light received by the at least one eye tracking sensor. The display may include a second display area corresponding to a position where the at least one eye tracking sensor is disposed and a first display area other than the second display area. The number of pixels per unit area of ​​the first display area may be greater than the number of pixels per unit area of ​​the second display area.

[0008] In an embodiment of the present disclosure, there is also provided an operating method of an HMD device, wherein the HMD device includes a display and an optical lens disposed adjacent to a first side of the display. The operating method of the HMD device may include emitting light toward an eye of a user by using at least one light source. The operating method of the HMD device may include receiving reflected light by using at least one eye tracking sensor disposed adjacent to a second side of the display, wherein the reflected light is obtained when light is emitted from at least one light source and reflected from an eye of the user. The operating method of the HMD device may include obtaining line of sight information about the user based on the received reflected light. The display may include a second display area corresponding to a position where the at least one eye tracking sensor is disposed and a first display area other than the second display area. The number of pixels per unit area of ​​the first display area may be greater than the number of pixels per unit area of ​​the second display area. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present disclosure will be easily understood from the following description taken in conjunction with the accompanying drawings, in which reference numerals represent structural elements.

[0010] Figure 1 is a conceptual diagram illustrating a head mounted display (HMD) device according to an embodiment of the present disclosure.

[0011] Figure 2 is a plan view showing a configuration of an HMD device according to an embodiment of the present disclosure.

[0012] Figure 3 is a block diagram for describing an HMD device and an operating method of the HMD device according to an embodiment of the present disclosure.

[0013] Figure 4 is a flowchart illustrating the operation of the HMD device according to an embodiment of the present disclosure.

[0014] Figure 5 is a conceptual diagram showing that an eye tracking sensor is disposed within a display area of ​​a display and an eye tracking light source is disposed outside the display area of ​​the display according to an embodiment of the present disclosure.

[0015] Figure 6 is an enlarged view showing that an eye tracking sensor is disposed within a display area of ​​a display and an eye tracking light source is disposed outside the display area of ​​the display according to an embodiment of the present disclosure.

[0016] Figure 7 is a conceptual diagram showing that each of an eye tracking sensor and an eye tracking light source is disposed within a display area according to an embodiment of the present disclosure.

[0017] Figure 8 is an enlarged view showing that each of the eye tracking sensor and the eye tracking light source is disposed within the display area according to an embodiment of the present disclosure.

[0018] Fig. 9 is a plan view showing the arrangement of an eye tracking sensor and an eye tracking light source according to an embodiment of the present disclosure.

[0019] Fig.10 is a plan view showing the arrangement of an eye tracking sensor and an eye tracking light source according to an embodiment of the present disclosure.

[0020] Fig.11 is a plan view showing the arrangement of an eye tracking sensor and an eye tracking light source according to an embodiment of the present disclosure.

[0021] Fig.12 is a plan view showing the arrangement of an eye tracking sensor and an eye tracking light source according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] Terms used in the present disclosure will now be briefly described, and embodiments of the present disclosure will be described in detail.

[0023] As the terms used herein, currently widely used general terms are selected by considering the functions in the present disclosure, but may be changed according to the intention of ordinary technicians in the field, precedents, the emergence of new technologies, etc. In addition, specific terms may be arbitrarily selected by the applicant, and in this case, the meaning of the selected terms will be described in detail in the detailed description of the corresponding embodiment of the present disclosure. Therefore, the terms used herein should not be defined by their simple titles, but are defined based on the meaning of the terms and the overall description of the present disclosure.

[0024] Unless the context clearly indicates otherwise, singular expressions used herein are intended to include plural expressions as well.All terms (including technical or scientific terms) used herein may have the same meanings as those generally understood by one of ordinary skill in the art of the present disclosure.

[0025] Throughout the disclosure, when a component "includes" or "comprises" an element, unless there is a specific description to the contrary, the component may also include other elements without excluding other elements. In addition, terms such as "part", "module", etc. used herein indicate a unit for processing at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software.

[0026] Depending on the context, the expression "configured to (or set to)" used in the present disclosure may be used interchangeably with, for example, expressions "suitable for", "having the ability to", "designed to", "adapted to", "manufactured to", or "capable of". The term "configured to (or set to)" may not necessarily mean only "specially designed to" in terms of hardware. Alternatively, the expression "a system configured to..." may mean in some contexts that the system, together with other devices or components, is "capable of". For example, the expression "a processor configured to (or set to) perform A, B, and C" may mean a dedicated processor (e.g., an embedded processor) for performing the corresponding operations or a general-purpose processor (e.g., a central processing unit (CPU) or an application processor (AP)) that is capable of performing the corresponding operations by running one or more software programs stored in a memory.

[0027] Furthermore, in the present disclosure, it should be understood that when a component is referred to as being “connected” or “coupled” to another component, unless there is a specific description to the contrary, the component may be directly connected or coupled to the other component, but may also be connected or coupled to the other component via another intermediate component between the component and the other component.

[0028] As used herein, an “electronic device” may be a head mounted display (HMD) device. However, the present disclosure is not limited thereto, and the “electronic device” may be implemented as various forms of electronic devices, such as a TV, a mobile device, a smart phone, a laptop computer, a desktop computer, a tablet PC, an e-book terminal, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a wearable device, etc.

[0029] In the present disclosure, "central vision" may refer to a field of view used when observing an area on which the user's sight is focused. Central vision may refer to a field of view used to observe an area set at the center of the user's visual field.

[0030] As used herein, "peripheral vision" may refer to a field of view used when observing an area surrounding an area on which a user's line of sight is focused. Peripheral vision may refer to a field of view used to observe an area adjacent to a center of a user's field of vision.

[0031] As used herein, the “reference angle” may be an angle set corresponding to an angle determined as a maximum angle based on human factors, wherein at the maximum angle, when a user viewing an image by using the HMD device focuses the user's sight on an image displayed on a display, the user moves his or her eyes to change the focus area without turning the head from the area of ​​the display on which the user's sight was initially focused.

[0032] As used herein, a “region of interest” is an area where a user can observe an image displayed on a display with central vision by moving only the user's eyes without moving the head.

[0033] The embodiments of the present disclosure will be described more fully below with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the embodiments. However, the embodiments of the present disclosure can be implemented in different forms and should not be construed as being limited to the embodiments of the present disclosure set forth herein. In addition, parts not related to the description are omitted to clearly explain the embodiments of the present disclosure in the accompanying drawings, and the same reference numerals always represent the same elements.

[0034] Hereinafter, embodiments of the present disclosure are described in detail with reference to the accompanying drawings.

[0035] Figure 1 is a conceptual diagram for describing an HMD device according to an embodiment of the present disclosure.

[0036] Reference Figure 1 According to an embodiment of the present disclosure, the electronic device may provide content to a user 150 wearing the electronic device. In an embodiment of the present disclosure, the electronic device is a device capable of providing content to the user 150, and may be the HMD device 100. In an embodiment of the present disclosure, Figure 1 An electronic device having a shape similar to glasses is shown, including a support portion across the face of the user 150. However, the present disclosure is not limited thereto, and the electronic device may include a support portion across the face and head of the user 150. In addition, in an embodiment of the present disclosure, the electronic device is not limited to the HMD apparatus, but may be implemented as various forms of electronic devices such as a TV, a mobile device, a smart phone, a laptop computer, a desktop, a tablet PC, an e-book terminal, a digital broadcast terminal, a PDA, a PMP, a wearable device, etc. Hereinafter, for ease of description, the electronic device is described as the HMD apparatus 100.

[0037] In an embodiment of the present disclosure, the HMD device 100 may include a display 110 displaying an image and an optical lens 120. The image provided by the display 110 may be refracted, reflected, or scattered by the optical lens 120 and provided to the user 150.

[0038] In an embodiment of the present disclosure, the HMD apparatus 100 may display an image generated based on externally provided data or data pre-stored in the HMD apparatus 100 on the display 110 and provide the image to the user 150. In an embodiment of the present disclosure, the HMD apparatus 100 may receive data including information about an image to be displayed on the display 110 from an external electronic device.

[0039] The HMD device 100 may be a video see-through electronic device that obtains an image of a physical environment space pointed to by the sight of the user 150 by capturing it with a camera and then provides the obtained image to the user 150 via the display 110 .

[0040] In an embodiment of the present disclosure, the HMD device 100 may include at least one light source 130. The at least one light source 130 may emit light 160 toward an eye 151 of a user 150 wearing the HMD device 100. In an embodiment of the present disclosure, the light 160 provided by the at least one light source 130 may be refracted, reflected, or scattered by the optical lens 120 to illuminate the eye 151 of the user.

[0041] In an embodiment of the present disclosure, the HMD device 100 may include at least one eye tracking sensor 140. The at least one eye tracking sensor 140 may receive reflected light 170 obtained when light emitted from the at least one light source 130 is reflected from the user's eye 151. The at least one eye tracking sensor 140 may receive reflected light 170 obtained when light provided by the at least one light source 130 for eye tracking is reflected from the user's eye 151. In an embodiment of the present disclosure, the at least one eye tracking sensor 140 may receive the reflected light 170 refracted, reflected, or scattered by the optical lens 120.

[0042] In an embodiment of the present disclosure, the HMD device 100 may obtain the sight line information of the user 150 by using at least one light source 130 and at least one eye tracking sensor 140. In an embodiment of the present disclosure, the HMD device 100 may obtain the biometric information about the user 150 by using at least one light source 130 and at least one eye tracking sensor 140. In an embodiment of the present disclosure, the sight line information of the user 150 may include the direction of the sight line of the user 150, the distance between the eye 151 of the user and the optical lens 120, the viewpoint at which the sight line of the user 150 intersects with the optical lens 120, and the viewpoint at which the sight line of the user 150 intersects with the display 110. In an embodiment of the present disclosure, the HMD device 100 may obtain the iris pattern of the eye 151 of the user or the image of the eyeball of the user 150 by using at least one eye tracking sensor 140. The HMD device 100 may obtain the biometric information about the user 150 based on the iris pattern of the eye 151 of the user or the image of the eyeball of the user 150. In an embodiment of the present disclosure, biometric information about the user 150 may include an image of the iris of the user's eye 151, an image of the retina of the user's eye 151, a vein pattern within the user's eye 151, the visual acuity of the user's eye 151, information about diseases present in the user's eye 151, and the like.

[0043] For ease of description, the at least one light source 130 is hereinafter referred to as at least one eye tracking light source 130 .

[0044] In an embodiment of the present disclosure, the HMD device 100 may track the sight line of the user 150 and obtain information about the direction of the sight line of the user 150 .

[0045] In an embodiment of the present disclosure, the HMD device 100 may calculate at least one of a distance between the user's eye 151 and the optical lens 120 or a distance between the user's eye 151 and the display 110 by using at least one eye tracking light source 130 and at least one eye tracking sensor 140 .

[0046] In an embodiment of the present disclosure, the HMD device 100 may identify a viewpoint at which the line of sight of the user 150 intersects the optical lens 120 based on the obtained information about the direction of the line of sight of the user 150 and the distance between the eye 151 of the user and the optical lens 120. The HMD device 100 may identify a viewpoint at which the line of sight of the user 150 intersects the display 110 based on the obtained information about the direction of the line of sight of the user 150 and the distance between the eye 151 of the user and the display 110.

[0047] exist Figure 1In the following drawings, a first direction axis 10, a second direction axis 20, and a third direction axis 30 are shown, and the directions indicated by the first direction axis 10, the second direction axis 20, and the third direction axis 30 as described in the present disclosure are relative concepts and can be changed to other directions. In addition, the directions indicated by the first direction axis 10, the second direction axis 20, and the third direction axis 30 are respectively described as the first direction, the second direction, and the third direction. In the present disclosure, the first direction axis 10 and the second direction axis 20 are directions orthogonal to each other, and the third direction axis 30 may be a direction perpendicular to a plane defined by the first direction axis 10 and the second direction axis 20.

[0048] In the present disclosure, the front (or top) and the rear (or bottom) of the HMD device 100 may be defined based on the first direction axis 10. As used herein, "overlapping" may mean overlapping with respect to the first direction axis 10. As used herein, "included in" may mean included in a corresponding area with respect to the first direction axis 10. In addition, as used herein, "disposed" or "arranged" may mean disposed or arranged with respect to the first direction axis 10. However, the present disclosure is not limited thereto, and may be described with respect to a case where components are overlapped or arranged with respect to the second direction axis 20 or the third direction axis 30, if necessary.

[0049] In an embodiment of the present disclosure, a first distance between the optical lens 120 and the user's eye 151 along the first direction axis 10 may be smaller than a second distance between the display 110 and the user's eye 151 along the first direction axis 10. Fig. 9 and Fig.10 The arrangement relationship between the optical lens 120 and the display 110 is described.

[0050] In the embodiments of the present disclosure, Figure 1 It is shown that at least one eye tracking light source 130 is arranged at a position spaced apart from the display 110 along the second direction axis 20. At least one eye tracking light source 130 arranged at a position spaced apart from the display 110 along the second direction axis 20 may not overlap with the display 110. At least one eye tracking light source 130 may be arranged outside the first display area 111 and the second display area 112. However, the present disclosure is not limited thereto. At least one eye tracking light source 130 may be arranged at a position spaced apart from the display 110 along the first direction axis 10. At least one eye tracking light source 130 arranged at a position spaced apart from the display 110 along the first direction axis 10 and the display 110 may overlap with the display 110. At least one eye tracking light source 130 may be included in the second display area 112.

[0051] In an embodiment of the present disclosure, at least one eye tracking light source 130 may overlap with the optical lens 120 .

[0052] Will refer to the following Figures 5 to 10 The arrangement relationship between at least one eye tracking light source 130, display 110 and optical lens 120 is described in the description.

[0053] In an embodiment of the present disclosure, at least one eye tracking sensor 140 may be arranged at a position spaced apart from the display 110 along the first direction axis 10. With respect to the first direction axis 10, a third distance between the at least one eye tracking sensor 140 and the user's eye 151 may be greater than a second distance between the display 110 and the user's eye 151.

[0054] In an embodiment of the present disclosure, a display area displaying an image on the display 110 may include a first display area 111 and a second display area 112. The resolution of the first display area 111 may be greater than the resolution of the second display area 112. The number of pixels per unit area of ​​the first display area 111 may be greater than the number of pixels per unit area of ​​the second display area 112.

[0055] At least one eye tracking sensor 140 may overlap with the second display area 112 of the display 110 with respect to the first direction axis 10. At least one eye tracking sensor 140 may be arranged on the inside of the second display area 112 of the display 110. In an embodiment of the present disclosure, the inside of the second display area 112 refers to an area included in the second display area 112 in a plane defined by the second direction axis 20 and the third direction axis 30. In an embodiment of the present disclosure, the second display area 112 may be an area corresponding to a position where the at least one eye tracking sensor 140 is located. The first display area 111 may not overlap with the at least one eye tracking sensor 140 with respect to the first direction axis 10 of the display 110. At least one eye tracking sensor 140 may be arranged outside the first display area 111. In an embodiment of the present disclosure, the first display area 111 may be an area other than the second display area 112. In an embodiment of the present disclosure, the at least one eye tracking sensor 140 and the optical lens 120 may overlap with respect to the first direction axis 10.

[0056] Refer to it later Figures 5 to 10 The arrangement relationship between at least one eye tracking sensor 140, the display 110 and the optical lens 120 is described in the description.

[0057] The HMD device 100 of the present disclosure may have at least one eye tracking sensor 140, wherein the at least one eye tracking sensor 140 is arranged to be included in the second display area 112 of the display 110 with respect to the first direction axis 10, so that the at least one eye tracking sensor 140 is close in distance to the user's eye 151 on the plane defined by the second direction axis 20 and the third direction axis 30. According to the present disclosure, even when the user 150 moves the user's eye 151 to observe an image displayed on the display 110 (for example, moves the eye 151 in a direction away from the position where the at least one eye tracking sensor 140 is arranged), the at least one eye tracking sensor 140 may receive the reflected light 170 which is light reflected from the user's eye 151. Therefore, the light receiving performance of the at least one eye tracking sensor 140 may be improved.

[0058] Furthermore, in the HMD device 100 of the present disclosure, as described later, the second display area 112 having a smaller number of pixels per unit area than the first display area 111 may be arranged in the area of ​​interest ( Figure 5 550) outside to improve the light receiving performance of at least one eye tracking sensor 140, thereby preventing the user 150 from visually perceiving the difference in image quality due to the lower resolution of the second display area 112.

[0059] Figure 2 1 is a plan view showing the configuration of an HMD device according to an embodiment of the present disclosure. Figure 1 The components described are the same components, and the description already provided above will be omitted.

[0060] Reference Figure 2 , the HMD device 100 may include a display 110, an optical lens 120, at least one eye tracking light source 130, at least one eye tracking sensor 140, a frame 181, temples 182, a nose support 183, a nose bridge 184, at least one processor 191, a memory 192, and a battery 193. In an embodiment of the present disclosure, in Figure 2 Only components for describing the structure of the HMD device 100 are shown in FIG. 1 , and components included in the HMD device 100 are not limited to Figure 2 In an embodiment of the present disclosure, the HMD device 100 may include a camera or the like for capturing an image of a physical environment space in the real world.

[0061] In an embodiment of the present disclosure, the display 110 and the optical lens 120 may be each disposed on a frame 181. The frame 181 may have a shape similar to that of a frame of a general eyeglass structure. The frame 181 may include a frame surrounding the display 110 and the optical lens 120.

[0062] In an embodiment of the present disclosure, the display 110 may include a first display 110_1 corresponding to a left eye 151_1 of a user and a second display 110_2 corresponding to a right eye 151_2 of the user.

[0063] In an embodiment of the present disclosure, the optical lens 120 may include a first optical lens 120_1 corresponding to the user's left eye 151_1 and a second optical lens 120_2 corresponding to the user's right eye 151_2. The first optical lens 120_1 may correspond to the first display 110_1, and the second optical lens 120_2 may correspond to the second display 110_2.

[0064] In an embodiment of the present disclosure, at least one eye tracking light source 130 may include a first eye tracking light source 130_1 corresponding to the user's left eye 151_1 and a second eye tracking light source 130_2 corresponding to the user's right eye 151_2. The first eye tracking light source 130_1 may correspond to the first display 110_1, and the second eye tracking light source 130_2 may correspond to the second display 110_2.

[0065] In an embodiment of the present disclosure, at least one eye tracking sensor 140 may include a first eye tracking sensor 140_1 corresponding to the user's left eye 151_1 and a second eye tracking sensor 140_2 corresponding to the user's right eye 151_2. The first eye tracking sensor 140_1 may correspond to the first display 110_1, and the second eye tracking sensor 140_2 may correspond to the second display 110_2.

[0066] In an embodiment of the present disclosure, the frame 181 may include a first frame 181_1 corresponding to the user's left eye 151_1 and a second frame 181_2 corresponding to the user's right eye 151_2. The first frame 181_1 may include a frame surrounding the first display 110_1 and the first optical lens 120_1. The second frame 181_2 may include a frame surrounding the second display 110_2 and the second optical lens 120_2.

[0067] In an embodiment of the present disclosure, the first display 110_1, the first optical lens 120_1, the first eye tracking light source 130_1, and the first eye tracking sensor 140_1 may be disposed on the first frame 181_1. In an embodiment of the present disclosure, the second display 110_2, the second optical lens 120_2, the second eye tracking light source 130_2, and the second eye tracking sensor 140_2 may be disposed on the second frame 181_2.

[0068] However, the present disclosure is not limited thereto. The first display 110_1 and the second display 110_2 may be integrally formed and disposed on the frame 181. The first optical lens 120_1 and the second optical lens 120_2 may be integrally formed and disposed on the frame 181.

[0069] In an embodiment of the present disclosure, a temple 182 may be connected to the frame 181. The temple 182 is a portion that supports the HMD device 100 by resting on the ear of the user 150 when the user 150 wears the HMD device 100. In an embodiment of the present disclosure, the temple 182 may include a first temple 182_1 connected to the first frame 181_1 and a second temple 182_2 connected to the second frame 181_2.

[0070] In an embodiment of the present disclosure, at least one processor 191, a memory 192, and a battery 193 included in the HMD device 100 may be embedded in the temple 182. Figure 2 At least one processor 191, a memory 192, and a battery 193 are shown to be embedded in the first temple 182_1, but this is merely an exemplary structure and the present disclosure is not limited to Figure 2 The structure shown.

[0071] In an embodiment of the present disclosure, a nose support 183 may be connected to the frame 181. The nose support 183 is a portion that supports the HMD device 100 by resting on the nose of the user 150 when the user 150 wears the HMD device 100. In an embodiment of the present disclosure, the nose support 183 may include a bridge and a nose pad. In addition, the bridge and the nose pad may be integrally formed, but are not limited thereto. In an embodiment of the present disclosure, the nose support 183 may be integrated with the frame 181.

[0072] In an embodiment of the present disclosure, the nose bridge piece 184 is a support piece connecting the first frame 181_1 to the second frame 181_2. The nose bridge piece 184 may be connected to the nose support piece 183. However, when the first frame 181_1 and the second frame 181_2 are integrally formed, the HMD device 100 may not include the nose bridge piece 184.

[0073] In an embodiment of the present disclosure, at least one eye tracking light source 130 may emit light 160 toward an eye 151 of a user wearing the HMD device 100 through the optical lens 120. Figure 3 The configuration, operation, and functionality of at least one eye-tracking light source 130 are described in detail.

[0074] In an embodiment of the present disclosure, at least one eye tracking sensor 140 may receive reflected light 170 passing through the optical lens 120, wherein the reflected light 170 is light reflected from the eye 151 of the user wearing the HMD device 100. Figure 3 The configuration, operation, and functionality of the at least one eye tracking sensor 140 are described in detail.

[0075] In an embodiment of the present disclosure, the battery 193 may be electrically and / or physically connected to at least one eye tracking light source 130, at least one eye tracking sensor 140, at least one processor 191, and the memory 192. The battery 193 may supply driving power to the at least one eye tracking light source 130 and the at least one eye tracking sensor 140 according to the control of the at least one processor 191.

[0076] In an embodiment of the present disclosure, the battery 193 may include at least one battery module composed of a rechargeable battery. The battery 193 may include, for example, a lithium (Li) ion battery, a Li ion polymer battery (LIPB), a nickel (Ni) cadmium (Cd) battery, a Ni metal hydride (MH) battery, etc., but is not limited thereto.

[0077] Reference Figure 3 The specific configuration, operation, and functions of the processor 191 and the memory 192 are described in detail.

[0078] Figure 3 1 is a block diagram for describing an HMD device and an operating method of the HMD device according to an embodiment of the present disclosure. Figure 1 and Figure 2 The components described are the same components, and the description already provided above will be omitted.

[0079] Reference Figures 1 to 3 , HMD device 100 may include display 110, optical lens 120, at least one eye tracking light source 130, at least one eye tracking sensor 140, at least one processor 191, memory 192 storing at least one instruction, and communication interface 194. However, Figure 3 All components shown in FIG. 1 are not essential components. Figure 3In an embodiment of the present disclosure, the HMD device 100 may further include at least one of a camera for capturing a real world space to which the sight line of the user 150 is directed or a battery 193 for supplying driving power to each of the display 110, the at least one eye tracking light source 130, the at least one eye tracking sensor 140, and the at least one processor 191.

[0080] In an embodiment of the present disclosure, the display 110, the optical lens 120, at least one eye tracking light source 130, at least one eye tracking sensor 140, at least one processor 191, the memory 192 and the communication interface 194 may be electrically and / or physically connected to each other.

[0081] In an embodiment of the present disclosure, the display 110 may include one of an organic light emitting diode (OLED) display and an inorganic LED display. However, the present disclosure is not limited thereto, and the display 110 may include other types of displays capable of providing content to a user.

[0082] In an embodiment of the present disclosure, the optical lens 120 may include one of glass and an electrically tunable liquid crystal lens including liquid crystal molecules. However, the present disclosure is not limited thereto, and the optical lens 120 may include a material capable of changing the optical properties of light incident on the optical lens 120 by refraction, reflection, or scattering, etc. In addition, although Figure 1 The optical lens 120 is shown to include a single lens, but the present disclosure is not limited thereto. The optical lens 120 may include a plurality of lenses, and in this case, each of the plurality of lenses may have a different shape, size, refractive index, etc.

[0083] In an embodiment of the present disclosure, at least one eye tracking light source 130 may be configured to emit light 160 toward an eye 151 of a user wearing the HMD device 100. In an embodiment of the present disclosure, the light 160 emitted from at least one eye tracking light source 130 toward the eye 151 of the user may be infrared (IR) rays.

[0084] In an embodiment of the present disclosure, when the at least one eye tracking light source 130 includes an IR light emitting diode (LED) that emits IR rays, the at least one eye tracking light source 130 may emit IR rays in the form of planar light toward the user's eyes 151 .

[0085] In an embodiment of the present disclosure, when the at least one eye tracking light source 130 includes an IR scanner (e.g., a micro-electromechanical system (MEMS) scanner) that emits IR light, the at least one eye tracking light source 130 may emit IR rays in the form of point light or line light toward the user's eyes 151. The at least one eye tracking light source 130 may emit IR rays while changing its emission direction to cover the space where the user's eyes 151 are located.

[0086] In an embodiment of the present disclosure, at least one eye tracking sensor 140 may be configured to receive reflected light 170, wherein the reflected light 170 is light reflected from the user's eye 151. In an embodiment of the present disclosure, the reflected light 170 from the user's eye 151 may be IR light. At least one eye tracking sensor 140 may include an IR detector (e.g., a photodiode) capable of receiving IR light.

[0087] In an embodiment of the present disclosure, at least one eye tracking sensor 140 may include an IR camera. At least one eye tracking sensor 140 may obtain an eye image by capturing an image of the user's eye 151 through the IR camera, and obtain line of sight information of the user 150 from the eye image.

[0088] In an embodiment of the present disclosure, the memory 192 may include at least one type of storage medium, i.e., at least one of a flash memory, a hard disk memory, a multimedia card micro memory, a card memory (e.g., an SD card or an XD memory), a random access memory (RAM), a static RAM (SRAM), a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), a PROM, a mask ROM, a flash ROM, a hard disk drive (HDD), or a solid-state drive (SSD). The memory 192 may store at least one instruction or program code for executing a function or operation of the HMD device 100. The at least one instruction, algorithm, data structure, program code, and application stored in the memory 192 may be implemented in a programming or scripting language such as C, C++, Java, assembler, etc.

[0089] In an embodiment of the present disclosure, the eye sensing module 192_3 and the image display module 192_4 may be stored in the memory 192. However, Figure 3 All modules shown in the figure are not essential modules. The memory 192 can store Figure 3 More or fewer modules than those shown in .

[0090] The “module” included in the memory 192 may refer to a unit for processing a function or operation performed by at least one processor 191. The “module” included in the memory 192 may be implemented as software such as at least one instruction, algorithm, data structure, or program code.

[0091] In an embodiment of the present disclosure, the eye sensing module 192_3 may include instructions or program codes related to the following operations or functions: obtaining the line of sight information of the user 150 or at least one of the biometric information about the user 150 based on the reflected light 170 obtained via at least one eye tracking sensor 140.

[0092] In an embodiment of the present disclosure, the eye sensing module 192_3 may include instructions or program codes related to the following operations or functions: obtaining information about the direction of the sight line of the user 150 based on the reflected light 170 obtained via at least one eye tracking sensor 140. The eye sensing module 192_3 may include instructions or program codes related to the following operations or functions: obtaining information about the direction of the sight line of the user 150 by detecting the direction or intensity of the reflected light 170, for example, as light reflected from the user's eye 151. In an embodiment of the present disclosure, the eye sensing module 192_3 may include instructions or program codes related to the following operations or functions: obtaining information about the direction of the sight line of the user 150 by detecting an image of an eyeball or pupil from an eye image obtained via at least one eye tracking sensor 140. In an embodiment of the present disclosure, the eye sensing module 192_3 may include instructions or program codes related to the following operations or functions: obtaining biometric information about the user 150 by detecting an image of the eyeball or pupil from an eye image obtained via at least one eye tracking sensor 140, wherein the biometric information includes an iris pattern of each eye 151 of the user, an image of the iris of each eye 151 of the user, an image of the retina of each eye 151 of the user, a vein pattern within the user's eye 151, the visual acuity of the user's eye 151, information about diseases present in the user's eye 151, and the like.

[0093] In an embodiment of the present disclosure, at least one processor 191 may execute instructions or program codes of the eye sensing module 192_3 to obtain the direction of the line of sight of the user 150 based on the reflected light 170 obtained via at least one eye tracking sensor 140.

[0094] In an embodiment of the present disclosure, the eye sensing module 192_3 may include instructions or program codes related to the following operations or functions: calculating at least one of the distance between the user's eye 151 and the optical lens 120 or the distance between the user's eye 151 and the display 110 based on the reflected light 170 obtained via at least one eye tracking sensor 140.

[0095] In an embodiment of the present disclosure, at least one processor 191 may execute instructions or program codes of the eye sensing module 192_3 to calculate at least one of the distance between the user's eye 151 and the optical lens 120 or the distance between the user's eye 151 and the display 110 based on the reflected light 170 obtained via at least one eye tracking sensor 140.

[0096] In an embodiment of the present disclosure, the eye sensing module 192_3 may include instructions or program codes related to the following operations or functions: identifying the viewpoint where the line of sight of the user 150 intersects with the optical lens 120 based on the direction of the line of sight of the user 150 and the distance between the user's eye 151 and the optical lens 120.

[0097] In an embodiment of the present disclosure, at least one processor 191 may execute instructions or program codes of the eye sensing module 192_3 to identify the viewpoint where the line of sight of the user 150 intersects with the optical lens 120 based on the direction of the line of sight of the user 150 and the distance between the user's eye 151 and the optical lens 120.

[0098] In an embodiment of the present disclosure, the eye sensing module 192_3 may include instructions or program codes related to the following operations or functions: identifying the viewpoint where the line of sight of the user 150 intersects with the display 110 based on the direction of the line of sight of the user 150 and the distance between the user's eye 151 and the display 110.

[0099] In an embodiment of the present disclosure, at least one processor 191 may execute instructions or program codes of the eye sensing module 192_3 to identify the viewpoint where the line of sight of the user 150 intersects with the display 110 based on the direction of the line of sight of the user 150 and the distance between the user's eye 151 and the display 110.

[0100] In an embodiment of the present disclosure, the image display module 192_4 may include instructions or program codes related to the following operations or functions: displaying an image on the display 110. In an embodiment of the present disclosure, the image display module 192_4 may include instructions or program codes related to the following operations or functions: changing the image displayed on the display 110 corresponding to at least one of the viewpoints at which the recognized line of sight of the user 150 intersects with the optical lens 120 or the viewpoints at which the recognized line of sight of the user 150 intersects with the display 110, and then displaying the changed image on the display 110.

[0101] In an embodiment of the present disclosure, the image display module 192_4 may include instructions or program codes regarding the operation or function of displaying an image, wherein the image corresponds to at least one viewpoint among multiple images where the line of sight of the identified user 150 intersects with the optical lens 120 or where the line of sight of the identified user 150 intersects with the display 110, so that at least one of the resolution, brightness or contrast is higher than at least one of the resolution, brightness or contrast of the remaining images that do not correspond to the at least one viewpoint.

[0102] In an embodiment of the present disclosure, at least one processor 191 may execute instructions or program codes of the image display module 192_4 to display an image on the display 110 and provide the image to the user 150 wearing the HMD device 100 .

[0103] In an embodiment of the present disclosure, at least one processor 191 may include but is not limited to a CPU, a microprocessor, a graphics processing unit (GPU), an application-specific integrated circuit (ASIC) AP, a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a neural processing unit (NPU), or at least one of a dedicated AI processor designed with a hardware structure dedicated to training and processing artificial intelligence (AI) models.

[0104] In an embodiment of the present disclosure, at least one processor 191 may control at least one eye tracking light source 130 to emit light 160 toward an eye 151 of a user wearing the HMD device 100. In this case, the light emitted from the at least one eye tracking light source 130 may be IR rays.

[0105] In an embodiment of the present disclosure, at least one processor 191 may control at least one eye tracking sensor 140 to receive reflected light 170, wherein the reflected light 170 is light reflected from the eye 151 of the user wearing the HMD device 100. In this case, the reflected light 170 received via the at least one eye tracking sensor 140 may be IR rays.

[0106] In an embodiment of the present disclosure, the communication interface 194 may perform data communication with an external server (not shown) according to the control of the at least one processor 191. In addition, the communication interface 194 may perform data communication with other nearby electronic devices and an external server.

[0107] In an embodiment of the present disclosure, the communication interface 194 can perform data communication with a server or other nearby electronic devices by using at least one of data communication methods including, for example, a wired local area network (LAN), a wireless LAN, Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), an infrared data association (IrDA), Bluetooth low energy (BLE), near field communication (NFC), wireless broadband Internet (WiBro), world interoperability for microwave access (WiMAX), shared wireless access protocol (SWAP), wireless Gigabit Alliance (WiGig) and radio frequency (RF) communication.

[0108] In an embodiment of the present disclosure, the communication interface 194 may receive data including an image displayed on the display 110 from a server or other nearby electronic devices. The at least one processor 191 may display an image on the display 110 based on the received data.

[0109] Figure 4 is a flowchart illustrating the operation of the HMD device according to an embodiment of the present disclosure.

[0110] Reference Figure 1 and Figure 4 In an embodiment of the present disclosure, the operating method of the HMD device 100 may include emitting light 160 toward the user's eye 151 by using at least one eye tracking light source 130 (S100). In the operation S100 of emitting light 160 toward the user's eye 151, at least one processor ( Figure 3 191) can emit light 160 toward the user's eye 151 via at least one eye tracking light source 130.

[0111] In an embodiment of the present disclosure, the operating method of the HMD device 100 may include receiving a reflected light 170 by using at least one eye tracking sensor 140, wherein the reflected light 170 is light reflected from the user's eye 151 (S200). In an embodiment of the present disclosure, the at least one eye tracking sensor 140 may be connected to the second side ( Fig. 9 115) is positioned adjacent to the second side of the display 110, wherein the second side is adjacent to the first side ( Fig. 9 In an embodiment of the present disclosure, in operation S200 of receiving reflected light 170 as light reflected from the user's eye 151, at least one processor 191 may receive, via at least one eye tracking sensor 140, reflected light 170 obtained when light emitted from at least one eye tracking light source 130 is reflected from the user's eye 151.

[0112] In an embodiment of the present disclosure, the operating method of the HMD device 100 may include obtaining line of sight information about the user 150 based on the received reflected light 170 (S300). In an embodiment of the present disclosure, in the operation S300 of obtaining the line of sight information about the user 150, the at least one processor 191 may obtain line of sight information including at least one of the following items: the direction of the line of sight of the user 150, the distance between the eye 151 of the user and the optical lens 120, the viewpoint at which the line of sight of the user 150 intersects with the optical lens 120, or the viewpoint at which the line of sight of the user 150 intersects with the display 110. However, the present disclosure is not limited thereto, and in the operation S300 of obtaining the line of sight information about the user 150 based on the received reflected light 170, biometric information of the user 150 may also be obtained. In an embodiment of the present disclosure, in the operation S300 of obtaining the line of sight information or biometric information about the user 150, the at least one processor 191 may obtain an iris pattern of the eye 151 of the user or an image of the eyeball of the user 150. The HMD device 100 may obtain biometric information about the user 150 based on an iris pattern of the user's eyes 151 or an image of the eyeball of the user 150. In an embodiment of the present disclosure, the biometric information about the user 150 may include an image of an iris of each eye 151 of the user, an image of a retina of each eye 151 of the user, a vein pattern within the user's eyes 151, a visual acuity of the user's eyes 151, information about a disease present in the user's eyes 151, and the like.

[0113] In an embodiment of the present disclosure, in operation S300 of obtaining sight line information or biometric information about the user 150 based on the received reflected light 170, at least one processor 191 may obtain sight line information of the user 150 or at least one of the biometric information about the user 150 based on the received reflected light 170.

[0114] In an embodiment of the present disclosure, the operating method of the HMD device 100 may further include displaying an image on the display 110. In the operation of displaying the image on the display 110, at least one of resolution, brightness, or contrast of the image displayed on the display 110 may be determined based on the obtained line of sight information about the user 150.

[0115] In an embodiment of the present disclosure, the operating method of operating the HMD device 100 may further include providing the obtained biometric information about the user 150 to the user 150 .

[0116] Figure 5 is a conceptual diagram showing that an eye tracking sensor is disposed within a display area of ​​a display and an eye tracking light source is disposed outside the display area of ​​the display according to an embodiment of the present disclosure.

[0117] Reference Figure 1 and Figure 5 , a user's eye 151, a display 110, and an optical lens 120 are shown to illustrate the arrangement of at least one eye tracking light source 510 and at least one eye tracking sensor 520. In the following, for ease of description, Figures 5 to 12 The display 110, the optical lens 120, the at least one eye tracking light source 510, and the at least one eye tracking sensor 520 are shown, each of which corresponds to one of the two eyes of the user 150. However, the present disclosure is not limited thereto, and the display, the optical lens, the at least one eye tracking light source, and the at least one eye tracking sensor each corresponding to the other eye of the user 150 may have the same Figures 5 to 12 Same shape and arrangement as shown.

[0118] In an embodiment of the present disclosure, the optical lens 120 may be disposed on the display 110 relative to the first direction axis 10. A side of the display 110 adjacent to the user's eye 151 may be defined as a first side 114, and a side opposite to the first side 114 may be defined as a second side 115. The optical lens 120 may be disposed adjacent to the first side 114 of the display 110. In an embodiment of the present disclosure, a first distance 540 between the optical lens 120 and the user's eye 151 relative to the first direction axis 10 may be smaller than a second distance between the display 110 and the user's eye 151.

[0119] In an embodiment of the present disclosure, in a plane defined by the second direction axis 20 and the third direction axis 30, the cross-sectional area of ​​the optical lens 120 may be greater than or equal to the cross-sectional area of ​​the display 110. The entire area of ​​the display 110 may overlap with the optical lens 120 relative to the first direction axis 210. In an embodiment of the present disclosure, the cross-sectional area of ​​the aperture stop of the optical lens 120 may be greater than or equal to the cross-sectional area of ​​the display area of ​​the display 110. In an embodiment of the present disclosure, the aperture stop of the optical lens 120 may be inscribed in the display area of ​​the display 110.

[0120] In an embodiment of the present disclosure, the field of view of the user 150 includes central vision and peripheral vision. "Central vision" may refer to a field of view used when observing an area on which the sight of the user 150 is focused. Central vision may refer to a field of view used to observe an area set in the center of the user's field of view. In an embodiment of the present disclosure, the user 150 may use central vision to observe an image displayed in an area of ​​the display 110 on which the sight of the user 150 is focused.

[0121] “Peripheral vision” may refer to a field of view used when observing an area surrounding an area on which the user 150's line of sight is focused. Peripheral vision may refer to a field of view used to observe an area adjacent to the center of the user's field of view. In an embodiment of the present disclosure, the user 150 may use peripheral vision to observe an image displayed in an area of ​​the display 110 adjacent to the area on which the user 150's line of sight is focused.

[0122] In this case, the visual acuity of the user when observing an image by using central vision is higher than that of the user when observing an image by using peripheral vision. Therefore, in order to clearly view the image displayed on the display 110, the user 150 can view the image displayed on the display 110 by changing the focus area on the display 110.

[0123] In an embodiment of the present disclosure, an image displayed on the display 110 may be refracted, reflected, or scattered by the optical lens 120 and then provided to the user 150. The distance between the user's eye 151 and the region where the image displayed on the display 110 is formed after being refracted, reflected, or scattered by the optical lens 120 may be less than or equal to a first distance between the optical lens 120 and the user's eye 151. Hereinafter, for ease of description, it is described that the distance between the user's eye 151 and the region where the image refracted, reflected, or scattered by the optical lens 120 is formed is equal to the first distance.

[0124] In an embodiment of the present disclosure, the user 150 may recognize an image provided by the HMD device 100 by observing an image refracted, reflected, or scattered through the optical lens 120 .

[0125] In an embodiment of the present disclosure, the region of interest 550 is an area among multiple areas where an image refracted, reflected or scattered by the optical lens 120 is formed, where the user 150 can observe the image with central vision by moving only the user's eyes 151 without moving the head.

[0126] In an embodiment of the present disclosure, the region of interest 550 may be determined based on a first distance 540 and a predetermined reference angle 530 , wherein the first distance 540 is a distance between the user's eye 151 and the optical lens 120 .

[0127] In an embodiment of the present disclosure, the first distance 540 may be determined based on the structure and shape of the HMD device 100 according to the present disclosure and a physical feature such as a skeleton of the user 150. In an embodiment of the present disclosure, the first distance 540 may be an average of a plurality of first distances calculated based on the structure and shape of the HMD device 100 according to the present disclosure and different physical features of a plurality of users who are expected to use the HMD device 100.

[0128] In an embodiment of the present disclosure, the reference angle 530 may be an angle set to correspond to an angle determined as a maximum angle based on human factors, wherein at the maximum angle, when the user 150 who views an image by using the HMD device 100 focuses his or her sight on an area where an image refracted, reflected, or scattered by the optical lens 120 is formed, the user 150 moves the user's eyes 151 to change the focus area without moving the head from the area where the user's sight was initially focused. In an embodiment of the present disclosure, the reference angle 530 may be an angle based on the center ( Fig. 9 931) sets the angle.

[0129] In an embodiment of the present disclosure, when the center 931 of the optical lens 120 corresponds to the area of ​​the display 110 on which the user 150 initially focuses the user's sight, the reference angle 530 may be set to an angle of at least 10° but not more than 40° relative to the center 931 of the optical lens 120. In an embodiment of the present disclosure, the reference angle 530 may be set to 30°.

[0130] In an embodiment of the present disclosure, when the user 150 attempts to change the focus area from the area on which the user's sight was initially focused to another area to observe an image, and the angle between the area on which the user's sight was initially focused and the other area relative to the user's eyes 151 is less than or equal to the reference angle 530, the user 150 can change the focus area by moving the user's eyes 151 without moving the user's 150 head.

[0131] In an embodiment of the present disclosure, when the user 150 attempts to change the focus area from the area on which the user's sight was initially focused to another area to observe an image, and the angle between the area on which the user's sight was initially focused and the other area relative to the user's eye 151 is greater than the reference angle 530, the user 150 can change the focus area by turning the user's 150 head.

[0132] In an embodiment of the present disclosure, when observing an image corresponding to the region of interest 550, the user 150 may observe the image by moving only the user's eyes 151. In this case, the sharpness of the image corresponding to the area other than the region of interest 550 and observed using the peripheral vision of the user 150 may be lower than the sharpness of the image corresponding to the region of interest 550.

[0133] In an embodiment of the present disclosure, when observing an image displayed in an area other than the area of ​​interest 550 within the display 110 , the user 150 may observe the image displayed on the display 110 by moving the head or moving the head and eyes 151 of the user 150 .

[0134] In this case, since the HMD device 100 of the present disclosure is worn on the head of the user 150, the HMD device 100 also moves in response to the user 150 moving his head. Therefore, even when the user 150 moves his head, the center of the optical lens 120 corresponding to the area of ​​the display 110 on which the user 150 initially focuses the user's sight may be the same as the center 931 of the optical lens 120 before the user 150 moves his or her head. In addition, the position of the region of interest 550 included in the display 110 when the user 150 moves his or her head may also be the same as the position of the region of interest 550 included in the display 110 before the user 150 moves his or her head.

[0135] In an embodiment of the present disclosure, when at least one processor ( Figure 3 When the at least one processor 191 recognizes via the eye sensing module 192_3 that the head of the user 150 is moving, the at least one processor 191 may change the image displayed on the display 110 in response to the movement of the head of the user 150, so that the user 150 can observe the image displayed in the area other than the area of ​​interest 550 before the user 150 moves his or her head. In detail, when it is recognized that the head of the user 150 is moving, the at least one processor 191 may change the image displayed in the area of ​​interest 550 in response to the movement of the head of the user 150, thereby reproducing the same effect as observing the image from a wide viewing angle according to the movement of the head of the user 150.

[0136] In this case, even when the user 150 moves the head, the area other than the region of interest 550 is observed using peripheral vision, and thus, the clarity of an image displayed in the area other than the region of interest 550 recognized by the user 150 may be low.

[0137] In an embodiment of the present disclosure, at least one eye tracking light source 510 may be disposed outside the first display area 111 and the second display area 112 relative to the first direction axis 10. The at least one eye tracking light source 510 may overlap with the optical lens 120 relative to the first direction axis 10. The light 160 emitted by the at least one eye tracking light source 510 may be irradiated onto the user's eye 151 through the optical lens 120. However, the present disclosure is not limited thereto. Even when the at least one eye tracking light source 510 and the optical lens 120 do not overlap relative to the first direction axis 10, the light 160 provided by the at least one eye tracking light source 510 may be emitted onto the user's eye 151 through the optical lens 120.

[0138] In an embodiment of the present disclosure, at least one eye tracking sensor 520 may be disposed inside the second display area 112 relative to the first direction axis 10. At least one eye tracking sensor 520 may be disposed adjacent to the second side 115 of the display 110. A third distance between the at least one eye tracking sensor 520 and the user's eye 151 relative to the first direction axis 10 may be greater than the first distance between the optical lens 120 and the user's eye 151.

[0139] In an embodiment of the present disclosure, at least one eye tracking sensor 520 may be disposed outside the region of interest 550 relative to the first direction axis 10. The at least one eye tracking sensor 520 may overlap with the optical lens 120 relative to the first direction axis 10. The at least one eye tracking sensor 520 may receive reflected light 170 passing through the optical lens 120, wherein the reflected light 170 is light reflected from the user's eye 151. However, the present disclosure is not limited thereto. Even when the at least one eye tracking sensor 520 and the optical lens 120 do not overlap relative to the first direction axis 10, the at least one eye tracking sensor 520 may receive reflected light 170 passing through the optical lens 120 and reflected from the user's eye 151.

[0140] When the at least one eye tracking sensor 520 is not disposed within the display area with respect to the first direction axis 10 and is spaced apart from the display 110 with respect to at least one of the second direction axis 20 or the third direction axis 30, the reflected light receiving performance of the at least one eye tracking sensor 520 may be reduced. In detail, in order to observe an image displayed on the display 110, the user 150 may move the user's eye 151 in a direction away from the position where the at least one eye tracking sensor 520 is disposed. In this case, since the distance between the user's eye 151 and the at least one eye tracking sensor 520 on a plane defined by the second direction axis 20 and the third direction axis 30 is large, the amount of reflected light 170 from the user's eye 151 received by the at least one eye tracking sensor 520 may be insufficient, and thus, the light receiving performance of the at least one eye tracking sensor 520 may be deteriorated.

[0141] On the other hand, when at least one eye tracking sensor 520 in the HMD device 100 of the present disclosure is arranged in the second display area ( Figure 6 612), even if the user 150 moves the user's eye 151 away from the position where the at least one eye tracking sensor 520 is provided to observe the image displayed on the display 110, since the distance between the user's eye 151 and the at least one eye tracking sensor 520 in the plane defined by the second direction axis 20 and the third direction axis 30 is very close, the light receiving performance of the at least one eye tracking sensor 520 can also be improved.

[0142] Figure 6 is an enlarged view showing that an eye tracking sensor is disposed within a display area of ​​a display and an eye tracking light source is disposed outside the display area of ​​the display according to an embodiment of the present disclosure.

[0143] Figure 6 yes Figure 5 An enlarged view of area 560 is shown in FIG.

[0144] In an embodiment of the present disclosure, a display 600 may include a substrate 630 , a circuit layer 620 disposed on the substrate 630 , and a plurality of pixels 610 arranged on the circuit layer 620 .

[0145] In an embodiment of the present disclosure, the substrate 630 may be a member providing a base surface on which the circuit layer 620 is disposed. The substrate 630 may be a stacked structure including a plastic substrate, an insulating film, glass, or a plurality of insulating layers.

[0146] In an embodiment of the present disclosure, the circuit layer 620 may be disposed on the substrate 630. The circuit layer 620 may include an insulating layer, a semiconductor pattern, a conductive pattern, and a signal line. In an embodiment of the present disclosure, the circuit layer 620 may include a thin film transistor and a plurality of insulating layers on the substrate 630. The circuit layer 620 may also include a plurality of connection electrode units formed to penetrate the plurality of insulating layers.

[0147] In an embodiment of the present disclosure, a plurality of pixels 610 may be disposed on a circuit layer 620. Each of the plurality of pixels 610 may generate light. In an embodiment of the present disclosure, each of the plurality of pixels 610 may include an OLED or an inorganic LED that generates light of different colors. However, the present disclosure is not limited thereto, and each of the plurality of pixels 610 may include an OLED or an inorganic LED that generates light of the same color, and the display 600 may further include a color filter layer, wherein the color filter layer includes a plurality of color filters having different colors. In an embodiment of the present disclosure, each of the plurality of pixels 610 may include a quantum dot LED (QLED).

[0148] In an embodiment of the present disclosure, the plurality of pixels 610 may be electrically connected to at least one of a semiconductor pattern, a conductive pattern, or a signal line included in the circuit layer 620 .

[0149] In an embodiment of the present disclosure, each of the plurality of pixels 610 may include an emission layer for generating light and an electrode layer (such as a cathode and an anode) for applying a voltage to the emission layer. A portion of the light irradiated from the outside to the plurality of pixels 610 may be refracted, reflected, etc. by at least one of the emission layer or the electrode layer included in each of the plurality of pixels 610. Therefore, the transmittance of the region where the plurality of pixels 610 are arranged may be lower than the transmittance of the region where the plurality of pixels 610 are not arranged.

[0150] In an embodiment of the present disclosure, the display 600 may include a first display area 611 and a second display area 612 for displaying an image. In an embodiment of the present disclosure, the plurality of pixels 610 may include a plurality of first pixels 613 included in the first display area 611 and a plurality of second pixels 614 included in the second display area 612.

[0151] In an embodiment of the present disclosure, the resolution of the first display area 611 may be higher than the resolution of the second display area 612. The number of the plurality of first pixels 613 per unit area of ​​the first display area 611 may be greater than the number of the plurality of second pixels 614 per unit area of ​​the second display area 612. In an embodiment of the present disclosure, the size of each of the first pixels 613 may be equal to the size of each of the second pixels 614. In an embodiment of the present disclosure, the interval between adjacent first pixels among the plurality of first pixels 613 may be smaller than the interval between adjacent second pixels among the plurality of second pixels 614.

[0152] In an embodiment of the present disclosure, the transmittance of the second display area 612 having a smaller number of pixels per unit area may be higher than the transmittance of the first display area 611 having a larger number of pixels per unit area.

[0153] However, the present disclosure is not limited thereto. In an embodiment of the present disclosure, the size of each of the plurality of second pixels 614 may be larger than the size of each of the plurality of first pixels 613, and the interval between adjacent first pixels among the plurality of first pixels 613 may be smaller than the interval between adjacent second pixels among the plurality of second pixels 614. Even in this case, in an embodiment of the present disclosure, the transmittance of the second display area 612 having a smaller number of pixels per unit area may be higher than the transmittance of the first display area 611 having a larger number of pixels per unit area.

[0154] In an embodiment of the present disclosure, the first side ( Figure 5 The first side 114 of the display 600 may be a side adjacent to the plurality of pixels 610. The first side 114 of the display 600 may refer to a top surface of the plurality of pixels 610.

[0155] In the embodiment of the present disclosure, the second side ( Figure 5 The second side 115 of the display 600 may be a side adjacent to the substrate 630. The second side 115 of the display 600 may refer to a bottom surface of the substrate 630.

[0156] In an embodiment of the present disclosure, the substrate 630 may be disposed above the at least one eye tracking sensor 640. In an embodiment of the present disclosure, the at least one eye tracking sensor 640 may be disposed below the display 600 and may be disposed within the second display area 612 relative to the first direction axis 10. In an embodiment of the present disclosure, the at least one eye tracking sensor 640 may be disposed outside the first display area 611 relative to the first direction axis 10. The first display area 611 does not overlap with the at least one eye tracking sensor 640 relative to the first direction axis 10. In an embodiment of the present disclosure, the at least one eye tracking sensor 640 may be disposed to overlap with an area of ​​the substrate 630 corresponding to the second display area 612. The at least one eye tracking sensor 640 may be disposed below an area of ​​the substrate 630 included in the second display area 612.

[0157] In an embodiment of the present disclosure, at least one eye tracking sensor 640 may be disposed inside the second display area 612 having a relatively high transmittance compared to the first display area 611 to receive the reflected light ( Figure 5 170), wherein the reflected light is from the user's eyes ( Figure 5 151) reflected light.

[0158] The HMD device of the present disclosure ( Figure 1 100) may have at least one eye tracking sensor 640, wherein the at least one eye tracking sensor 640 is arranged to be close to the user's eye 151 in a plane defined by the second direction axis 20 and the third direction axis 30, and is included in a second display area 612 with high transmittance in the display 600, thereby improving the light receiving performance of the at least one eye tracking sensor 640.

[0159] Figure 7 1 is a conceptual diagram showing that each of the eye tracking sensor and the eye tracking light source according to an embodiment of the present disclosure is disposed within the display area. Figure 5 The components described are the same components, and the description already provided above will be omitted.

[0160] Reference Figure 1 and Figure 7 , a user's eye 151 , a display 110 , and an optical lens 120 are shown to illustrate the arrangement of at least one eye tracking light source 710 and at least one eye tracking sensor 720 .

[0161] In an embodiment of the present disclosure, the optical lens 120 may be disposed on the display 110 relative to the first direction axis 10. The optical lens 120 may be disposed adjacent to the first side 114 of the display 110. In an embodiment of the present disclosure, a first distance 740 between the optical lens 120 and the user's eye 151 relative to the first direction axis 10 may be smaller than a second distance between the display 110 and the user's eye 151.

[0162] In an embodiment of the present disclosure, in a plane defined by the second direction axis 20 and the third direction axis 30, the cross-sectional area of ​​the optical lens 120 may be greater than or equal to the cross-sectional area of ​​the display 110. The cross-sectional area of ​​the aperture stop of the optical lens 120 may be greater than or equal to the cross-sectional area of ​​the display area of ​​the display 110. With respect to the first direction axis 10, the entire area of ​​the display 110 may overlap with the optical lens 120. The aperture stop of the optical lens 120 may be inscribed in the display area of ​​the display 110.

[0163] In an embodiment of the present disclosure, the region of interest 750 may be determined based on a first distance 740 and a predetermined reference angle 730 , wherein the first distance 740 is a distance between the user's eye 151 and the optical lens 120 .

[0164] In an embodiment of the present disclosure, the region of interest 750 is a region where the user 150 can observe an image with central vision without moving the head by moving only the user's eyes 151. In an embodiment of the present disclosure, when observing an image corresponding to the region of interest 750, the user 150 can observe the image by moving only the user's eyes 151. In this case, the clarity of the image corresponding to the region other than the region of interest 750 and observed using the user's 150 peripheral vision may be lower than the clarity of the image corresponding to the region of interest 750.

[0165] In an embodiment of the present disclosure, at least one eye tracking light source 710 may be disposed inside the second display area 112 relative to the first direction axis 10. At least one eye tracking light source 710 may be disposed adjacent to the second side 115 of the display 110. A fourth distance between the at least one eye tracking light source 710 and the user's eye 151 relative to the first direction axis 10 may be greater than a first distance 740 between the optical lens 120 and the user's eye 151.

[0166] At least one eye tracking light source 710 may overlap with the optical lens 120 with respect to the first direction axis 10. Light 160 emitted by the at least one eye tracking light source 710 may pass through the display 110 and be irradiated onto the user's eye 151 through the optical lens 120. However, the present disclosure is not limited thereto. Even when the at least one eye tracking light source 710 and the optical lens 120 do not overlap with respect to the first direction axis 10, the light 160 provided by the at least one eye tracking light source 710 may be emitted onto the user's eye 151 through the optical lens 120.

[0167] In an embodiment of the present disclosure, at least one eye tracking light source 710 may be disposed outside the region of interest 750 relative to the first direction axis 10 .

[0168] In an embodiment of the present disclosure, at least one eye tracking sensor 720 may be disposed inside the second display area 112 relative to the first direction axis 10. At least one eye tracking sensor 720 may be disposed adjacent to the second side 115 of the display 110.

[0169] In an embodiment of the present disclosure, at least one eye tracking sensor 720 may be disposed outside the region of interest 750 relative to the first direction axis 10. At least one eye tracking sensor 520 may overlap with the optical lens 120 relative to the first direction axis 10. At least one eye tracking sensor 720 may receive reflected light 170 through the optical lens 120, which is light reflected from the user's eye 151. However, the present disclosure is not limited thereto. Even when at least one eye tracking sensor 720 and the optical lens 120 do not overlap relative to the first direction axis 10, at least one eye tracking sensor 720 may receive reflected light 170 through the optical lens 120, which is light reflected from the user's eye 151.

[0170] Figure 8 1 is an enlarged view showing that each of the eye tracking sensor and the eye tracking light source according to an embodiment of the present disclosure overlaps with the display area. Figure 6 The components described are the same components, and the description already provided above will be omitted.

[0171] Reference Figure 7 and Figure 8 , Figure 8 yes Figure 7 An enlarged view of the region 760 shown in FIG. 8 . In an embodiment of the present disclosure, the display 800 may include a substrate 830 , a circuit layer 820 disposed on the substrate 630 , and a plurality of pixels 810 arranged on the circuit layer 820 .

[0172] In an embodiment of the present disclosure, the display 800 may include a first display area 811 and a second display area 812 for displaying an image. In an embodiment of the present disclosure, the plurality of pixels 810 may include a plurality of first pixels 813 included in the first display area 811 and a plurality of second pixels 814 included in the second display area 812.

[0173] In an embodiment of the present disclosure, the resolution of the first display area 811 may be higher than the resolution of the second display area 812. The number of the plurality of first pixels 813 per unit area of ​​the first display area 811 may be greater than the number of the plurality of second pixels 814 per unit area of ​​the second display area 812. In an embodiment of the present disclosure, the transmittance of the second display area 812 having a smaller number of pixels per unit area may be higher than the transmittance of the first display area 811 having a larger number of pixels per unit area.

[0174] In an embodiment of the present disclosure, the second display area 812 may include a first sub-display area 812_1 overlapping with at least one eye tracking light source 840 relative to the first direction axis and a second sub-display area 812_2 overlapping with at least one eye tracking sensor 850 relative to the first direction axis.

[0175] In an embodiment of the present disclosure, at least one eye tracking light source 840 may be disposed below the display 800 and may be disposed inside the first sub display area 812_1 relative to the first direction axis 10. At least one eye tracking light source 840 may be disposed outside the first display area 811 relative to the first direction axis 10. At least one eye tracking light source 840 may be disposed below a region of the substrate 830 included in the first sub display area 812_1.

[0176] In an embodiment of the present disclosure, at least one eye tracking light source 840 may be disposed inside the first sub display area 812_1 to emit light 160 toward the user's eye 151, wherein the first sub display area 812_1 has a relatively high transmittance compared to the first display area 811. The number of the plurality of second pixels 814_1 per unit area of ​​the first sub display area 812_1 may be less than the number of the plurality of first pixels 813 per unit area of ​​the first display area 811 adjacent to the first sub display area 812_1. Therefore, the emission performance of the at least one eye tracking light source 840 that provides the light 160 to the user's eye 151 through the first sub display area 812_1 may be improved.

[0177] In an embodiment of the present disclosure, at least one eye tracking sensor 850 may be disposed below the display 800 and may be disposed inside the second sub display area 812_2 relative to the first direction axis 10. At least one eye tracking sensor 850 may be disposed outside the first display area 811 relative to the first direction axis 10. At least one eye tracking light source 850 may be disposed below a region of the substrate 830 included in the second sub display area 812_2.

[0178] In an embodiment of the present disclosure, at least one eye tracking sensor 850 may be disposed in the second sub display area 812_2 having a relatively high transmittance compared to the first display area 811 to receive the reflected light 170, which is light reflected from the user's eye 151. The number of the plurality of second pixels 814_2 per unit area of ​​the second sub display area 812_2 may be made smaller than the number of the plurality of first pixels 813 per unit area of ​​the first display area 811 adjacent to the second sub display area 812_2, thereby improving the light receiving performance of the at least one eye tracking sensor 850 that receives the reflected light 170 through the second sub display area 812_2.

[0179] Reference Figure 8 , the first sub display area 812_1 and the second sub display area 812_2 are spaced apart from each other. The first sub display area 812_1 and the second sub display area 812_2 may each be surrounded by the first display area 811. The first sub display area 812_1 and the second sub display area 812_2 may be included in the display 110 while being spaced apart from each other in a plane defined by the second direction axis 20 and the third direction axis 30. However, the present disclosure is not limited thereto, and the first sub display area 812_1 and the second sub display area 812_2 may contact each other and be integrated in the display 110 as a single area.

[0180] The plurality of second pixels 814 may include a plurality of first sub-pixels 814_1 included in the first sub-display area 812_1 and a plurality of second sub-pixels 814_2 included in the second sub-display area 812_2. Figure 8 , the number of the plurality of first sub-pixels 814_1 per unit area is equal to the number of the plurality of second sub-pixels 814_2 per unit area. However, the present disclosure is not limited thereto, and the number of the first sub-pixels 814_1 per unit area may be different from the number of the second sub-pixels 814_2 per unit area.

[0181] Fig. 9 1 is a plan view showing the arrangement of an eye tracking sensor and an eye tracking light source according to an embodiment of the present disclosure. Figures 5 to 8 The components described are the same components, and the description already provided above will be omitted.

[0182] Fig. 9 The display 110 , the optical lens 120 , the at least one eye-tracking light source 910 , and the at least one eye-tracking sensor 920 are shown as viewed from a first direction axis 10 .

[0183] In an embodiment of the present disclosure, the optical lens 120 may be disposed on the display 110 relative to the first direction axis 10. In an embodiment of the present disclosure, the optical lens 120 may have a circular shape in a plane defined by the second direction axis 20 and the third direction axis 30. In an embodiment of the present disclosure, the display 110 may have a rectangular shape in a plane defined by the second direction axis 20 and the third direction axis 30.

[0184] In an embodiment of the present disclosure, the entire cross-section of the display 110 in the plane defined by the second direction axis 20 and the third direction axis 30 may overlap with the cross-section of the optical lens 120 in the plane defined by the second direction axis 20 and the third direction axis 30. In an embodiment of the present disclosure, the cross-sectional area of ​​the display 110 in the plane defined by the second direction axis 20 and the third direction axis 30 may be smaller than the cross-sectional area of ​​the optical lens 120 in the plane defined by the second direction axis 20 and the third direction axis 30. The cross-sectional area of ​​the first display area 111 and the second display area 112 of the display 110 in the plane defined by the second direction axis 20 and the third direction axis 30 may be smaller than the cross-sectional area of ​​the aperture stop of the optical lens 120. The aperture stop of the optical lens 120 in the plane defined by the second direction axis 20 and the third direction axis 30 may be inscribed in the first display area 111 and the second display area 112 of the display 110.

[0185] In an embodiment of the present disclosure, the display 110 may include a first display area 111 and a second display area 112. The first display area 111 may correspond to a central portion of the display 110. The second display area 112 may be adjacent to the first display area 111. In an embodiment of the present disclosure, the second display area 112 may be surrounded by the first display area 111. However, the present disclosure is not limited thereto, and some of the boundaries of the second display area 112 may contact the first display area 111, and the remaining boundaries of the second display area 112 may not contact the first display area 111.

[0186] In an embodiment of the present disclosure, at least one eye tracking light source 910 may be disposed below the optical lens 120 and overlap the optical lens 120 relative to the first direction axis 10. The at least one eye tracking light source 910 may not overlap the display 110 relative to the first direction axis 10. The at least one eye tracking light source 910 may be disposed outside the first display area 111 and the second display area 112. However, the present disclosure is not limited thereto, and when the display 110 includes a non-display area (e.g., a frame) where no image is displayed, the at least one eye tracking light source 910 may be disposed outside the first display area 111 and the second display area 112, and may also be disposed inside the non-display area.

[0187] In an embodiment of the present disclosure, at least one eye tracking sensor 920 may be disposed below the display 110 and may be disposed inside the second display area 112 of the display 110 relative to the first direction axis 10. In an embodiment of the present disclosure, at least one eye tracking sensor 920 may be disposed outside the first display area 111 of the display 110.

[0188] Reference Figure 5 and Fig. 9 , an area of ​​interest 930 having a reference radius 932 may be defined based on a center 931 of the optical lens 120. The reference radius 932 may be set based on a predetermined reference angle 530 and a first distance 540. In an embodiment of the present disclosure, the area of ​​interest 930 may be an area formed after an image displayed on the display 110 is refracted, reflected, or scattered by the optical lens 120. The user 150 may recognize an image provided via the HMD device 100 based on an image formed in the area of ​​interest 930. In an embodiment of the present disclosure, at least one processor ( Figure 3 191) can identify a viewpoint where the line of sight of the user 150 intersects the area of ​​interest 930, and control the image displayed on the display 110 and the operation of the HMD device 100 based on the identified viewpoint. Figure 5 It is shown that the distance between the human eye 151 and the region of interest 550 is equal to the first distance 540 between the user's eye 151 and the optical lens 120, but the present disclosure is not limited thereto. Based on the shape, configuration, and refractive index of the optical lens 120, the distance between the user's eye 151 and the region of interest 550 may be smaller than the first distance 540 between the user's eye 151 and the optical lens 120.

[0189] In an embodiment of the present disclosure, the region of interest 930 in a plane defined by the second direction axis 20 and the third direction axis 30 may be different from a cross section of the display 110 defined by the second direction axis 20 and the third direction axis 30 .

[0190] In an embodiment of the present disclosure, the user 150 may move his or her eyes to view an image displayed in an area corresponding to the area of ​​interest 930 by using central vision having relatively high visual acuity compared to peripheral vision. Therefore, the image displayed in the area corresponding to the area of ​​interest 930 may be clearly viewed by the user 150.

[0191] On the other hand, the user 150 may view images displayed in areas other than the area corresponding to the area of ​​interest 930 by using peripheral vision having relatively low visual acuity compared to central vision. Therefore, images displayed in areas other than the area corresponding to the area of ​​interest 930 may appear blurry to the user 150.

[0192] In an embodiment of the present disclosure, the first display area 111 may correspond to the area of ​​interest 930. The first display area 111 may overlap the area of ​​interest 930 with respect to the first direction axis 10. A portion of the first display area 111 may overlap the area of ​​interest 930, and a portion of the first display area 111 may not overlap the area of ​​interest 930.

[0193] In an embodiment of the present disclosure, the second display area 112 may be disposed outside the area of ​​interest 930. The second display area 112 may not overlap the area of ​​interest 930 with respect to the first direction axis 10. Because the image displayed in the second display area 112 is an image displayed in an area of ​​the display 110 other than the area corresponding to the area of ​​interest 930, the image may appear blurred to the user 150. Therefore, even when the resolution of the second display area 112 is lower than that of the first display area 111, the user 150 may be prevented from perceiving the difference between the quality of the image displayed in the second display area 112 and the quality of the image displayed in the first display area 111.

[0194] Therefore, the HMD device 100 of the present disclosure can set at least one eye tracking sensor 920 close to the user's eye 151 in the plane defined by the second direction axis 20 and the third direction axis 30, and be included in the second display area 112 with high transmittance in the display 110, thereby improving the light receiving performance of the at least one eye tracking sensor 920 while preventing the user 150 from perceiving the difference in image quality caused by the second display area 112.

[0195] Fig.10 1 is a plan view showing the arrangement of an eye tracking sensor and an eye tracking light source according to an embodiment of the present disclosure. Fig. 9 The components described are the same components, and the description already provided above will be omitted.

[0196] Fig.10A display 110 , an optical lens 120 , at least one eye-tracking light source 1010 , and at least one eye-tracking sensor 1020 are shown as viewed from a first directional axis 10 .

[0197] In an embodiment of the present disclosure, the optical lens 120 may be disposed on the display 110 relative to the first direction axis 10. In an embodiment of the present disclosure, the optical lens 120 may have a circular shape in a plane defined by the second direction axis 20 and the third direction axis 30. In an embodiment of the present disclosure, the display 110 may have a rectangular shape in a plane defined by the second direction axis 20 and the third direction axis 30.

[0198] In an embodiment of the present disclosure, at least one eye tracking light source 1010 may be disposed below the optical lens 120 and overlap the optical lens 120 relative to the first direction axis 10. At least one eye tracking light source 1010 may be disposed outside the first display area 111 and the second display area 112.

[0199] although Fig.10 The HMD device ( Figure 1 100) includes four eye tracking light sources 1010, but the present disclosure is not limited thereto. The HMD device 100 may include three or fewer eye tracking light sources, or five or more eye tracking light sources. In addition, although Fig.10 The four eye tracking light sources 1010 are shown to be spaced apart from each other on the top, bottom, left, and right sides of the optical lens 120, but the present disclosure is not limited thereto. The arrangement of the at least one eye tracking light source 1010 included in the HMD device 100 may be based on the light from the user's eyes ( Figure 1 151) varies with the pattern of reflected light to be measured.

[0200] In an embodiment of the present disclosure, at least one eye tracking sensor 1020 may be disposed below the display 110, and may be disposed inside the second display area 112 of the display 110 relative to the first direction axis 10. In an embodiment of the present disclosure, at least one eye tracking sensor 1020 may be disposed outside the first display area 111 of the display 110. In an embodiment of the present disclosure, at least one eye tracking sensor 1020 may be disposed outside the region of interest 930.

[0201] In the embodiments of the present disclosure, Fig.10 The HMD device 100 is shown to include two eye tracking sensors 1020. The two eye tracking sensors 1020 may be respectively disposed inside the second display area 112. However, the present disclosure is not limited thereto. The HMD device 100 may include one eye tracking sensor or three or more eye tracking sensors. In addition, although Fig.10It is shown that the two eye tracking sensors 1020 are spaced apart from each other, but the present disclosure is not limited thereto. Two or more eye tracking sensors included in the HMD device 100 may be arranged adjacent to each other and overlap a common second display area.

[0202] Fig.11 1 is a plan view showing the arrangement of an eye tracking sensor and an eye tracking light source according to an embodiment of the present disclosure. Fig. 9 and Fig.10 The components described are the same components, and the description already provided above will be omitted.

[0203] Fig.11 A display 110 , an optical lens 120 , at least one eye-tracking light source 1110 , and at least one eye-tracking sensor 1120 are shown as viewed from a first direction axis 10 .

[0204] In an embodiment of the present disclosure, the optical lens 120 may be disposed on the display 110 relative to the first direction axis 10. In an embodiment of the present disclosure, the optical lens 120 may have a circular shape in a plane defined by the second direction axis 20 and the third direction axis 30. In an embodiment of the present disclosure, the display 110 may have a rectangular shape in a plane defined by the second direction axis 20 and the third direction axis 30.

[0205] In an embodiment of the present disclosure, at least one eye tracking light source 1110 may be disposed below the display 110, so that the at least one eye tracking light source 1110 may be disposed inside the second display area 112 of the display 110 relative to the first direction axis 10. At least one eye tracking light source 1110 may be disposed outside the first display area 111 of the display 110. In an embodiment of the present disclosure, at least one eye tracking light source 1110 may be disposed outside the region of interest 930. At least one eye tracking light source 1110 may overlap with the optical lens 120 relative to the first direction axis 10.

[0206] Fig.11 The HMD device 100 is shown to include three eye tracking light sources 1110. The second display area 112 may include three first sub-display areas 112_1 respectively overlapping the three eye tracking light sources 1110. Fig.11 It is shown that three eye tracking light sources 1110 and three first sub display areas 112_1 are arranged to be spaced apart from each other, but the present disclosure is not limited thereto. The three eye tracking light sources 1110 may be arranged adjacent to each other and may be provided in one common first sub display area 112 .

[0207] In an embodiment of the present disclosure, at least one eye tracking sensor 1120 may be disposed below the display 110, so that at least one eye tracking sensor 1120 may be disposed inside the second display area 112 of the display 110 relative to the first direction axis 10. At least one eye tracking sensor 1120 may be disposed outside the region of interest 930.

[0208] Fig.11 The HMD device is shown to include an eye tracking sensor 1120. The second display area 112 may include a second sub-display area 112_2 overlapping the eye tracking sensor 1120. Fig.11 One eye tracking sensor 1120 and three eye tracking light sources 1110 are shown to be spaced apart from each other, but the present disclosure is not limited thereto. One eye tracking sensor 1120 and three eye tracking light sources 1110 may be arranged adjacent to each other and may be disposed within one common second display area 112 .

[0209] Fig.12 1 is a plan view showing the arrangement of an eye tracking sensor and an eye tracking light source according to an embodiment of the present disclosure. Figures 9 to 11 The components described are the same components, and the description already provided above will be omitted.

[0210] Fig.12 A display 110 , an optical lens 120 , at least one eye-tracking light source 1210 , and at least one eye-tracking sensor 1220 are shown as viewed from a first direction axis 10 .

[0211] In an embodiment of the present disclosure, the optical lens 120 may be disposed on the display 110 relative to the first direction axis 10. In an embodiment of the present disclosure, the optical lens 120 may have a circular shape in a plane defined by the second direction axis 20 and the third direction axis 30. In an embodiment of the present disclosure, the display 110 may have a circular shape in a plane defined by the second direction axis 20 and the third direction axis 30. However, the present disclosure is not limited thereto, and the optical lens 120 and the display 110 may have other shapes, such as a quadrilateral shape.

[0212] In an embodiment of the present disclosure, at least one eye tracking light source 1210 may be disposed below the optical lens 120 and overlap the optical lens 120 relative to the first direction axis 10. At least one eye tracking light source 1210 may be disposed outside the first display area 111 and the second display area 112 relative to the first direction axis 10.

[0213] In an embodiment of the present disclosure, at least one eye tracking sensor 1220 may be disposed below the display 110, and may be disposed inside the second display area 112 of the display 110 relative to the first direction axis 10. In an embodiment of the present disclosure, at least one eye tracking sensor 1220 may be disposed outside the first display area 111 of the display 110. In an embodiment of the present disclosure, at least one eye tracking sensor 1220 may be disposed outside the region of interest 1230, wherein the region of interest 1230 has a reference radius 1232 based on the center 1231 of the optical lens 120.

[0214] In order to solve the above problems, according to an embodiment of the present disclosure, there is provided an HMD device including a display and an optical lens disposed adjacent to a first side of the display. The HMD device may include at least one light source. The HMD device may include at least one eye tracking sensor, wherein the at least one eye tracking sensor is disposed adjacent to a second side of the display and is configured to obtain line of sight information about the user by receiving reflected light, wherein the reflected light is obtained when light emitted from the at least one light source is reflected from the user's eyes. The display may include a second display area corresponding to a position where the at least one eye tracking sensor is disposed and a first display area other than the second display area. The number of pixels per unit area of ​​the first display area may be greater than the number of pixels per unit area of ​​the second display area.

[0215] In an embodiment of the present disclosure, the first display area may correspond to an area of ​​interest, wherein the area of ​​interest is an area having a reference radius corresponding to a reference angle, wherein the reference angle is predetermined based on the center of the optical lens and the distance between the user's eye and the optical lens. The second display area may be adjacent to the first display area and may be disposed outside the area of ​​interest.

[0216] In an embodiment of the present disclosure, the predetermined reference angle may be set to an angle of at least 10° but not greater than 40°.

[0217] In an embodiment of the present disclosure, the cross-sectional area of ​​the display may be smaller than the cross-sectional area of ​​the optical lens. In an embodiment of the present disclosure, the cross-sectional area of ​​the aperture stop of the optical lens may be greater than or equal to the cross-sectional area of ​​the display area of ​​the display. In an embodiment of the present disclosure, the aperture stop of the optical lens may be inscribed in the display area of ​​the display.

[0218] In an embodiment of the present disclosure, at least one eye tracking sensor may obtain biometric information about a user by receiving reflected light, wherein the reflected light is obtained when light emitted from at least one light source is reflected from the user's eyes.

[0219] In an embodiment of the present disclosure, a display may include a substrate, a circuit layer disposed on the substrate, and a plurality of pixels disposed on the circuit layer, wherein a first side of the display is a side adjacent to the plurality of pixels, and a second side of the display is a side adjacent to the substrate.

[0220] In an embodiment of the present disclosure, the substrate may be disposed above the at least one eye tracking sensor, and the at least one eye tracking sensor may be disposed to overlap with a region of the substrate corresponding to an outside of the first display area.

[0221] In an embodiment of the present disclosure, the second display area may include a first sub-display area and a second sub-display area, wherein at least one eye tracking sensor is disposed in the first sub-display area, and at least one light source is disposed in the second sub-display area.

[0222] In an embodiment of the present disclosure, a first distance between the user's eyes and the optical lens may be smaller than a second distance between the user's eyes and the display.

[0223] In an embodiment of the present disclosure, the display may include a first display corresponding to a left eye of a user and a second display corresponding to a right eye of the user. The optical lens may include a first optical lens corresponding to the first display and a second optical lens corresponding to the second display. The at least one eye tracking sensor may include at least one first eye tracking sensor corresponding to the first display and at least one second eye tracking sensor corresponding to the second display.

[0224] In an embodiment of the present disclosure, the at least one light source may include at least one first eye-tracking light source corresponding to the first display and at least one second eye-tracking light source corresponding to the second display.

[0225] In an embodiment of the present disclosure, the transmittance of the second display area may be greater than the transmittance of the first display area.

[0226] In an embodiment of the present disclosure, each of the plurality of pixels may include an OLED.

[0227] In order to solve the above problems, according to an embodiment of the present disclosure, there is also provided an HMD device including a display and an optical lens disposed adjacent to a first side of the display. The HMD device may include at least one light source. The HMD device may include at least one eye tracking sensor, wherein the at least one eye tracking sensor is disposed adjacent to a second side of the display and is configured to receive reflected light, wherein the reflected light is obtained when light emitted from the at least one light source is reflected from an eye of a user. The HMD device may include a memory storing at least one instruction and at least one processor configured to execute at least one instruction stored in the memory. The at least one processor may be configured to obtain line of sight information about the user based on the reflected light received by the at least one eye tracking sensor. The display may include a second display area corresponding to a position where the at least one eye tracking sensor is disposed and a first display area other than the second display area. The number of pixels per unit area of ​​the first display area may be greater than the number of pixels per unit area of ​​the second display area.

[0228] In an embodiment of the present disclosure, the first display area may correspond to an area of ​​interest, wherein the area of ​​interest is an area having a reference radius corresponding to a reference angle, wherein the reference angle is predetermined based on the center of the optical lens and the distance between the user's eye and the optical lens, and the second display area may be adjacent to the first display area and may be set outside the area of ​​interest. The predetermined reference angle may be set to an angle of at least 10° but not greater than 40°.

[0229] In an embodiment of the present disclosure, the display may include a first display corresponding to a left eye of a user and a second display corresponding to a right eye of the user. The optical lens may include a first optical lens corresponding to the first display and a second optical lens corresponding to the second display. The at least one eye tracking sensor may include at least one first eye tracking sensor corresponding to the first display and at least one second eye tracking sensor corresponding to the second display. The at least one light source may include at least one first eye tracking light source corresponding to the first display and at least one second eye tracking light source corresponding to the second display.

[0230] In order to solve the above problems, according to an embodiment of the present disclosure, there is also provided an operating method of an HMD device, wherein the HMD device includes a display and an optical lens disposed adjacent to a first side of the display. The operating method of the HMD device may include emitting light toward an eye of a user by using at least one light source. The operating method of the HMD device may include receiving reflected light by using at least one eye tracking sensor disposed adjacent to a second side of the display, wherein the reflected light is obtained when light is emitted from the at least one light source and reflected from an eye of the user. The operating method of the HMD device may include obtaining line of sight information about the user based on the received reflected light. The display may include a second display area corresponding to a position where the at least one eye tracking sensor is disposed and a first display area other than the second display area. The number of pixels per unit area of ​​the first display area may be greater than the number of pixels per unit area of ​​the second display area.

[0231] In an embodiment of the present disclosure, the first display area may correspond to an area of ​​interest, wherein the area of ​​interest is an area having a reference radius corresponding to a reference angle, wherein the reference angle is predetermined based on the center of the optical lens and the distance between the user's eye and the optical lens, and the second display area may be adjacent to the first display area and may be set outside the area of ​​interest. The predetermined reference angle may be set to an angle of at least 10° but not greater than 40°.

[0232] In an embodiment of the present disclosure, the display may include a first display corresponding to a left eye of a user and a second display corresponding to a right eye of the user. The optical lens may include a first optical lens corresponding to the first display and a second optical lens corresponding to the second display. The at least one eye tracking sensor may include at least one first eye tracking sensor corresponding to the first display and at least one second eye tracking sensor corresponding to the second display. The at least one light source may include at least one first eye tracking light source corresponding to the first display and at least one second eye tracking light source corresponding to the second display.

[0233] In an embodiment of the present disclosure, there is provided a computer-readable recording medium having a program recorded thereon, wherein the program is for executing at least one of the operating methods of the HMD device 100 according to the embodiment of the present disclosure on a computer.

[0234] The program executed by the HMD device 100 described in this specification may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. The program may be executed by any system capable of executing computer-readable instructions.

[0235] The software may include a computer program, a piece of code, an instruction, or a combination of one or more of them, and configures the processing device to operate as desired or instructs the processing device, either independently or collectively.

[0236] Software may be implemented as a computer program including instructions stored in a computer-readable storage medium. Examples of computer-readable recording media include magnetic storage media (e.g., ROM, RAM, floppy disk, hard disk, etc.), optical recording media (e.g., compact disk (CD)-ROM, digital versatile disk (DVD), etc.), etc. Computer-readable recording media may be distributed on computer systems connected via a network so that computer-readable code may be stored and executed in a distributed manner. The recording medium may be read by a computer, stored in a memory, and executed by a processor.

[0237] The computer-readable storage medium may be provided in the form of a non-transitory storage medium. In this regard, the term "non-transitory" simply means that the storage medium does not include a signal (e.g., an electromagnetic wave) and is a tangible device, and the term does not distinguish between data being semi-permanently stored in the storage medium and data being temporarily stored in the storage medium. For example, a "non-transitory storage medium" may include a buffer that temporarily stores data.

[0238] In addition, when provided, the program according to the embodiment of the present disclosure set forth in this specification may be included in a computer program product. The computer program product may be traded between a seller and a buyer as a product.

[0239] The computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, the computer program product may include a computer program that is provided by a manufacturer of the HMD device 100 or through an electronic market (e.g., Samsung Galaxy Store). TM ) A product in the form of a software program that is electronically distributed (e.g., a downloadable application). For such electronic distribution, at least a portion of the software program may be stored in a storage medium or may be temporarily generated. In this case, the storage medium may be a storage medium of a server of a manufacturer of the HMD device 100, a server of an electronic market, or a relay server for temporarily storing software programs.

Claims

1. A head mounted display (HMD) device (100), include: Display (110); an optical lens (120) disposed adjacent to a first side of the display (110); at least one light source (130) configured to emit light; as well as at least one eye tracking sensor (140) adjacent to a second side of the display (110) and configured to obtain line of sight information about the user by receiving reflected light, wherein the reflected light is at least a portion of light emitted from at least one light source (130) and reflected from an eye of the user, The display (110) includes a second display area corresponding to the position of at least one eye tracking sensor (140) and a first display area other than the second display area, and The number of pixels per unit area of ​​the first display area is greater than the number of pixels per unit area of ​​the second display area.

2. The HMD device (100) according to claim 1, in, The first display area corresponds to an area of ​​interest having a reference radius corresponding to a reference angle, wherein the reference angle is predetermined based on a center of the optical lens (120) and a distance between the user's eye and the optical lens (120), The second display area is adjacent to the first display area and is outside the area of ​​interest.

3. The HMD device (100) according to claim 2, in, The reference angle is set to an angle within a range of 10° to 40°.

4. The HMD device (100) according to any one of claims 1 to 3, in, The cross-sectional area of ​​the display (110) is smaller than the cross-sectional area of ​​the optical lens (120).

5. The HMD device (100) according to any one of claims 1 to 4, in, At least one eye tracking sensor (140) is also configured to obtain biometric information about the user by receiving reflected light, wherein the reflected light is obtained when light emitted from at least one light source (130) is reflected from the user's eyes.

6. The HMD device (100) according to any one of claims 1 to 5, in, The display (100) comprises: substrate; A circuit layer on the substrate; and A plurality of pixels on the circuit layer, wherein a first side of the display is adjacent to the plurality of pixels, and A second side of the display is adjacent to the substrate.

7. The HMD device (100) according to claim 6, in, The substrate is above at least one eye tracking sensor (140), and At least one eye tracking sensor (140) overlaps with an area of ​​the substrate corresponding to the outside of the first display area.

8. The HMD device (100) as claimed in any one of claims 6 and 7, in, The substrate is above at least one light source (130), and Wherein, at least one light source (130) overlaps with a region of the substrate corresponding to the second display area.

9. The HMD device (100) according to claim 8, in, The second display area includes a first sub-display area provided with at least one eye tracking sensor (140) and a second sub-display area provided with at least one light source (130).

10. The HMD device (100) according to any one of claims 1 to 9, in, A first distance between the user's eye and the optical lens (120) is smaller than a second distance between the user's eye and the display (110).

11. The HMD device (100) according to any one of claims 1 to 10, in, The display (110) includes a first display corresponding to the left eye of the user and a second display corresponding to the right eye of the user, The optical lens (120) includes a first optical lens corresponding to the first display and a second optical lens corresponding to the second display. At least one eye tracking sensor (140) includes at least one first eye tracking sensor corresponding to the first display and at least one second eye tracking sensor corresponding to the second display.

12. The HMD device (100) according to claim 11, in, The at least one light source (130) includes at least one first eye-tracking light source corresponding to the first display and at least one second eye-tracking light source corresponding to the second display.

13. The HMD device (100) according to any one of claims 1 to 12, in, The transmittance of the second display area is greater than the transmittance of the first display area.

14. A head mounted display (HMD) device (100), include: Display (110); an optical lens (120) adjacent to the first side of the display; at least one light source (130) configured to emit light; at least one eye tracking sensor (140) adjacent to the second side of the display and configured to receive reflected light, wherein the reflected light is light emitted from the at least one light source (130) and reflected from an eye of the user; a memory (192) storing at least one instruction; and at least one processor (191) configured to execute the at least one instruction stored in the memory (192) to obtain line of sight information about the user based on the reflected light received by at least one eye tracking sensor (140), The display comprises a second display area corresponding to the position of at least one eye tracking sensor (140) and a first display area other than the second display area, and The number of pixels per unit area of ​​the first display region is greater than the number of pixels per unit area of ​​the second display region.

15. A method for operating a head mounted display (HMD) device (100), in, The HMD device (100) includes a display (110) and an optical lens (120) disposed adjacent to a first side of the display (110), and the operating method includes: At least one light source (130) emits light toward eyes of a user (S100); receiving reflected light by at least one eye tracking sensor (140) adjacent to a second side of the display, wherein the reflected light is light emitted from at least one light source (130) and reflected from an eye of the user (S200); and obtaining line of sight information about the user based on the received reflected light (S300), The display (110) includes a second display area corresponding to a position where at least one eye tracking sensor (140) is arranged and a first display area other than the second display area, and The number of pixels per unit area of ​​the first display area is greater than the number of pixels per unit area of ​​the second display area.

Images