Optical device, electronic device including same, display method, control method, and control device

Abstract

Disclosed in an embodiment is an optical device comprising: a plurality of lenses; a lens barrel in which the plurality of lenses are arranged; a first light guide disposed in the lens barrel; an opening formed in a side surface of the lens barrel; a light source device coupled to the opening; a through hole formed in a side surface of the lens barrel; and a light receiving portion disposed adjacent to the through hole.

Description

Technical Field

[0001] Embodiments disclosed herein relate to an optical device, an electronic device including the optical device, a display method, a control method, and a control device. Background Art

[0002] Virtual Reality (VR) refers to a specific environment or situation created by adopting artificial technology using a computer or the like, and thus simulates reality but is not real, or refers to the artificial technology itself.

[0003] Augmented Reality (AR) refers to a technology that superimposes virtual objects or information onto the real environment and makes the virtual objects appear to exist in the original environment.

[0004] Mixed Reality (MR) or Hybrid Reality refers to creating a new environment or new information by mixing the virtual world with the real world. In particular, mixed reality refers to the real-time interaction between the real-time existence of objects in the real world and the real-time existence of objects in the virtual world.

[0005] At this time, the created virtual environment, situation, etc. stimulates the five senses of the user, and enables the user to experience a virtual world similar to the real world in space or time, thereby freely blurring the boundary between reality and imagination. In addition, the user can not only immerse himself in such a situation, but also participate in mutual interaction with the virtual world, for example, by performing operations or issuing commands using devices in the real world.

[0006] In recent years, active research has been conducted on devices (or gear) used in this technical field. However, there is an increasing demand for detecting whether a user is wearing these devices and monitoring the user's biosignals.

[0007] In addition, these devices need to be adjusted for the user's vision. There is an increasing demand for such adjustments for the user's vision. In addition, there is an increasing demand for simple controllers for these devices. In addition, there is an increasing demand for these devices to be equipped with additional features. Summary of the invention

[0008] [Technical issues]

[0009] An object of an embodiment is to provide an optical device capable of accurately detecting a user's biosignal using augmented reality (AR) or signal technology, and an electronic device including the optical device.

[0010] Another object of an embodiment is to provide an optical device capable of reducing noise when sensing a signal (biological signal), and an electronic device including the optical device.

[0011] Still another object of an embodiment is to provide an optical device capable of accurately detecting whether a user is wearing an electronic device, and an electronic device including the optical device.

[0012] Another object of an embodiment is to provide an optical device capable of achieving improved power efficiency, and an electronic device including the optical device.

[0013] Still another object of an embodiment is to provide an electronic device and a display method capable of generating an image based on the vision of a user.

[0014] Still another object of an embodiment is to provide an electronic device and a display method capable of comfortably enabling a user to recognize a virtual image or the like through an image selected by the user.

[0015] Another object of an embodiment is to provide a control device and a control method capable of reducing malfunctions by activating input types in a manner corresponding to a state of a user.

[0016] Still another object of an embodiment is to provide a control device and a control method capable of intuitively controlling an electronic device without attracting the attention of others.

[0017] Another object of an embodiment is to provide an optical device capable of easily suppressing a fogging phenomenon, and an electronic device including the optical device.

[0018] Still another object of the embodiment is to provide an optical device capable of improving reliability through heat dissipation, and an electronic device including the optical device.

[0019] Still another object of the present embodiment is to provide an optical device capable of achieving improved energy efficiency, and an electronic device including the optical device.

[0020] The embodiments are not limited to the above-mentioned objects, and additional objects and effects may also be identified from means for solving the problems or from the embodiments, both of which are described below.

[0021] [Technical solution]

[0022] According to one aspect of an embodiment, an optical device is provided, comprising: a plurality of lenses; a lens barrel, wherein the plurality of lenses are arranged in the lens barrel; a first light guide, wherein the first light guide is arranged in the lens barrel; an opening portion, wherein the opening portion is formed in a side surface of the lens barrel; a light source device, wherein the light source device is coupled to the opening portion; a through hole, wherein the through hole is formed in the side surface of the lens barrel; and a light receiving unit, wherein the light receiving unit is arranged adjacent to the through hole.

[0023] The optical device may include a barrier unit arranged between the through hole and the light receiving unit.

[0024] The optical device may include: an optical signal generating unit which generates an optical signal including an image signal, wherein the light receiving unit may be arranged between the first light guide and the optical signal generating unit.

[0025] In the optical device, the plurality of lenses may include a first lens and a second lens disposed between the first light guide and the optical signal generating unit, and the first lens may have a smaller size than the second lens.

[0026] In the optical device, the light receiving unit may be disposed between the first lens and the second lens.

[0027] In the optical device, the through hole may be arranged between the first lens and the second lens.

[0028] In the optical device, the light receiving unit may protrude further than an outer surface of the lens barrel.

[0029] In the optical device, the barrier unit may protrude further than an outer surface of the lens barrel.

[0030] The optical device may include: a housing having an opening portion formed therein; a second light guide disposed in the housing; and a light source emitting light toward the second light guide.

[0031] In the optical device, the light receiving unit may be driven while light is emitted from the light source.

[0032] The optical device may include a control unit that controls the light source device, the optical signal generating unit, and the light receiving unit.

[0033] In the optical device, the control unit may receive a first light receiving signal from the light receiving unit when no current is applied to the light source device.

[0034] In the optical device, the control unit may determine whether to drive the light source device by comparing the first light receiving signal with a first set value.

[0035] In the optical device, in a case where the first light reception signal is lower than a first set value, the control unit may drive the light source and receive a second light reception signal from the light reception unit.

[0036] In the optical device, the control unit may compare the second light reception signal with the second set value and accordingly receive a third light reception signal from the light reception unit in a manner corresponding to respective operations of the optical signal generation unit and the light source.

[0037] According to another aspect of the embodiment, there is provided an electronic device, comprising: a frame; an input unit, which is arranged on the frame and outputs an input signal; an optical device, which is arranged on the frame and generates an image; a display unit, which is connected to the optical device and outputs the image generated by the optical device; and a control unit, which controls an image signal of the image, wherein the control unit provides an image signal of a test image with different focal lengths and determines the user's vision based on the image selected by the input unit.

[0038] In the electronic device, the test images having different focal lengths may be adjusted through the input unit.

[0039] In the electronic device, a plurality of test images having different focal lengths may be provided, and the plurality of test images may have respective different focal lengths.

[0040] In the electronic device, at least one image may be selected from a plurality of test images.

[0041] In the electronic device, the control unit may determine the eyesight of the user, and then adjust the focus of the image based on the determined eyesight of the user, thereby outputting the resulting image.

[0042] In the electronic device, in a case where a plurality of images are selected, the control unit may determine the eyesight of the user based on the maximum focal length.

[0043] In the electronic device, the control unit may determine that the test images having different focal lengths may each include a plurality of objects having different focal lengths.

[0044] In the electronic device, the control unit may adjust the focus of the test image by moving a lens within the optical device.

[0045] The electronic device may include an optical element disposed on a frame, wherein the optical element may include a variable focus lens.

[0046] In the electronic device, the control unit may change the curvature of the optical element.

[0047] According to another aspect of the embodiment, a display method is provided, which includes the steps of outputting test images with different focal lengths; determining the user's vision based on an image selected from the test images with different focal lengths; and outputting a virtual image based on the determined user's vision.

[0048] The display method may include a step of storing the determined eyesight of the user after the step of determining the eyesight of the user.

[0049] In the display method, the step of determining the eyesight of the user may include the step of outputting a plurality of test images having different focal lengths.

[0050] In the display method, the step of outputting the virtual image may include the step of changing a focal length of the virtual image based on the determined eyesight of the user.

[0051] In the display method, the step of determining the user's vision may include: the step of selecting at least one image from a plurality of test images having different focal lengths; and the step of determining the determined user's vision based on a maximum focal length when a plurality of images are selected.

[0052] According to another aspect of the embodiment, a control device is provided, including: a detection unit that outputs a detection signal including exercise information; and a controller, wherein the controller determines a user's state based on the exercise information and determines activation of gesture motion and touch motion in a manner corresponding to the user's state.

[0053] In the control device, when the intensity of the detection signal is higher than the first set value, the controller may compare whether the amplitude of the detection signal is higher than the second set value.

[0054] In the control device, when the amplitude of the detection signal is higher than a second set value, the controller may determine the state of the user as an exercise state.

[0055] In the control device, when the amplitude of the detection signal is lower than the second set value, the controller may compare whether the intensity of the detection signal is higher than a third set value.

[0056] In the control device, the controller may determine the state of the user as the riding state when the intensity of the detection signal is higher than a third setting value, and the third setting value may be higher than the first setting value.

[0057] In the control device, when the intensity of the detection signal is lower than the first set value, the controller may compare whether the amplitude of the detection signal is higher than a fourth set value, and the fourth set value may be lower than the second set value.

[0058] In the control device, when the amplitude of the detection signal is higher than a fourth set value, the controller may determine the state of the user as a moving state.

[0059] In the control device, when the amplitude of the detection signal is lower than a fourth set value, the controller may determine the state of the user as a stationary state.

[0060] In the control device, when the touch motion is activated, the controller may configure the initial input position of the user to be one of: up, down, left, or right with respect to the center portion.

[0061] According to another embodiment of the implementation, a control method is provided, which includes: a step of receiving a detection signal; a step of determining a user's state; a step of determining activation of a gesture movement or a touch movement in a manner corresponding to the determined user's state; and a step of sending a command in a manner corresponding to the activated movement.

[0062] In the control method, the step of determining the state of the user may include the step of comparing whether the amplitude of the detection signal is higher than a second setting value when the intensity of the detection signal is higher than the first setting value.

[0063] The control method may include the step of determining the state of the user as an exercise state when the amplitude of the detection signal is higher than a second set value.

[0064] The control method may include the step of comparing whether the intensity of the detection signal is higher than a third set value when the amplitude of the detection signal is lower than the second set value.

[0065] The control method may include the step of determining the state of the user as a riding state when the intensity of the detection signal is higher than a third setting value,

[0066] The third setting value may be higher than the first setting value.

[0067] The control method may include the step of comparing whether the amplitude of the detection signal is higher than a fourth set value when the intensity of the detection signal is lower than the first set value, wherein the fourth set value may be lower than the second set value.

[0068] The control method may include the step of determining the state of the user as a moving state when the amplitude of the detection signal is higher than a fourth set value.

[0069] The control method may include the step of determining the state of the user as a stationary state when the amplitude of the detection signal is lower than a fourth set value.

[0070] According to yet another aspect of the embodiment, there is provided an electronic device including: a frame; an optical device arranged on the frame and generating an image; a display unit connected to the optical device and outputting the image generated by the optical device; and a heat dissipation unit connected to the optical device and arranged within the frame.

[0071] In the electronic device, the optical device may include: a lens barrel on which a plurality of lenses are arranged; an opening portion formed in a side surface of the lens barrel; and a light source device coupled to the opening portion.

[0072] Wherein, the light source device may include:

[0073] a housing having an opening formed therein; a light source emitting light toward the second light guide; and

[0074] A substrate is connected to the light source.

[0075] In the electronic device, the heat dissipation unit may be in contact with the substrate.

[0076] In the electronic device, the heat dissipation unit may extend from an outer surface of the housing to the frame.

[0077] In the electronic device, the optical device may include an optical signal generating unit that generates an optical signal including image information, wherein the heat dissipation unit may extend adjacent to the optical signal generating unit.

[0078] The electronic device may include a heating unit disposed on the frame.

[0079] In the electronic device, the heat supply unit and the heat dissipation unit may be arranged to be spaced apart from each other.

[0080] In the electronic device, the frame may include: an upper region; and a lower region disposed below the upper region, wherein one of the heat supply unit and the heat dissipation unit is disposed in the upper region, and the remaining units may be disposed in the lower region.

[0081] In the electronic device, the heat dissipation unit may be arranged along an outer surface of the frame.

[0082] [Beneficial Effects]

[0083] According to an embodiment, an optical device capable of accurately detecting a user's biosignal using augmented reality (AR) or signal technology, and an electronic device including the optical device may be implemented.

[0084] In addition, an optical device capable of reducing noise when sensing a signal (biological signal), and an electronic device including the optical device can be realized.

[0085] In addition, an optical device capable of accurately detecting whether a user is wearing an electronic device, and an electronic device including the optical device can be realized.

[0086] In addition, an optical device capable of achieving improved power efficiency and an electronic device including the optical device can be realized.

[0087] In addition, an electronic device and a display method capable of generating an image based on the user's vision using augmented reality (AR) or signal technology may be realized.

[0088] In addition, an electronic device and a display method capable of comfortably enabling a user to recognize a virtual image or the like through an image selected by the user can be realized.

[0089] In addition, a control device and a control method capable of reducing malfunctions by activating an input type in a manner corresponding to a state of a user using augmented reality (AR) or signal technology may be realized.

[0090] In addition, a control device and a control method capable of intuitively controlling an electronic device without attracting the attention of others can be realized.

[0091] In addition, an optical device capable of easily suppressing a fogging phenomenon and an electronic device including the optical device can be realized.

[0092] In addition, an optical device capable of achieving improved reliability through heat dissipation and an electronic device including the optical device can be realized.

[0093] In addition, an optical device capable of achieving improved energy efficiency and an electronic device including the optical device can be realized.

[0094] Various useful advantages and effects of the present invention are not limited to those mentioned above and can be better understood from the detailed description of the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 is a conceptual diagram illustrating an embodiment of an AI device.

[0096] Figure 2 is a block diagram illustrating a configuration of an extended reality electronic device according to an embodiment of the present invention.

[0097] Figure 3 is a perspective view showing an augmented reality electronic device according to a first embodiment of the present invention.

[0098] Figures 4 to 6 is a conceptual diagram cited to describe various display technologies applicable to a display unit according to an embodiment of the present invention.

[0099] Figure 7 is a conceptual diagram illustrating an electronic device and an optical device thereof according to an embodiment.

[0100] Figure 8 is a conceptual diagram showing an optical device according to an embodiment.

[0101] Fig. 9 is a perspective view showing an optical device according to an embodiment.

[0102] Fig.10 is an exploded perspective view showing an optical device according to an embodiment.

[0103] Fig.11 is along Fig. 9 A cross-sectional view taken along line AA'.

[0104] Fig.12 It is shown Fig.11 An enlarged view of part K1.

[0105] Fig.13 It is shown Fig.11 An enlarged view of part K2.

[0106] Fig.14 is a view showing another example of a light source device according to an embodiment.

[0107] Fig.15 is a flowchart illustrating an operating method of an electronic device or an optical device thereof according to an embodiment.

[0108] Fig.16 is a timing chart referenced to describe the respective operations of the light source, the optical signal generating unit, and the light receiving unit.

[0109] Fig.17 is a flowchart illustrating a display method according to an embodiment.

[0110] Fig.18 is an additional flow chart illustrating a display method according to an embodiment.

[0111] Fig.19 and Fig. 20 is a view referred to to describe vision determination in a display method according to an embodiment.

[0112] Fig.21 is a view showing an output of an image in a display method according to an embodiment.

[0113] Fig. 22 is a flowchart illustrating a display method according to another embodiment.

[0114] Fig.23 is a view cited to describe visual acuity determination in the display method of the same embodiment.

[0115] Fig.24 is a flowchart illustrating a display method according to still another embodiment.

[0116] Fig.25 is a perspective view showing an augmented reality electronic device as a modified example according to the embodiment.

[0117] Fig.26 is a block diagram showing a control device according to an embodiment of the present invention.

[0118] Fig. 27 : is a conceptual diagram showing a control device according to an embodiment of the present invention.

[0119] Fig.28is a flowchart illustrating a control method according to an embodiment of the present invention.

[0120] Fig.29 is a specific flowchart showing a control method according to an embodiment of the present invention.

[0121] Fig.30 are views referred to for describing the operation of the control device and the control method according to the embodiment of the present invention.

[0122] Fig.31 1 is a view referred to to describe the operation and control between the control device and the electronic device according to the embodiment of the present invention.

[0123] Fig.32 is a view showing an interface of an electronic device according to an embodiment of the present invention.

[0124] Fig.33 are views referred to for describing the operation of an electronic device using a control device according to an embodiment of the present invention.

[0125] Fig.34 2 is a view cited to describe a control device according to an embodiment of the present invention.

[0126] Fig.35 is a side view showing an electronic device according to an embodiment.

[0127] Fig.36 is a plan view showing an electronic device according to an embodiment.

[0128] Fig.37 is a conceptual diagram showing an optical device according to an embodiment.

[0129] Fig.38 is a perspective view showing an optical device according to an embodiment.

[0130] Fig.39 is an exploded perspective view showing an optical device according to an embodiment.

[0131] Fig.40 is along Fig.38 A view taken along line AA'.

[0132] Fig.41 It is shown Fig.40 An enlarged view of part K1.

[0133] Fig.42 It is shown Fig.40 An enlarged view of part K2.

[0134] Fig.43 is a side view showing an electronic device according to another embodiment.

[0135] Fig.44 is a plan view showing an electronic device according to the same embodiment.

[0136] Fig.45 is a side view showing an electronic device according to still another embodiment.

[0137] Fig.46 is a plan view showing an electronic device according to the same embodiment. DETAILED DESCRIPTION

[0138] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0139] However, the technical concept of the present invention is not limited to the one or several embodiments described, and can be implemented in various forms. One or more constituent elements of these embodiments can be used by selective coupling and replacement within the scope of the technical concept of the present invention.

[0140] In addition, unless otherwise clearly defined and used, the terms (including technical terms and scientific terms) used in the embodiments of the present invention may be interpreted as having a meaning that is generally understandable to a person of ordinary skill in the art to which the present invention belongs. Commonly used terms such as those defined in dictionaries may be interpreted as having a meaning defined by considering the contextual meaning in the relevant technology.

[0141] In addition, the terms used in the embodiments of the present invention are for describing them and are not intended to impose any limitations on the present invention.

[0142] Throughout the specification, singular nouns are intended to include their plural forms unless the context clearly dictates otherwise. Where the expression "at least one (or one or more) of A, B or C" is used, the expression may be interpreted as including one or more of a combination consisting of A, B and C individually or collectively.

[0143] In addition, in describing constituent elements according to embodiments of the present invention, the terms first, second, etc., A, B, etc., and (a), (b), etc. may be used.

[0144] These terms are used only to distinguish one constituent element from another constituent element, and do not impose limitations on the nature, order or sequence of the constituent elements.

[0145] When a component is described as being "connected to", "coupled to" or "able to access" another component, it may be directly connected to, directly coupled to or able to access the other component. In addition, it may be connected to, coupled to or able to directly access the other component with an intermediate component in between.

[0146] In addition, where one component element is described as being formed or arranged "on top" or "under" another component element, it may be in direct contact with the other component element. In addition, where one component element is described as being formed or arranged "above" or "under" another component element, it may be formed or arranged to have one or more intermediate component elements therebetween. In addition, where one component element is described as being on top (above) or under (under) another component element, the upward direction and the downward direction may be defined relative to the position of one of these components.

[0147] Figure 1 is a conceptual diagram illustrating an embodiment of an AI device.

[0148] Reference Figure 1 In the AI ​​system, at least one of the AI ​​server (16), the robot (11), the autonomous vehicle (12), the XR device (13), the smartphone (14), or the home appliance (15) is connected to the cloud network (10). In this case, all the robots (11), the autonomous vehicle (12), the XR device (13), the smartphone (14), the home appliance (15), and the like that apply AI technology are referred to as AI devices (11 to 15).

[0149] The cloud network (10) may constitute a part of a cloud computing infrastructure or refer to a network existing within a cloud computing infrastructure. In this case, the cloud network (10) may be configured using a 3G network, a 4G or Long Term Evolution (LTE) network, a 5G network or a similar network.

[0150] That is, the devices (11 to 16) constituting the AI ​​system can be connected to each other through the cloud network (10). In particular, the devices (11 to 16) can communicate with each other through the base station, and can also communicate with each other directly without the base station in between.

[0151] The AI ​​server (16) may include a server that performs AI processing and a server that performs calculations on big data.

[0152] The AI ​​server (16) can be connected to at least one of the AI ​​devices constituting the AI ​​system, including a robot (11), an autonomous driving vehicle (12), an XR device (13), a smartphone (14), and a home appliance (15) through a cloud network (10). The AI ​​server (16) can assist the AI ​​devices (11 to 15) to which the AI ​​server (16) is connected in performing at least a part of AI processing.

[0153] At this time, the AI ​​server (16) can train the artificial intelligence network according to the machine learning algorithm on behalf of the AI ​​devices (11 to 15), and directly store the learning model or send the learning model to the AI ​​devices (11 to 15).

[0154] The AI server (16) can receive input data from the AI devices (11 to 15), derive a result value of the received input data using a learning model, generate a response or control command based on the derived result value, and send the generated response or control command to the AI devices (11 to 15).

[0155] Alternatively, the AI devices (11 to 15) can directly use the learning model to derive a result value of the input data and also generate a response or control command based on the derived result value.

[0156] <AI + Robot>

[0157] The robot (11) can be implemented as a guiding robot, a handling robot, a cleaning robot, a wearable robot, an entertainment robot, a pet robot, an unmanned aerial robot, etc., each of which applies AI technology.

[0158] The robot (11) can include a robot control module for controlling its operation, and the robot control module can refer to a software module or a chip generated by implementing the software module in hardware.

[0159] Using sensor information obtained from various types of sensors, the robot (11) can obtain its status information, detect (recognize) its surrounding environment and objects, generate map data, determine its movement path and driving plan, determine a response to an interaction with a user, or determine its operation.

[0160] At this time, in order to determine its movement path and driving plan, the robot (11) can use sensor information obtained from at least one sensor among a lidar (LiDAR), a radar, or a camera device.

[0161] The robot (11) can use a learning model configured with at least one artificial intelligence network to perform the above operations. For example, the robot (11) can use the learning model to recognize its surrounding environment and objects, and use information about its recognized surrounding environment or object information to determine its operation. At this time, the learning model can be derived from direct learning in the robot (11) or from learning in an external device such as the AI server (16).

[0162] At this time, the robot (11) can directly use the learning model to generate a result and perform its operation. However, the robot (11) can also send sensor information to an external device such as the AI server (16), receive a result generated based on the sensor information, and perform its operation.

[0163] The robot (11) can determine its movement path and driving plan and control the driving unit using at least one of the following: map data, object information detected from sensor information, or object information obtained from an external device. Thus, the robot (11) can drive according to the determined movement path and driving plan.

[0164] The map data can include object recognition information about various objects arranged in the space where the robot (11) moves. For example, the map data can include object recognition information about fixed objects such as walls and doors and about movable objects such as flower pots and tables. Then, the object recognition information can include the name, type, distance, location, etc. of the object.

[0165] In addition, the robot (11) can control the driving unit under the control of the user or based on interaction with the user, and thus can perform its operations or drive. At this time, the robot (11) can obtain intention information of the interaction interpreted from the movement and voice utterances of the user, determine a response based on the obtained intention information, and perform its operations.

[0166] <AI + Autopilot>

[0167] The autonomous vehicle (12) can be implemented as a mobile robot, vehicle, unmanned aerial vehicle, etc. each applying AI technology.

[0168] The autonomous vehicle (12) can include an autonomous driving control module for controlling its autonomous driving function, and the autonomous driving control module can refer to a software module or a chip generated by implementing the software module in hardware. The autonomous driving control module can be included as a component within the autonomous vehicle (12). However, the autonomous driving control module can also be configured as a separate hardware component and externally connected to the autonomous vehicle (12).

[0169] Using sensor information obtained from various types of sensors, the autonomous vehicle (12) can obtain its status information, detect (identify) its surrounding environment and objects, generate map data, determine its movement path and driving plan, or determine its operations.

[0170] At this time, in order to determine its movement path and driving plan, the autonomous vehicle (12) can use sensor information obtained from at least one of the following, like the robot (11): lidar, radar, or a camera device.

[0171] In particular, the autonomous vehicle (12) can use sensor information from an external device to identify an area that is blocked from view, or the environment or objects in an area located at a predetermined distance away. Alternatively, the autonomous vehicle (12) can directly receive the identified information from the external device.

[0172] The autonomous vehicle (12) may perform the above operations using a learning model configured with at least one artificial intelligence network. For example, the autonomous vehicle (12) may use the learning model to identify its surrounding environment and objects, and use information about the environment or objects it identifies to determine its driving route. In this case, the learning model may be derived from direct learning in the autonomous vehicle (12), or from learning in an external device such as an AI server (16).

[0173] At this time, the autonomous driving vehicle (12) can directly use the learning model to generate results and perform its operations. However, the autonomous driving vehicle (12) can also send sensor information to an external device such as an AI server (16), receive results generated based on the sensor information, and perform its operations.

[0174] The autonomous driving vehicle (12) can determine its moving path and driving plan and control the driving unit using at least one of the following: map data, object information detected from sensor information, or object information acquired from an external device. Therefore, the autonomous driving vehicle (12) can be driven according to its determined moving path and driving plan.

[0175] The map data may include object recognition information about various objects arranged in a space (e.g., a road) in which the autonomous driving vehicle (12) moves. For example, the map data may include object recognition information about fixed objects such as street lights, rocks, and buildings, and about movable objects such as vehicles and pedestrians. The object recognition information may then include the name, type, distance, location, etc. of the object.

[0176] In addition, the autonomous driving vehicle (12) can control the driving unit under the control of the user or based on the interaction with the user, and thus can perform its operation or travel. At this time, the autonomous driving vehicle (12) can obtain the intention information of the interaction interpreted from the user's movement and voice utterance, determine a response based on the obtained intention information, and perform its operation.

[0177] <AI+XR>

[0178] The XR device (13) can be implemented as a head-mounted display (HMD), a head-up display (HUD) set in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a fixed robot, a mobile robot, etc., each of which applies AI technology.

[0179] The XR device (13) can generate position data and attribute data of 3D points by analyzing 3D point cloud data or image data obtained through various sensors or from external devices. Therefore, the XR device (13) can obtain information about the surrounding space or real objects, render the XR object to be output, and output the XR object. For example, the XR device (13) can output an XR object including additional information about the recognized object in a manner corresponding to the recognized object.

[0180] The XR device (13) can perform the above operations using a learning model configured with at least one artificial intelligence network. For example, the XR device (13) can use the learning model to recognize real objects from 3D point cloud data or image data and provide information corresponding to the recognized real objects. At this time, the learning model can be derived from direct learning in the XR device (13) or from learning in an external device such as the AI server (16).

[0181] At this time, the XR device (13) can directly use the learning model to generate results and perform its operations. However, the XR device (13) can also send sensor information to an external device such as the AI server (16), receive the results generated based on the sensor information, and perform its operations.

[0182] <AI + Robot + Autopilot>

[0183] The robot (11) can be implemented as a guiding robot, a handling robot, a cleaning robot, a wearable robot, an entertainment robot, a pet robot, an unmanned aerial vehicle, etc., each of which applies AI technology and autopilot technology.

[0184] The robot (11) applying AI technology and autopilot technology can refer to the robot itself equipped with an autopilot function, the robot (11) interacting with the autonomous vehicle (12), or robots of a similar type.

[0185] The robots (11) each equipped with an autopilot function can be collectively referred to as devices that move along a given route by themselves without user control or independently determine their routes and move along the determined routes.

[0186] The robots (11) and the autonomous vehicle (12) both equipped with an autopilot function can use a common sensing method to determine one or more of the following: a movement path or a driving plan. For example, the robots (11) and the autonomous vehicle (12) both equipped with an autopilot function can use the information sensed by LiDAR, radar, and camera devices to determine at least one of the following: a movement path or a driving plan.

[0187] A robot (11) that interacts with an autonomous vehicle (12) can exist separately from the autonomous vehicle (12) and can operate in conjunction with the autonomous driving function inside or outside the autonomous vehicle (12), or perform operations with a user riding in the autonomous vehicle (12).

[0188] At this time, the robot (11) that interacts with the autonomous vehicle (12) can obtain sensor information on behalf of the autonomous vehicle (12) and provide the obtained sensor information to the autonomous vehicle (12). Alternatively, the robot (11) can obtain sensor information, generate information on the surrounding environment or object information, and provide the generated information to the autonomous vehicle (12). Therefore, the robot (11) can control or assist the autonomous vehicle (12) in performing its autonomous driving function.

[0189] Alternatively, the robot (11) that interacts with the autonomous vehicle (12) can monitor a user riding in the autonomous vehicle (12) or control the functions of the autonomous vehicle (12) through interaction with the user. For example, in the case of determining that the driver is drowsy, the robot (11) can activate the autonomous driving function of the autonomous vehicle (12) or assist the autonomous vehicle (12) in controlling the drive unit. At this time, the functions of the autonomous vehicle (12) controlled by the robot (11) can include not only the autonomous driving function but also the functions provided in the navigation system or audio system provided in the autonomous vehicle (12).

[0190] Alternatively, the robot (11) that interacts with the autonomous vehicle (12) can provide information to the autonomous vehicle (12) from outside the autonomous vehicle (12) or assist the autonomous vehicle (12) in performing its functions. For example, the robot (11) such as an intelligent traffic signal can also provide traffic information including signal information or the like to the autonomous vehicle (12). In addition, the robot (11) such as an automatic charging station for an electric vehicle can automatically connect the charger to the charging inlet by interacting with the autonomous vehicle (12).

[0191] <AI + Robot + XR>

[0192] The robot (11) can be implemented as a guiding robot, a handling robot, a cleaning robot, a wearable robot, an entertainment robot, a pet robot, an unmanned flying robot, a drone, etc., each of which applies AI technology and XR technology.

[0193] The robot (11) applying XR technology can refer to a robot that is controlled or interacted with within the XR image. In this case, the robot (11) is distinguished from the XR device (13) and can operate in conjunction with the XR device (13).

[0194] When a robot (11) that undergoes control or interaction within an XR image obtains sensor information from a sensor including a camera device, the robot (11) or the XR device (13) may generate an XR image based on the sensor information, and the XR device (13) may output the generated XR image. Then, the robot (11) may operate based on a control signal input through the XR device (13) or interaction with the user.

[0195] For example, a user may remotely check an XR image corresponding to a time point for a robot (11) that is combined with an external device such as the XR device (13). In addition, through interaction, the user may adjust the autonomous driving path of the robot (11), control the operation or driving of the robot (11), or check information about nearby objects.

[0196] <AI + Autopilot + XR>

[0197] An autonomous driving vehicle (12) may be implemented as a mobile robot, a vehicle, an unmanned aerial vehicle, etc., each of which applies AI technology and XR technology.

[0198] An autonomous driving vehicle (12) that applies XR technology may refer to an autonomous driving vehicle equipped with means for providing an XR image, an autonomous driving vehicle that undergoes control or interaction within an XR image, or a vehicle of a similar type. In particular, an autonomous driving vehicle (12) that undergoes control or interaction within an XR image is different from the XR device (13) and may operate in combination with the XR device (13).

[0199] An autonomous driving vehicle (12) equipped with means for providing an XR image may acquire sensor information from a sensor including a camera device and output an XR image generated based on the acquired sensor information. For example, the autonomous driving vehicle (12) may include a HUD and output an XR image. Therefore, the autonomous driving vehicle (12) may provide an XR object corresponding to a real object or an object within a screen to an occupant.

[0200] At this time, in the case of outputting an XR object on the HUD, at least a part of the XR object may be output in a manner that overlaps with a real object on which the occupant keeps his / her gaze. In contrast, in the case of outputting an XR object on a display provided within the autonomous driving vehicle (12), at least a part of the XR object may be output in a manner that overlaps with an object within the screen. For example, the autonomous driving vehicle (12) may output XR objects corresponding to objects such as a vehicle road, other vehicles, traffic signals, traffic signs, two-wheeled vehicles, pedestrians, and buildings, respectively.

[0201] When the autonomous driving vehicle (12) undergoing control or interaction within the XR image obtains sensor information from a sensor including a camera device, the autonomous driving vehicle (12) or the XR device (13) can generate an XR image based on the sensor information, and the XR device (13) can output the generated XR image. Then, the autonomous driving vehicle (12) can operate based on a control signal input through an external device such as the XR device (13) or an interaction with a user.

[0202] [Extended Reality Technology]

[0203] Extended reality (XR) is a general term for virtual reality (VR), augmented reality (AR), and mixed reality (MR). Virtual technology only provides objects or backgrounds in the real world as CG images. AR technology additionally provides virtually generated CG images on the actual object images. MR technology is a computer graphics technology that presents virtual objects in a way that is seamlessly integrated into the real world.

[0204] MR technology presents real objects and virtual objects together. In this respect, MR technology is similar to AR technology. However, AR technology is used in the form of supplementing real objects with virtual objects. In contrast, MR technology is used with the concept of virtual objects and real objects coexisting and interacting equally. In this respect, AR technology is different from MR technology.

[0205] XR technology can be applied to head-mounted displays (HMDs), heads-up displays (HUDs), mobile phones, tablet PCs, laptop computers, desktop computers, TVs, digital signage, etc. Devices applying XR technology may be referred to as XR devices.

[0206] The following describes an electronic device for providing extended reality according to an embodiment of the present invention, and in particular, describes in detail an optical device applied in augmented reality and an electronic device including the optical device.

[0207] Figure 2 is a block diagram showing the configuration of an extended reality electronic device ( 20 ) according to an embodiment of the present invention.

[0208] Reference Figure 2 The extended reality electronic device (20) may include a wireless communication unit (21), an input unit (22), a sensing unit (23), an output unit (24), an interface unit (25), a memory (26), a control unit (27), a power supply unit (28) and other similar units. Figure 2 The constituent elements shown in are not essential for realizing the electronic device (20). The electronic device (20) described in this specification may have more or fewer constituent elements than those listed above.

[0209] More specifically, among the constituent elements listed above, the wireless communication unit (21) may include one or more modules for implementing wireless communication between the electronic device (20) and a wireless communication system, between the electronic device (20) and another electronic device, or between the electronic device (20) and an external server. In addition, the wireless communication unit (21) may include one or more modules for connecting the electronic device (20) to one or more networks.

[0210] The wireless communication unit (21) may include at least one of the following: a broadcast receiving module, a mobile communication module, a wireless Internet module, a short-range communication module or a location information module.

[0211] The input unit (22) may include a camera or an image input unit for inputting an image signal, a microphone or an audio input unit for inputting an audio signal, and a user input unit (e.g., a touch key, a key (mechanical key), etc.) for receiving information from a user as input. Voice data or image data collected by the input unit (22) may be analyzed and processed into a user's control command.

[0212] The sensing unit (23) may include at least one sensor to sense at least one of the following: information within the electronic device (20), information about the surrounding environment of the electronic device (20), or user information.

[0213] For example, the sensing unit (23) may include at least one of the following: a proximity sensor, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, a G sensor, a gyro sensor, a motion sensor, an RGB sensor, an infrared (IR) sensor, a finger scan sensor, an ultrasonic sensor, an optical sensor (e.g., an image capture device), a microphone, a battery gauge, an environmental sensor (e.g., a barometer, a hygrometer, a thermometer, a radiation detection sensor, a thermal sensor, a gas sensor, etc.), or a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). The electronic device (20) disclosed in the present specification may utilize information fragments sensed in at least two of these sensors in combination.

[0214] The output unit (24) is used to generate an output associated with vision, hearing or touch, and may include at least one of the following: a display unit, a sound output unit, a tactile module or a light output unit. The display unit is configured to be structurally layered with the touch sensor or formed integrally with the touch sensor, thereby realizing a touch screen. The touch screen can be used as a user input device, which provides an input interface between the extended reality electronic device (20) and the user, and can also provide an output interface between the extended reality electronic device (20) and the user.

[0215] The interface unit (25) serves as a path to various external devices connected to the electronic device (20). Virtual reality or augmented reality content can be provided from the external device to the electronic device (20) through the interface unit (25), and the electronic device (20) can perform mutual interaction by exchanging various input signals, sensing signals and data with the external device.

[0216] For example, the interface unit (25) may include at least one of the following: a wired / wireless head mounted device port, an external charger port, a wired or wireless data port, a memory card port, a port for connecting to a device including an identification module, an audio input / output (I / O) port, a video input / output (I / O) port, or a headphone port.

[0217] In addition, data for supporting various functions of the electronic device (20) is stored in the memory (26). Examples of the memory (26) may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and other storage devices. A plurality of applications (or programs) executed on the electronic device (20), data for operating the electronic device (20), and commands may be stored in the memory (26). At least one or more of these applications may be downloaded from an external server via wireless communication. At least one or more of these applications may be pre-installed on the electronic device (20) before being shipped from a factory to perform basic functions of the electronic device (20) (e.g., receiving a call, making a call, receiving a message, and sending a message).

[0218] The control unit (27) generally controls the overall operation of the electronic device (20) in addition to the operation associated with the application program. The control unit (27) can process signals, data, information, etc. input or output through the above-mentioned constituent elements.

[0219] In addition, the control unit (27) can control at least one or more of the constituent elements by executing an application program stored in the memory (26), thereby providing or executing information or functions suitable for the user, respectively. In addition, the control unit (27) can operate at least two of the constituent elements included in the electronic device (20) in combination to execute the application program. In addition, the control unit (27) can perform the processing of the driving method described below. The control unit (27) can be represented as a processor, a control unit, etc. In addition, the control unit (27) can include an application-specific integrated circuit (ASIC), a different chipset, a logic circuit and / or a data processing device.

[0220] In addition, the control unit (27) may use a gyro sensor, a gravity sensor, a motion sensor, and other sensors included in the sensing unit (23) to detect the movement of the electronic device (20) or the movement of the user. Alternatively, the control unit (27) may also use a proximity sensor, an illumination sensor, a magnetic sensor, an infrared (IR) sensor, an ultrasonic sensor, an optical sensor, and other sensors included in the sensing unit (23) to detect an object approaching the vicinity of the electronic device (20) or the user. In addition, the control unit (27) may also detect the movement of the user by means of a sensor provided in a controller operating in conjunction with the electronic device (20).

[0221] In addition, the control unit (27) can use the application program stored in the memory (26) to perform the operation (or function) of the electronic device (20).

[0222] Under the control of the control unit (27), power is supplied to the power supply unit (28) from an external or internal power source, and the power supply unit (28) supplies power to the constituent elements included in the electronic device (20). The power supply unit (28) may include a battery, and the battery may be provided in a built-in or replaceable form.

[0223] At least one or more of the constituent elements can operate in coordination to perform the operation or control of the electronic device according to various embodiments or to perform the method of controlling the electronic device. In addition, the operation or control of the electronic device or the method of controlling the electronic device can be implemented on the electronic device by executing at least one application stored in the memory (26).

[0224] The following describes an electronic device by taking an embodiment of the present invention applied to a head mounted display (HMD) as an example. However, embodiments of the electronic device (20) according to the present invention may include a mobile phone, a smart phone, a laptop computer, a digital broadcast dedicated terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC, a tablet PC, an ultrabook computer, a wearable device, etc. In addition to the HMD, the wearable device may also include a watch-type terminal (smart watch), contact lenses, VR / AR / MR glasses, etc.

[0225] Figure 3 is a perspective view showing an augmented reality electronic device according to an embodiment of the present invention.

[0226] like Figure 3 As shown, the electronic device according to the embodiment of the present invention may include a frame (100), an optical device (200), and a display unit (300).

[0227] The electronic device may be provided in the form of smart glasses. The electronic device (20) in the form of smart glasses may be configured to be wearable on a person's head and may include a frame (housing, case, etc.) (100) for this purpose. The frame (100) may be formed of a flexible material in an easy-to-wear manner.

[0228] The frame (100) is supported on the head of a person and provides a space for mounting various types of components. As shown in the figure, electronic components such as an optical device (200), a user input unit (130) or a sound output unit (140) can be mounted on the frame (100). In addition, a lens positioned in front of at least one of the left eye and the right eye of the user can be removably attached to the frame (100).

[0229] As shown, the frame (100) may be in the form of glasses worn on the user's face, but need not be limited thereto. The frame (100) may also be in the form of goggles or a similar configuration worn closely on the user's face.

[0230] The frame (100) may include a front frame (110) and a pair of side frames (120), wherein the front frame (110) includes at least one opening portion, and the pair of side frames (120) are arranged in a manner similar to that of a front frame (110). Figure 3 The two frames extend parallel to each other in the y direction and intersect with the front frame (110).

[0231] The frame (100) may have a length in the x-direction (DI) and a length in the y-direction (LI) that are the same as or different from each other.

[0232] The optical device (200) is configured to control various types of electronic components provided in an electronic device.

[0233] The optical device (200) can generate an image or image sequence visible to a user. The optical device (200) can include an image source panel, a plurality of lenses, etc. The image source panel generates an image. The plurality of lenses diffuse or converge the light generated from the image source panel.

[0234] The optical device (200) may be fixed to any one of the pair of lateral frames (120). For example, the optical device (200) may be fixed to the inside or outside of any lateral frame (120), or may be integrally formed in a manner of being built into the lateral frame (120). Alternatively, the optical device (200) may be fixed to the front frame (110), or may be provided separately from the electronic device.

[0235] The display unit (300) can be implemented in the form of a head-mounted display (HMD). The form of the HMD involves a display technology that makes an image directly visible in front of the user's eyes. The display unit (300) can be arranged in a manner corresponding to at least one of the user's left eye and right eye, so that when the user wears the electronic device (20), the image is provided directly in front of the user's eyes. This figure shows that the display unit (300) is positioned in a portion corresponding to the user's right eye, so that the image is output toward the user's right eye. However, the display unit (300) is not limited to the portion described above. The display unit (300) can also be arranged in portions corresponding to the user's right eye and left eye, respectively.

[0236] The display unit (300) may be configured to allow a user to visually recognize the surrounding environment, and at the same time, the image generated by the optical device (200) is visible to the user. For example, the display unit (300) may project an image onto a display area using a prism.

[0237] In addition, the display unit (300) may be formed of a transparent material so that the projected image and the natural field of view in the forward direction (the range of the user's visual perception) are simultaneously visible. For example, the display unit (300) may be translucent and formed of an optical member including glass.

[0238] Then, the display unit (300) can be fixed to the front frame (110) by being inserted into the opening portion included in the front frame (110). Alternatively, the display unit (300) can be fixed to the front frame (110) by being positioned on the rear surface of the opening portion (i.e., between the opening portion and the user). As an example, the accompanying drawings show a case where the display unit (300) is positioned on the rear surface of the opening portion and thus fixed to the front frame (110). In addition, the display unit (300) can be arranged and fixed at various positions on the frame (100).

[0239] like Figure 3 As shown, in the electronic device, when the optical device (200) guides the image light of the image toward one side of the display unit (300), the image light is emitted from the other side of the display unit (300). Therefore, the image generated by the optical device (200) can be visible to the user.

[0240] Therefore, the user can view the external environment through the opening in the frame (100) and simultaneously view the image generated by the optical device (200). That is, the image output by the display unit (300) can be viewed in a manner overlapping with the natural field of view. The electronic device can provide augmented reality (AR) that uses these display characteristics to superimpose a virtual image on a real image or background and display the superimposed result as one image.

[0241] In addition to performing the driving operation, the electronic device can also provide the user with an external environment and an image generated by the optical device (200) with a time lag of a short period of time that is imperceptible to humans. For example, the external environment is provided to the user in one section of a frame, and in another section, the image from the optical device (200) can be provided to the user.

[0242] Alternatively, both overlap and time lag may be provided.

[0243] Figures 4 to 6 is a conceptual diagram cited to describe various display technologies applicable to a display unit according to an embodiment of the present invention.

[0244] Specifically, Figure 4 is a view cited to describe an embodiment of an optical member employing prism technology. Figure 5 are views cited to describe an embodiment of an optical member employing a waveguide technology. Figure 6 is a view cited to describe an embodiment of an optical member employing a surface reflection technology.

[0245] like Figure 4 As shown, an optical member using prism technology can be used in the display unit (300-1) according to the embodiment of the present invention.

[0246] As an embodiment of an optical component using prism technology, Figure 4 As shown in (a), a flat glass optical member having a flat surface on which image light is incident and a flat surface (300a) from which image light is emitted may be used. Figure 4 As shown in (b), a free-form glass optical member may be used, which has a surface (300b) from which image light is emitted, wherein the surface (300b) is formed as a curved surface that does not have a constant radius of curvature.

[0247] The image light generated by the optical device (200) can be incident on the flat side of the flat glass optical component, reflected from the total reflection mirror (300a) provided in the flat glass optical component, and emitted toward the user. At this time, the total reflection mirror (300a) provided in the flat glass optical component can be formed in the flat glass optical component using a laser.

[0248] The free-form glass optical component is configured in such a way that its thickness decreases as the distance from the surface on which light is incident decreases. Therefore, image light generated by the optical device (200) can be incident on the curved side of the free-form glass optical component, totally reflected internally, and emitted toward the user.

[0249] like Figure 5As shown, an optical member or a light guide optical element (LOE) using a waveguide technology may be used in a display unit (300-2) according to another embodiment of the present invention.

[0250] As an embodiment of an optical member using waveguide or lightguide technology, it is possible to provide Figure 5 (a) The glass optical component using the segmented beam splitter technology, such as Figure 5 (b) A glass optical component using sawtooth prism technology, such as Figure 5 (c) A glass optical component using a diffractive optical element (DOE), such as Figure 5 (d) A glass optical component having a holographic optical element (HOE), such as Figure 5 (e) A glass optical component with a passive grating as shown in Figure 5 (f) Glass optical component with active grating shown.

[0251] As shown in the figure, Figure 5 The glass optical component using the segmented beam splitter technology shown in (a) may include a total reflection mirror (301a) on the side where the optical image is incident and a segmented beam splitter (301b) on the side where the optical image is emitted in the glass optical component.

[0252] Therefore, the optical image generated by the optical device (200) can be totally reflected from the total reflection mirror (301a) in the glass optical member. Then, the totally reflected optical image can be partially separated and emitted by the segmented beam splitter (301b) while being light-guided along the length direction of the glass, so as to be visually recognized by the user.

[0253] In such Figure 5 In the glass optical component using the sawtooth prism technology shown in (b), image light from the optical device (200) can be incident on the side of the glass in an inclined direction, totally reflected inside the glass, and emitted from the glass through a sawtooth-shaped protrusion or groove portion 302 arranged on one side of the outgoing optical image, thereby being visually recognized by the user.

[0254] like Figure 5 The glass optical member having a diffractive optical element (DOE) shown in (c) may include a first diffractive portion (303a) on the surface on the side where the optical image is incident and a second diffractive portion (303b) on the surface on the side where the optical image is emitted. The first diffractive portion (303a) and the second diffractive portion (303b) may be provided in a form that allows a specific pattern to be formed on the glass surface or allows a separate diffractive film to be attached to the glass surface.

[0255] Therefore, the optical image generated by the optical device (200) can be diffracted when incident through the first diffraction part (303a), and guided along the length direction of the glass while being totally reflected. Then, the optical image is emitted through the second diffraction part (303b), so as to be visually recognized by the user.

[0256] like Figure 5 The glass optical component with a holographic optical element (HOE) shown in (d) may include an output coupler (304). The output coupler (304) is positioned on one side of the optical image emitted from the glass. Therefore, the optical image can enter from the optical device (200) through the side of the glass in an oblique direction, be totally reflected, be light-guided along the length direction of the glass, and be output by the output coupler (304), so as to be visually recognized by the user. The structure of the holographic optical element can be gradually changed, and thus can be subdivided into a structure with a passive grating and a structure with an active grating.

[0257] like Figure 5 The glass optical component with a passive grating shown in (e) may include an incoupler (305a) on a surface opposite to the glass surface on the side where the optical image is incident and an outcoupler (305b) on a surface opposite to the glass surface on the side where the optical image is emitted. In this case, the incoupler (305a) and the outcoupler (305b) may be provided in the form of a film with a passive grating.

[0258] Therefore, the optical image on the glass surface on the side where the optical image incident on the glass is incident can be totally reflected by the coupler (305a) positioned on the opposite surface, guided along the length direction of the glass, and emitted by the coupler (305b) through the opposite surface of the glass, thereby being visually recognized by the user.

[0259] like Figure 5 The glass optical component with active grating shown in (f) may include an input coupler (306a) and an output coupler (306b). The input coupler (306a) is formed in the form of an active grating on the side of the glass where the optical image is incident. The output coupler (306b) is formed in the form of an active grating on the side of the glass where the optical image is emitted.

[0260] Therefore, the optical image incident on the glass can be totally reflected by the coupler (306a), guided along the length direction of the glass, and emitted from the glass by the coupler (306b), so as to be visually recognized by the user.

[0261] As the display unit according to the modified example, an optical member adopting the pin mirror technology may be used.

[0262] In addition, in order to use as a coupler, such as Figure 6As shown in (a), a free-form combiner glass having a plurality of flat surfaces can be used as an optical member using a surface reflection technology belonging to the free-form combiner technology, wherein the plurality of flat surfaces respectively have different incident angles for an optical image and are formed as one glass. The optical image can be incident on the free-form combiner glass (300) in a manner that the incident angle varies on a per-area basis, thereby being emitted to a user.

[0263] like Figure 6 As shown in (b), the optical component using the surface reflection technology using the flat HOE technology may include a holographic optical element (HOE) (311) which is coated or patterned on the flat surface of the glass. The optical image incident on the optical device (200) may pass through the holographic optical element (311), be reflected from the surface of the glass and pass through the holographic optical element (311) again, so as to be output toward the user.

[0264] like Figure 6 As shown in (c), the optical component using the surface reflection technology belonging to the free-form HOE technology may include a holographic optical element (HOE) (313) which is coated or patterned on the surface of the glass in a free form. The operating principle and reference behind the optical component Figure 6 (b) Same as described.

[0265] Figure 7 is a conceptual diagram illustrating an electronic device and an optical device thereof according to an embodiment. Figure 8 is a conceptual diagram showing an optical device according to an embodiment. Fig. 9 is a perspective view showing an optical device according to an embodiment. Fig.10 is an exploded perspective view showing an optical device according to an embodiment. Fig.11 is along Fig. 9 A cross-sectional view taken along line AA'. Fig.12 It is shown Fig.11 An enlarged view of part K1. Fig.13 It is shown Fig.11 An enlarged view of part K2.

[0266] First, refer to Figure 7 As described above, the electronic device according to the embodiment may include a frame (100), an optical device (200), and a display unit (300).

[0267] The optical device (200) may be arranged on the frame (100). The optical device (200) may be connected to the display unit (300), and thus may provide a video or image to the display unit (300). The display unit (300) may be positioned on a transparent lens (e.g., a glass lens) connected to the frame (100).

[0268] In addition, as described below, the optical device (200) may include a lens barrel (210), a light source device (220) arranged in the lens barrel and including a light source, an optical signal generating unit (230) for converting an optical signal into a signal for an image signal, and a detection unit (250). The details provided below can be applied to the description of these constituent elements in the same manner.

[0269] In particular, the detection unit (250) in the optical device (200) according to the embodiment may include a light receiving unit (251). In addition, the detection unit (250) may include a barrier unit (252) adjacent to the light receiving unit (251) and a substrate unit (253) connected to the light receiving unit. At this time, the lens barrel (210) in the optical device (200) may include a through hole (210rs) formed on the side.

[0270] At this time, the detection unit (250) can be arranged adjacent to the through hole (210rs). Therefore, the light passing through the multiple lenses in the lens barrel (210) can be emitted to the outside through the through hole (210rs). For example, in the case where the user (or wearer) wears the electronic device, the light emitted through the through hole (210rs) can be emitted to the skin of the wearer. The emitted light (IL) can pass through the tissue of the wearer and propagate into the blood vessel. At this time, the amount of light absorbed varies according to the amount of blood (e.g., red blood cells) flowing through the blood vessel (e.g., artery). For example, the amount of blood passing through the blood vessel alternately increases or decreases according to the cycle or frequency of the heart beat (or pulse). For example, when the amount of blood is large, most of the light (e.g., green wavelength light) propagated into the skin is absorbed by the red blood cells. Therefore, the amount of light reflected from the skin can be relatively reduced. Therefore, the light received by the light receiving unit (251) can be relatively reduced. In addition, when the amount of blood is small, the light propagated into the skin is absorbed by the red blood cells. Therefore, the amount of light reflected from the skin can be relatively increased. Therefore, the light received by the light receiving unit can be relatively increased. In other words, the amount of light received by the light receiving unit can be less when the amount of blood is large compared to when the amount of blood is small. The light receiving unit (251) can convert the received light into an electrical signal, and the electrical signal generated from the conversion can be sent to the control unit of the optical device or the control unit of the electronic device. Therefore, the pulse of the wearer can be detected by the detection unit (250). Therefore, the electronic device or the optical device can use the detection unit (250) to detect whether the user is wearing the electronic device.

[0271] The barrier unit (252) may be arranged adjacent to the light receiving unit (251), and thus may prevent the light passing through the through hole (210rs) from directly propagating to the light receiving unit (251). Therefore, the optical device or electronic device according to the embodiment may perform sensing in a more accurate manner. For example, the optical device or electronic device may accurately detect a pulse or an oxygen saturation level.

[0272] The substrate unit (253) may support the barrier unit (252) and the light receiving unit (251). In addition, the substrate unit (253) may be electrically connected to the light receiving unit (251) and may be electrically connected to a control unit of an optical device or an electronic device.

[0273] Further references Figures 8 to 11 According to an embodiment, an optical device (200) includes a lens barrel (210), a lens (L), and a first light guide (LG1). In addition, the optical device (200) may include a light source device (220) positioned in an opening portion (OP) formed in a side of the lens barrel (210) or coupled to the opening portion (OP), and an optical signal generating unit (230) adjacent to the lens barrel (210). In addition, the optical device (200) may further include a cover (CV) covering the lens barrel (210), the light source device (220), and the optical signal generating unit (230), and a substrate (not shown) including a connector.

[0274] The lens (L) may include a plurality of lenses. For example, the lens (L) may include a plurality of lenses arranged sequentially from the top of the lens barrel (210). For example, the lens (L) may include a first lens (L1) arranged at the uppermost position, a second lens (L2) arranged below the first lens (L1), and so on, until the Nth lens (Ln) arranged at the lowermost position. At this time, N may be a natural number greater than or equal to 2. Among the plurality of lenses (L), the Nth lens (Ln) may be positioned closest to the optical signal generating unit (230), which is positioned below the lower end of the lens barrel (210) or below the lowermost lens among the plurality of lenses (L).

[0275] In addition, in an embodiment of the present invention, the first direction (X-axis direction) may correspond to the optical axis. In addition, the first direction (X-axis direction) may correspond to the direction in which light emitted from the light source device (220) and reflected from the optical signal generating unit (230) is emitted to the above-mentioned display unit. The second direction (Y-axis direction) is a direction perpendicular to the first direction (Y-axis direction). In addition, the second direction (Y-axis direction) may correspond to the direction from the first light guide (LG1) toward the opening portion (OP). In addition, hereinafter in this specification, or in this embodiment, the second direction (Y-axis direction) may correspond to the direction from the first light guide (LG1) toward the second light guide (LG2). The first light guide (LG1) may be referred to as a first light guide unit or a first guide member. In addition, the second light guide (LG2) may be referred to as a second light guide unit or a second guide member.

[0276] The lens barrel (210) may also have a hole corresponding to the opening (OP). As an embodiment, the lens barrel (210) may have a lens barrel hole or an additional hole (210h). The additional hole (210h) may be positioned in a manner facing the opening (OP). Alternatively, the additional hole (210h) may overlap with the opening (OP) in the second direction (Y-axis direction). Alternatively, the distance from the Nth lens (Ln) to the additional hole (210h) in the first direction may be the same as the distance between the opening (OP) and the Nth lens (Ln). Alternatively, the additional hole (210h) may be positioned in an area corresponding to the position of the opening (OP) on the inner surface of the lens barrel (210). With this configuration, a joining member may be easily applied to the light source device (220), which is coupled to, inserted into, fixed to or connected to the opening (OP) through the additional hole (210h). Therefore, the bonding force between the lens barrel (210) and the light source device (220) can be improved. Therefore, the optical device (200) according to the embodiment can have improved durability, persistence or reliability. Alternatively, it is also possible to easily perform an optical test on the light source device (220) positioned in the opening (OP) through the additional hole (210h). Alternatively, the lens barrel (210) and the light source device (200) can be easily discharged or distributed through the additional hole (210h) through the additional hole (210h). The presence or absence of the additional hole (210h) can be applied in a different manner from one embodiment to another. For example, as described above, the additional hole (210h) can be arranged on the side of the lens barrel (210). Alternatively, considering durability or similar factors, the additional hole (210h) may not exist on the side of the lens barrel (210).

[0277] A plurality of lenses (L) may be positioned within the lens barrel (210). In addition, a first light guide (LG1) may be positioned within the lens barrel (210).

[0278] The lens barrel (210) according to the embodiment may include an opening portion (OP) formed in a side thereof. The opening portion (OP) may have various shapes, such as a circular shape and a polygonal shape. The opening portion (OP) may correspond to the position of the first light guide (LG1).

[0279] As an embodiment, the opening portion (OP) may overlap the first light guide (LG1) in a direction perpendicular to the optical axis. With this configuration, light emitted from the light source device (220) positioned on the side of the lens barrel (210) may be easily incident on the first light guide (LG1).

[0280] The first light guide (LG1) may be arranged between two lenses of the plurality of lenses. For example, the first light guide (LG1) may be arranged between the first lens (L1) and the N-th lens (Ln). In addition, the first light guide (LG1) may be arranged between the first lens (L1) and the optical signal generating unit (230). In addition, the first light guide (LG1) may be arranged between the first lens (L1) and the second lens (L2), or between the second lens (L2) and the N-th lens (Ln). In various embodiments, these different positions of the first light guide (LG1) are described below.

[0281] In the present embodiment, the first light guide (LG1) may be positioned between the second lens (L2) and the N-th lens (Ln). Therefore, the N-th lens (Ln) may be arranged between the first light guide (LG1) and the optical signal generating unit (230). With this configuration, light reflected from the first light guide (LG1) may be provided to the optical signal generating unit (230), thereby ensuring a suitable optical path. In addition, refraction of reflected light may be achieved. Therefore, minimization of the first light guide (LG1) may be achieved.

[0282] The first light guide (LG1) may include a first prism. The first prism may be a polarizing prism. Alternatively, the first prism may be a polarization separating prism.

[0283] The first prism may reflect the first polarized light and transmit the second polarized light. For example, a portion of the light (first polarized light) provided from the light source device (220) or incident on the first light guide (LG1) may be reflected from the first light guide (LG1) and provided to the optical signal generating unit (230). Another portion of the light (second polarized light) incident on the first light guide (LG1) may pass through the first light guide (LG1) and be absorbed into the lens barrel (210).

[0284] Light emitted from the light source device (220) and passing through the opening portion (OP) may be incident on the first light guide (LG1). To this end, as described above, the opening portion (OP) may be arranged in the region where the first light guide (LG1) is positioned. For example, the incident surface of the first light guide (LG1) may be positioned in a manner facing the opening portion (OP). Alternatively, the position of the first light guide (LG1) in the lens barrel (210) may be the same as the position of the first light guide (LG1).

[0285] The light source device (220) may include a light source (223) and thus generate (produce or provide) light. The light source device (220) according to an embodiment may be positioned in or coupled to the opening (OP). That is, the light source device (220) may be connected or coupled to the lens barrel (210).

[0286] The light source device (220) may include a housing (222) having an opening (222h), a second light guide (LG2) arranged in the housing (2220), and a light source (223) providing light to the second light guide (LG2).

[0287] In addition, the light source device (220) may include a light source assembly (221) arranged on the outside of the light source device (220) and covering the shell (222), a light source lens (224) adjacent to the light source (223), and a dielectric lens (MO) positioned within the shell (222).

[0288] The light source assembly (221) may be arranged at the outermost edge of the light source device (220). In the case where the light source (223) is difficult to install in the housing (222) or an additional lens (light source lens) needs to be installed, the light source assembly (221) may be positioned outside the housing (222). The light source assembly (221) may have a structure that is integrally formed with the housing (222) or is separated from the housing (222).

[0289] The housing (222) may have an opening (222h). The housing (222) may be positioned adjacent to the opening (OP) in the lens barrel (210). For example, the opening (222h) in the housing (222) may be positioned in a manner corresponding to the opening (OP) in the lens barrel (210). Therefore, the opening (222h) in the housing (222) may overlap with the opening (OP) in the lens barrel (210) in the second direction (Y-axis direction).

[0290] The light source (223) may be positioned in the housing (222) or the light source assembly (221). The light source (223) may emit light. For example, the light from the light source (223) may be incident on the second light guide (LG2) in the housing (222). The second light guide (LG2) may be positioned in the housing (222). Therefore, the second light guide (LG2) may transmit the light emitted from the light source (223) to the opening (222h) or the first light guide (LG1).

[0291] The light source (223) may include one or more light sources. That is, in the light source device (220), the light source (223) may refer to a single light source, and may include a plurality of light sources. The light source (223) may include a plurality of light sources, such as a first light source (223a), a second light source (223b), and a third light source (223c). The first light source (223a) to the third light source (223c) may emit light in the same direction, or may emit light in different directions. For example, the first light source (223a) and the third light source (223c) may be positioned facing each other. The first light source (223a) and the third light source (223c) may be positioned overlapping each other in a first direction (X-axis direction). The second light guide (LG2) may be positioned between the first light source (223a) and the third light source (223c). Therefore, the second light guide (LG2) may overlap with the first light source (223a) and the third light source (223c). The second light source (223b) may be positioned between the first light source (223a) and the third light source (223c). The first light source (223a) to the third light source (223c) may emit light toward the second light guide (LG2). The second light source (223b) may overlap with the second light guide (LG2) in the second direction. With this configuration, the optical device (200) may have a compact light source device (220).

[0292] In addition, the first light source (223a), the second light source (223b) and the third light source (223c) can emit light of the same wavelength or color, or emit light of different wavelengths or different colors. For example, the first light source (223a), the second light source (223b) and the third light source (223c) can emit red light, green light and blue light respectively.

[0293] The second light guide (LG2) may include a second prism. The second prism serves as a reflective member and may include, for example, an X-prism. As an embodiment, the second light guide (LG2) or the second prism may have a structure in which at least two prisms are coupled to each other.

[0294] The second prism may have at least two coated surfaces (reflective member or reflective sheet). One of the at least two coated surfaces may reflect light of a first wavelength and light of a second wavelength and transmit light of a third wavelength. That is, the coated surface may reflect light in a predetermined wavelength band. Therefore, light in a desired wavelength band emitted from each of the plurality of light sources (223) may be reflected from the second light guide (LG2). For example, the light passing through the second light guide (LG2) may be provided to the first light guide (LG1) or the dielectric lens (MO).

[0295] The light source lens (224) may be positioned adjacent to the light source (223). For example, the light source lens (224) may include a plurality of light source lenses. The light source lens (224) may be positioned on a path of light emitted from each light source (223).

[0296] As an embodiment, the light source lens (224) may be positioned between the second light guide (LG2) and the light source (223). In addition, the light source lens (224) may include a plurality of light source lenses. The plurality of light source lenses (224) may be positioned between the second light guide (LG2) and the light source (223). Alternatively, the light source lenses (224) may be arranged in a manner corresponding to the plurality of light sources (223), respectively.

[0297] For example, the light source lens (224) may include a first light source lens (224a), a second light source lens (224b), and a third light source lens (224c). The first light source lens (224a) may be positioned between the first light source (223a) and the second light guide (LG2). The second light source lens (224b) may be positioned between the second light source (223b) and the second light guide (LG2). The third light source lens (224c) may be positioned between the third light source (223c) and the second light guide (LG2).

[0298] In addition, a plurality of light source lenses, such as a first light source lens (224a), a second light source lens (224b), and a third light source lens (224c), can be respectively located above a plurality of light sources, such as a first light source (223a), a second light source (223b), and a third light source (223c). For example, any one of the plurality of first light source lenses (224a) can be located above the first light source (223a) and coupled to the light source assembly (221). Another one of the plurality of first light source lenses (224a) can be located between the aforementioned first light source lens (224a) and the second light guide (LG2) and coupled to the housing (222).

[0299] The first light source lens (224a), the second light source lens (224b) and the third light source lens (224c) may include a collimating lens or a collimator.

[0300] In addition, a substrate (225) connected to the light source (223) may be arranged in the light source assembly (221). The substrate (225) may refer to a single substrate, or may include a plurality of substrates in a manner corresponding to the light source (223). For example, the plurality of substrates (225) may include a first substrate (225a), a second substrate (225b), and a third substrate (225c). As described above, the plurality of substrates (225) may form a single integrated substrate or correspond to a plurality of light sources.

[0301] In addition, the substrate (225) can be electrically connected to the control unit or processor (or display unit) in the above-mentioned frame. Therefore, the substrate (225) can include a connector for communicating with an external device or a connected device. In addition, the substrate (225) can be arranged outside the light source (223) and dissipate the heat generated in the light source to the outside. Therefore, the reliability of the light source device (220) can be improved.

[0302] The dielectric lens (MO) may be positioned between the opening (222h) and the second light guide (LG2). In addition, the dielectric lens (MO) may be positioned between the first light guide (LG1) and the second light guide (LG2).

[0303] The dielectric lens (MO) may include a plurality of lenses. For example, the dielectric lens (MO) may include a first dielectric lens (MO1) and a second dielectric lens (MO2). However, as described below, in the absence of the dielectric lens (MO), the light reflected from the second light guide (LG2) may also be directly provided to the first light guide (LG1).

[0304] The first dielectric lens (MO1) may have a first surface (MO1s1) adjacent to the first light guide (LG1) and a second surface (MO1s2) corresponding to the first surface (MO1s1). Light emitted from the light source (223) may pass through the second light guide (LG2) and then sequentially pass through the first surface (MO1s1) and the second surface (MO1s2).

[0305] The second surface (MO1s2) may be convex toward the second light guide (LG2). Alternatively, the second surface (MO1s2) may be concave toward the first light guide (LG1). In addition, the first surface (MO1s1) may be convex or concave toward the second light guide (LG2). For example, the first dielectric lens (MO1) may have the shape of a meniscus lens. Since the second surface (MO1s2) is convex toward the second light guide (LG2) in this manner, light can be converged while passing through the dielectric lens (MO). Therefore, the regularity or uniformity of the light provided to the optical signal generating unit (230) can be improved. In other words, the light incident on the optical signal generating unit (230) can be surface light. Then, the uniformity of the surface light can be improved. Therefore, the accuracy or resolution of the image signal or image emitted to the display unit by the optical device (200) according to the embodiment can be improved.

[0306] The second dielectric lens (MO2) may include a micro lens array (MLA). Therefore, surface illumination may be partially performed on the light passing through the second light guide (LG2).

[0307] Furthermore, the second dielectric lens (MO2) may have a different size than the second light guide (LG2). Additionally, the first dielectric lens (MO1) may have a different size than the second light guide (LG2). For example, the dielectric lens (MO2) may have a size larger than the size of the second light guide (LG2).

[0308] In addition, the positions of the plurality of lenses (L), the positions of the plurality of dielectric lenses (MO), and the positions of the plurality of light source lenses (224) can be maintained by spacers (SP). For example, a plurality of spaces (SP) respectively adjacent to the plurality of lenses (L) can exist in the lens barrel (210). In addition, the plurality of spaces (SP) can be arranged in a housing (222) or a light source assembly (221) of the light source device (220) in a manner adjacent to each of the dielectric lenses (MO) and the light source lenses (224). As an embodiment, the plurality of spaces (SP) can be arranged above or below the above-mentioned lenses, and thus fix or maintain the positions of the lenses.

[0309] The optical signal generating unit (230) may be located below the lower end of the lens barrel (210). The optical signal generating unit (230) may overlap with the lens (L) on the optical axis or in the first direction (X-axis direction).

[0310] The optical signal generating unit (230) may convert light incident on the first light guide (LG1), reflected, and then passing through the Nth lens into an optical signal including image information.

[0311] The optical signal generating unit (230) may reflect the light (first polarized light) reflected from the first light guide (LG1). The optical signal generating unit (230) may generate an optical signal including image information. That is, the light emitted from the optical signal generating unit (230) may be light including image information.

[0312] The optical signal generating unit (230) may include a liquid crystal on silicon (LCoS) device.

[0313] The LCOS device may have a structure in which liquid crystal is distributed between a silicon wafer (a thin circular substrate material for semiconductors) having a complementary metal oxide semiconductor (CMOS) array and an anti-reflection (AR) member coated with a transparent electrode (indium tin oxide (ITO)).

[0314] An alignment layer may be formed on top of the wafer (silicon wafer) to form an initial liquid crystal alignment.

[0315] In addition, a reflective layer or a reflective electrode formed as an aluminum layer and having high optical reflectivity can be positioned below the alignment layer. The reflective electrode can be positioned on top of the silicon wafer. A semiconductor array (CMOS array) can be formed on top of the silicon wafer. Data signals can be sent to the semiconductor array through the panel.

[0316] According to the driving of the optical signal generating unit (230), the optical signal generating unit (230) can reflect at least a portion of the incident light. The optical signal generating unit (230) can reflect the light incident from the surface light source on a per-pixel basis. In addition, the intensity of the reflected light can also be adjusted according to the degree of modulation. For example, the optical signal generating unit (230) can modulate a portion of the first polarized light into the second polarized light. Therefore, the second polarized light generated by the modulation can pass through the first light guide (LG1) and the first lens (L1), and then be provided to the display unit.

[0317] That is, the optical signal generating unit (230) can modulate the delay of the light generated by the modulation, that is, the polarized light. The optical signal generating unit (230) can perform the delay of the first polarized light in various ways. That is, the electric field can be formed by adjusting the voltage (the voltage of the electrode) for each pixel. The degree of liquid crystal distortion can also be adjusted according to the adjusted voltage. For example, when the maximum voltage is applied, the light reflected from the optical signal generating unit (230) can be completely reflected from the first light guide (LG1). When the minimum voltage is applied, the light reflected from the optical signal generating unit (230) can all pass through the second light guide (LG1). However, the light reflected from the optical signal generating unit (230) can also propagate in the reverse direction according to the electric field. In addition, when the intermediate level voltage is applied, a part of the light can pass through the first light guide (LG1). That is, the intensity (e.g., brightness) of the light provided to the display unit can be approximately at an intermediate level.

[0318] In this way, the optical signal generated by the optical signal generating unit (230) can be transmitted to the first lens (L1) through the Nth lens (Ln) and the first light guide (LG1). In addition, at least a portion of the optical signal generated by the optical signal generating unit (230) can pass through the first lens (L1) and be incident on the display unit.

[0319] In addition, the transparent member (240) can also be arranged between the optical signal generating unit (230) and the Nth lens (Ln) (or the first light guide). The transparent member (240) can be a glass panel. The transparent member (240) can be coupled to the lens barrel (210) or the cover (CV). In addition, the transparent member (240) can be positioned above the optical signal generating unit (230). Therefore, it is easy to prevent foreign matter from being introduced into the optical signal generating unit (230). The transparent member (240) can have the same size as the optical signal generating unit (230) or a different size. In addition, the transparent member (240) can overlap with at least a portion of the optical signal generating unit (230) on the optical axis or in the first direction (X-axis direction).

[0320] In addition, the barrier unit (252) may be located between the through hole (210rs) in the lens barrel (210) and its light receiving unit (251). The light receiving unit (251) may be located in an area (DA) located between the first light guide (LG1) and the optical signal generating unit (230). For example, the area (DA) may be an area in a first direction (first X-axis direction) between the through hole (210rs) and the optical signal generating unit (230). In addition, the through hole (210rs) may be located within the area (DA). With this configuration, a portion of the light emitted before the optical signal is incident on the area (230) may be effectively emitted to the outside (e.g., the skin of the wearer) through the through hole (210rs). That is, increasing the amount of light passing through the through hole (210rs) may improve the sensing accuracy of the detection unit (250).

[0321] The light receiving unit (251) may be positioned within the lens barrel (210). For example, the light receiving unit (251) may be positioned within a groove formed in the outer surface of the lens barrel (210). In addition, the outer surface of the light receiving unit (251) and the outer surface of the lens barrel (210) may be stepped. For example, the light receiving unit (251) may protrude further outward than the outer surface of the lens barrel (210). Therefore, the light receiving unit (251) may be arranged in a manner closer to the user. Therefore, the sensing accuracy may be further improved by the light receiving unit (251).

[0322] Similarly, the barrier unit (252) can protrude further outward than the outer surface of the lens barrel (210). Therefore, the barrier unit (252) can prevent the light emitted through the through hole (210rs) from being directly emitted to the light receiving unit (251). In other words, noise can be reduced.

[0323] Furthermore, in the optical device according to the embodiment, the first reference lens (L n ) and the second reference lens (L n-1 ) may be positioned in a manner adjacent to the optical signal generating unit (230) in sequence. Although not shown, the second reference lens may be located between the first light guide (LG1) and the first reference lens (L n ). The first reference lens (L n ) may be larger than the second reference lens (L n-1 ) size (d1). At this time, the through hole (210rs) can be located at the first reference lens (L n ) and the second reference lens (L n-1). Therefore, light can be more easily emitted from the first reference lens toward the second reference lens through the through hole (210rs). Therefore, the accuracy of the detection unit can be improved. Similarly, the light receiving unit (251) can also be positioned between the first reference lens (L n ) and the second reference lens (L n-1 )between.

[0324] Further references Fig.12 , the lights (La, Lb and Lc) emitted from the light source (223) can pass through the light source lens (224), the second light guide (LG2) and the dielectric lens (MO). For example, the first light (La), the second light (Lb) and the third light (Lc) emitted from the first light source (223a) to the third light source (223b) can be emitted from the second light guide (LG2) in the same direction.

[0325] For example, the second light guide (LG2) may have a first coated surface (LG2a) and a second coated surface (LG2b). As described above, at least one of the two coated surfaces may reflect one or more of: light of a first wavelength, light of a second wavelength, or light of a third wavelength. For example, the first wavelength includes a wavelength band of red light. The second wavelength includes a wavelength band of green light. The third wavelength includes a wavelength band of blue light.

[0326] In addition, the first coated surface (LG2a) can reflect the first light (La) or the light of the first wavelength. That is, the first coated surface (LG2a) can transmit the second light (Lb) and the third light (Lc). In other words, the first coated surface (LG2a) can transmit the light of the second wavelength and the light of the third wavelength.

[0327] The second coated surface (LG2b) can reflect the second light (Lb) or the light of the second wavelength. That is, the second coated surface (LG2b) can transmit the first light (La) and the third light (Lc). In other words, the second coated surface (LG2b) can transmit the light of the first wavelength and the light of the third wavelength.

[0328] Therefore, the first light (La) can be reflected from the second light guide (LG2) in a manner of propagating toward the dielectric lens (MO) or the opening (OP) or incident on the dielectric lens (MO) or the opening (OP). The second light (Lb) can be reflected from the second light guide (LG2) in a manner of propagating toward the dielectric lens (MO) or the opening (OP) or incident on the dielectric lens (MO) or the opening (OP). In addition, the third light (Lc) can be reflected from the second light guide (LG2) in a manner of propagating toward the dielectric lens (MO) or the opening (OP) or incident on the dielectric lens (MO) or the opening (OP).

[0329] Therefore, the light (IL) emitted from the light source (223) (or the light indicated by the combination of La, Lb and Lc) may be incident on the first light guide (LG). At this time, the light incident on the first light guide (LG) may be the first incident light (IL).

[0330] Further references Fig.13 , a portion of the first incident light (IL) may be reflected from the first light guide (LG1), and the remaining portion may pass through the first light guide (LG1). That is, the first light guide (LG1) may reflect the first polarized light (ILa) of the first incident light (IL) and transmit the second polarized light (ILb) thereof. For example, the first polarized light (ILa) and the second polarized light (ILb) may each be S polarized light or P polarized light. Therefore, the first polarized light (ILa) as a portion of the first incident light (IL) may be provided to the optical signal generating unit (230) through the Nth lens (Ln). The second polarized light (ILb) may be absorbed into the lens barrel (210) or provided to the additional hole (210h).

[0331] Fig.14 is a view showing another example of a light source device according to an embodiment.

[0332] Reference Fig.14 , the light source (223) in the light source device (220) may be at least a single light source, as described above. That is, the light source (223) may include a first light source (223a), a second light source (223b), and a third light source (223c). In addition, in another example, instead of the second light guide, there are multiple mirrors (e.g., mirrors (M1 to M3)). That is, the optical device may include multiple mirrors (e.g., mirrors (M1 to M3)), and multiple mirrors (e.g., mirrors (M1 to M3)) may be used to provide light to multiple lenses or the first light guide. Other optical devices may have various structures.

[0333] Fig.15 is a flowchart illustrating an operating method of an electronic device or an optical device thereof according to an embodiment. Fig.16 is a timing chart referenced to describe the respective operations of the light source, the optical signal generating unit, and the light receiving unit.

[0334] First, the light receiving unit (251) can operate while emitting light from a light source. However, as described below, when detecting whether the electronic device is worn, the light receiving unit (251) may not operate when light is not emitted from the light source. In addition, the operating method described below can be performed by a control unit (or controller) of an optical device or a control unit (or controller) of an electronic device. That is, in addition to the details described below, the processing in each step of the operating method described below can be performed by a control unit. The operating method according to the disclosed embodiment can be implemented in the form of program instructions that can be executed by various computer devices, and can therefore be recorded on a computer-readable medium. In addition, an embodiment of the present disclosure may be a computer-readable recording medium having recorded thereon one or more programs containing instructions for executing the operating method.

[0335] Reference Fig.15 and Fig.16 According to the embodiment, the operating method includes: a step of turning off the optical device (S1010); a step of operating the light receiving unit (S1020); a step of comparing the first light receiving signal with the first set value (S1030); a step of turning on the light source device (S1050); a step of operating the light receiving unit (S1060); a step of comparing the second light receiving signal with the second set value (S1070) and a step of driving the detection unit (S1080).

[0336] At this time, in order to operate (for example, turn on) the light source unit, the control unit can send a high (HIGH) signal to the light receiving unit. That is, the control unit can control the operation of the light receiving unit. In addition, the control unit can control the turning on (HIGH state) or off (LOW state) of the light source. In addition, the control unit can control the turning on or off of the optical signal generating unit by controlling the turning on or off of the light source. In other words, the control unit can control the light source device, the optical signal generating unit, and the light receiving unit.

[0337] First, when the electronic device is powered on, the control unit can keep the light source turned off. That is, no current may be applied to the light source. For example, in the first operating section (T1), as described above, when the optical device is turned off, the control unit can operate the light receiving unit (so that the light receiving unit can enter the HIGH state) (S1020). In the first operating section (T1), the control unit can receive a first light receiving signal from the light receiving unit. In other words, without applying current to the light source device (entering the LOW state), the control unit can enable the light receiving unit to operate and receive a first light receiving signal from the light receiving unit. In the first operating section (T1), the control unit can compare the first light receiving signal with a first set value (S1030).

[0338] The control unit can determine whether to drive the light source device by comparing the first light receiving signal with the first set value. At this time, when the first light receiving signal is the first set value or higher than the first set value, the control unit can determine that the non-wearing state of the user without introducing ambient light has been reached. Therefore, the control unit can output a non-wearing alarm, etc., and return to the above step (S1010). In addition, the control unit can start again from step (S1020).

[0339] In the first operation section ( T1 ), when the first light reception signal is lower than the first set value, the operation may proceed to the second operation section ( T2 ).

[0340] In the second operating section (T2), the control unit may send a HIGH signal to the light source device. As an embodiment, the control unit may send a HIGH signal to a light source in the light source device (e.g., a second light source emitting light of a green wavelength). For example, the control unit may send a HIGH signal to at least one light source in the light source device. Therefore, the first to third light sources may emit different light or emit light in a non-overlapping manner. Therefore, energy efficiency may be improved.

[0341] For example, during the first light source time segment (T2aa), the first light source may be turned on. In other words, during the first light source time segment (T2aa), the control unit may send a HIGH signal to the first light source. In addition, during the first generation unit time segment (T2ba), the optical signal generation unit may be turned on by receiving a HIGH signal. In addition, during the first light receiving time segment (T2ca), the control unit may be turned on by receiving a HIGH signal. For example, the light receiving unit may receive red light, and the control unit may use the optical signal (second light receiving signal) received from the light receiving unit to detect the user's oxygen saturation level. In other words, when the first light receiving signal is lower than the first set value, the control unit may drive the light source and receive a second light receiving signal from the light receiving unit.

[0342] In addition, during the second light source time section (T2ab), the second light source can be turned on. In other words, during the second light source time section (T2ab), the control unit can send a HIGH signal to the second light source. In addition, during the second generation unit time section (T2bb), the optical signal generation unit can be turned on by receiving a HIGH signal. In addition, during the second light receiving time section (T2cb), the control unit can be turned on by receiving a HIGH signal. For example, the light receiving unit can receive green light, and the control unit can use the optical signal (second light receiving signal) received from the light receiving unit to detect the user's pulse.

[0343] In addition, during the third light source time segment (T2ac), the third light source can be turned on. In other words, during the third light source time segment (T2ac), the control unit can send a HIGH signal to the third light source. In addition, during the third generation unit time segment (T2bc), the optical signal generation unit can be turned on by receiving a HIGH signal. In addition, during the third light receiving time segment (T2cc), the control unit can also be turned off by receiving a LOW signal.

[0344] For example, the light source time segment and the generation unit time segment may be synchronized with each other.

[0345] In this way, in the second operation section (T2), the control unit can detect or determine whether the user is in the wearing state using the optical signal (second light receiving signal) received from the light receiving unit. In addition, only when the specific light source is turned on (in the HIGH state), the control unit can receive the optical signal (second light receiving signal) by driving the light receiving unit.

[0346] In addition, in the present embodiment, when the light source is turned on (in a HIGH state), the optical signal generating unit can be turned on (in a HIGH state) in a manner corresponding to the light source time section. Therefore, the precision level of the operation can be improved.

[0347] Subsequently, the control unit may receive a second light receiving signal output by the turned-on light receiving unit (S1060). The control unit may compare the second light receiving signal with the second set value (S1070). At this time, when the second light receiving signal is lower than the second set value, the control unit may determine that the user or the wearer is not accurately in the wearing state. For example, in a case where at least a portion of the electronic device is properly in close contact with the skin, the second light receiving signal may be lower than the second set value. Therefore, it may be determined whether the user is accurately in the wearing state. In this case, it may be determined that the user is in a non-wearing state, or in particular in an inappropriate wearing state, and therefore may enter a waiting state (S1090). For example, it may be possible to return to step (S1010) or step (S1050).

[0348] On the contrary, when the second light receiving signal is higher than the second set value, it may proceed to the third operation section (T3).Therefore, the control unit may send a driving signal (HIGH signal) to the light source and the optical signal generating unit at a predetermined period.

[0349] The control unit may turn on the light receiving unit (enable the light receiving unit to enter the HIGH state) in a manner corresponding to the turning on of the light source and the optical signal generating unit (HIGH state) (S1080). In other words, the control unit may compare the second light receiving signal with the second set value, and accordingly receive a third light receiving signal from the light receiving unit in a manner corresponding to the corresponding operation of the light source and the optical signal generating unit. That is, the control unit may drive the detection unit (S1080). For example, in order to detect the wearer's pulse in the third operating section (T3), the control unit may turn on the light receiving unit for a predetermined period of time to operate (enable the light receiving unit to enter the HIGH state) in a manner corresponding to the turning on of the light source and the optical signal generating unit (e.g., corresponding to the second light source time section and the second generating unit time section).

[0350] For example, in order to detect the wearer's oxygen saturation level in the third operating section (T3), the control unit can turn on the light receiving unit for a predetermined period of time for operation (enabling the light receiving unit to enter a HIGH state) in a manner corresponding to the turning on of the light source and the optical signal generating unit (for example, corresponding to the first light source time section and the first generating unit time section).

[0351] In the operating method and electronic device according to the various embodiments described above, a test image may be selected for each of the right eye and the left eye of the user. In addition, a test image may be selected for both eyes of the user. Alternatively, a test image may be selected sequentially for the right eye, the left eye, and both eyes of the user.

[0352] First, refer to Figures 17 to 24 The display method described can be executed by the control unit of the above-mentioned electronic device. That is, in addition to the details described below, the processing in each step of the display method described below can be performed by the control unit. The display method according to the disclosed embodiment can be implemented in the form of program instructions that can be executed by various computer devices, and can therefore be recorded on a computer-readable medium. In addition, an embodiment of the present disclosure may be a computer-readable recording medium having recorded thereon one or more programs containing instructions for executing the display method.

[0353] Fig.17 is a flowchart illustrating a display method according to an embodiment. Fig.18 is an additional flow chart illustrating a display method according to an embodiment. Fig.19 and Fig. 20 is a view referred to to describe vision determination in a display method according to an embodiment. Fig.21 is a view showing an output of an image in a display method according to an embodiment.

[0354] Reference Fig.17 and Fig.18 According to an embodiment, the display method includes: a step of outputting test images with different focal lengths (S1040); a step of determining the user's vision using a test image selected from the test images with different focal lengths as a reference (S1060); and a step of outputting a virtual image based on the determined user's vision (S1080).

[0355] In addition, according to the embodiment, the display method may further include, before the step (S1040) of outputting test images with different focal lengths: a step (S1010) of detecting a user (or a wearer of an electronic device); a step (S1020) of checking whether an input (via a button or a similar mechanism) is detected; and a step (S1030) of checking whether the user has been stored when an input is detected.

[0356] Preferably, refer to Figure 2 When the user or wearer wears the display device, the control unit (27) (see Figure 2 ) can be used by sensing unit (23) (see Figure 2 ) to detect or determine whether the user is wearing the display device. When the user wears the display device, the control unit (27) can determine whether an input is detected based on the signal received by the sensing unit (23) (S1020). For example, the user can use the user input unit (130) (see Figure 3 ) or input unit (22) (see Figure 2 ) generates an input signal. When the user touches the input unit, it can be recognized that the user is using the electronic device. If the user's input is not detected within a predetermined time, the control unit can re-execute the step of detecting whether the user is wearing the electronic device.

[0357] In addition, when the input is detected through the input unit, the control unit may check whether the user has been stored (S1030). In the case where the user has been stored, the control unit may retrieve the stored Figure 2 Alternatively, although the user has been stored, the control unit can re-determine the user's vision according to the user's input and re-store the user's vision information. Therefore, any changes in the user's body can also be easily taken into account.

[0358] In the case where the user has not been stored or the user is tested again, as described above, the control unit may output a test image with a different focal length (S1040). For example, the control unit may generate or provide an image signal of a test image with a different focal length. Based on the image signal of the test image generated by the control unit, the optical device may generate an image, and the image generated by the optical device may be output on the display unit.

[0359] Further references Fig.19 , the test images (TI1, TI2, and TI3) may have different focal lengths. A plurality of test images may be provided, such as test images (TI1, TI2, and TI3). As an embodiment, the test images may include a first test image (TI1), a second test image (TI2), and a third test image (TI3). At this time, the first test image (TI1) has the longest focal length relative to the user's eyeball, followed by the second test image (TI2), and then the third test image (TI3). For example, the focal length (f1) of the first test image (TI1) may be greater than the focal length (f2) of the second test image (TI2). The focal length (f2) of the second test image (TI2) may be greater than the focal length (f3) of the third test image (TI3).

[0360] In addition, refer to Fig. 20 , the first test image (TI1), the second test image (TI2) and the third test image (TI3) are described as Fig.19 The test images in the image display device are the same. In addition, the first test image (TI1), the second test image (TI2) and the third test image (TI3) can be output according to the user's selection (input) (S1051). One of the multiple test images can be output by the control unit. For example, the control unit can output the first test image (TI1). Subsequently, the user can input an input signal (S1051) by touching or otherwise interacting with the input unit (130). For example, the user can generate an input signal by touching one of the first button (BT1), the second button (BT2) or the third button (BT3) on the input unit arranged on the frame. At this time, the first button (BT1) can correspond to an increase in the focal length of the test image. The second button (BT2) can correspond to a decrease in the focal length of the test image. The third button (BT3) can correspond to the selection of the test image. In addition, similar to the case where the first test image (TI1), the second test image (TI2) and the third test image (TI3) have different focal lengths, one or more test images can have different focal lengths. In addition, in the case where multiple test images with different focal lengths are provided, the multiple test images can have respective different focal lengths.

[0361] As an example, when the control unit receives the user's input (S1051), the control unit may adjust the focus of the test image in a manner corresponding to the user's input, i.e., the input of the application (S1052). The control unit may determine the user's vision in a manner corresponding to the user's selection and input (S1053 and S1060).

[0362] For example, in the case where the first test image (T1) is identified in a fuzzy manner, the user can change the focal length of the test image by touching the first button (BT1) or the second button (BT2). For example, when the user touches the second button (BT2), the control unit can output a second test image (TI2) having a shorter focal length than the first test image (TI1). In the case where the second test image (TI2) is identified in a fuzzy manner, the user can reduce the focal length of the test image by touching the second button (BT2) again. Therefore, the control unit can output a third test image (TI3) having a focal length smaller than that of the second test image (TI2). Subsequently, in the case where the third test image (TI3) is clearly identified, the user can select the third test image (TI3) as an image for determining the user's vision by touching the third button (BT3). In this way, the control unit adjusts the focal length of the test image in a manner corresponding to the user's input (S1050).

[0363] At this time, the image or test image selected by the user's input may be at least one of a plurality of test images. For example, a plurality of test images that can be seen clearly according to the user's vision may be provided. In this case, if a plurality of images are selected, the control unit may determine the user's vision based on the maximum focal length. Therefore, the control unit may determine the user's accurate vision and output a virtual image corresponding to the user's vision. Therefore, the electronic device according to the embodiment can effectively eliminate the inconvenience of the user.

[0364] The control unit may determine the user's eyesight using the image selected through the input unit as a reference (S1060). The control unit may determine the user's eyesight and then adjust the focus of the image based on the determined user's eyesight, thereby outputting the resulting image (S1080). Therefore, the user can clearly recognize the virtual image output from the optical unit.

[0365] Further references Fig.21 The control unit may send a control signal to the optical device in a manner of outputting a plurality of virtual images such as virtual images (VI, VI2, etc., until VIN) (collectively referred to as virtual images (VI)). Therefore, the optical device may generate images corresponding to the plurality of virtual images.

[0366] As shown, multiple virtual images can be configured as layers that vary according to distance. Virtual objects can be positioned across virtual images configured as varying layers. These virtual images are transmitted to the user by the optical device (200). The user can adjust the focal length of the virtual images based on the user's determined vision. For example, multiple virtual images can be output by reducing the focal length for a nearsighted user.

[0367] In addition, the optical elements (OD1 and OD2) may be arranged on the front and rear sides of the display unit (300), respectively. At this time, the front side refers to the side of the display unit (300) facing the object or the real image, and the rear side refers to the side of the display unit (300) facing the user. Alternatively, the display unit (300) may also include optical elements (OD1 and OD2). The optical elements (OD1 and OD2) may be located on the frame of the electronic device. In addition to the diffraction portion, the optical elements (OD1 and OD2) may have a variable focus lens. Therefore, the curvature of the optical element can be changed in a manner corresponding to the eyesight of a determined user. That is, the focal length can be changed.

[0368] For example, the first optical element (OD1) may be arranged on the outer side of the display unit (300) and include a diffraction portion or a polarization portion. Therefore, the real image (RI) may be provided to the user in a predetermined frame period or through a predetermined frame. As described above, the focus or curvature of the first optical element (OD1) may be changed in a manner corresponding to the determined vision of the user. Therefore, the user may recognize the real image clearly rather than in a blurred manner. That is, the real image or the like may appear on the user's eyeball as it would for normal vision.

[0369] Furthermore, the focus or curvature of the second optical element (OD2) may also be changed in a manner corresponding to the vision of the user.

[0370] As a modified example, there are multiple lenses in the optical device (200), and the distance between each of the multiple lenses in the optical device can also be changed by an actuator, etc. In addition, the control unit can adjust the focus of the test image by moving the lens in the optical device.

[0371] After determining the user's vision (S1060), the control unit may store the vision information in the memory (S1070). In addition to the vision information, specific detection information of the user acquired by the sensing unit may also be stored in the memory together.

[0372] Fig. 22 is a flowchart illustrating a display method according to another embodiment. Fig.23 is a view cited to describe visual acuity determination in the display method of the same embodiment.

[0373] Reference Fig. 22 and Fig.23 According to the same embodiment, the display method may include: a step of outputting a test image including multiple objects (S2010); a step of selecting at least one object from the multiple objects (S2020); a step of determining the user's vision based on the selected object (S2030); and a step of storing vision information (S2040).

[0374] The step (S2010) of outputting a test image including a plurality of objects may correspond to the step (S1040) of outputting a test image. In addition, the steps before the step (S1040) of outputting a test image are applicable to the display method according to the present embodiment. Except for the details described below, the details in the above steps may be applied to the display method according to the present embodiment in the same manner.

[0375] According to this embodiment, the test image may include a plurality of objects. In this case, the plurality of objects may have different focal lengths. That is, the test images with different focal lengths may each include a plurality of objects with different focal lengths.

[0376] As shown in the figure, the test image (TI4) may include multiple objects, such as objects (OJ1 to OJ3). For example, the multiple objects may include a first object (OJ1), a second object (OJ2), and a third object (OJ3). At this time, the first object (OJ1) may have a larger distance or focal length than the second object (OJ2). The second object (OJ2) may have a larger distance or focal length than the third object (OJ3).

[0377] For example, a selection window (SD) for an object within a test image, i.e., a virtual image, may be moved in a manner corresponding to an input by a user (via a first button, a second button, or a third button). For example, the first button may correspond to the left. The second button may correspond to the right. The third button may correspond to "select". One of a plurality of objects may be selected according to the user's input. Alternatively, a test image may be selected from test images having different focal lengths and configured with a plurality of layers. Thus, the user's vision may be determined. Subsequently, the determined user's vision may be stored as described above. Alternatively, based on the determined user's vision, the focal length of the virtual image may be adjusted and output. That is, in the case of outputting a virtual image, the focus of the virtual image may vary or change based on the determined user's vision.

[0378] Fig.24 is a flowchart illustrating a display method according to still another embodiment. Fig.25 is a perspective view showing an augmented reality electronic device as a modified example according to the embodiment.

[0379] Reference Fig.24 According to the same embodiment, the display method may include: a step of outputting a plurality of test images (S3010); a step of selecting at least one image (S3020); a step of determining the user's vision (S3030) and a step of storing vision information (S3040).

[0380] In the case of outputting a plurality of test images (S3010), the above-mentioned various examples may be applicable to the display method according to the present embodiment. In addition, at least one image may be selected by receiving an input signal of a user through a control unit (S3020). Visual acuity may be selected based on the selected test image (S3030). At this time, the step of determining visual acuity may include: a step of selecting at least one image from a plurality of test images having different focal lengths, and a step of determining the visual acuity of the user based on the maximum focal length when a plurality of images are selected.

[0381] For example, the control unit may output a plurality of images selected from the test images according to the user's input. The user's vision may be determined based on the maximum focal length of the selected plurality of images. The control unit may output the focus of the virtual image by changing the focus of the virtual image based on the user's vision determined according to the maximum focal length.

[0382] Reference Fig.25 , the electronic device may further include a light receiving element (PD) arranged on the frame (100). Therefore, when determining the above-mentioned vision, the test light (TL) (image, etc.) can be output to the user's eyeball through the optical device (200) and the display unit (300), and the light (image) reflected from the human eyeball can be transmitted to the light receiving element (PD). At this time, the control unit can determine the refractive index of the user's eyeball or the user's vision by providing the light (RL) to the light receiving element (PD). Therefore, as described above, the electronic device can output a virtual image by changing the focal length based on the user's vision.

[0383] Reference Figures 26 to 34 The control method described can be executed by a controller or control unit in the above-mentioned control device or electronic device. That is, in addition to the details described below, the controller can perform the processing in each step of the control method. The control method according to the disclosed embodiment can be implemented in the form of program instructions that can be executed by various computer devices, and can therefore be recorded on a computer-readable medium. In addition, an embodiment of the present disclosure can be a computer-readable recording medium having recorded thereon one or more programs containing instructions for executing the control method.

[0384] Fig.26 is a block diagram showing a control device according to an embodiment of the present invention. Fig. 27: is a conceptual diagram showing a control device according to an embodiment of the present invention. Fig.28 is a flowchart illustrating a control method according to an embodiment of the present invention. Fig.29 is a specific flowchart showing a control method according to an embodiment of the present invention. Fig.30 are views referred to for describing the operation of the control device and the control method according to the embodiment of the present invention. Fig.31 1 is a view referred to to describe the operation and control between the control device and the electronic device according to the embodiment of the present invention. Fig.32 is a view showing an interface of an electronic device according to an embodiment of the present invention. Fig.33 are views referred to for describing the operation of an electronic device using a control device according to an embodiment of the present invention. Fig.34 2 is a view cited to describe a control device according to an embodiment of the present invention.

[0385] Reference Fig.26 and Fig. 27 According to an embodiment, the control device (500) may include a touch input unit or a first input unit (510), a button input unit or a second input unit (520), a detection unit (530), a communication unit (540), a battery unit or a power supply unit (550), a memory unit or a storage unit (560), and a controller (570).

[0386] The touch input (510) may detect a user's touch. For example, the touch input (510) may include a decompression sensor, etc. The touch input (510) may output a signal regarding the detected user's touch to the controller (570), etc. In an embodiment, the touch input (510) may set the position of the control device (500) by detecting the user's touch, and detect various touch movements.

[0387] The upward, downward, leftward and rightward directions of the control device (500) according to the embodiment can be configured by the technology described below. That is, the upward, downward, leftward and rightward directions of the control device (500) may not be structurally configured. For example, the control device (500) may be configured in a circular or similar shape, and thus has a structure symmetrical with respect to an imaginary line. The touch input (510) may be positioned externally relative to the button input unit (520) described below. For example, the touch input (510) may surround the button input unit (520) positioned at the center. The touch input (510) may also be configured as a touch screen. With this configuration, the control device according to the embodiment is used as a control device (e.g., a remote controller) of an electronic device, and the operation or control of the electronic device can be easily performed using a human sensory organ (e.g., a tactile organ). That is, the user's vision is not necessary to control the control device, and therefore the electronic device can be easily controlled in various environments and spaces.

[0388] For example, the touch input ( 510 ) receives various inputs (eg, a slide input and a tap), etc., from a user and may output signals corresponding to the inputs.

[0389] The button input unit (520) may detect a button input from a user or other individual. For example, the button input unit (520) may be configured to have various structures and output a signal corresponding to the pressure applied by the user to the controller (570), etc. In an embodiment, an image or information within an image output by the electronic device through the button input unit (520) may be selected.

[0390] The button input unit (520) may be positioned at the center of the control device (500). Therefore, the user can easily find the button input unit (520). In addition, the button input unit (520) may have a shape protruding outward or a groove shape. Therefore, the user can use their fingers (e.g., their thumbs) to control the electronic device. For example, the button input unit (520) may be used to control whether the electronic device or the control device is powered on. Alternatively, the button input unit (520) may be used to control whether an operation combined with the electronic device is performed or whether an input waiting mode in the electronic device is activated.

[0391] The detection unit (530) may be configured with various motion sensors. The detection unit (530) may output a detection signal including motion information. For example, the detection unit (530) may include an IMU sensor. For example, the detection unit (530) may include a gyroscope sensor, an acceleration sensor, an angle sensor, an angular velocity sensor, and a geomagnetic field sensor. The motion information may include information about speed or acceleration. The detection unit (530) may exist in the control device, or may also correspond to a sensing unit of the electronic device. Therefore, the control unit of the control device may also receive a detection signal from the sensing unit of the electronic device.

[0392] As an embodiment, the detection unit (530) can also detect the gravity direction of the control device (500) and the degree to which the control device (500) is tilted. In addition, the posture information from the detection unit (530) can be used similarly to the sliding information through the touch input (510) (input). Therefore, regardless of the direction of the control device (500), the user can easily control the electronic device.

[0393] For example, when the control device (500) is tilted to the left or right, a command corresponding to the direction key may be transmitted to the electronic device.

[0394] The communication unit (540) may transmit and receive data to and from the electronic device. Alternatively, the communication unit (540) may include at least one module that enables wireless communication with an external server.

[0395] The battery unit (550) can supply power to the control device. Data supporting various functions can be stored in the memory unit (560) used as a storage unit. Examples of the memory unit (560) may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, and / or other storage devices.

[0396] The controller (570) may perform the processing of the control method described below. The controller (570) may refer to a processor, a control unit, etc. In addition, the controller (570) may include an application specific integrated circuit (ASIC), a different chipset, a logic circuit, and / or a data processing device.

[0397] In an embodiment, the controller (570) is described as executing the control method described below.

[0398] Further references Fig.28 and Fig.29According to the implementation method, the control method includes: a step of receiving a user's input (S1010); a step of activating a waiting mode (S1020); a step of receiving a detection signal (S1030); a step of determining a user's state (S1040); a step of determining activation of a gesture motion or a touch motion in a manner corresponding to the determined user's state (S1050); and a step of sending a command corresponding to the activated motion (S1060).

[0399] The controller (570) may receive a user's input through a button input unit or a touch input unit (S1010). Therefore, the controller (570) may wake up the control device, that is, the controller (570) may activate a waiting mode of the control device (S1020).

[0400] Subsequently, the controller (570) may receive a detection signal from the detection unit (530) (S1030). At this time, the controller (570) may determine the state of the user using the detection signal received from the detection unit (530) (S1040). Examples of the state of the user may include a stationary state, a moving state, an exercise state, and a riding state. These states are exemplary and may also be represented as a plurality of states (e.g., state 1 to state 4).

[0401] Subsequently, the controller (570) may determine the activation of the gesture motion or touch motion in a manner corresponding to the determined state of the user (S1050). The controller (570) may send a command to the electronic device in a manner corresponding to the activated motion (S1060). For example, the controller (570) may activate only the gesture motion. In this case, although the user performs the touch motion, the output or command corresponding to the touch motion may not be sent to the electronic device. In addition, the controller (570) may activate only the touch motion. In this case, although the user performs the gesture motion, the output or command corresponding to the gesture motion may not be sent to the electronic device.

[0402] More specifically, in the control method according to the embodiment, the step (S1040) of determining the state of the user includes the step (S1041) of comparing whether the amplitude of the detection signal is higher than a second setting value when the intensity of the detection signal is higher than a first setting value.

[0403] First, refer to Fig.30 , in the stationary state (state 1), the intensity (A1) (eg, average intensity) and amplitude (AM1) of the detection signal may be the lowest (compared to cases in other states).

[0404] In the moving state (state 2), the intensity of the detection signal may be higher than that in state 1, and the amplitude (AM2) may be lower than that in states 1 and 4. In addition, in the moving state, the intensity of the detection signal may be lower than that in states 3 and 4. In the moving state, the amplitude (AM2) of the detection signal may be lower than that in state 3.

[0405] In the exercise state (state 3), the intensity (A3) of the detection signal may be higher than in states 1 and 2, and lower than in state 4. In the exercise state, the amplitude (AM3) of the detection signal may be the highest compared to the amplitudes in other states.

[0406] In addition, in the riding state (state 4), the intensity (A4) of the detection signal can be the highest compared to the intensity in other states. In the riding state, the amplitude (AM4) of the detection signal can be higher than the amplitude in state 1, and lower than the amplitude in states 2 and 3.

[0407] Therefore, when the strength of the detection signal is higher than the first set value, it can be compared whether the amplitude of the detection signal is higher than the second set value. At this time, the first set value can be within the range between the value of A3 and the value of A2.

[0408] The control method according to the embodiment may include a step of determining the state of the user as an exercise state when the amplitude of the detection signal is higher than a second set value ( S1042 ).

[0409] That is, when the amplitude of the detection signal is higher than the second value, the controller may determine the state of the user as an exercise state (S1042). The second set value may be within a range between a value of AM3 and a value of AM4.

[0410] The control method may include a step of comparing whether the intensity of the detection signal is higher than a third set value when the amplitude of the detection signal is lower than the second set value (S1043).

[0411] For example, when the amplitude of the detection signal is lower than the second set value, the controller may compare whether the intensity of the detection signal is higher than the third set value (S1043).

[0412] Subsequently, the control method may include a step of determining the state of the user as a sitting state when the intensity of the detection signal is higher than a third set value (S1404).

[0413] That is, when the strength of the detection signal is higher than the third setting value, the controller may determine the state of the user as the sitting state (S1044). At this time, the third setting value may be higher than the value of A3. The third setting value may be higher than the first setting value.

[0414] The control method may include a step of comparing whether the amplitude of the detection signal is higher than a fourth set value when the strength of the detection signal is lower than the first set value (S1045). The fourth set value may be within a range between the value of AM1 and the value of AM2. The fourth set value may be lower than the second set value.

[0415] Subsequently, the control method may include a step of determining the state of the user as a moving state when the amplitude of the detection signal is higher than a fourth set value (S1047).

[0416] When the amplitude of the detection signal is higher than the fourth value, the controller may determine the state of the user as a moving state (S1047). That is, based on the amplitude, the controller may determine the state of the user as a non-stationary state.

[0417] In addition, the control method may include a step of determining the state of the user as a stationary state when the amplitude of the detection signal is lower than a fourth set value (S1046).

[0418] When the amplitude of the detection signal is lower than the fourth set value, the controller may determine the state of the user as a stationary state in which the user hardly moves (S1046).

[0419] After determining the state of the user as described, the controller may determine activation of gesture motion and touch motion accordingly.

[0420] When the amplitude of the detection signal is higher than the second value, the controller may determine the state of the user as an exercise state (S1042). Subsequently, the controller may block both the gesture motion and the touch motion (S1051).

[0421] When the intensity of the detection signal is higher than the third set value, the controller may determine the state of the user as a riding state (S1044). Subsequently, the controller may activate the gesture motion and the touch motion (S1052).

[0422] When the amplitude of the detection signal is lower than the fourth set value, the controller may determine the state of the user as a stationary state in which the user hardly moves (S1046). Subsequently, the controller may activate the gesture motion and the touch motion (S1052).

[0423] When the amplitude of the detection signal is higher than the fourth value, the controller may determine the state of the user as a moving state (S1047). Subsequently, the controller may stop the gesture motion and activate the touch motion.

[0424] Therefore, it is easy to prevent the user from accidentally inputting a touch or gesture (error input situation). In other words, even when the user is stationary and does not move, the control method according to the embodiment can accurately send a control command to the electronic device according to the user's input without any error input.

[0425] Further references Fig.31 , the electronic device as described above may include a sensing unit. Therefore, the electronic device or the control device may send and receive detection signals to each other. For example, in the case where the difference between the detection signal (sig1) from the sensing unit of the electronic device and the detection signal (sig2) from the detection unit of the control device falls within a predetermined range, a command from the control device may be provided to the electronic device.

[0426] Further references Fig.32 , the user can recognize various objects on the real image through the electronic device. In addition, the electronic device can provide virtual objects and the like in a manner corresponding to the objects in the real image. For example, the electronic device can output information corresponding to the real image (e.g., a building) as a virtual object to the user.

[0427] Further references Fig.33 , the user can use the control device according to the embodiment to output a command for moving or selecting a virtual object. That is, the user can Fig.32 The user interface shown in the figure selects an object (e.g., a virtual object or information (sticky note picture)) by controlling the device. For example, when selecting Fig.32 When an object in the form of a sticky note is shown, the screen can be switched to Fig.33 .

[0428] Then, for example, in the case where the user is stationary, the controller can activate gesture motion and touch motion as described above. Thus, in the case where the user taps the control device, the control device can move the selection target or selection object from "yes" to "no" (SL'), or send a command to select a specific object to the electronic device.

[0429] In addition, when the user shakes the control device in a specific manner (eg, a specific number of times), the control device may move a selection target or selection object from "Yes" to "No" (SL') or send a command to select a specific object to the electronic device.

[0430] In addition, further reference Fig.34 , when the touch motion is activated, the controller may configure the user's initial input position to be one of: above, below, to the left, or to the right relative to the center portion.

[0431] For example, the touch motion can be activated in any of the above-mentioned user's states. At this time, the controller can select one of the following: upward, downward, left or right relative to the area (AR1) (initial input) that the user initially inputs to the touch input unit. At this time, the controller can configure a first imaginary line (VL1) connecting the button input unit (520) as the center part and the initial input, and a second imaginary line (VL2) perpendicular to the first imaginary line (VL1) that divides the touch input unit (510) into two halves. The controller can configure the direction to be one of the following: upward, downward, left or right relative to the first imaginary line (VL1) and the second imaginary line (VL2).

[0432] For example, if the user specifically touches the "left" side of the second imaginary line (VL2) after the initial input, the control device may send a command to the electronic device to move "left". The electronic device may execute the command in a manner corresponding to the user's touch (input).

[0433] In this way, the user can provide input to the electronic device or execute a command for the electronic device through the control device while moving. At this time, even if the user's entire body is shaken, the control device can easily prevent erroneous input. In addition, the user can also experience convenience while moving.

[0434] The control method according to the disclosed embodiment can be implemented in the form of program instructions that can be executed by various computer devices, and thus can be recorded on a computer-readable medium. In addition, an embodiment of the present disclosure can be a computer-readable recording medium having recorded thereon one or more programs containing instructions for executing the control method.

[0435] Fig.35 is a side view showing an electronic device according to an embodiment. Fig.36 is a plan view showing an electronic device according to an embodiment. Fig.37 is a conceptual diagram showing an optical device according to an embodiment. Fig.38 is a perspective view showing an optical device according to an embodiment. Fig.39 is an exploded perspective view showing an optical device according to an embodiment. Fig.40 is along Fig.38 A view taken along line AA'. Fig.41 It is shown Fig.40 An enlarged view of part K1. Fig.42 It is shown Fig.40 An enlarged view of part K2.

[0436] Reference Fig.35 and Fig.37As described above, the electronic device according to the embodiment may include a frame (100), an optical device (200), a display unit (300), and a heat dissipation unit (HD).

[0437] As described, the frame (100) may be an eyeglass frame.

[0438] The optical device (200) may be arranged on the frame (100) and generate an image. Therefore, the optical device (200) may include a light source.

[0439] The display unit (300) may be connected to the optical device (200) and output an image generated by the optical device (200). To this end, the optical device (200) may include an optical signal generating unit (230).

[0440] The heat dissipation unit (HD) may be connected to the optical device (200). The heat dissipation unit (HD) may be arranged within the frame (100). With this configuration, fogging or the like caused by environmental conditions can be easily prevented by the heat dissipation unit (HD).

[0441] The heat dissipation unit (HD) may be in contact with the light source of the optical device. Therefore, it is possible to easily dissipate heat generated by the light source of the optical device. Therefore, when the user wears the frame, the user's skin can be prevented from being damaged by the optical device. In addition, the reliability of the optical device can be improved.

[0442] The heat dissipation unit (HD) may be in contact with a substrate of a light source device in the optical device (200). More specifically, the heat dissipation unit (HD) may be in contact with a substrate in contact with the light source. To this end, the substrate may be exposed outside the housing of the light source device. That is, through the exposed surface, the heat dissipation unit (HD) may easily come into contact with the substrate on which the light source is mounted, and the heat dissipation unit may easily perform heat dissipation.

[0443] Further references Figures 37 to 40 According to an embodiment, an optical device (200) includes a lens barrel (210), a lens (L), and a first light guide (LG1). In addition, the optical device (200) may include a light source device (220) positioned in an opening portion (OP) formed in a side of the lens barrel (210) or coupled to the opening portion (OP), and an optical signal generating unit (230) adjacent to the lens barrel (210). In addition, the optical device (200) may further include a cover (CV) covering the lens barrel (210), the light source device (220), and the optical signal generating unit (230), and a substrate (not shown) including a connector.

[0444] The lens (L) may include a plurality of lenses. For example, the lens (L) may include a plurality of lenses arranged sequentially from the top of the lens barrel (210). For example, the lens (L) may include a first lens (L1) arranged at the uppermost position, a second lens (L2) arranged below the first lens (L1), and so on, until the Nth lens (Ln) arranged at the lowermost position. At this time, N may be a natural number greater than or equal to 2. Among the plurality of lenses (L), the Nth lens (Ln) may be positioned closest to the optical signal generating unit (230), which is positioned below the lower end of the lens barrel (210) or below the lowermost lens among the plurality of lenses (L).

[0445] In addition, in an embodiment of the present invention, the first direction (X-axis direction) may correspond to the optical axis. In addition, the first direction (X-axis direction) may correspond to the direction in which light emitted from the light source device (220) and reflected from the optical signal generating unit (230) is emitted to the above-mentioned display unit. The second direction (Y-axis direction) is a direction perpendicular to the first direction (Y-axis direction). In addition, the second direction (Y-axis direction) may correspond to the direction from the first light guide (LG1) toward the opening portion (OP). In addition, hereinafter in this specification, or in this embodiment, the second direction (Y-axis direction) may correspond to the direction from the first light guide (LG1) toward the second light guide (LG2). The first light guide (LG1) may be referred to as a first light guide unit or a first guide member. In addition, the second light guide (LG2) may be referred to as a second light guide unit or a second guide member.

[0446] The lens barrel (210) may also have a hole corresponding to the opening (OP). As an embodiment, the lens barrel (210) may have a lens barrel hole or an additional hole (210h). The additional hole (210h) may be positioned in a manner facing the opening (OP). Alternatively, the additional hole (210h) may overlap with the opening (OP) in the second direction (Y-axis direction). Alternatively, the distance from the Nth lens (Ln) to the additional hole (210h) in the first direction may be the same as the distance between the opening (OP) and the Nth lens (Ln). Alternatively, the additional hole (210h) may be positioned in an area corresponding to the position of the opening (OP) on the inner surface of the lens barrel (210). With this configuration, a joining member may be easily applied to the light source device (220), which is coupled to, inserted into, fixed to or connected to the opening (OP) through the additional hole (210h). Therefore, the bonding force between the lens barrel (210) and the light source device (220) can be improved. Therefore, the optical device (200) according to the embodiment can have improved durability, persistence or reliability. Alternatively, it is also possible to easily perform an optical test on the light source device (220) positioned in the opening (OP) through the additional hole (210h). Alternatively, the lens barrel (210) and the light source device (200) can be easily discharged or distributed through the additional hole (210h) through the additional hole (210h). The presence or absence of the additional hole (210h) can be applied in a different manner from one embodiment to another. For example, as described above, the additional hole (210h) can be arranged on the side of the lens barrel (210). Alternatively, considering durability or similar factors, the additional hole (210h) may not exist on the side of the lens barrel (210).

[0447] A plurality of lenses (L) may be positioned within the lens barrel (210). In addition, a first light guide (LG1) may be positioned within the lens barrel (210).

[0448] The lens barrel (210) according to the embodiment may include an opening portion (OP) formed in a side thereof. The opening portion (OP) may have various shapes, such as a circular shape and a polygonal shape. The opening portion (OP) may correspond to the position of the first light guide (LG1).

[0449] As an embodiment, the opening portion (OP) may overlap the first light guide (LG1) in a direction perpendicular to the optical axis. With this configuration, light emitted from the light source device (220) positioned on the side of the lens barrel (210) may be easily incident on the first light guide (LG1).

[0450] The first light guide (LG1) may be arranged between two lenses of the plurality of lenses. For example, the first light guide (LG1) may be arranged between the first lens (L1) and the N-th lens (Ln). In addition, the first light guide (LG1) may be arranged between the first lens (L1) and the optical signal generating unit (230). In addition, the first light guide (LG1) may be arranged between the first lens (L1) and the second lens (L2), or between the second lens (L2) and the N-th lens (Ln). In various embodiments, these different positions of the first light guide (LG1) are described below.

[0451] In the present embodiment, the first light guide (LG1) may be positioned between the second lens (L2) and the N-th lens (Ln). Therefore, the N-th lens (Ln) may be arranged between the first light guide (LG1) and the optical signal generating unit (230). With this configuration, light reflected from the first light guide (LG1) may be provided to the optical signal generating unit (230), thereby ensuring a suitable optical path. In addition, refraction of reflected light may be achieved. Therefore, minimization of the first light guide (LG1) may be achieved.

[0452] The first light guide (LG1) may include a first prism. The first prism may be a polarizing prism. Alternatively, the first prism may be a polarization separating prism.

[0453] The first prism may reflect the first polarized light and transmit the second polarized light. For example, a portion of the light (first polarized light) provided from the light source device (220) or incident on the first light guide (LG1) may be reflected from the first light guide (LG1) and provided to the optical signal generating unit (230). Another portion of the light (second polarized light) incident on the first light guide (LG1) may pass through the first light guide (LG1) and be absorbed into the lens barrel (210).

[0454] Light emitted from the light source device (220) and passing through the opening portion (OP) may be incident on the first light guide (LG1). To this end, as described above, the opening portion (OP) may be arranged in the region where the first light guide (LG1) is positioned. For example, the incident surface of the first light guide (LG1) may be positioned in a manner facing the opening portion (OP). Alternatively, the position of the first light guide (LG1) in the lens barrel (210) may be the same as the position of the first light guide (LG1).

[0455] The light source device (220) may include a light source (223) and thus generate (produce or provide) light. The light source device (220) according to an embodiment may be positioned in or coupled to the opening (OP). That is, the light source device (220) may be connected or coupled to the lens barrel (210).

[0456] The light source device (220) may include a housing (222) having an opening (222h), a second light guide (LG2) arranged in the housing (2220), and a light source (223) providing light to the second light guide (LG2).

[0457] In addition, the light source device (220) may include a light source assembly (221) arranged on the outside of the light source device (220) and covering the shell (222), a light source lens (224) adjacent to the light source (223), and a dielectric lens (MO) positioned within the shell (222).

[0458] The light source assembly (221) may be arranged at the outermost edge of the light source device (220). In the case where the light source (223) is difficult to install in the housing (222) or an additional lens (light source lens) needs to be installed, the light source assembly (221) may be positioned outside the housing (222). The light source assembly (221) may have a structure that is integrally formed with the housing (222) or is separated from the housing (222).

[0459] The housing (222) may have an opening (222h). The housing (222) may be positioned adjacent to the opening (OP) in the lens barrel (210). For example, the opening (222h) in the housing (222) may be positioned in a manner corresponding to the opening (OP) in the lens barrel (210). Therefore, the opening (222h) in the housing (222) may overlap with the opening (OP) in the lens barrel (210) in the second direction (Y-axis direction).

[0460] The light source (223) may be positioned in the housing (222) or the light source assembly (221). The light source (223) may emit light. For example, the light from the light source (223) may be incident on the second light guide (LG2) in the housing (222). The second light guide (LG2) may be positioned in the housing (222). Therefore, the second light guide (LG2) may transmit the light emitted from the light source (223) to the opening (222h) or the first light guide (LG1).

[0461] The light source (223) may include one or more light sources. That is, in the light source device (220), the light source (223) may refer to a single light source, and may include a plurality of light sources. The light source (223) may include a plurality of light sources, such as a first light source (223a), a second light source (223b), and a third light source (223c). The first light source (223a) to the third light source (223c) may emit light in the same direction, or may emit light in different directions. For example, the first light source (223a) and the third light source (223c) may be positioned facing each other. The first light source (223a) and the third light source (223c) may be positioned to overlap each other in a first direction (X-axis direction). The second light guide (LG2) may be positioned between the first light source (223a) and the third light source (223c). Therefore, the second light guide (LG2) may overlap with the first light source (223a) and the third light source (223c). The second light source (223b) may be positioned between the first light source (223a) and the third light source (223c). The first light source (223a) to the third light source (223c) may emit light toward the second light guide (LG2). The second light source (223b) may overlap with the second light guide (LG2) in the second direction. With this configuration, the optical device (200) may have a compact light source device (220).

[0462] In addition, the first light source (223a), the second light source (223b) and the third light source (223c) can emit light of the same wavelength or color, or emit light of different wavelengths or different colors. For example, the first light source (223a), the second light source (223b) and the third light source (223c) can emit red light, green light and blue light respectively.

[0463] The second light guide (LG2) may include a second prism. The second prism serves as a reflective member and may include, for example, an X-prism. As an embodiment, the second light guide (LG2) or the second prism may have a structure in which at least two prisms are coupled to each other.

[0464] The second prism may have at least two coated surfaces (reflective member or reflective sheet). One of the at least two coated surfaces may reflect light of a first wavelength and light of a second wavelength and transmit light of a third wavelength. That is, the coated surface may reflect light in a predetermined wavelength band. Therefore, light in a desired wavelength band emitted from each of the plurality of light sources (223) may be reflected from the second light guide (LG2). For example, the light passing through the second light guide (LG2) may be provided to the first light guide (LG1) or the dielectric lens (MO).

[0465] The light source lens (224) may be positioned adjacent to the light source (223). For example, the light source lens (224) may include a plurality of light source lenses. The light source lens (224) may be positioned on a path of light emitted from each light source (223).

[0466] As an embodiment, the light source lens (224) may be positioned between the second light guide (LG2) and the light source (223). In addition, the light source lens (224) may include a plurality of light source lenses. The plurality of light source lenses (224) may be positioned between the second light guide (LG2) and the light source (223). Alternatively, the plurality of light source lenses (224) may be arranged in a manner corresponding to the plurality of light sources (223), respectively.

[0467] For example, the light source lens (224) may include a first light source lens (224a), a second light source lens (224b), and a third light source lens (224c). The first light source lens (224a) may be positioned between the first light source (223a) and the second light guide (LG2). The second light source lens (224b) may be positioned between the second light source (223b) and the second light guide (LG2). The third light source lens (224c) may be positioned between the third light source (223c) and the second light guide (LG2).

[0468] In addition, a plurality of light source lenses, such as a first light source lens (224a), a second light source lens (224b), and a third light source lens (224c), can be respectively located on a plurality of light sources, such as a first light source (223a), a second light source (223b), and a third light source (223c). For example, any one of the plurality of first light source lenses (224a) can be located on the first light source (223a) and coupled to the light source assembly (221). Another one of the plurality of first light source lenses (224a) can be located between the aforementioned first light source lens (224a) and the second light guide (LG2) and coupled to the housing (222).

[0469] The first light source lens (224a), the second light source lens (224b) and the third light source lens (224c) may include a collimating lens or a collimator.

[0470] In addition, a substrate (225) connected to the light source (223) may be arranged in the light source assembly (221). The substrate (225) may refer to a single substrate, or may include a plurality of substrates in a manner corresponding to the light source (223). For example, the plurality of substrates (225) may include a first substrate (225a), a second substrate (225b), and a third substrate (225c). As described above, the plurality of substrates (225) may form a single integrated substrate or correspond to a plurality of light sources.

[0471] In addition, the substrate (225) can be electrically connected to the control unit or processor (or display unit) in the above-mentioned frame. Therefore, the substrate (225) can include a connector for communicating with an external device or a connected device. In addition, the substrate (225) can be arranged outside the light source (223) and dissipate the heat generated in the light source to the outside. Therefore, the reliability of the light source device (220) can be improved.

[0472] The dielectric lens (MO) may be positioned between the opening (222h) and the second light guide (LG2). In addition, the dielectric lens (MO) may be positioned between the first light guide (LG1) and the second light guide (LG2).

[0473] The dielectric lens (MO) may include a plurality of lenses. For example, the dielectric lens (MO) may include a first dielectric lens (MO1) and a second dielectric lens (MO2). However, as described below, in the absence of the dielectric lens (MO), the light reflected from the second light guide (LG2) may also be directly provided to the first light guide (LG1).

[0474] The first dielectric lens (MO1) may have a first surface (MO1s1) adjacent to the first light guide (LG1) and a second surface (MO1s2) corresponding to the first surface (MO1s1). Light emitted from the light source (223) may pass through the second light guide (LG2) and then sequentially pass through the first surface (MO1s1) and the second surface (MO1s2).

[0475] The second surface (MO1s2) may be convex toward the second light guide (LG2). Alternatively, the second surface (MO1s2) may be concave toward the first light guide (LG1). In addition, the first surface (MO1s1) may be convex or concave toward the second light guide (LG2). For example, the first dielectric lens (MO1) may have the shape of a meniscus lens. Since the second surface (MO1s2) is convex toward the second light guide (LG2) in this manner, light can be converged while passing through the dielectric lens (MO). Therefore, the regularity or uniformity of the light provided to the optical signal generating unit (230) can be improved. In other words, the light incident on the optical signal generating unit (230) can be surface light. Then, the uniformity of the surface light can be improved. Therefore, the accuracy or resolution of the image signal or image emitted to the display unit by the optical device (200) according to the embodiment can be improved.

[0476] The second dielectric lens (MO2) may include a micro lens array (MLA). Therefore, surface illumination may be partially performed on the light passing through the second light guide (LG2).

[0477] Furthermore, the second dielectric lens (MO2) may have a different size than the second light guide (LG2). Additionally, the first dielectric lens (MO1) may have a different size than the second light guide (LG2). For example, the dielectric lens (MO2) may have a size larger than the size of the second light guide (LG2).

[0478] In addition, the positions of the plurality of lenses (L), the positions of the plurality of dielectric lenses (MO), and the positions of the plurality of light source lenses (224) can be maintained by spacers (SP). For example, a plurality of spaces (SP) respectively adjacent to the plurality of lenses (L) can exist in the lens barrel (210). In addition, the plurality of spaces (SP) can be arranged in a housing (222) or a light source assembly (221) of the light source device (220) in a manner adjacent to each of the dielectric lenses (MO) and the light source lenses (224). As an embodiment, the plurality of spaces (SP) can be arranged above or below the above-mentioned lenses, and thus fix or maintain the positions of the lenses.

[0479] The optical signal generating unit (230) may be located below the lower end of the lens barrel (210). The optical signal generating unit (230) may overlap with the lens (L) on the optical axis or in the first direction (X-axis direction).

[0480] The optical signal generating unit (230) may convert light incident on the first light guide (LG1), reflected, and then passing through the Nth lens into an optical signal including image information.

[0481] The optical signal generating unit (230) may reflect the light (first polarized light) reflected from the first light guide (LG1). The optical signal generating unit (230) may generate an optical signal including image information. That is, the light emitted from the optical signal generating unit (230) may be light including image information.

[0482] The optical signal generating unit (230) may include a liquid crystal on silicon (LCoS) device.

[0483] The LCOS device may have a structure in which liquid crystal is distributed between a silicon wafer (a thin circular substrate material for semiconductors) having a complementary metal oxide semiconductor (CMOS) array and an anti-reflection (AR) member coated with a transparent electrode (indium tin oxide (ITO)).

[0484] An alignment layer may be formed on top of the wafer (silicon wafer) and thus an initial liquid crystal orientation may be formed.

[0485] In addition, a reflective layer or a reflective electrode formed as an aluminum layer and having a high optical reflectivity can be positioned below the alignment layer. The reflective electrode can be positioned on top of the silicon wafer. A semiconductor array (CMOS array) can be formed on top of the silicon wafer. It is possible to transmit data signals to the semiconductor array through the panel.

[0486] According to the driving of the optical signal generating unit (230), the optical signal generating unit (230) can reflect at least a portion of the incident light. The optical signal generating unit (230) can reflect the light incident from the surface light source on a per-pixel basis. In addition, the intensity of the reflected light can also be adjusted according to the degree of modulation. For example, the optical signal generating unit (230) can modulate a portion of the first polarized light into the second polarized light. Therefore, the second polarized light generated by the modulation can pass through the first light guide (LG1) and the first lens (L1), and then be provided to the display unit.

[0487] That is, the optical signal generating unit (230) can modulate the delay of the light generated by the modulation, that is, the polarized light. The optical signal generating unit (230) can perform the delay of the first polarized light in various ways. That is, the electric field can be formed by adjusting the voltage (the voltage of the electrode) for each pixel. The degree of liquid crystal distortion can also be adjusted according to the adjusted voltage. For example, when the maximum voltage is applied, the light reflected from the optical signal generating unit (230) can be completely reflected from the first light guide (LG1). When the minimum voltage is applied, the light reflected from the optical signal generating unit (230) can all pass through the second light guide (LG1). However, the light reflected from the optical signal generating unit (230) can also propagate in the reverse direction according to the electric field. In addition, when the intermediate level voltage is applied, a part of the light can pass through the first light guide (LG1). That is, the intensity (e.g., brightness) of the light provided to the display unit can be approximately at an intermediate level.

[0488] In this way, the optical signal generated by the optical signal generating unit (230) can be transmitted to the first lens (L1) through the Nth lens (Ln) and the first light guide (LG1). In addition, at least a portion of the optical signal generated by the optical signal generating unit (230) can pass through the first lens (L1) and be incident on the display unit.

[0489] In addition, the transparent member (240) can also be arranged between the optical signal generating unit (230) and the Nth lens (Ln) (or the first light guide). The transparent member (240) can be a glass panel. The transparent member (240) can be coupled to the lens barrel (210) or the cover (CV). In addition, the transparent member (240) can be positioned above the optical signal generating unit (230). Therefore, it is easy to prevent foreign matter from being introduced into the optical signal generating unit (230). The transparent member (240) can have the same size as the optical signal generating unit (230) or a different size. In addition, the transparent member (240) can overlap with at least a portion of the optical signal generating unit (230) on the optical axis or in the first direction (X-axis direction).

[0490] In addition, the heat dissipation unit (HD) may extend from the outer surface of the housing (222) toward the frame (100). As described, the heat dissipation unit (HD) may be arranged on the outer surface of the housing (222) in a manner of contacting the substrate (225) exposed to the outside of the housing (222). At this time, the heat dissipation unit (HD) may not be positioned on the inner surface (200IS) of the optical device (200). That is, the heat dissipation unit (HD) may be positioned on a surface opposite to the inner surface (200IS) of the optical device (200). In addition, the heat dissipation unit (HD) may be positioned on a surface opposite to the inner surface (100IS) of the frame (100). That is, the heat dissipation unit (HD) may not be arranged along the inner surface (100IS) of the frame (100).

[0491] The heat dissipation unit (HD) may extend along the outer surface of the housing (222) so as to advance toward the frame at the front side. In addition, the heat dissipation unit (HD) may extend inwardly from the frame along the edge of the glass lens or lens (GL).

[0492] In addition, the heat dissipation unit (HD) may extend adjacent to the optical signal generating unit (230). For example, the heat dissipation unit (HD) may extend from the light source device (220) to below the optical signal generating unit (230). Therefore, the heat generated by the optical signal generating unit (230) and by the light source may be easily dissipated to the outside. At this time, as described above, the heat dissipation unit (HD) may extend along the surface opposite to the inner surface (200IS) of the optical device, i.e., the outer surface of the optical device. Therefore, this configuration ensures that the wearer does not feel the heat. That is, the heat dissipation unit (HD) may extend upward from below the optical signal generating unit (230) along the outer surface of the optical device (200), i.e., along the side of the optical device.

[0493] Further references Fig.41, the lights (La, Lb and Lc) emitted from the light source (223) can pass through the light source lens (224), the second light guide (LG2) and the dielectric lens (MO). For example, the first light (La), the second light (Lb) and the third light (Lc) emitted from the first light source (223a) to the third light source (223b) can be emitted from the second light guide (LG2) in the same direction.

[0494] For example, the second light guide (LG2) may have a first coated surface (LG2a) and a second coated surface (LG2b). As described above, at least one of the two coated surfaces may reflect one or more of: light of a first wavelength, light of a second wavelength, or light of a third wavelength. For example, the first wavelength includes a wavelength band of red light. The second wavelength includes a wavelength band of green light. The third wavelength includes a wavelength band of blue light.

[0495] In addition, the first coated surface (LG2a) can reflect the first light (La) or the light of the first wavelength. That is, the first coated surface (LG2a) can transmit the second light (Lb) and the third light (Lc). In other words, the first coated surface (LG2a) can transmit the light of the second wavelength and the light of the third wavelength.

[0496] The second coated surface (LG2b) can reflect the second light (Lb) or the light of the second wavelength. That is, the second coated surface (LG2b) can transmit the first light (La) and the third light (Lc). In other words, the second coated surface (LG2b) can transmit the light of the first wavelength and the light of the third wavelength.

[0497] Therefore, the first light (La) can be reflected from the second light guide (LG2) in a manner of propagating toward the dielectric lens (MO) or the opening (OP) or incident on the dielectric lens (MO) or the opening (OP). The second light (Lb) can be reflected from the second light guide (LG2) in a manner of propagating toward the dielectric lens (MO) or the opening (OP) or incident on the dielectric lens (MO) or the opening (OP). In addition, the third light (Lc) can be reflected from the second light guide (LG2) in a manner of propagating toward the dielectric lens (MO) or the opening (OP) or incident on the dielectric lens (MO) or the opening (OP).

[0498] Therefore, the light (IL) emitted from the light source (223) (or the light indicated by the combination of La, Lb and Lc) may be incident on the first light guide (LG). At this time, the light incident on the first light guide (LG) may be the first incident light (IL).

[0499] Further references Fig.42, a portion of the first incident light (IL) may be reflected from the first light guide (LG1), and the remaining portion may pass through the first light guide (LG1). That is, the first light guide (LG1) may reflect the first polarized light (ILa) of the first incident light (IL) and transmit the second polarized light (ILb) thereof. For example, the first polarized light (ILa) and the second polarized light (ILb) may each be S polarized light or P polarized light. Therefore, the first polarized light (ILa) as a portion of the first incident light (IL) may be provided to the optical signal generating unit (230) through the Nth lens (Ln). The second polarized light (ILb) may be absorbed into the lens barrel (210) or provided to the additional hole (210h).

[0500] Fig.43 is a side view showing an electronic device according to another embodiment. Fig.44 is a plan view showing an electronic device according to the same embodiment. Fig.45 is a side view showing an electronic device according to still another embodiment. Fig.46 1 is a plan view showing an electronic device according to the same embodiment. Except for the details described below, the corresponding above descriptions of the frame, the optical device, the display unit, and the heat dissipation unit may be applied in the same manner.

[0501] Reference Fig.43 and Fig.44 , the electronic device according to the embodiment may further include a heating unit (HL) arranged on the frame (100).

[0502] The heating unit (HL) may be positioned within the frame (100). That is, similar to the heat dissipation unit (HD), the heating unit (HL) within the frame (100) may surround the edge of the lens (GL). Therefore, fogging or the like may be more effectively prevented.

[0503] The heating unit (HL) may also be electrically connected to the optical device (200). In addition, the heating unit (HL) may be electrically connected to a controller or control unit in the electronic device. Alternatively, the heating unit (HL) may be electrically connected to a controller or control unit in the optical device.

[0504] The heat dissipation unit (HD) and the heating unit (HL) may be sealed with a sealing member (e.g., epoxy resin) to reduce the influence of external moisture. In addition, in the case where fogging occurs on the lens, the heating unit (HL) may compensate for insufficient heat when the fogging is not sufficiently removed by the heat dissipation unit (HD).

[0505] In addition, the heating unit (HL) and the heat dissipation unit (HD) can be arranged to be spaced apart from each other. Therefore, the influence of the heat generated by the heating unit (HL) on the heat dissipation unit (HD) can be minimized. That is, the heat dissipation unit (HD) can efficiently dissipate the heat of the light source and the substrate, rather than the heat generated by the heating unit (HL).

[0506] For example, although the heating unit and the heat dissipation unit (HD) are spaced apart from each other, they may extend in parallel within the frame. Therefore, the occurrence of the fogging phenomenon may be more effectively prevented.

[0507] Further references Fig.45 and Fig.46 As described above, the heating unit (HL) may be positioned within the frame (100). That is, similar to the heat dissipation unit (HD), the heating unit (HL) within the frame (100) may surround the edge of the lens (GL). Therefore, fogging or the like may be more effectively prevented.

[0508] The heating unit (HL) may also be electrically connected to the optical device (200). In addition, the heating unit (HL) may be electrically connected to a controller or control unit in the electronic device. Alternatively, the heating unit (HL) may be electrically connected to a controller or control unit in the optical device.

[0509] The heat dissipation unit (HD) and the heating unit (HL) may be sealed with a sealing member (e.g., epoxy resin) to reduce the influence of external moisture. In addition, in the case where fogging occurs on the lens, the heating unit (HL) may compensate for insufficient heat when the fogging is not sufficiently removed by the heat dissipation unit (HD).

[0510] In addition, the heating unit (HL) and the heat dissipation unit (HD) may be arranged to be spaced apart from each other. Therefore, it is possible to minimize the influence of heat generated by the heating unit (HL) on the heat dissipation unit (HD).

[0511] In this embodiment, the frame (100) may include an upper region (US) and a lower region (BS). The lens (GL) may be divided into two halves with respect to the upper region (US) and the lower region (BS). At this time, one of the heat dissipation unit (HD) and the heating unit (HL) may be arranged in the upper region (US) or the lower region (BS). The remaining units may be arranged in another region. For example, the heat dissipation unit (HD) may be positioned in the upper region (US). The heating unit (HL) may be positioned in the lower region (BS).

[0512] In addition, as a modified example, the heat dissipation unit (HD) may be arranged in the frame (100) in a manner surrounding the lens (GL) along the edge of the lens (GL), and the heating unit (HL) may be positioned in one area of ​​the frame (100). Therefore, in order to effectively remove the fogging phenomenon, the heating unit (HL) may be relatively mainly positioned in the lower area or the upper area.

[0513] With this configuration, energy efficiency can be achieved, and fogging can be effectively suppressed.

[0514] The operating method according to the disclosed embodiment can be implemented in the form of program instructions that can be executed by various computer devices, and thus can be recorded on a computer-readable medium. In addition, an embodiment of the present disclosure can be a computer-readable recording medium having recorded thereon one or more programs containing instructions for executing the operating method.

[0515] Program instructions, data files, data structures, etc. are recorded on computer-readable media, either individually or in combination. The program instructions recorded on the medium may be program instructions specifically designed and configured for the present invention, or program instructions known and available to those of ordinary skill in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVD drives, magneto-optical media such as floppy disks, and hardware devices such as ROMs, RAMs, and flash memories, which are specifically configured to store program instructions to be executed. Examples of program instructions include machine language codes, such as those generated by a compiler, and high-level language codes that can be executed by a computer using an interpreter or similar tool.

[0516] Computer-readable storage media may be provided in the form of non-transitory storage media. "Non-transitory storage media" refers to tangible media that do not include signals (e.g., electromagnetic waves). The term is used ambiguously to refer to storage media on which data is semi-permanently stored and storage media on which data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0517] As an example, the operating methods according to the various embodiments described in this specification may be provided in the form of a computer program product. The computer program product may be sold as a commercial product. The computer program product may be distributed in the form of a computer-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be downloaded through an application store (e.g., PlayStore). TM) online distribution (e.g., download or upload) or distribution between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable application) may be at least temporarily stored or temporarily generated in a computer-readable storage medium such as a memory, in a server located at a manufacturing company, in a server located at an application store, or in a relay server.

[0518] Specifically, at least a part of the computer program product may be implemented in the form of a computer program product including a recording medium on which a program for executing the operating method according to the disclosed embodiment is recorded.

[0519] The above describes the embodiments in detail, but does not impose limitations on the scope of the invention claimed. Various modifications and improvements made by ordinary technicians in this field using the basic concept of the invention defined in the claims are also included in the scope of the invention claimed.

[0520] The term "unit" constituting the name of each component element described in the present embodiment refers to a software component element or a hardware component element such as a field programmable gate array (FPGA) or an ASIC, and performs a specific function. However, the term "unit" is not limited to a software component element or a hardware component element. The component element whose name includes the term "unit" can also be configured to reside in an addressable storage medium, and can also be configured to enable one or more processors to perform processing. Therefore, the example of the component element whose name includes the term "unit" includes components such as software component elements, object-oriented software component elements, class component elements, task component elements, processing, functions, attributes, processes, subroutines, program code segments, drivers, firmware, microcodes, circuits, data, databases, data structures, tables, arrays, and variable components. Multiple components can be functionally combined into a smaller number of components, and the components can be further divided into additional components functionally. A plurality of constituent elements whose names include the term “unit” may be functionally combined into a smaller number of constituent elements whose names include the term “unit”, and a constituent element whose name includes the term “unit” may be further functionally divided into additional constituent elements whose names include the term “unit”. Furthermore, constituent elements or constituent elements whose names include the term “unit” may also be implemented in a manner that enables one or more CPUs or secure multimedia cards within a device to perform processing.

[0521] Although the present invention has been described above by focusing on its embodiments, the present invention is not limited to these embodiments. It is obvious to those skilled in the art that various modifications and applications of these embodiments are possible without departing from the scope of its essential features. For example, each component element specifically adopted in the embodiment can be realized in a modified manner. Differences related to these modifications and applications should be interpreted as falling within the scope of the present invention defined by the claims.

Claims

1. An optical device, comprising: Multiple lenses; a lens barrel, wherein the plurality of lenses are arranged in the lens barrel; a first light guide disposed in the lens barrel; an opening portion formed in a side surface of the lens barrel; a light source device, the light source device being coupled to the opening; a through hole formed in a side surface of the lens barrel; as well as A light receiving unit is disposed adjacent to the through hole.

2. The optical device according to claim 1, comprising: A barrier unit is arranged between the through hole and the light receiving unit.

3. The optical device according to claim 1, comprising: an optical signal generating unit, the optical signal generating unit generating an optical signal including image information, Wherein, the light receiving unit is arranged between the first light guide and the optical signal generating unit.

4. The optical device according to claim 3, wherein: The plurality of lenses include a first lens and a second lens disposed between the first light guide and the optical signal generating unit, and the first lens has a smaller size than the second lens.

5. The optical device according to claim 4, wherein: The light receiving unit is disposed between the first lens and the second lens.

6. The optical device according to claim 4, wherein: The through hole is disposed between the first lens and the second lens.

7. The optical device according to claim 1, wherein: The light receiving unit protrudes further than an outer surface of the lens barrel.

8. The optical device according to claim 2, wherein: The barrier unit protrudes further than an outer surface of the lens barrel.

9. The optical device according to claim 1, wherein: The light source device comprises: a housing having an opening formed therein; a second light guide disposed in the housing; and A light source emits light toward the second light guide.

10. The optical device according to claim 9, wherein: The light receiving unit is driven while light is emitted from the light source.

Images