Optical guide device and electronic device including same

Abstract

Provided are an optical guide device and an electronic device including same. The optical guide device is used in augmented reality (AR) in which a grid structure with degrees of freedom for all angles of an optical device and an observer can be designed, the optical guide device comprising: a first substrate; a first input diffractive element which is arranged on the first substrate and successively struck with light emitted from a projector; and a first transmission diffraction element and a first output diffraction element.

Description

Light guide device and electronic device including the same

[0001] The embodiment relates to a light guide device and an electronic device including the same.

[0002] Virtual Reality (VR) refers to a specific environment or situation, or the technology itself, that is similar to reality but not real, created using artificial technology such as computers.

[0003] Augmented Reality (AR) is a technology that synthesizes virtual objects or information into the real environment to make them appear as objects that exist in the original environment.

[0004] Mixed reality (MR) or hybrid reality refers to the creation of new environments or information by merging the virtual and real worlds. In particular, it refers to real-time interaction between real and virtual worlds.

[0005] At this time, the created virtual environments and situations stimulate the user's five senses, allowing them to experience spatial and temporal experiences similar to reality, freely moving between reality and imagination. Furthermore, users can not only immerse themselves in these environments, but also interact with the objects embodied within them, using real devices to control and command them.

[0006] Recently, active research has been conducted on the gear and devices used in these fields. However, the need for miniaturization and improved optical performance of these devices is emerging.

[0007] In addition, traffic accidents caused by drivers while driving are increasing significantly along with the increase in vehicles, and there is a problem that the occurrence rate of these driver-caused accidents increases significantly depending on the physical condition of the driver.

[0008] In particular, the number of accidents caused by distracted driving, such as smartphone use while driving, has been increasing significantly. To address this growing problem of traffic accidents caused by driver inattention or drowsiness, automakers have recently been installing Driver Monitoring Systems (DMS) in their vehicles to monitor drivers' concentration and physical condition.

[0009] DMS was generally implemented in a form that simply predicted the driver's condition based on the driver's steering wheel operation status or the steering wheel operation status considering the driving time and alerted the driver with a warning sound or warning message. However, recently, with the development of intelligent driver assistance systems (ADAS: Advanced Driver Assistance Systems) that provide various functions that directly assist the driver's driving, such as ACC (Adaptive Cruise Control) that automatically maintains the distance from the car in front according to the speed set by the driver and drives, and LKAS (Lane Keeping Assist System) that recognizes the lane with a camera and helps maintain the lane, various methods are being implemented, such as using a camera to capture the driver's physical movement through the video to determine the driver's concentration state or physical state such as drowsiness, or monitoring the movement of the driver's eyelids to determine the driver's concentration state or drowsiness, and vibrating the steering wheel or warning sound to the driver to avoid danger, or slowing down the vehicle below a certain speed.

[0010] Meanwhile, as the autonomous driving performance of automobiles becomes more advanced, the DMS does not necessarily require the driver to hold the steering wheel, and as the driver's hands-off state occurs more frequently, a driver monitoring system with a hands-off monitoring function is required. As a result, an advanced DMS that can determine the driver's concentration state or drowsiness based on the driver's various movements due to the driver's hands-off is required.

[0011] However, as drivers and others do not prefer to have themselves filmed or monitored, the need for technology that does not make drivers aware of such surveillance is also increasing.

[0012] The embodiment provides a light guide device and an electronic device including the same, which can design a grid structure having degrees of freedom for all angles of an optical device and an observer, when using the light guide device used in AR (Augmented Reality) and the like.

[0013] The embodiments provide a light guide device and an electronic device with improved optical performance.

[0014] The embodiment provides a light guide device and a camera module that enable easier creation of images of passengers, etc., within a vehicle, etc.

[0015] The problem to be solved in the embodiment is not limited to this, and it can be said that the purpose or effect that can be understood from the solution or embodiment of the problem described below is also included.

[0016] A light guide device according to an embodiment includes a first substrate; and a first input diffractive element, a first transmission diffractive element, and a first output diffractive element, which are arranged on the first substrate and onto which light emitted from a projector is sequentially incident; and can satisfy mathematical expressions 1 and 2.

[0017] [Mathematical Formula 1]

[0018]

[0019] [Equation 2]

[0020]

[0021] (Here θ projector is the angle between the projector and the first substrate, is the angle at which the projector faces the first substrate, γ is the wrap angle of the first substrate, λ is the wavelength of light, α is the pantoscopic angle of the first substrate, Λ IC is the grating period of the first input diffractive element, Λ OC is the grating period of the first-order diffractive element, Λ FG is the grating period of the first transmission diffraction element, is the grating angle of the first input diffractive element, is the grating angle of the first diffractive element, is the grating angle of the first transmission diffraction element, FoV is the field of view of the light guide device, x and y are the resolutions of the light guide device)

[0022] The wrap angle of the first substrate is an angle formed by the first substrate with respect to a first direction, and the first direction may be a direction perpendicular to a direction in which a user views the light guide device.

[0023] The pantospheric angle of the first substrate is an angle formed by the first substrate with respect to a second direction, and the second direction may be a direction perpendicular to the first direction and a direction in which the user views the light guide device.

[0024] The angle between the projector and the first substrate may be an angle formed between the direction in which the projector outputs the light and a plane perpendicular to the first substrate.

[0025] The angle at which the projector faces the first substrate is the angle formed by the direction in which the projector outputs the light with respect to the first axis on the first substrate, and the first axis may be the long axis of the first substrate.

[0026] The first input diffractive element may include a first protrusion, the first transmission diffractive element may include a second protrusion, the first output diffractive element may include a third protrusion, and the grating period of the first input diffractive element may be the shortest distance between the same side surfaces of adjacent first protrusions, the grating period of the first transmission diffractive element may be the shortest distance between the same side surfaces of adjacent second protrusions, and the grating period of the first output diffractive element may be the shortest distance between the same side surfaces of adjacent third protrusions.

[0027] The grating angle of the first input diffractive element may be an angle formed by the separation direction between the first protrusions of the first input diffractive element and the first axis, the grating angle of the first transmission diffractive element may be an angle formed by the separation direction between the second protrusions of the first transmission diffractive element and the first axis, and the grating angle of the first output diffractive element may be an angle formed by the separation direction between the third protrusions of the first output diffractive element and the first axis.

[0028] The second substrate may further include: a second substrate arranged to overlap the first substrate; a second input diffraction element, a second transmission diffraction element, and a second output diffraction element arranged on the second substrate, onto which light is sequentially incident;

[0029] The grating angles of the first input diffractive element and the second input diffractive element may be the same, the grating angles of the first transmission diffractive element and the second transmission diffractive element may be the same, and the grating angles of the first output diffractive element and the second output diffractive element may be the same.

[0030] The viewing angle of the above light guide device may be the angle at which the light is incident on the first input diffraction element and transmitted to the first transmission diffraction element.

[0031] The grating periods of the second input diffractive element, the second transmission diffractive element, and the second output diffractive element may be different from the grating periods of the first input diffractive element, the first transmission diffractive element, and the first output diffractive element, respectively.

[0032] The light guide device according to the embodiment can satisfy mathematical expressions 3 and 4.

[0033] [Equation 3]

[0034]

[0035] [Equation 4]

[0036]

[0037] (Here θ projector is the angle between the projector and the second substrate, is the angle at which the projector faces the second substrate, γ is the wrap angle of the second substrate, λ is the wavelength of light, α is the pantoscopic angle of the second substrate, Λ IC is the grating period of the second input diffractive element, Λ OC is the grating period of the second-order diffractive element, Λ FG is the grating period of the second transmission diffraction element, is the grating angle of the second input diffractive element, is the grating angle of the second diffractive element, is the grating angle of the second transmission diffraction element, FoV is the field of view of the light guide device, x and y are the resolutions of the light guide device)

[0038] The wrap angle of the first substrate and the wrap angle of the second substrate may be the same.

[0039] The pantospheric angle of the first substrate and the pantospheric angle of the second substrate may be the same.

[0040] The light sequentially incident on the first input diffraction element, the first transmission diffraction element, and the first output diffraction element may have a different wavelength from the light sequentially incident on the second input diffraction element, the second transmission diffraction element, and the second output diffraction element.

[0041] It is possible to provide a light guide device and an electronic device capable of designing a grating structure having degrees of freedom for all angles of the optical device and the observer.

[0042] In addition, a light guide device and electronic device with improved optical performance can be provided.

[0043] In addition, a light guide device and a camera module can be provided that enable easier creation of images of passengers, etc., within a vehicle.

[0044] The various advantageous and beneficial effects of the present invention are not limited to the above-described contents, and will be more easily understood in the course of explaining specific embodiments of the present invention.

[0045] Figure 1 is a conceptual diagram showing an embodiment of an AI device,

[0046] FIG. 2 is a block diagram showing the configuration of an extended reality electronic device according to an embodiment of the present invention.

[0047] Figure 3 is a perspective view of an augmented reality electronic device according to an embodiment of the present invention;

[0048] Figure 4 is a schematic diagram of a light guide device according to an embodiment of the present invention.

[0049] FIG. 5 is a drawing showing the angle of the substrate of the light guide device according to an embodiment of the present invention;

[0050] FIG. 6 is a drawing showing the angle between the projector and the substrate of the light guide device according to an embodiment of the present invention.

[0051] FIG. 7 is a drawing showing a grating pattern of a diffraction element of a light guide device according to an embodiment of the present invention.

[0052] Figure 8 is a schematic diagram of a light guide device according to another embodiment of the present invention.

[0053] FIG. 9 is a drawing showing the configuration of a camera module according to an embodiment of the present invention.

[0054] FIG. 10 is a drawing showing the angle of the substrate of the light guide device according to an embodiment of the present invention.

[0055] Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the attached drawings.

[0056] However, the technical idea of ​​the present invention is not limited to some of the embodiments described, but can be implemented in various different forms, and within the scope of the technical idea of ​​the present invention, one or more of the components between the embodiments can be selectively combined or substituted for use.

[0057] In addition, terms (including technical and scientific terms) used in the embodiments of the present invention may be interpreted as having a meaning that can be generally understood by a person of ordinary skill in the technical field to which the present invention belongs, unless explicitly and specifically defined and described, and terms that are commonly used, such as terms defined in a dictionary, may be interpreted in consideration of the contextual meaning of the relevant technology.

[0058] Additionally, the terms used in the embodiments of the present invention are intended to describe the embodiments and are not intended to limit the present invention.

[0059] In this specification, the singular may also include the plural unless specifically stated otherwise in the phrase, and when it is described as “A and / or at least one (or more) of B, C”, it may include one or more of all combinations that can be combined with A, B, C.

[0060] Additionally, in describing components of embodiments of the present invention, terms such as first, second, A, B, (a), (b), etc. may be used.

[0061] These terms are intended only to distinguish one component from another, and are not intended to limit the nature, order, or sequence of the component.

[0062] And, when a component is described as being 'connected', 'coupled' or 'connected' to another component, it may include not only cases where the component is directly connected, coupled or connected to the other component, but also cases where the component is 'connected', 'coupled' or 'connected' by another component between the component and the other component.

[0063] Additionally, when described as being formed or arranged "above or below" each component, "above" or "below" includes not only cases where the two components are in direct contact with each other, but also cases where one or more other components are formed or arranged between the two components. Furthermore, when expressed as "above" or "below", it can include the meaning of a downward direction as well as an upward direction based on one component.

[0064] Figure 1 is a conceptual diagram illustrating an embodiment of an AI device.

[0065] Referring to FIG. 1, an AI system is connected to a cloud network (10) by at least one of an AI server (16), a robot (11), an autonomous vehicle (12), an XR device (13), a smartphone (14), or an appliance (15). Here, a robot (11), an autonomous vehicle (12), an XR device (13), a smartphone (14), or an appliance (15) to which AI technology is applied may be referred to as an AI device (11 to 15).

[0066] A cloud network (10) may refer to a network that constitutes part of a cloud computing infrastructure or exists within a cloud computing infrastructure. Here, the cloud network (10) may be configured using a 3G network, a 4G or LTE (Long Term Evolution) network, a 5G network, etc.

[0067] That is, each device (11 to 16) constituting the AI ​​system can be connected to each other through a cloud network (10). In particular, each device (11 to 16) can communicate with each other through a base station, but can also communicate with each other directly without going through a base station.

[0068] The AI ​​server (16) may include a server that performs AI processing and a server that performs operations on big data.

[0069] The AI ​​server (16) is connected to at least one of the AI ​​devices constituting the AI ​​system, such as a robot (11), an autonomous vehicle (12), an XR device (13), a smartphone (14), or a home appliance (15), through a cloud network (10), and can assist at least part of the AI ​​processing of the connected AI devices (11 to 15).

[0070] At this time, the AI ​​server (16) can train an artificial neural network according to a machine learning algorithm on behalf of the AI ​​devices (11 to 15), and can directly store the learning model or transmit it to the AI ​​devices (11 to 15).

[0071] At this time, the AI ​​server (16) can receive input data from the AI ​​devices (11 to 15), infer a result value for the received input data using a learning model, and generate a response or control command based on the inferred result value and transmit it to the AI ​​devices (11 to 15).

[0072] Alternatively, the AI ​​device (11 to 15) may infer a result value for input data using a direct learning model and generate a response or control command based on the inferred result value.

[0073] <AI+로봇>

[0074] Robots (11) can be implemented as guide robots, transport robots, cleaning robots, wearable robots, entertainment robots, pet robots, unmanned flying robots, etc. by applying AI technology.

[0075] The robot (11) may include a robot control module for controlling movement, and the robot control module may mean a software module or a chip that implements the same as hardware.

[0076] The robot (11) can obtain status information of the robot (11), detect (recognize) the surrounding environment and objects, generate map data, determine a movement path and driving plan, determine a response to user interaction, or determine an action using sensor information obtained from various types of sensors.

[0077] Here, the robot (11) can use sensor information acquired from at least one sensor among lidar, radar, and camera to determine a movement path and driving plan.

[0078] The robot (11) can perform the above-described operations using a learning model comprised of at least one artificial neural network. For example, the robot (11) can recognize its surroundings and objects using the learning model, and determine operations using the recognized surrounding environment information or object information. Here, the learning model may be learned directly by the robot (11) or by an external device such as an AI server (16).

[0079] At this time, the robot (11) may perform an action by generating a result using a direct learning model, but it may also perform an action by transmitting sensor information to an external device such as an AI server (16) and receiving the result generated accordingly.

[0080] The robot (11) can determine a movement path and a driving plan using at least one of map data, object information detected from sensor information, or object information acquired from an external device, and control a driving unit to drive the robot (11) according to the determined movement path and driving plan.

[0081] Map data may include object identification information for various objects positioned in the space where the robot (11) moves. For example, map data may include object identification information for fixed objects such as walls and doors, as well as movable objects such as flower pots and desks. Furthermore, object identification information may include name, type, distance, location, etc.

[0082] Additionally, the robot (11) can perform actions or drive by controlling the driving unit based on the user's control / interaction. At this time, the robot (11) can acquire intention information regarding the interaction based on the user's actions or voice utterances, and determine a response based on the acquired intention information to perform the action.

[0083] <AI+자율주행>

[0084] Autonomous vehicles (12) can be implemented as mobile robots, vehicles, unmanned aerial vehicles, etc. by applying AI technology.

[0085] An autonomous vehicle (12) may include an autonomous driving control module for controlling autonomous driving functions. The autonomous driving control module may refer to a software module or a chip implementing the same as hardware. The autonomous driving control module may be included internally as a component of the autonomous vehicle (12), but may also be configured as separate hardware and connected to the exterior of the autonomous vehicle (12).

[0086] An autonomous vehicle (12) can obtain status information of the autonomous vehicle (12), detect (recognize) the surrounding environment and objects, generate map data, determine a movement path and driving plan, or determine an action by using sensor information obtained from various types of sensors.

[0087] Here, the autonomous vehicle (12) can use sensor information acquired from at least one sensor among lidar, radar, and camera, similar to the robot (11), to determine the movement path and driving plan.

[0088] In particular, an autonomous vehicle (12) can recognize an environment or object in an area where the field of vision is obscured or an area beyond a certain distance by receiving sensor information from external devices, or can receive information recognized directly from external devices.

[0089] An autonomous vehicle (12) can perform the above-described operations using a learning model comprised of at least one artificial neural network. For example, the autonomous vehicle (12) can recognize its surroundings and objects using the learning model, and determine a driving route using the recognized surrounding environment information or object information. Here, the learning model may be learned directly by the autonomous vehicle (12) or by an external device such as an AI server (16).

[0090] At this time, the autonomous vehicle (12) may perform an action by generating a result using a direct learning model, but may also perform an action by transmitting sensor information to an external device such as an AI server (16) and receiving the result generated accordingly.

[0091] An autonomous vehicle (12) can determine a movement path and a driving plan using at least one of map data, object information detected from sensor information, or object information acquired from an external device, and control a driving unit to drive the autonomous vehicle (12) according to the determined movement path and driving plan.

[0092] Map data may include object identification information for various objects located in the space (e.g., road) where the autonomous vehicle (12) travels. For example, map data may include object identification information for fixed objects such as streetlights, rocks, and buildings, as well as movable objects such as vehicles and pedestrians. Furthermore, object identification information may include name, type, distance, location, and the like.

[0093] Additionally, the autonomous vehicle (12) can perform actions or drive by controlling the driving unit based on the user's control / interaction. At this time, the autonomous vehicle (12) can acquire intention information regarding the interaction based on the user's actions or voice utterances, and determine a response based on the acquired intention information to perform the action.

[0094] <AI+XR>

[0095] The XR device (13) can be implemented as an HMD (Head-Mount Display), a HUD (Head-Up Display) equipped in a vehicle, a television, a mobile phone, a smart phone, a computer, a wearable device, a home appliance, digital signage, a vehicle, a fixed robot, or a mobile robot by applying AI technology.

[0096] The XR device (13) can obtain information about the surrounding space or real objects by analyzing 3D point cloud data or image data acquired through various sensors or from an external device to generate location data and attribute data for 3D points, and can render and output an XR object to be output. For example, the XR device (13) can output an XR object including additional information about a recognized object in correspondence with the recognized object.

[0097] The XR device (13) can perform the above-described operations using a learning model composed of at least one artificial neural network. For example, the XR device (13) can recognize a real-world object from 3D point cloud data or image data using the learning model, and provide information corresponding to the recognized real-world object. Here, the learning model may be learned directly in the XR device (13) or learned from an external device such as an AI server (16).

[0098] At this time, the XR device (13) may perform an action by generating a result using a direct learning model, but may also perform an action by transmitting sensor information to an external device such as an AI server (16) and receiving the result generated accordingly.

[0099] <AI+로봇+자율주행>

[0100] Robots (11) can be implemented as guide robots, transport robots, cleaning robots, wearable robots, entertainment robots, pet robots, unmanned flying robots, etc. by applying AI technology and autonomous driving technology.

[0101] A robot (11) to which AI technology and autonomous driving technology are applied may refer to a robot itself with autonomous driving functions, or a robot (11) that interacts with an autonomous vehicle (12).

[0102] A robot (11) with autonomous driving function can be a general term for devices that move on their own along a given path without user control or move by determining the path on their own.

[0103] A robot (11) and a self-driving vehicle (12) with autonomous driving capabilities may use a common sensing method to determine one or more of a movement path or a driving plan. For example, a robot (11) and a self-driving vehicle (12) with autonomous driving capabilities may use information sensed through lidar, radar, and cameras to determine one or more of a movement path or a driving plan.

[0104] A robot (11) interacting with an autonomous vehicle (12) may exist separately from the autonomous vehicle (12), and may be linked to autonomous driving functions inside or outside the autonomous vehicle (12), or may perform actions linked to a user riding in the autonomous vehicle (12).

[0105] At this time, the robot (11) interacting with the autonomous vehicle (12) can control or assist the autonomous driving function of the autonomous vehicle (12) by acquiring sensor information on behalf of the autonomous vehicle (12) and providing it to the autonomous vehicle (12), or by acquiring sensor information and generating surrounding environment information or object information and providing it to the autonomous vehicle (12).

[0106] Alternatively, a robot (11) interacting with an autonomous vehicle (12) may monitor a user riding in the autonomous vehicle (12) or control the functions of the autonomous vehicle (12) through interaction with the user. For example, if the robot (11) determines that the driver is drowsy, it may activate the autonomous driving function of the autonomous vehicle (12) or assist in controlling the driving unit of the autonomous vehicle (12). Here, the functions of the autonomous vehicle (12) controlled by the robot (11) may include not only the autonomous driving function, but also functions provided by a navigation system or audio system installed inside the autonomous vehicle (12).

[0107] Alternatively, a robot (11) interacting with an autonomous vehicle (12) may provide information to the autonomous vehicle (12) or assist functions from outside the autonomous vehicle (12). For example, the robot (11) may provide traffic information, including signal information, to the autonomous vehicle (12), such as a smart traffic light, or may interact with the autonomous vehicle (12) to automatically connect an electric charger to a charging port, such as an automatic electric charger for an electric vehicle.

[0108] <AI+로봇+XR>

[0109] Robots (11) can be implemented as guide robots, transport robots, cleaning robots, wearable robots, entertainment robots, pet robots, unmanned flying robots, drones, etc. by applying AI technology and XR technology.

[0110] A robot (11) to which XR technology is applied may refer to a robot that is the subject of control / interaction within an XR image. In this case, the robot (11) is distinct from the XR device (13) and can be linked with each other.

[0111] When a robot (11) that is the target of control / interaction within an XR image obtains sensor information from sensors including a camera, the robot (11) or the XR device (13) can generate an XR image based on the sensor information, and the XR device (13) can output the generated XR image. In addition, the robot (11) can operate based on a control signal input through the XR device (13) or a user's interaction.

[0112] For example, a user can check an XR image corresponding to the viewpoint of a remotely connected robot (11) through an external device such as an XR device (13), and through interaction, adjust the autonomous driving path of the robot (11), control the operation or driving, or check information on surrounding objects.

[0113] <AI+자율주행+XR>

[0114] Autonomous vehicles (12) can be implemented as mobile robots, vehicles, unmanned aerial vehicles, etc. by applying AI technology and XR technology.

[0115] An autonomous vehicle (12) to which XR technology is applied may refer to an autonomous vehicle equipped with a means for providing XR images, an autonomous vehicle that is the subject of control / interaction within an XR image, etc. In particular, an autonomous vehicle (12) that is the subject of control / interaction within an XR image is distinct from an XR device (13) and can be linked with each other.

[0116] An autonomous vehicle (12) equipped with a means for providing XR images can acquire sensor information from sensors including cameras and output XR images generated based on the acquired sensor information. For example, the autonomous vehicle (12) can be equipped with a HUD to output XR images, thereby providing passengers with XR objects corresponding to real objects or objects on the screen.

[0117] At this time, when the XR object is output to the HUD, at least a part of the XR object may be output so as to overlap with an actual object toward which the passenger's gaze is directed. On the other hand, when the XR object is output to a display provided inside the autonomous vehicle (12), at least a part of the XR object may be output so as to overlap with an object on the screen. For example, the autonomous vehicle (12) may output XR objects corresponding to objects such as a road, another vehicle, a traffic light, a traffic sign, a two-wheeled vehicle, a pedestrian, a building, etc.

[0118] When an autonomous vehicle (12) that is the target of control / interaction within an XR image acquires sensor information from sensors including a camera, the autonomous vehicle (12) or the XR device (13) generates an XR image based on the sensor information, and the XR device (13) can output the generated XR image. In addition, the autonomous vehicle (12) can operate based on a control signal input through an external device such as the XR device (13) or a user's interaction.

[0119] [Augmented Reality Technology]

[0120] Extended Reality (XR) is a general term for Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). VR technology provides real-world objects and backgrounds only as CG images, AR technology provides virtually created CG images on top of images of real objects, and MR technology is a computer graphics technology that provides virtual objects mixed and combined in the real world.

[0121] MR technology is similar to AR in that it presents both real and virtual objects simultaneously. However, while AR uses virtual objects to complement real objects, MR uses virtual and real objects on an equal footing.

[0122] XR technology can be applied to HMD (Head-Mount Display), HUD (Head-Up Display), mobile phones, tablet PCs, laptops, desktops, TVs, digital signage, etc., and devices to which XR technology is applied can be called XR devices.

[0123] Below, an electronic device providing augmented reality according to an embodiment of the present invention will be described. In particular, a light guide device applicable to augmented reality and an electronic device including the same will be described in detail.

[0124] FIG. 2 is a block diagram showing the configuration of an extended reality electronic device according to an embodiment of the present invention.

[0125] Referring to FIG. 2, the extended reality electronic device (20) may include a wireless communication unit (21), an input unit (22), a sensing unit (23), an output unit (24), an interface unit (25), a memory (26), a control unit (27), and a power supply unit (28). The components illustrated in FIG. 1 are not essential for implementing the electronic device (20), and thus, the electronic device (20) described in this specification may have more or fewer components than the components listed above.

[0126] More specifically, among the above components, the wireless communication unit (21) may include one or more modules that enable wireless communication between the electronic device (20) and a wireless communication system, between the electronic device (20) and another electronic device, or between the electronic device (20) and an external server. In addition, the wireless communication unit (21) may include one or more modules that connect the electronic device (20) to one or more networks.

[0127] This wireless communication unit (21) may include at least one of a broadcast reception module, a mobile communication module, a wireless Internet module, a short-range communication module, and a location information module.

[0128] The input unit (22) may include a camera or video input unit for inputting video signals, a microphone or audio input unit for inputting audio signals, and a user input unit (e.g., a touch key, a mechanical key, etc.) for receiving information from a user. Voice data or image data collected from the input unit (22) may be analyzed and processed into a user's control command.

[0129] The sensing unit (23) may include one or more sensors for sensing at least one of information within the electronic device (20), information about the surrounding environment surrounding the electronic device (20), and user information.

[0130] For example, the sensing unit (23) may include at least one of a proximity sensor, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, a G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor (IR sensor), a fingerprint recognition sensor, an ultrasonic sensor, an optical sensor (e.g., a photographing device), a microphone, a battery gauge, an environmental sensor (e.g., a barometer, a hygrometer, a thermometer, a radiation detection sensor, a heat detection sensor, a gas detection sensor, etc.), and a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric recognition sensor, etc.). Meanwhile, the electronic device (20) disclosed in the present specification may utilize information sensed by at least two or more of these sensors in combination.

[0131] The output unit (24) is for generating output related to visual, auditory, or tactile sensations, and may include at least one of a display unit, an audio output unit, a haptic module, and an optical output unit. The display unit may be formed as a touch screen by forming a mutual layer structure with a touch sensor or by forming an integral structure. This touch screen may function as a user input means that provides an input interface between the augmented reality electronic device (20) and the user, and at the same time, may provide an output interface between the augmented reality electronic device (20) and the user.

[0132] The interface unit (25) serves as a passageway for various types of external devices connected to the electronic device (20). Through the interface unit (25), the electronic device (20) can receive virtual reality or augmented reality content from the external device, and can perform mutual interaction by exchanging various input signals, sensing signals, and data.

[0133] For example, the interface unit (25) may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port.

[0134] In addition, the memory (26) stores data that supports various functions of the electronic device (20). The memory (26) can store a plurality of application programs (or applications) that run on the electronic device (20), data for the operation of the electronic device (20), and commands. At least some of these application programs can be downloaded from an external server via wireless communication. In addition, at least some of these application programs can exist on the electronic device (20) from the time of shipment for the basic functions of the electronic device (20) (e.g., call receiving and making functions, message receiving and making functions).

[0135] In addition to operations related to the application program, the control unit (27) typically controls the overall operation of the electronic device (20). The control unit (27) can process signals, data, information, etc. input or output through the components discussed above.

[0136] In addition, the control unit (27) can control at least some of the components by driving an application program stored in the memory (26) to provide appropriate information to the user or process a function. Furthermore, the control unit (27) can operate at least two or more of the components included in the electronic device (20) in combination with each other to drive the application program.

[0137] In addition, the control unit (27) can detect the movement of the electronic device (20) or the user by using a gyroscope sensor, gravity sensor, motion sensor, etc. included in the sensing unit (23). Alternatively, the control unit (27) can detect an object approaching the electronic device (20) or the user by using a proximity sensor, a light sensor, a magnetic sensor, an infrared sensor, an ultrasonic sensor, a light sensor, etc. included in the sensing unit (23). In addition, the control unit (27) can also detect the movement of the user by using sensors provided in a controller that operates in conjunction with the electronic device (20).

[0138] Additionally, the control unit (27) can perform operations (or functions) of the electronic device (20) using an application program stored in the memory (26).

[0139] The power supply unit (28) receives external power or internal power under the control of the control unit (27) and supplies power to each component included in the electronic device (20). The power supply unit (28) includes a battery, and the battery may be provided in a built-in or replaceable form.

[0140] At least some of the above components may cooperate with each other to implement the operation, control, or control method of the electronic device according to various embodiments described below. In addition, the operation, control, or control method of the electronic device may be implemented on the electronic device by driving at least one application program stored in the memory (26).

[0141] Hereinafter, an electronic device described as an example of the present invention will be described based on an embodiment applied to an HMD (Head Mounted Display). However, embodiments of the electronic device according to the present invention may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (personal digital assistant), a PMP (portable multimedia player), a navigation device, a slate PC, a tablet PC, an ultrabook, and a wearable device. In addition to an HMD, the wearable device may include a smart watch, a contact lens, VR / AR / MR Glass, and the like.

[0142] FIG. 3 is a perspective view of an augmented reality electronic device according to an embodiment of the present invention.

[0143] As illustrated in FIG. 3, an electronic device according to an embodiment of the present invention may include a frame (100), a projector device (200), and a display unit (300).

[0144] The electronic device may be provided as a glass type (smart glass). The glass type electronic device is configured to be worn on the head of the human body and may include a frame (case, housing, etc.) (100) for this purpose. The frame (100) may be formed of a flexible material to facilitate wearing.

[0145] The frame (100) is supported by the head and provides a space for mounting various components. As illustrated, electronic components such as a projector device (200), a user input unit (130), or an audio output unit (140) may be mounted on the frame (100). In addition, a lens covering at least one of the left and right eyes may be detachably mounted on the frame (100).

[0146] The frame (100) may have a shape of glasses worn on the face of the user's body as shown in the drawing, but is not necessarily limited thereto, and may also have a shape of goggles or the like worn in close contact with the user's face.

[0147] Such a frame (100) may include a front frame (110) having at least one opening, and a pair of side frames (120) extending in the y direction (in FIG. 2) intersecting the front frame (110) and being parallel to each other.

[0148] The frame (100) may have the same or different length (DI) in the x direction and length (LI) in the y direction.

[0149] The project device (200) is provided to control various electronic components provided in an electronic device. The project device (200) may be used interchangeably with 'optical output device', 'optical projector device', 'light irradiation device', 'optical device', etc.

[0150] The projector device (200) can generate an image or a video of a sequence of images that are displayed to the user. The projector device (200) can include an image source panel that generates an image and a plurality of lenses that diffuse and converge light generated from the image source panel.

[0151] The project device (200) may be fixed to one of the two side frames (120). For example, the project device (200) may be fixed to the inside or outside of one of the side frames (120), or may be integrally formed by being built into the inside of one of the side frames (120). Alternatively, the project device (200) may be fixed to the front frame (110) or may be provided separately from the electronic device.

[0152] The display unit (300) may be implemented in the form of a head-mounted display (HMD). The HMD form refers to a display method that is mounted on the head and directly displays an image in front of the user's eyes. When the user wears the electronic device, the display unit (300) may be positioned to correspond to at least one of the left and right eyes so that the image can be directly displayed in front of the user's eyes. In this drawing, the display unit (300) is positioned in a portion corresponding to the right eye so as to output an image toward the user's right eye. However, as described above, the present invention is not limited thereto and may be positioned for both the left and right eyes.

[0153] The display unit (300) can allow the user to visually perceive the external environment while simultaneously displaying images generated by the projector device (200). For example, the display unit (300) can project images onto the display area using a prism.

[0154] The display unit (300) may be formed to be translucent so that the projected image and the general field of view in front (the range that the user sees through his eyes) can be viewed simultaneously. For example, the display unit (300) may be translucent and formed of an optical member including glass.

[0155] And the display unit (300) can be inserted into and fixed to an opening included in the front frame (110), or can be positioned on the back surface of the opening (i.e., between the opening and the user) and fixed to the front frame (110). In the drawing, an example in which the display unit (300) is positioned on the back surface of the opening and fixed to the front frame (110) is shown, but the display unit (300) can be positioned and fixed to various positions of the frame (100).

[0156] As illustrated in FIG. 2, when the electronic device projects image light from the projector device (200) onto one side of the display unit (300), the image light is emitted to the other side through the display unit (300), thereby allowing the user to see the image generated from the projector device (200).

[0157] Accordingly, the user can view the external environment through the opening of the frame (100) and simultaneously view the image generated by the projector device (200). That is, the image output through the display unit (300) can be seen to overlap with the general field of view. By utilizing these display characteristics, electronic devices can provide augmented reality (AR) that superimposes a virtual image on a real image or background and shows it as a single image.

[0158] Furthermore, in addition to these operations, images generated by the external environment and the projector device (200) may be provided to the user with a time difference for a short period of time that is not recognized by the user. For example, within a single frame, the external environment may be provided to the user during one section, and images from the projector device (200) may be provided to the user during another section.

[0159] Alternatively, both overlap and time difference may be provided.

[0160] In addition, the projector device according to the embodiment may have the structure described below, or may be formed by a structure further including a waveguide and / or glass in the structure. In addition, the projector device may include a DLP (Digital Light Processing) projector or a projector device. Hereinafter, the projector device may be expressed as a projector. The projector according to the embodiment may correspond to the projector device included in the augmented reality electronic device according to the embodiment.

[0161] The following display unit may be expressed as a light guide device. The light guide device according to the embodiment may correspond to the display unit included in the augmented reality electronic device according to the above embodiment.

[0162] Figure 4 is a schematic diagram of a light guide device according to an embodiment of the present invention.

[0163] Referring to FIG. 4, the light guide device (300) according to the embodiment may include a first substrate (310), a first input diffraction element (320), a first transmission diffraction element (330), a first output diffraction element (340), and a cover (301).

[0164] The light guide device (300) can change the path of the light output from the projector (200) and output the light to the outside again. The light guide device (300) can output the light to the outside so that it reaches the user's eyes. The light can sequentially enter the first input diffraction element (320), the first transmission diffraction element (330), and the first output diffraction element (340) and be output to the outside again. The direction in which the light enters the light guide device (300) can be perpendicular to the first substrate (310) or can be a direction forming a certain angle with the direction perpendicular to the first substrate (310).

[0165] The first substrate (310) can serve as a path for transmitting light. A first input diffractive element (320), a first transmission diffractive element (330), and a first output diffractive element (340) can be arranged on the first substrate (310). The light can be totally reflected within the first substrate (310) and travel along the interior of the first substrate (310). The first substrate (310) can include a waveguide. The first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340) can be arranged spaced apart from each other on the first substrate (310).

[0166] The first substrate (310) may include a plurality of surfaces. The first substrate (310) may include a first surface and a second surface. The first surface may be a surface onto which light enters. In addition, the second surface may be a surface from which light exits. The second surface may be a surface spaced apart from the first surface. The first surface and the second surface may be parallel to each other. A first input diffraction element (320), a first transmission diffraction element (330), and a first output diffraction element (340) may be arranged on the first surface or the second surface.

[0167] The first input diffraction element (320) can serve as a path through which light is incident. The first input diffraction element (320) can be placed on the first substrate (310). Light can be incident from the outside onto the light guide device (300) through the first input diffraction element (320) and transmitted through the first substrate (310). The first input diffraction element (320) can change the path of light by diffracting the light.

[0168] The first transmission diffraction element (330) can serve to change the path of light. The first transmission diffraction element (330) can be arranged on the first substrate (310). The first transmission diffraction element (330) can change the path of light incident through the first input diffraction element (320). The first transmission diffraction element (330) can change the path of light so that it is directed toward the first output diffraction element (340). The first transmission diffraction element (330) can change the path of light by diffracting the light.

[0169] The first exit diffraction element (340) can serve as a path through which light is emitted. The first exit diffraction element (340) can be disposed on the first substrate (310). Light can be emitted to the outside of the light guide device (300) through the first exit diffraction element (340). The first exit diffraction element (340) can receive light whose path has been changed from the first transmission diffraction element (330) and emitted it to the outside. The first exit diffraction element (340) can change the path of the light and emitted it to the outside. The first exit diffraction element (340) can change the path of the light by diffracting the light.

[0170] The cover (301) may be disposed on the first substrate (310), the first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340). The cover (301) may be disposed adjacent to the projector (200) on the first substrate (310), the first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340). Light may pass through the cover (301) and enter the first input diffractive element (320). The cover (301) may have the effect of protecting the interior of the light guide device (300).

[0171] The first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340) may include a plurality of protrusions. The plurality of protrusions may have a constant width, period, and height and may be arranged on the first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340). The plurality of protrusions may protrude in a direction perpendicular to the first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340). The plurality of protrusions may be arranged to be spaced apart from each other in the vector direction of the pattern including the protrusions. Depending on the width, period, and height of the plurality of protrusions, the path of light after passing through the first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340) may be changed differently.

[0172] FIG. 5 is a drawing showing the angle of the substrate of the light guide device according to an embodiment of the present invention.

[0173] In FIG. 5, the first coordinate axis (x1, y1, z1) may be a coordinate axis set based on the user. The user's gaze direction may be defined as the z1 axis. The x1 axis and the y1 axis may be axes set in a direction perpendicular to the z1 axis. The x1 axis and the y2 axis may be perpendicular to each other. In addition, the second coordinate axis (x2, y2, z2) may be a coordinate axis set based on the light guide device. The direction perpendicular to the first substrate (310) of the light guide device may be defined as the z2 axis. The x2 axis and the y2 axis may be axes set in a direction perpendicular to the z2 axis. The x2 axis and the y2 axis may be perpendicular to each other.

[0174] Referring to FIG. 5A, the wrap angle (γ) may be defined as the angle formed by the first substrate (310) of the light guide device and the first direction (x1). The first direction (x1) may be a direction perpendicular to the direction in which the user views the light guide device. The direction in which the user views the light guide device may be a third direction (z1). The first direction (x1) may be a direction perpendicular to the third direction (z3).

[0175] Referring to FIG. 5b, the pantoscopic angle (α) may be defined as the angle formed by the first substrate (310) of the light guide device and the second direction (y1). The second direction (y1) may be a direction perpendicular to the direction in which the user views the light guide device. The second direction (y1) may be a direction perpendicular to the first direction (x1) and the third direction (z1).

[0176] The placement angle of the light guide device may vary depending on the wrap angle and pantoscapic angle.

[0177] FIG. 6 is a drawing showing the angle between the projector and the substrate of the light guide device according to an embodiment of the present invention.

[0178] Referring to FIG. 6A, a first angle (θprojector) may be defined as an angle between the projector (200) and the first substrate (310). The angle between the projector (200) and the first substrate (310) may be an angle formed between a direction in which the projector (200) outputs light and a direction (z2) perpendicular to the first substrate (310). The first angle (θprojector) may be 0° to 90°.

[0179] Referring to Fig. 6b, the second angle ( ) can be defined as the angle at which the projector (200) faces the first substrate (310). The angle at which the projector (200) faces the first substrate (310) can be the angle at which the direction in which the projector (200) outputs light forms with the first axis (x2) on the first substrate (310). The first axis (x2) can be the long axis of the first substrate (310). The second angle ( ) may be an angle formed counterclockwise from the first axis (x2). The second angle ( ) can be from 0˚ to 360˚.

[0180] The arrangement relationship between the projector and the light guide device may vary depending on the first and second angles.

[0181] FIG. 7 is a drawing showing a grating pattern of a diffraction element of a light guide device according to an embodiment of the present invention.

[0182] Referring to FIG. 7, the first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340) may each include a grating pattern in which a plurality of protrusions are repeatedly arranged. The first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340) may each include a plurality of protrusions, and the plurality of protrusions may be arranged in parallel and spaced apart from each other by a predetermined distance. In this case, the grating period of the diffractive element may be defined as the shortest distance between one side of a protrusion and one side of an adjacent protrusion. The grating period of the diffractive element may be the shortest distance between the same side surfaces of the protrusions. The grating vector of the diffractive element may be set in a direction perpendicular to the direction in which the protrusions are arranged. The grating angle of the diffractive element may be an angle formed by the grating vector of the diffractive element with the first axis (x2). Additionally, the grating angle of the diffractive element may be the angle formed by the separation direction between the protrusions of the diffractive element and the first axis (x2).

[0183] The first input diffractive element (320) may include a plurality of first protrusions (321). The grating period (λ) of the first input diffractive element (320) IC ) may be the shortest distance between one side of the first protrusion (321) and one side of the adjacent first protrusion (321). In addition, the grating period (λ) of the first input diffraction element (320) IC ) may be the shortest distance between the same side surfaces of adjacent first protrusions (321). The grating angle ( ) may be the angle formed by the grating vector of the first input diffraction element (320) with the first axis (x2). In addition, the grating angle ( ) may be the angle formed by the separation direction between the first protrusions (321) of the first input diffraction element (320) and the first axis (x2). The grating period (λ) of the first input diffraction element (320) IC ) may be 480 nm to 520 nm. For example, the grating period (λ) of the first input diffraction element (320) IC ) may be 500 nm. The grating angle ( of the first input diffraction element (320) ) may be 310˚ to 320˚. For example, the grating angle ( of the first input diffractive element (320) ) can be 315˚.

[0184] The first transmission diffraction element (330) may include a plurality of second protrusions (331). The grating period (λ) of the first transmission diffraction element (330) FG ) may be the shortest distance between one side of the second protrusion (331) and one side of the adjacent second protrusion (331). In addition, the grating period (λ) of the first transmission diffraction element (330) FG ) may be the shortest distance between the same side surfaces of adjacent second protrusions (331). The grating angle ( ) may be the angle formed by the grating vector of the first transmission diffraction element (330) with the first axis (x2). In addition, the grating angle ( ) may be the angle formed by the separation direction between the second protrusions (331) of the first transmission diffraction element (330) and the first axis (x2). The grating period (λ) of the first transmission diffraction element (330) FG ) may be 380 nm to 420 nm. For example, the grating period (λ) of the first transmission diffraction element (330) FG ) may be 400 nm. The grating angle ( of the first transmission diffraction element (330) ) may be 65˚ to 75˚. For example, the grating angle ( of the first transmission diffraction element (330) ) can be 70˚.

[0185] The first diffractive element (340) may include a plurality of third protrusions (341). The grating period (λ) of the first diffractive element (340) OC ) may be the shortest distance between one side of the third protrusion (341) and one side of the adjacent third protrusion (341). In addition, the grating period (λ) of the first diffraction element (340) OC ) may be the shortest distance between the same sides of adjacent third protrusions (341). The grating angle ( of the first diffractive element (340) ) may be the angle formed by the grating vector of the first diffraction element (340) with the first axis (x2). In addition, the grating angle ( ) may be the angle formed by the separation direction between the third protrusions (341) of the first diffraction element (340) and the first axis (x2). The grating period (λ) of the first diffraction element (340) OC ) may be 360 ​​nm to 400 nm. For example, the grating period (λ) of the first diffractive element (340) OC ) may be 378 nm. The grating angle of the first diffraction element (340) ) can be 180˚ to 190˚. For example, the grating angle ( of the first diffractive element (340) ) can be 185.35˚.

[0186] The light guide device (300) according to the embodiment can satisfy mathematical expressions 1 and 2.

[0187] [Mathematical Formula 1]

[0188]

[0189] [Equation 2]

[0190]

[0191] Here θ projector is the angle between the projector and the first substrate, is the angle at which the projector faces the first substrate, γ is the wrap angle of the first substrate, λ is the wavelength of light, α is the pantoscopic angle of the first substrate, Λ IC is the grating period of the first input diffractive element, Λ OC is the grating period of the first-order diffractive element, Λ FG is the grating period of the first transmission diffraction element, is the grating angle of the first input diffractive element, is the grating angle of the first diffractive element, is the grating angle of the first transmission diffractive element, FoV is the field of view of the light guide device, and x and y may be the resolutions of the light guide device. The field of view of the light guide device may be the angle at which light is incident on the first input diffractive element and transmitted to the first transmission diffractive element. It is possible to design a grating structure of the diffractive element that can be applied to all angles of the light guide device and the projector.

[0192] The light guide device according to one embodiment is Λ IC is 500 nm, Λ OC is 378 nm, Λ FG is 400nm, is 315˚, is 185.35˚, is 70˚, λ is 525 nm, θ projector is 5˚, can be 7˚, α can be 6˚, and γ can be 8˚.

[0193] Figure 8 is a schematic diagram of a light guide device according to another embodiment of the present invention.

[0194] Referring to FIG. 8, the light guide device (300) may further include a second substrate (350), a second input diffraction element (360), a second transmission diffraction element (370), and a second output diffraction element (380).

[0195] The second substrate (350) may be disposed spaced apart from the first substrate (310) in the direction of the optical axis. The second substrate (350) may be disposed at the rear end of the first substrate (310) on the path of light. The second input diffraction element (360), the second transmission diffraction element (370), and the second exit diffraction element (280) may be disposed on the second substrate (350). Light may be incident through the second input diffraction element (370) and may be emitted through the second exit diffraction element (380). The second input diffraction element (360) may diffract light that passes through the first input diffraction element (320) without being diffracted by the first input diffraction element (320). The wrap angle of the first substrate and the wrap angle of the second substrate may be the same. In addition, the pantospheric angle of the first substrate and the pantospheric angle of the second substrate may be the same. The second input diffractive element (360) may diffract light of a different wavelength band from the first input diffractive element (320). Accordingly, the wavelength band of light transmitted through the second substrate (350) may be different from the wavelength band of light transmitted through the first substrate (310). Accordingly, light sequentially incident on the first input diffractive element, the first transmission diffractive element, and the first exit diffractive element may have a different wavelength from light sequentially incident on the second input diffractive element, the second transmission diffractive element, and the second exit diffractive element. Light of different wavelength bands may be emitted from the first exit diffractive element (340) and the second exit diffractive element (380), respectively, and reach the user's eyes.

[0196] The grating angle of the first input diffraction element (320) and the grating angle of the second input diffraction element (360) are the same, the grating angle of the first transmission diffraction element (330) and the grating angle of the second transmission diffraction element (370) are the same, and the grating angles of the first output diffraction element (340) and the second output diffraction element (380) may be the same. As a result, light of different wavelength bands can be transmitted and output in the same manner.

[0197] The grating periods of the second input diffractive element (360), the second transmission diffractive element (370), and the second output diffractive element (380) may be different from the grating periods of the first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340), respectively. Since the grating periods of the second input diffractive element (360), the second transmission diffractive element (370), and the second output diffractive element (380) are different from the grating periods of the first input diffractive element (320), the first transmission diffractive element (330), and the first output diffractive element (340), light of different wavelength bands can be diffracted.

[0198] The light guide device according to the embodiment can satisfy mathematical expressions 3 and 4.

[0199] [Equation 3]

[0200]

[0201] [Equation 4]

[0202]

[0203] Here θ projector is the angle between the projector and the second substrate, is the angle at which the projector faces the second substrate, γ is the wrap angle of the second substrate, λ is the wavelength of light, α is the pantoscopic angle of the second substrate, Λ IC is the grating period of the second input diffractive element, Λ OC is the grating period of the second-order diffractive element, Λ FG is the grating period of the second transmission diffraction element, is the grating angle of the second input diffractive element, is the grating angle of the second diffractive element, is the grating angle of the second transmission diffractive element, FoV is the viewing angle of the light guide device, and x and y can be the resolutions of the light guide device. It is possible to design a grating structure of the diffractive element that can be applied to all angles of the light guide device and the projector.

[0204] A vehicle system (or environment) according to an embodiment may include a vehicle, a passenger (driver), and an electronic device. While the electronic device is described below as a device separately installed within the vehicle (1), the present invention is not limited thereto. For example, the electronic device may be implemented as a component of the vehicle.

[0205] A vehicle may include a body and various devices for moving the body (e.g., wheels, a drive unit for driving the wheels, a starter for starting the drive unit, an engine for generating power and transmitting the generated power to the drive unit, a steering unit for controlling the direction of the vehicle, an accelerator for controlling the speed of the vehicle, etc.). In addition, the vehicle may include various electrical systems. For example, the electrical system may include an engine control unit for controlling the engine, a temperature control unit for controlling the temperature inside the vehicle, a light control unit for controlling the lights according to external conditions, etc.

[0206] In particular, the vehicle may include a communication interface capable of communicating with an electronic device, and may include an additional processor for analyzing data transmitted through the communication interface and performing preset functions based on the analysis results.

[0207] The processor may be implemented, for example, as the aforementioned engine control unit or motor control unit. The communication interface may support at least one of various communication methods, such as CAN communication, which supports data transmission and reception within the vehicle, or wired communication via a cable connected to an electronic device. As an example, the vehicle may receive images acquired by an electronic device or analysis results of the images, and perform a designated function based on the received results.

[0208] According to an embodiment, an electronic device may be connected to a camera module to acquire an image of a driver, analyze the acquired image, and then perform various set function processing (e.g., deceleration processing, turning on or off emergency lights, controlling a horn, controlling vehicle vibration, controlling window opening and closing, etc.) based on the analysis results. In addition to this function processing, various other function processing may be additionally implemented.

[0209] And the driver, as a person who can sit in the driver's seat and control the steering device, can be the subject of video capture by an electronic device. In the present invention, the electronic device acquires an image of a driver seated in the driver's seat as a representative example, but the present invention is not limited thereto. For example, the monitoring system can be applied to acquire images of not only the driver but also passengers seated in the passenger seat or other seats, and to adjust the image acquisition method according to various behaviors of the passengers. In this regard, at least one electronic device can be arranged in the vehicle. For example, a camera module and an electronic device can be arranged to acquire an image of a driver seated in the driver's seat that only supports driver monitoring.

[0210] Alternatively, if monitoring capabilities for the driver and passengers are supported, multiple electronic devices may be placed within the vehicle to enable image acquisition of the driver and passenger seats.

[0211] An electronic device can receive image information about the driver's face via a camera module. Accordingly, the electronic device can determine the driver's drowsiness, etc., in a predetermined manner and provide various feedback (e.g., an alarm) in response. For example, the electronic device can determine the driver's drowsiness based on the user's eye track, iris size, or size changes. Furthermore, the electronic device can receive image information about the driver's hands as well as the user's face from the camera module. Therefore, the electronic device can easily determine the driver's hands-off status. In other words, a driver monitoring system with a hands-off monitoring function can be implemented. This allows the driver's concentration and drowsiness to be easily determined based on various driver movements resulting from the driver's hands-off.

[0212] The camera module can be positioned within the vehicle to easily capture images of occupants. For example, it can be positioned in various locations within the vehicle, such as the windshield (e.g., where the head-up display is positioned), the underside of the windshield, the dashboard, or the instrument panel, to capture images of a subject seated in the driver's seat. Furthermore, the camera module can be positioned in locations that are difficult for occupants to easily perceive.

[0213] Additionally, a camera module connected to an electronic device can be positioned at a specific location within the vehicle to receive image information about passengers other than the driver. For example, there may be at least one camera module, and the camera module may be positioned in a rearview mirror (or rearview mirror) or similar location to detect all passengers other than the driver. This allows the camera module to generate an image of all passengers.

[0214] This allows the camera module to generate image information about other occupants in the vehicle, not just the driver. Furthermore, the electronic device can receive image information including occupants other than the driver. This configuration facilitates the implementation of a monitoring system for not only the driver but also other occupants.

[0215] A camera module according to an embodiment may include a light source unit, a light guide device, a light receiving unit, and a control unit.

[0216] First, the light source can output light according to a control signal. Finally, the light output from the light source can be irradiated onto an object. Then, the light irradiated onto the object can be reflected and provided to the light receiving unit.

[0217] The light source unit may include at least one light source. The at least one light source may emit light of a predetermined wavelength range or light having a predetermined central wavelength. Furthermore, the light source unit may emit light in a predetermined pattern based on a pre-designed algorithm. The light source unit may output light under the control of the control unit.

[0218] Hereinafter, output light or incident light refers to light output from a light source and provided to an object, and input light or reflected light refers to light output from a light source, reaches an object, is reflected from the object, and is input to a light receiving unit. That is, from the object's perspective, output light can be incident light, and input light can be reflected light.

[0219] At least one light source of the light source unit can output light of a predetermined wavelength band. For example, the wavelength of the light output from the light source may be infrared light of 770 nm to 3000 nm. Additionally, the wavelength of the light output from the light source may be visible light of 380 nm to 770 nm. Furthermore, the light source of the light source unit may emit light outside the wavelength range described above. In particular, the light source may irradiate light of a specific wavelength band so as not to be harmful to passengers such as the driver and other passengers in the vehicle, as described above, or irradiate light of a specific energy or lower so as not to be harmful.

[0220] The light source may include a light emitting diode (LED), an organic light emitting diode (OLED), a laser diode (LD), a vertical-cavity surface-emitting laser (VCSEL), a plasma lamp, a fluorescent lamp, a xenon lamp, a halogen lamp, a neon lamp, etc. It may output a wavelength of about 800 nm to 1000 nm, for example, a wavelength of about 850 nm or about 940 nm.

[0221] The light guide device can be positioned adjacent to the light source and the light receiver. The light guide device can guide light emitted from the light source and transmit it to an object. In addition, the light guide device can guide light reflected from the object back to the light receiver. In this way, the light guide device can be configured to control the light and direct it along a desired path. In other words, the light guide device can both transmit light to the object and receive reflected light. Accordingly, the light guide device can be configured to help the light in the camera module accurately reach the sensor or guide it along a specific path so that optical information can be accurately transmitted.

[0222] Such light guide devices may be made of materials such as glass, polymer, or silicone. The light guide devices may also include various other light-guiding materials.

[0223] The light guide device can transmit light in a desired direction using diffraction. Accordingly, the light guide device may include optical elements for determining the path of light on a substrate, which is a waveguide. The optical elements may include various elements that operate based on diffraction.

[0224] As an example, the light guide device may include a diffractive element, which is a holographic optical element (HOE). The light guide device may include an input diffractive element, an input / output diffractive element, and an output diffractive element, as described below. For example, the input diffractive element, the input / output diffractive element, and the output diffractive element may be formed of a holographic optical element.

[0225] Holographic optical elements use interference patterns generated by laser interference to diffract light (or light), allowing them to control light of a specific wavelength or diffract it in a desired direction. This diffraction process is governed by Bragg's Law, and the diffraction angle can be determined based on the wavelength of the light and the structure of the holographic optical element.

[0226] A holographic optical element can be composed of an interference pattern recorded on a transparent substrate. As described above, the transparent substrate is a waveguide and can be made of various materials, such as glass, plastic, or polymer. The interference pattern of the holographic optical element can be precisely designed within or on the substrate to guide light in a specific direction. Furthermore, holographic optical elements can be categorized into transmissive, which diffracts light as it passes, and reflective, which diffracts light as it reflects. Therefore, the holographic optical element can be positioned variably on the substrate. These holographic optical elements enable precise control of light, resulting in high-resolution images. Furthermore, holographic optical elements can support high-speed data transmission in optical communications through wavelength separation and combination. Furthermore, holographic optical elements are lighter and thinner than conventional lenses or mirrors, enabling the provision of miniaturized camera modules. A detailed description of the diffractive elements in the light guide device—the input diffractive element, the input / output diffractive element, and the output diffractive element—will be provided below.

[0227] The light receiving unit can receive light transmitted through the light guide device. The light receiving unit can include an image sensor. The image sensor can receive light reflected from an object. Accordingly, the image sensor can detect the light and convert it into an electrical signal. For example, the image sensor can convert the electrical signal to generate a digital image. The image sensor can include a charge-coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS), an indium gallium arsenide (InGaAs) sensor, a mercury cadmium telluride (HgCdTe) sensor, a microbolometer, etc. The light receiving unit may also include an image sensor that receives light of various wavelength bands other than those described above or as examples.

[0228] The light receiver may be positioned adjacent to the light guide device, similar to the light source. Alternatively, an additional lens may be positioned between the light receiver and the light guide device. This can also be applied to the area between the light source and the light guide device.

[0229] The control unit can control the operation of the light source and light receiver. Furthermore, the control unit can generate depth information based on the image generated by the light receiver, or transmit and receive image information with other electronic devices, such as vehicles. This control unit can control the operation within the camera module and communicate with processors within external electronic devices, such as vehicles.

[0230] The control unit may include a processor, a microcontroller (MCU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc., and may also be implemented in the form of an application processor (AP) of various electronic devices.

[0231] FIG. 9 is a drawing showing the configuration of a camera module according to an embodiment of the present invention, and FIG. 10 is a drawing showing the angle of a substrate of a light guide device according to an embodiment of the present invention.

[0232] Fig. 9a shows a state in which light incident from the outside enters the light receiving unit (500) through the light guide device (400), and Fig. 9b shows a state in which light irradiated from the light source unit (600) is emitted to the outside through the light guide device (400). Fig. 10a shows a pantospheric angle (α), Fig. 10b shows a wrap angle (γ), and Fig. 10c can show a vertical distance (d, Vertex distance) between an object and the light guide device.

[0233] Referring to FIGS. 9 and 10, the light guide device (400) may include a cover (401), a first substrate (410), a first input diffraction element (420), a first transmission diffraction element (430), a first input / output diffraction element (440), and a first output diffraction element (450).

[0234] The first input diffraction element (420), the first transmission diffraction element (430), the first input / output diffraction element (440), and the first output diffraction element (450) may be arranged on a first substrate (410) which is a waveguide. The first input diffraction element (420), the first transmission diffraction element (430), the first input / output diffraction element (440), and the first output diffraction element (450) may be either a transmissive type or a reflective type, and may be positioned on either one surface (e.g., an upper surface) or the other surface (e.g., a lower surface) of the first substrate (410).

[0235] And the first input diffraction element (420), the first transmission diffraction element (430), the first input / output diffraction element (440), and the first output diffraction element (450) are diffraction elements as described above, and can be spaced apart from each other.

[0236] The first input diffraction element (420) diffracts light provided from the light source unit (600) and guides it to the first substrate (410), and the light guided into the first substrate (410) can be provided to the first input / output diffraction element (440) through the first transmission diffraction element (430). The first input / output diffraction element (440) diffracts light guided from the first input diffraction element (420) to the first substrate (410) onto an object, and diffracts light reflected from the object and guides it into the substrate (410). The first output diffraction element (450) diffracts light reflected from the object and guided to the substrate (410) by the first input / output diffraction element (440) and guides or provides it to the light receiving unit (500). At this time, the light diffracted by the first emitting diffraction element (450) and guided to the light receiving unit (500) may be incident on the light receiving unit (500) and converted into image information. In the case of a light guide device according to another embodiment, the first transmission diffraction element (430) may not be included.

[0237] The camera module according to the embodiment can satisfy the above mathematical expressions 1 and 2 when light is irradiated from the light source unit (600) and emitted to an object. (In this case, the projector of the mathematical expressions 1 and 2 can correspond to the light source unit (600), and the first output diffraction element of the mathematical expression 1 can correspond to the first input / output diffraction element (440).) In addition, the camera module according to the embodiment can satisfy not only the mathematical expressions 1 and 2 when light is irradiated from the light source unit (600) and emitted to an object, but can also satisfy the mathematical expressions 5 and 6 when light is reflected from the object and received by the light receiving unit (500).

[0238] The light guide device of the camera module according to the embodiment can satisfy mathematical expressions 5 and 6.

[0239] [Equation 5]

[0240]

[0241] [Equation 6]

[0242]

[0243] Here, θcamera is the angle between the photodetector and the first substrate, is the angle at which the light-receiving portion faces the first substrate, γ is the wrap angle of the first substrate, λ is the wavelength of light, α is the pantoscopic angle of the first substrate, Λ IOC is the grating period of the first input / output diffractive element, Λ OC is the grating period of the first-order diffractive element, Λ FG is the grating period of the first transmission diffraction element, is the grating angle of the first input / output diffractive element, is the grating angle of the first diffractive element, is the grating angle of the first transmission diffraction element, FoV is the viewing angle of the light guide device, and x and y may be the resolutions of the light guide device. The viewing angle of the light guide device may be the angle at which light is emitted from the first exit diffraction element and transmitted to the light receiving unit. The grating structure of the diffraction element that can be applied to all angles of the light guide device and the light receiving unit can be designed. (In the case where the first transmission diffraction element is not included in the light guide device according to another embodiment, the grating period (Λ) of the first transmission diffraction element in Equations 5 and 6 FG ) and grid angle ( ) can be ignored.)

[0244] The angles shown in FIGS. 5 and 6 can also be applied to Equations 5 and 6. Referring to FIG. 5a, the wrap angle (γ) of Equation 5 can be defined as the angle formed by the first substrate of the light guide device and the first direction (x1). The first direction (x1) may be a direction perpendicular to the direction in which the user views the light guide device. Referring to FIG. 5b, the pantospheric angle (α) of Equation 6 can be defined as the angle formed by the first substrate of the light guide device and the second direction (y1). The second direction (y1) may be a direction perpendicular to the direction in which the user views the light guide device. The pantospheric angle (α) according to an embodiment can have a range of -(90˚ - Fov) to +(90˚ - Fov). The pantospheric angle (α) according to an embodiment can be 15˚ to 25˚. For example, the pantospheric angle (α) may be 20°. Referring to Fig. 6a, θcamera may be defined as the angle between the light-receiving unit (corresponding to 200 in Fig. 6a) and the first substrate. The angle between the light-receiving unit and the first substrate may be the angle formed by the direction in which the light-receiving unit receives light and the direction (z2) perpendicular to the first substrate. Referring to Fig. 6b, The angle at which the light receiving unit (corresponding to 200 in FIG. 6b) faces on the first substrate may be defined as the angle at which the light receiving unit faces on the first substrate. The angle at which the light receiving unit faces on the first substrate may be the angle formed by the direction in which the light receiving unit receives light with the first axis (x2) on the first substrate. The first axis (x2) may be the major axis of the first substrate. The light guide device of the camera module according to one embodiment may have a wrap angle (γ) and may be 0. (The first substrate (310) of FIGS. 5 and 6 may correspond to the first substrate (410) of FIG. 9, and the projector (200) of FIG. 6 may correspond to the light receiving unit (500) of FIG. 9.)

[0245] The camera module according to the embodiment can design a diffractive element grating structure that can be applied to all angles of the light guide device and the light source unit, and all angles of the light guide device and the light receiving unit. Accordingly, a light guide device and camera module can be provided that more easily generate images of passengers, etc., within a vehicle.

[0246] Although the above description focuses on examples, these are merely examples and do not limit the present invention. Those skilled in the art will appreciate that various modifications and applications not exemplified above are possible without departing from the essential characteristics of the present invention. For example, each component specifically shown in the examples can be modified and implemented. In addition, differences related to such modifications and applications should be construed as being included within the scope of the present invention defined in the appended claims.

Claims

1. First substrate; and It comprises a first input diffraction element, a first transmission diffraction element, and a first output diffraction element, which are arranged on the first substrate and onto which light emitted from the project is sequentially incident; An optical guide device satisfying mathematical expressions 1 and 2. [Mathematical Formula 1] [Equation 2] (Here θ projector is the angle between the projector and the first substrate, is the angle at which the projector faces the first substrate, γ is the wrap angle of the first substrate, λ is the wavelength of light, α is the pantoscopic angle of the first substrate, Λ IC is the grating period of the first input diffractive element, Λ OC is the grating period of the first-order diffractive element, Λ FG is the grating period of the first transmission diffraction element, is the grating angle of the first input diffractive element, is the grating angle of the first diffractive element, is the grating angle of the first transmission diffraction element, FoV is the field of view of the light guide device, x and y are the resolutions of the light guide device) 2. In paragraph 1, The wrap angle of the first substrate is the angle that the first substrate forms with the first direction, A light guide device in which the first direction is perpendicular to the direction in which the user views the light guide device.

3. In paragraph 2, The pantospheric angle of the first substrate is the angle formed by the first substrate with the second direction, A light guide device in which the second direction is perpendicular to the first direction and the direction in which the user views the light guide device.

4. In paragraph 1, A light guide device in which the angle between the projector and the first substrate is the angle formed between the direction in which the projector outputs the light and a plane perpendicular to the first substrate.

5. In paragraph 1, The angle at which the projector faces the first substrate is the angle at which the direction in which the projector outputs the light forms with the first axis on the first substrate, The above first axis is a light guide device that is the long axis of the first substrate.

6. In paragraph 1, The first input diffractive element includes a first protrusion, the first transmission diffractive element includes a second protrusion, and the first output diffractive element includes a third protrusion. A light guide device in which the grating period of the first input diffractive element is the shortest distance between the same side surfaces of adjacent first protrusions, the grating period of the first transmission diffractive element is the shortest distance between the same side surfaces of adjacent second protrusions, and the grating period of the first output diffractive element is the shortest distance between the same side surfaces of adjacent third protrusions.

7. In paragraph 5, A light guide device in which the grating angle of the first input diffractive element is the angle formed by the separation direction between the first protrusions of the first input diffractive element and the first axis, the grating angle of the first transmission diffractive element is the angle formed by the separation direction between the second protrusions of the first transmission diffractive element and the first axis, and the grating angle of the first output diffractive element is the angle formed by the separation direction between the third protrusions of the first output diffractive element and the first axis.

8. In paragraph 1, A second substrate arranged to overlap the first substrate; A light guide device further comprising a second input diffraction element, a second transmission diffraction element, and a second output diffraction element, which are arranged on the second substrate and onto which light is sequentially incident.

9. In paragraph 8, A light guide device in which the grating angles of the first input diffractive element and the second input diffractive element are the same, the grating angles of the first transmission diffractive element and the second transmission diffractive element are the same, and the grating angles of the first exit diffractive element and the second exit diffractive element are the same.

10. In paragraph 1, A light guide device in which the viewing angle of the light guide device is the angle at which the light is incident on the first input diffraction element and transmitted to the first transmission diffraction element.

Images