Wearable electronic device implemented distributed system for content and vision processing

Abstract

A wearable electronic device is disclosed. The wearable electronic device comprises: a frame; a first arm and a second arm each extending from the frame; a screen display unit disposed in the frame and including a display; at least one camera disposed in the frame; a first processor located within the first arm and operably connected to the display; and a second processor located within the second arm and operably connected to at least one camera, wherein the first processor is configured to display content on the screen display unit using the display, and the second processor may be configured to perform an operation associated with at least one camera. Other embodiments are possible.

Description

Technology Field

[0001] The embodiments disclosed in this document relate to a wearable electronic device in which a distributed system for content and vision processing is implemented. Background Technology

[0002] Wearable electronic devices, such as AR glasses that provide augmented reality (AR), are being developed. These wearable electronic devices may include a display provided corresponding to both lenses for the playback of content and a plurality of cameras for identifying AR movements. The problem to be solved

[0003] A system related to content playback of a wearable electronic device and a system for identifying AR movements can be implemented as a single system. In this case, power consumption is concentrated at the location where the single system is mounted, and a hot spot where heat is concentrated in the periphery of the single system may occur.

[0004] To eliminate hot spots, additional heat dissipation structures may be provided. However, these additional heat dissipation structures cause weight deviations, and to eliminate this weight deviation, additional structures may be installed on the opposite side to balance the weight. These additional structures for weight balance may increase the overall weight of the wearable electronic device. Hot spots, weight imbalances, and heavy weight of the wearable electronic device can degrade the user's wearing comfort.

[0005] In addition, since the signal lines used in a wearable electronic device with a single system are also arranged in a biased manner, it may be difficult to design various form factors of the wearable electronic device.

[0006] According to one embodiment of the present disclosure, an electronic device is provided that implements a distribution system capable of maintaining weight balance and preventing unnecessary weight increase by eliminating hot spots. means of solving the problem

[0007] A wearable electronic device according to one embodiment comprises: a frame; a first arm and a second arm each extending from the frame; a screen display unit disposed in the frame and including a display; at least one camera disposed in the frame; a first processor located within the first arm and operably connected to the display; and a second processor located within the second arm and operably connected to the at least one camera, wherein the first processor is configured to display content on the screen display unit using the display, and the second processor may be configured to perform an operation associated with the at least one camera.

[0008] A wearable electronic device according to one embodiment comprises: a frame; a first arm extending from an edge portion of the frame in a first direction; a second arm extending from an edge portion of the frame in a direction opposite to the first direction; a screen display unit disposed on the frame and including a display; a sensor unit disposed on the frame and including a head tracking sensor and a gaze tracking sensor; a first system for processing content displayed on the screen display unit; and a second system for processing a value detected by the sensor unit, wherein the first system is disposed on the first arm and the second system may be disposed on the second arm. Effects of the invention

[0009] According to the embodiments disclosed in this document, through a distributed system for content and vision processing, heat generated by an electronic device can be evenly distributed and hot spots can be eliminated.

[0010] According to the embodiments disclosed in this document, through a distributed system for content and vision processing, design constraints related to the volume of electronic devices and the mounting space of signal lines can be reduced.

[0011] According to the embodiments disclosed in this document, through a distributed system for content and vision processing, weight balance can be maintained without unnecessary weight increase and the user's wearing comfort can be improved.

[0012] In addition, various effects that can be identified directly or indirectly through this document may be provided. Brief explanation of the drawing

[0013] FIG. 1a is a schematic diagram of an electronic device according to one embodiment. FIG. 1b is a schematic diagram of an electronic device according to one embodiment. FIG. 2 is a schematic diagram of a gaze tracking and display method through a transparent member according to one embodiment. FIG. 3 is a block diagram of an electronic device in a network environment according to one embodiment. FIG. 4 shows an example of a distributed system for content processing and vision processing implemented in an electronic device according to one embodiment. FIG. 5 is a schematic diagram illustrating a content processing and vision processing method of a distributed system implemented in an electronic device according to one embodiment. FIG. 6 shows the heating temperature of an electronic device in which a distributed system is implemented according to one embodiment. Figure 7 shows the heating temperature of an electronic device according to a comparative example. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Specific details for implementing the invention

[0014] Hereinafter, various embodiments of the present invention are described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present invention.

[0015] FIG. 1a is a schematic diagram of an electronic device (101) according to one embodiment. FIG. 1b is a schematic diagram of an electronic device (101) according to one embodiment. In FIG. 1a, the illustration of the third PCB (printed circuit board) (487) of FIG. 1b has been omitted, but this is for convenience of explanation and is not limited to the embodiment illustrated in FIG. 1a or FIG. 1b. Similarly, in FIG. 1b, the illustration of some components of the electronic device (101), excluding the third PCB (487), has been omitted.

[0016] In the examples of FIGS. 1a and 1b, the electronic device (101) may be referred to as a head-mounted display (HMD) device, a wearable device, smart glasses, eyewear, an augmented reality device (AR) device, or a virtual reality device (VR) device. The form of the electronic device (101) shown in FIGS. 1a and 1b is exemplary and the embodiments of this document are not limited thereto.

[0017] Referring to FIG. 1a, an electronic device (101) according to one embodiment (e.g., the electronic device (301) of FIG. 3) may include a frame (or frame structure) (197), a first arm (193-1), and a second arm (193-2). The frame (197) may include a first member (197-1) on which a first transparent member (296-1) is disposed, a second member (197-2) on which a second transparent member (296-2) is disposed, and a third member (197-3) connecting the first member (197-1) and the second member (197-2). At least some of the first member (197-1), the second member (197-2), and / or the third member (197-3) may be formed integrally, but are not limited thereto. The first arm (193-1) and the second arm (193-2) may each extend from the frame (197). For example, the first arm (193-1) may extend from the first side edge portion of the first member (197-1) (e.g., the edge portion in the +Y direction of the first member (197-1)), and the second arm (193-2) may extend from the second side edge portion of the second member (197-2) opposite the first side (e.g., the edge portion in the -Y direction of the second member (197-2). The first arm (193-1) and the second arm (193-2) may be physically connected to the frame (197). For example, the first arm (193-1) may be connected to the first member (197-1), and the second arm (193-2) may be connected to the second member (197-2). When the user wears the electronic device (101), the frame (197) may be positioned on the front of the user's head (e.g., +X direction), the first arm (193-1) may be positioned on the first side of the user's head (e.g., +Y direction), and the second arm (193-2) may be positioned on the second side opposite the first side (e.g., -Y direction). Additionally, when the user wears the electronic device (101), the first member (197-1) of the frame (197) may correspond to the user's right eye, and the second member (197-2) may correspond to the user's left eye.Additionally, the first arm (193-1) and the second arm (193-2) can support the frame (197) when the electronic device (101) is worn.

[0018] In one embodiment, the first arm (193-1) may include a first sub-arm (195-1), a second sub-arm (or first temple) (198-1), and a first hinge portion (199-1). The first sub-arm (195-1) may be connected to a first member (197-1) of the frame (197). The second sub-arm (198-1) may be rotatably connected to the first sub-arm (195-1) through the first hinge portion (199-1). For example, the second sub-arm (198-1) may be folded inward or unfolded outward relative to the first sub-arm (195-1) in response to the operation of the first hinge portion (199-1). In one embodiment, the second arm (193-2) may include a third sub-arm (195-2), a fourth sub-arm (or second temple) (198-2), and a second hinge portion (199-2). The third sub-arm (195-2) may be connected to a second member (197-2) of the frame (197). The fourth sub-arm (198-2) may be rotatably connected to the third sub-arm (195-2) through the second hinge portion (199-2). For example, the fourth sub-arm (198-2) may be folded inward or unfolded outward relative to the third sub-arm (195-2) in response to the operation of the second hinge portion (199-2). In the present disclosure, the frame (197), the first arm (193-1), and the second arm (193-2) may be referred to as the “housing” of the electronic device (101).

[0019] In one embodiment, at least a portion of the surface of the housing of the electronic device (101) may include a sensing area for detecting a user's touch input. For example, the surface (1981) facing the first side (e.g., +Y direction) of the first arm (193-1) may be used at least partially as the sensing area.

[0020] An electronic device (101) according to one embodiment may include a display (161) (e.g., a display module (360) of FIG. 3). For example, the electronic device (101) may include a first display (161-1) and / or a second display (161-2). The first display (161-1) may be placed on a first member (197-1) of a frame (197) and / or a first sub-arm (195-1) of a first arm (193-1). The second display (161-2) may be placed on a second member (197-2) of a frame (197) and / or a third sub-arm (195-2) of a second arm (193-2).

[0021] In one embodiment, the first display (161-1) and / or the second display (161-2) may include at least one of a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon device (LCoS device), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED). The display (161) of the electronic device (101) may include at least one light source for irradiating light. For example, if the first display (161-1) and / or the second display (161-2) includes one of a liquid crystal display, a digital mirror device, or a silicon liquid crystal display, the electronic device (101) may include at least one light source for irradiating light to a screen output area (160-1 and / or 160-2) of the display (161). In another embodiment, if the display (161) of the electronic device (101) can generate light on its own, the display (161) may not include a separate light source in addition to the light source included in the display (161). For example, if the first display (161-1) and / or the second display (161-2) includes at least one of an organic light-emitting diode or a micro LED, the electronic device (101) may provide an image to the user without including a separate light source. If the display (161) is implemented with an organic light-emitting diode or a micro LED, a separate light source is unnecessary, so the weight of the electronic device (101) may be reduced.

[0022] An electronic device (101) according to one embodiment may include a first transparent member (196-1) and a second transparent member (196-2). The first transparent member (196-1) may be at least partially received in a window defined by the first member (197-1) of the frame (197). The second transparent member (196-2) may be at least partially received in a window defined by the second member (197-2) of the frame (197). When a user wears the electronic device (101), the user can see through the first transparent member (196-1) and / or the second transparent member (196-2). The first transparent member (196-1) and / or the second transparent member (196-2) may be formed from at least one of a glass plate, a plastic plate, or a polymer, and may be substantially transparent or translucent. When the electronic device (101) is worn, the first transparent member (196-1) may be positioned facing the user's right eye, and the second transparent member (196-2) may be positioned facing the user's left eye.

[0023] In one embodiment, at least a portion of the first transparent member (196-1) and / or the second transparent member (196-2) may be a waveguide. The waveguide may transmit an image generated by the display (161) to the user's eye. The waveguide may be formed of glass, plastic, or polymer. For example, the waveguide may include a nano-pattern formed on the interior or on one surface (e.g., a polygonal or curved grating structure). Light incident on one end of the waveguide may propagate within the waveguide through the nano-pattern and be provided to the user's eye. For example, the waveguide composed of a free-form prism may be configured to provide the incident light to the user through a reflective mirror.

[0024] According to one embodiment, the optical waveguide may include at least one of a diffractive element (e.g., DOE (Diffractive Optical Element), HOE (Holographic Optical Element)) or a reflective element (e.g., a reflective mirror). The optical waveguide may guide light emitted from a light source to the user's eye using at least one diffractive element or reflective element included in the optical waveguide. For example, the diffractive element may include an input optical member (e.g., 162-1 and / or 162-2) and / or an output optical member (not shown). The first input optical member (162-1) and / or the second input optical member (162-2) may be referred to as an input grating area, and the output optical member (not shown) may be referred to as an output grating area. The input grating area can diffract or reflect light to transmit light output from a light source (e.g., Micro LED) to a transparent member of the screen display (e.g., a first transparent member (196-1) and / or a second transparent member (196-2)). The output grating area can diffract or reflect light transmitted to a transparent member of the optical waveguide (e.g., a first transparent member (196-1) and / or a second transparent member (196-2)) toward the user's eyes. For example, the reflective element may include a total internal reflection optical element or a total internal reflection waveguide for total internal reflection (TIR). Total internal reflection may be referred to as a method of inducing light and may mean creating an angle of incidence such that light (e.g., an image) input through the input grating area is 100% reflected from one side (e.g., a specific side) of the optical waveguide, thereby transmitting 100% to the output grating area. In one embodiment, the light path of light emitted from the display (161) can be guided into a light waveguide by an input optical member.Light traveling inside the optical waveguide can be guided toward the user's eye through an output optical element. The screen output area (160-1 and / or 160-2) can be determined based on the light emitted toward the eye.

[0025] Although FIG. 1a describes an electronic device (101) providing an image to a user using an optical waveguide, the embodiments of this document are not limited thereto. For example, the display (161) of the electronic device (101) may be a transparent or translucent display. In this case, the first display (161-1) and the second display (161-2) may each be positioned facing the user's eyes (e.g., the first screen output area (160-1) and / or the second screen output area (160-2)). Alternatively, the first display (161-1) and the second display (161-2) may each be positioned to overlap at least partially with the first transparent member (196-1) and / or the second transparent member (196-2). For example, the first display (161-1) may be placed on the first transparent member (196-1), or the first display (161-1) may be interposed within the first transparent member (196-1) if the first transparent member (196-1) is composed of multiple layers. Additionally or optionally, the second display (161-2) may be placed on the second transparent member (196-2), or the second display (161-2) may be interposed within the second transparent member (196-2) if the second transparent member (196-2) is composed of multiple layers. In the present disclosure, at least one of the input optical member (162), the transparent member (196), and the display (161), which outputs light and redirects the output light to provide a screen viewable by a user, may be referred to as a "screen display unit."

[0026] According to one embodiment, the electronic device (101) may include at least one camera (e.g., the camera module (380) of FIG. 3). For example, the electronic device (101) may include a first camera (180-1), a second camera (180-2), and / or a third camera (180-3) disposed in a frame (197). The first camera (180-1) may be disposed in a first member (197-1) of the frame (197) so as to be adjacent to a first display (161-1). The second camera (180-2) may be disposed in a second member (197-2) of the frame (197) so as to be adjacent to a second display (161-2). The third camera (180-3) may be disposed in a third member (197-3) of the frame (197). Each of the first camera (180-1), the second camera (180-2), and the third camera (180-3) may include a single or multiple cameras.

[0027] The first camera (180-1) and the second camera (180-2) may be used, for example, to recognize external images. The first camera (180-1) and the second camera (180-2) may be configured to acquire images corresponding to a direction (e.g., +X direction) corresponding to the gaze of a user wearing the electronic device (101). The electronic device (101) may perform head tracking (e.g., 3-degrees of freedom or 6-degrees of freedom (DoF) tracking), hand image detection, hand image tracking, and / or spatial recognition using the first camera (180-1) and the second camera (180-2). For example, the first camera (180-1) and the second camera (180-2) may be GS (global shutter) cameras having the same specifications and performance (e.g., angle of view, shutter speed, resolution, and / or number of color bits, etc.). The electronic device (101) can support simultaneous localization and mapping (SLAM) technology by using stereo cameras positioned on the left and right to perform spatial recognition (e.g., 6-degrees-of-freedom spatial recognition) and / or depth information acquisition. Additionally, the electronic device (101) can recognize user gestures using stereo cameras positioned on the left and right. The electronic device (101) can detect faster hand movements and fine movements by using a GS camera, which has relatively less distortion compared to a rolling shutter (RS) camera.

[0028] The third camera (180-3) may be used, for example, for external image recognition. The third camera (180-3) may be configured to acquire an image corresponding to the direction corresponding to the user's gaze (e.g., +X direction). In one example, the third camera (180-3) may be a camera having a relatively higher resolution compared to the first camera (180-1) and the second camera (180-2). The third camera (180-3) may be referred to as a high-resolution (HR) camera or a photo-video (PV) camera. The third camera (180-3) may support functions for acquiring high-quality images, such as auto focus (AF) and / or optical image stabilization (OIS). The third camera (180-3) may be a GS camera or an RS camera.

[0029] According to one embodiment, the electronic device (101) may include at least one eye-tracking sensor (e.g., sensor module (376) of FIG. 3). For example, the electronic device (101) may include a first eye-tracking sensor (176-1) and a second eye-tracking sensor (176-2). Each of the first eye-tracking sensor (176-1) and the second eye-tracking sensor (176-2) may include at least one camera (e.g., third camera (480-3) and fourth camera (480-4) of FIG. 4) configured to acquire an image in a direction corresponding to the user's eyes, for example. The first eye-tracking sensor (176-1) and the second eye-tracking sensor (176-2) may be configured to acquire an image of the user's right eye and an image of the user's left eye, respectively. The electronic device (101) may be configured to detect the user's pupil based on an eye image obtained using a first eye tracking sensor (176-1) and a second eye tracking sensor (176-2). For example, the electronic device (101) may detect the position (or gaze) and / or change in position (or change in gaze) of the user's pupil based on the eye image. The electronic device (101) may obtain the user's gaze (or data regarding the gaze) based on the user's eye image and display content (e.g., an image) on the screen display based on the obtained gaze. For example, the electronic device (101) may use a display (161) to display an image in a first transparent area (196-1) and / or a second transparent area (196-2) so that the image is positioned in the direction of the user's gaze. For example, the first eye tracking sensor (176-1) and the second eye tracking sensor (176-2) may be GS (global shutter) cameras having the same specifications and performance (e.g., angle of view, shutter speed, resolution, and / or number of color bits, etc.), but are not limited thereto.

[0030] According to one embodiment, the electronic device (101) may include at least one illumination unit. The illumination unit may include, for example, at least one LED. For example, the electronic device (101) may include a first illumination unit (181-1) and a second illumination unit (181-2). The first illumination unit (181-1) and the second illumination unit (181-2) may be placed in a frame (197). For example, the first illumination unit (181-1) may be placed in a first member (197-1) of the frame (197) and the second illumination unit (181-2) may be placed in a second member (197-2) of the frame (197). Each of the first illumination unit (181-1) and the second illumination unit (181-2) may be adjacent to a first camera (180-1) and a second camera (180-2). The first lighting unit (181-1) and the second lighting unit (181-2) may provide auxiliary lighting for the first camera (180-1) and the second camera (180-2). Although not illustrated, the electronic device (101) may include a third lighting unit positioned adjacent to the third camera (180-3). The third lighting unit may provide auxiliary lighting for the third camera (180-3).

[0031] In one example, the electronic device (101) may include lighting units (not shown) respectively positioned around the first transparent member (196-1) and the second transparent member (196-2) in the frame (197). For example, each of the lighting units may be positioned in the third member (197-3) of the frame (197) so as to be adjacent to the first eye tracking sensor (176-1) and the second eye tracking sensor (176-2). Alternatively, each of the lighting units may be included in the first eye tracking sensor (176-1) and the second eye tracking sensor (176-2) and implemented integrally with them. The lighting units may be positioned toward the user's eyeball to provide lighting (e.g., light of infrared wavelength) for the first eye tracking sensor (176-1) and the second eye tracking sensor (176-2) to acquire a pupil image. The lighting unit may include an LED configured to emit light of infrared wavelength. Each of the first eye tracking sensor (176-1) and the second eye tracking sensor (176-2) may include an image sensor for acquiring an infrared wavelength image.

[0032] According to one embodiment, the electronic device (101) may include at least one battery (e.g., battery (389) of FIG. 3). For example, the electronic device (101) may include a first battery (189-1) located at one end of a third sub-arm (198-1) and a second battery (189-2) located at one end of a fourth sub-arm (198-2). The first battery (189-1) and the second battery (189-2) may be configured to supply power to the components of the electronic device (101).

[0033] According to one embodiment, the electronic device (101) may include at least one speaker (e.g., the acoustic output module (355) of FIG. 3). For example, the electronic device (101) may include a first speaker (170-1) and a second speaker (170-2). The first speaker (170-1) and the second speaker (170-2) may be placed on a second sub-arm (198-1) and a fourth sub-arm (198-2), respectively. The electronic device (101) may be configured to provide stereo sound using speakers located on the left and right sides.

[0034] According to one embodiment, the electronic device (101) may include at least one microphone (e.g., the audio module (370) of FIG. 3). For example, the electronic device (101) may include a first microphone (171-1), a second microphone (171-2), and / or a third microphone (171-3). The first microphone (171-1), the second microphone (171-2), and the third microphone (171-3) may be placed on the first member (197-1), the second member (197-2), and the third member (197-3) of the frame (197), respectively. For example, the first microphone (171-1) and the second microphone (171-2) may be placed on the lower ends of the first member (197-1) and the second member (197-2), respectively, but are not limited thereto.

[0035] Referring to FIGS. 1a and 1b, an electronic device (101) according to one embodiment may include a first PCB (187-1), a second PCB (187-2), and a third PCB (487). The first PCB (187-1) and the second PCB (187-2) may be placed in the second sub-arm (298-1) of the first arm (193-1) and the fourth sub-arm (198-2) of the second arm (193-2), respectively. The first PCB (187-1) and the second PCB (187-2) may be electrically connected to other components of the electronic device (101) through signal lines and / or a flexible PCB (FPCB). Each of the first PCB (187-1) and / or the second PCB (187-2) may be implemented as a substrate assembly comprising a plurality of PCBs and an interposer disposed between the plurality of PCBs, but is not limited thereto.

[0036] In one embodiment, the third PCB (487) may be placed within the first arm (193-1), the frame (197), and the second arm (193-2). The third PCB (487) may extend from the first arm (193-1), beyond the frame (197), to the second arm (193-2). For example, the third PCB (487) may extend along the outer surface of the housing within the first arm (193-1), the frame (197), and the second arm (193-2). The third PCB (487) may include at least a partially flexible portion to bend according to the shape of the housing. For example, a fourth portion (4874) of the third PCB (487) extending from the first sub-arm (195-1) along the edge of the first member (197-1) of the frame (197) may be configured to be flexible. The above-mentioned fourth portion (4874) may include a portion bent at the boundary between the first sub-arm (195-1) and the first member (197-1) and a portion bent according to the shape of the first member (197-1). Additionally, the fifth portion (4875) of the third PCB (487), which extends from the third sub-arm (195-2) along the edge of the second member (197-2) of the frame (197), may be configured to be flexible. The above-mentioned fifth portion (4875) may include a portion bent at the boundary between the third sub-arm (195-2) and the second member (197-2) and a portion bent according to the shape of the second member (197-2).

[0037] In one embodiment, the first part (4871), the second part (4872), and the third part (4873) of the third PCB (487) may be formed rigidly. The first part (4871) and the second part (4872) may be located at both ends of the fourth part (4874) and may be connected by the fourth part (4874). The first part (4871) may be connected to the first PCB (187-1) through a connecting member (1) passing through the first hinge part (199-1). The fourth part (4874) may be located between the second part (4872) and the third part (4873) to connect the second part (4872) and the third part (4873) to each other. The third part (4873) may be connected to the second PCB (187-2) through a connecting member (2) passing through the second hinge part (199-2). The connecting members (1, 2) may be formed flexibly. For example, the connecting members (1, 2) may include a flexible printed circuit (FPC) (e.g., a board-to-board connector) having connecting members (e.g., connectors) provided at both ends, but are not limited thereto. As another example, the connecting member (1) may be joined to the first PCB (187-1) by a hot-bar method rather than the connector. As another example, the connecting member (1) may include a flexible cable. The description of the connecting member (1) described above may apply equally to the connecting member (2). In one embodiment, the connecting members (1, 2) can be bent to correspond to the folding operation of the first hinge portion (199-1) and the second hinge portion (199-2).

[0038] In one embodiment, components of an electronic device (101) placed on the first PCB (187-1) and the second PCB (187-2) can be operatively connected to components placed on the frame (197) via the third PCB (487). This will be described later with reference to FIG. 4.

[0039] The configuration of the electronic device (101) described above is exemplary and the embodiments of this document are not limited thereto. For example, the electronic device (101) may not include at least some of the components described in relation to FIGS. 1a and 1b, or may include additional components other than those described. For example, the electronic device (101) may include at least one of the components of the electronic device (301) of FIG. 3 to be described later. For example, the electronic device (101) may include at least one sensor (e.g., accelerometer, gyroscope, and / or touch sensor, etc.) and / or an antenna.

[0040] FIG. 2 is a schematic diagram of a method for eye tracking and displaying through a transparent member according to one embodiment. Referring to FIG. 2, a display (261) (e.g., the display (161) of FIG. 1a) can provide an image of a transparent member (296) (e.g., the first transparent member (196-1) or the second transparent member (196-2) of FIG. 1a). According to one embodiment, the display (261) can input light corresponding to the image to an input optical member (262) (e.g., the first input optical member (162-1) or the second input optical member (162-2) of FIG. 1a) through a lens (251). The input optical member (262) can reflect or diffract the incident light and input it to an optical waveguide (260). An output optical member (264) can output the light transmitted through the optical waveguide (260) toward the user's eye (299). In one example, the lens (251) may be included in the display (261). In one example, the position of the lens (251) may be determined based on the distance between the transparent member (296) and the user's eye (299).

[0041] The eye tracking sensor (271) (e.g., the first eye tracking sensor (176-1) or the second eye tracking sensor (176-2) of FIG. 1a) can acquire an image corresponding to at least a portion of the user's eye (299). For example, light corresponding to the image of the user's eye (299) can be reflected and / or diffracted through the first splitter (381) and input into the optical waveguide (282). Light transmitted to the second splitter (283) through the optical waveguide (282) can be reflected and / or diffracted by the second splitter (283) and output in the direction of the eye tracking sensor (271).

[0042] FIG. 3 is a block diagram of an electronic device (301) in a network environment (300) according to one embodiment. Referring to FIG. 3, in the network environment (300), the electronic device (301) may communicate with an electronic device (302) through a first network (398) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (304) or a server (308) through a second network (399) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (301) may communicate with the electronic device (304) through the server (308). According to one embodiment, the electronic device (301) may include a processor (320), memory (330), input module (350), sound output module (355), display module (360), audio module (370), sensor module (376), interface (377), connection terminal (378), haptic module (379), camera module (380), power management module (388), battery (389), communication module (390), subscriber identification module (396), or antenna module (397). In some embodiments, at least one of these components (e.g., connection terminal (378)) may be omitted from the electronic device (301), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (376), camera module (380), or antenna module (397)) may be integrated into a single component (e.g., display module (360)).

[0043] The processor (320) can control at least one other component (e.g., a hardware or software component) of the electronic device (301) connected to the processor (320) by executing software (e.g., a program (340)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (320) can store commands or data received from other components (e.g., a sensor module (376) or a communication module (390)) in volatile memory (332), process the commands or data stored in volatile memory (332), and store the resulting data in non-volatile memory (334). According to one embodiment, the processor (320) may include a main processor (321) (e.g., a central processing unit or an application processor) or an auxiliary processor (323) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (301) includes a main processor (321) and an auxiliary processor (323), the auxiliary processor (323) may be configured to use lower power than the main processor (321) or to be specialized for a designated function. The auxiliary processor (323) may be implemented separately from the main processor (321) or as part thereof.

[0044] The auxiliary processor (323) may control at least some of the functions or states associated with at least one component of the electronic device (301) (e.g., display module (360), sensor module (376), or communication module (390)) on behalf of the main processor (321) while the main processor (321) is in an inactive (e.g., sleep) state, or together with the main processor (321) while the main processor (321) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (323) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (380) or communication module (390)). According to one embodiment, the auxiliary processor (323) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (301) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (308)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0045] The memory (330) can store various data used by at least one component of the electronic device (301) (e.g., processor (320) or sensor module (376)). The data may include, for example, software (e.g., program (340)) and input data or output data for related commands. The memory (330) may include volatile memory (332) or non-volatile memory (334).

[0046] The program (340) may be stored as software in memory (330) and may include, for example, an operating system (342), middleware (344), or an application (346).

[0047] The input module (350) can receive commands or data to be used for a component of the electronic device (301) (e.g., processor (320)) from outside the electronic device (301) (e.g., user). The input module (350) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0048] The sound output module (355) can output a sound signal to the outside of the electronic device (301). The sound output module (355) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0049] The display module (360) can visually provide information to an external (e.g., user) of the electronic device (301). The display module (360) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (360) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0050] The audio module (370) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (370) can acquire sound through the input module (350) or output sound through the sound output module (355) or an external electronic device (e.g., electronic device (302)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (301).

[0051] The sensor module (376) can detect the operating state of the electronic device (301) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (376) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0052] The interface (377) may support one or more specified protocols that can be used for the electronic device (301) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (302)). According to one embodiment, the interface (377) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0053] The connection terminal (378) may include a connector through which the electronic device (301) can be physically connected to an external electronic device (e.g., electronic device (302)). According to one embodiment, the connection terminal (378) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0054] The haptic module (379) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (379) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0055] The camera module (380) can capture still images and video. According to one embodiment, the camera module (380) may include one or more lenses, image sensors, image signal processors, or flashes.

[0056] The power management module (388) can manage power supplied to the electronic device (301). According to one embodiment, the power management module (388) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0057] The battery (389) can supply power to at least one component of the electronic device (301). According to one embodiment, the battery (389) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0058] The communication module (390) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (301) and an external electronic device (e.g., electronic device (302), electronic device (304), or server (308)), and the performance of communication through the established communication channel. The communication module (390) may include one or more communication processors that operate independently of the processor (320) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (390) may include a wireless communication module (392) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (394) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (304) through a first network (398) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (399) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (392) can identify or authenticate the electronic device (301) within a communication network such as the first network (398) or the second network (399) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (396).

[0059] The wireless communication module (392) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (392) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (392) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (392) can support various requirements specified in the electronic device (301), external electronic device (e.g., electronic device (304)), or network system (e.g., second network (399)). According to one embodiment, the wireless communication module (392) can support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0060] An antenna module (397) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (397) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (397) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (398) or a second network (399), may be selected from the plurality of antennas, for example, by a communication module (390). A signal or power may be transmitted or received between the communication module (390) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (397).

[0061] According to one embodiment, the antenna module (397) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0062] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0063] According to one embodiment, commands or data may be transmitted or received between the electronic device (301) and an external electronic device (304) through a server (308) connected to a second network (399). Each of the external electronic devices (302, or 304) may be the same or a different type of device as the electronic device (301). According to one embodiment, all or part of the operations performed on the electronic device (301) may be performed on one or more of the external electronic devices (302, 304, or 308). For example, if the electronic device (301) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (301) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (301). The electronic device (301) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (301) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (304) may include an Internet of Things (IoT) device. The server (308) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (304) or the server (308) may be included within a second network (399).The electronic device (301) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0064] FIG. 4 shows an example of a distributed system for content processing and vision processing implemented in an electronic device according to one embodiment.

[0065] Referring to FIG. 4, an electronic device (401) according to one embodiment may include a content system (410) and a vision system (412). The content system (410) may include hardware(s), software(s), and / or a group thereof associated with displaying content on a screen display of the electronic device (401). In one embodiment, the content system (410) may include a first memory (432-1), a first PMIC (488-1), a first communication circuit (492-1), a touch circuit (460), and a first processor (420-1) operatively or electrically connected to them. Additionally, the electronic device (401) may include a third PMIC (488-3), a first amplifier (457-1), a first speaker (455-1), a first sensor (476-1), and a first battery (489-1).

[0066] In one embodiment, the first processor (420-1), first PMIC (488-1), first communication circuit (492-1), and touch circuit (460) of the content system (410) may be placed on the first PCB (187-1). The first processor (420-1) (e.g., processor (320) of FIG. 3) may be electrically connected to the first display (161-1) and the second display (161-2). The first display (161-1) may be electrically connected to the first part (4871) of the third PCB (487) through a connecting member (3), and the second display (161-2) may be electrically connected to the third part (4873) of the third PCB (487) through a connecting member (4). The first part (4871) and the third part (4873) may be electrically connected to the first PCB (187-1) through a connecting member (1). The first processor (420-1) placed on the first PCB (187-1) may be electrically connected to the first display (161-1) and the second display (161-2) through an electrical path provided by the first PCB (187-1), the third PCB (487), and the connecting members (1, 3, 4). The electrical path may include a plurality of transmission lines formed of a conductive material. The plurality of transmission lines may be configured to transmit and receive data signals according to a designated interface, such as a display interface of the MIPI (Mobile Industry Processor Interface) standard, and communication signals (or control signals) according to I2C (inter integrated circuit) or SPI (serial peripheral interface).

[0067] In one embodiment, the first communication circuit (492-1) (e.g., the communication module (390) of FIG. 3) may include a wired communication circuit (e.g., the wired communication module (394) of FIG. 3) and / or a wireless communication circuit (e.g., the wireless communication module (392) of FIG. 3). If the first communication circuit (492-1) includes a wired communication circuit, a connector (e.g., the connection terminal (378) of FIG. 3) for providing a wired connection to the first PCB (287-1) may be additionally provided. The first communication circuit (492-1) may be configured to transmit and receive communication signals with an external device (e.g., the electronic device (302, 304) or server (308) of FIG. 3). For example, the first communication circuit (492-1) may transmit and receive wireless signals with the external device via a wireless network (e.g., the first network (398) or the second network (399) of FIG. 3). As another example, the first communication circuit (492-1) may be configured to support a tethering connection with an external device via wired or wireless means. The first processor (420-1) may use the first communication circuit (492-1) to receive data for outputting a screen to the display (161).

[0068] The first PMIC (488-1) (e.g., the power management module (388) of FIG. 3) can manage power supplied to the first processor (420-1). Additionally, the first PMIC (488-1) and / or the third PMIC (488-3) can manage power supplied to components placed on the first PCB (187-1), such as the first communication circuit (492-1), the touch circuit (460), the first sensor (476-1), and the first amplifier (457-1). The third PMIC (488-3) (e.g., the power management module (388) of FIG. 3) can be placed on the first PCB (187-1) and electrically connected to the first battery (489-1) and the first processor (420-1). The third PMIC (488-3) may include, for example, a fuel gauge integrated circuit for monitoring the power of the first battery (489-1), and a charger IC including a limiter and an over-voltage protection IC. Alternatively or optionally, the third PMIC (488-3) may be integrated into the first PMIC (488-1). In this case, the first PMIC (488-1) may perform substantially the same function as the third PMIC (488-3). The first battery (489-1) (e.g., battery (389) in FIG. 3) may be connected to the first PCB (187-1) via a connecting member (10) to provide power to the components of the electronic device (401).

[0069] In one embodiment, a first amplifier (amp) (457-1) may be placed on a first PCB (187-1). The first amplifier (457-1) may be electrically connected to a first processor (420-1). A first speaker (455-1) may be placed within a first arm (193-1) and connected to the first PCB (187-1) via a connecting member (9). The first speaker (455-1) may be electrically connected to the first amplifier (457-1). The first amplifier (457-1) may amplify an audio signal provided from the first processor (420-1) and transmit it to the first speaker (455-1), and the first speaker (455-1) may output the provided audio signal. The first amplifier (457-1) and the first speaker (455-1) can be referred to as the acoustic output module (355) of FIG. 3.

[0070] In one embodiment, the first sensor (476-1) may be placed on the first PCB (187-1) and electrically connected to the first processor (420-1). The first sensor (476-1) may include, for example, an inertial measurement unit (IMU). The first processor (420-1) may perform a latest stage re-projection (LSR) operation based on the value detected using the first sensor (476-1). This reduces the motion to photon (MTP), which refers to the delay time until the screen is updated according to the user's movement (or displacement of the electronic device (401)). Alternatively or optionally, the first sensor (476-1) may be placed on the second PCB (187-2). The first sensor (476-1) placed on the second PCB (187-2) is electrically connected to the second processor (420-2), so that operations associated with the first sensor (476-1) can be performed by the second processor (420-2).

[0071] In one embodiment, a touch circuit (or touch sensor IC) (460) may be configured to detect a touch input from a user. Although not illustrated, the touch circuit (460) may detect a touch input on a sensing area (e.g., the surface (1981) of the first arm (193-1) in FIG. 1a) and process the detected signal and provide it to a first processor (420-1). Based on the signal provided by the touch circuit (460), the first processor (420-1) may obtain information regarding the coordinates, time, and pressure of the user touch input. Additionally, a sensing area for detecting a touch signal may be formed on the second arm (193-2) symmetrically to the first arm (193-1). In this case, another touch circuit (not illustrated) distinct from the touch circuit (460) may be placed on the second PCB (187-2), but is not limited thereto.

[0072] The vision system (412) may include related hardware(s), software(s), and / or a group thereof to detect information about the user of the electronic device (401). In one embodiment, the vision system (412) may include a second PMIC (488-2), a second communication circuit (492-2), and a second processor (420-2) operatively or electrically connected to them. Additionally, the electronic device (401) may include a fourth PMIC (488-4), a second amplifier (457-2), a second speaker (455-2), a second sensor (476-2), a second battery (489-2), and a lighting unit driving circuit (481).

[0073] In one embodiment, the second processor (420-2), second PMIC (488-2), and second communication circuit (492-2) of the vision system (412) may be placed on the second PCB (187-2). The second processor (420-2) (e.g., processor (320) of FIG. 3) may be electrically connected to at least one camera. For example, the second processor (420-2) may be electrically connected to the first to fourth cameras (480-1 to 480-4). The first camera (480-1) (e.g., first camera (180-1) of FIG. 1a) may be connected to the first part (4871) of the third PCB (487) through a connecting member (5). The second camera (480-2) (e.g., the second camera (180-2) of FIG. 1a) can be connected to the third part (4873) of the third PCB (487) via a connecting member (6). The third camera (480-3) (e.g., the first eye-tracking sensor (176-1) of FIG. 1a) and the fourth camera (480-4) (e.g., the second eye-tracking sensor (176-2)) can each be connected to the second part (4872) of the third PCB (487) via connecting members (7, 8). A second processor (420-2) placed on a second PCB (187-2) can be electrically connected to the first to fourth cameras (480-1 to 480-4) through an electrical path provided by the second PCB (187-2), the third PCB (487), and connecting members (5, 6, 7, 8). The electrical path may include a plurality of transmission lines formed of a conductive material. The plurality of transmission lines may be configured to transmit and receive data signals according to a designated interface, such as a camera interface of the MIPI standard, and communication signals (or control signals) according to I2C.

[0074] In one embodiment, the second communication circuit (492-2) (e.g., the communication module (390) of FIG. 3) may include a wired communication circuit (e.g., the wired communication module (394) of FIG. 3) and / or a wireless communication circuit (e.g., the wireless communication module (392) of FIG. 3). If the second communication circuit (492-2) includes a wired communication circuit, a connector (e.g., the connection terminal (378) of FIG. 3) for providing a wired connection to the second PCB (287-2) may be additionally provided. The second communication circuit (492-2) may be configured to transmit and receive communication signals with an external device (e.g., the electronic device (302, 304) or server (308) of FIG. 3). For example, the second communication circuit (492-2) may transmit and receive wireless signals with the external device via a wireless network (e.g., the first network (398) or the second network (399) of FIG. 3). As another example, the second communication circuit (492-2) may be configured to support a tethering connection with an external device via wired or wireless means. The second processor (420-2) may use the second communication circuit (492-2) to transmit a value detected by the first to fourth cameras (480-1 to 480-4) or a signal based thereon to the external device.

[0075] In one embodiment, the second PMIC (488-2) (e.g., the power management module (388) of FIG. 3) can manage power supplied to the second processor (420-2). Additionally, the second PMIC (488-2) and / or the fourth PMIC (488-4) can manage power supplied to components placed on the second PCB (187-2), such as the second communication circuit (492-2), the second sensor (476-2), the third microphone (471-3), the second amplifier (457-2), and the lighting unit driving circuit (481). The fourth PMIC (488-4) (e.g., the power management module (388) of FIG. 3) can be placed on the second PCB (187-2) and electrically connected to the second battery (489-2) and the second processor (420-2). For the fourth PMIC (488-4), the description provided with reference to the second PMIC (488-2) may be applied in a corresponding manner. For example, the fourth PMIC (488-4) can prevent overcharging of the second battery (489-2), manage the output voltage, and monitor the remaining amount. Alternatively, the fourth PMIC (488-4) may be integrated into the second PMIC (488-2), in which case the second PMIC (488-2) may perform substantially the same functions as the fourth PMIC (488-4). The second battery (489-2) (e.g., battery (389) of FIG. 3) may be connected to the second PCB (187-2) via a connecting member (12) to provide power to the components of the electronic device (401).

[0076] In one embodiment, the second amplifier (457-2) may be placed on the second PCB (187-2). The second amplifier (457-2) may be electrically connected to the second processor (420-2). The second speaker (455-2) may be placed within the second arm (193-2) and connected to the second PCB (187-2) via a connecting member (11). For the second amplifier (457-2) and the second speaker (455-2), the description provided with reference to the first amplifier (457-1) and the first speaker (455-1) may be applied in a corresponding manner. For example, the second amplifier (457-2) and the second speaker (455-2) may be referred to as the acoustic output module (355) of FIG. 3.

[0077] In one embodiment, the second sensor (476-2) may be placed on the second PCB (187-2) and electrically connected to the second processor (420-2). The second sensor (476-2) may include, for example, a proximity sensor. The second processor (420-2) may determine whether the electronic device (401) is worn by a user based on a value detected using the second sensor (476-2). In another embodiment, the second sensor (476-2) may be placed on the first PCB (187-1) and controlled by the first processor (420-1). Optionally or alternatively, the electronic device (401) may not include the second sensor (476-2). In this case, the electronic device (401) can determine whether the electronic device (401) is worn by a user by using at least one of the first to fourth cameras (480-1 to 480-4).

[0078] In one embodiment, the lighting unit driving circuit (481) is placed on the second PCB (187-2) and can be electrically connected to the second processor (420-2). The lighting unit driving circuit (481) can drive a first IR LED and a second IR LED, which are not illustrated. The first IR LED is placed adjacent to the third camera (480-3) and can be connected to the third PCB (487) via a connecting member (7). The second IR LED is placed adjacent to the fourth camera (480-4) and can be connected to the third PCB (487) via a connecting member (8). Through the third PCB (487), a power signal for driving the first IR LED and the second IR LED can be provided. Additionally, through the third PCB (487), a control signal for operating the first IR LED and the second IR LED can be provided. The control signal can be provided from the lighting unit driving circuit (481). The first IR LED and the second IR LED can emit IR light for the third camera (480-3) and the fourth camera (480-4) to detect the user's gaze. In another embodiment, the first IR LED and the second IR LED may be integrated into the third camera (480-3) and the fourth camera (480-4), respectively.

[0079] In one embodiment, the third microphone (471-3) may be placed on the second PCB (187-2) and electrically connected to the second processor (420-2). In another embodiment, as with the third microphone (171-3) of FIG. 1a, the third microphone (471-3) may be placed on the second part (4872) of the third PCB (487) or on the third member (197-3) of the frame (197). In yet another embodiment, the third microphone (471-3) may be placed on the first PCB (187-1) and electrically connected to the first processor (420-1).

[0080] In one embodiment, the first processor (420-1) of the content system (410) can execute instructions stored in the first memory (432-1) (e.g., memory (330) of FIG. 3). The electronic device (401) can control the content system (410) by executing instructions established in the first memory (432-1) using the first processor (420-1).

[0081] The second processor (420-2) of the vision system (412) (e.g., processor (320) of FIG. 3) can execute instructions stored in the second memory (432-2) (e.g., memory (330) of FIG. 3). The electronic device (401) can control the vision system (412) by executing instructions established in the second memory (432-2) using the second processor (420-2).

[0082] In one embodiment, the content system (410) and the vision system (412) may operate independently. Optionally, a transmission line may be provided for synchronizing the content system (410) and the vision system (412). The transmission line may, for example, electrically connect the first processor (420-1) and the second processor (420-2) to each other, and communication between the first processor (420-1) and the second processor (420-2) may be performed through the transmission line. The transmission line may be implemented via a separate cable passing through the third PCB (487) or frame (197). The transmission line may be configured to support interface protocols such as SPI, UART (universal asynchronous receiver transmitter), or USB (universal serial bus), but is not limited to the examples described above.

[0083] The operation of the vision system (412) and the content system (410) will be described below with reference to FIG. 5. In the following description, the content processing method and the vision processing method of the distributed system implemented in the electronic device (401) are described as being performed by the vision system (412) and the content system (410), but are not limited thereto. The content processing method may be performed by the electronic device (401) or the first processor (420-1), and the vision processing method may be understood as being performed by the electronic device (401) or the second processor (420-2).

[0084] FIG. 5 is a schematic diagram illustrating a content processing and vision processing method of a distributed system implemented in an electronic device according to one embodiment. Referring to FIG. 5, according to one embodiment, a vision system (412) can acquire at least one image using at least one camera (480) (e.g., the first to fourth cameras (480-1 to 480-4) of FIG. 4). The vision system (412) can perform head tracking and gaze tracking actions of a user based on the acquired at least one image and values ​​detected using at least one sensor (476) (e.g., the first to third sensors (476-1 to 476-3) of FIG. 4). The vision system (412) can acquire first data (or first information) regarding gaze movement and second data (or second information) regarding head movement of a user. Additionally, the vision system (412) can acquire third information about an external object using at least one camera (480) and at least one sensor (476). For example, the third information may include information about the location of an external object (e.g., a TV) and tracking information about the speed and direction of the external object to the electronic device (401). The vision system (412) can transmit at least one of the first information, the second information, or the third information to an external electronic device (403). For example, the vision system (412) can transmit at least one of the first information, the second information, or the third information to an external electronic device (403) through a first communication network (N1) established using a second communication circuit (492-2). Additionally, the vision system (412) may optionally transmit information about the content of an application executed on the electronic device (401) (or an application running in the foreground) along with the at least one piece of information to an external electronic device (403).In one embodiment, the first communication network (N1) may include the first network (398) of FIG. 3 or the second network (399) of FIG. 3. Additionally, the first communication network (N1) may include a wired / wireless tethering connection between an electronic device (401) and an external electronic device (403).

[0085] In one embodiment, the external electronic device (403) may obtain screen information by rendering content based on data (or information) received from the electronic device (401). For example, the external electronic device (403) may render content related to the external object based on third information about the external object received from the electronic device (401). The content may include, for example, visual objects representing information related to the external object (e.g., specifications of a TV) and / or control menus related to the external object (e.g., a menu for accessing a web page to purchase a TV), but is not limited to the examples described above. The visual objects may be displayed on the screen display unit adjacent to or at least partially overlapping the external object, but are not limited thereto. The rendering of the content to be displayed may be performed on a server (408) (e.g., a cloud server) rather than on the external electronic device (403). In this case, the external electronic device (403) can transmit data received from the electronic device (401) to the server (408) via the second communication network (N2). Screen information obtained by the server (408) can be transmitted from the server (408) to the external electronic device (403) via the second communication network (N2), and the external electronic device (403) can transmit the screen information to the electronic device (401). Alternatively, unlike what is illustrated, the external electronic device (403) may be omitted, and the obtained data regarding user movement and the screen information based thereon may be transmitted and received between the electronic device (401) and the server (408) via the first communication network (N1) or the second communication network (N2).

[0086] The second communication network (N2) may include, for example, a short-range communication network such as Wi-Fi (e.g., the first network (398) of FIG. 3) established via a wireless access point (A). As another example, the second communication network (N2) may include a 4G (4) established via a base station (B). th generation), 5G(5 th It may include a generation) or a next-generation cellular network (e.g., the second network (399) of FIG. 3).

[0087] In one embodiment, the content system (410) may receive screen information of the content to be displayed from an external electronic device (403) or a server (408). For example, the content system (410) may receive the screen information using a first communication circuit (492-1). In one embodiment, the content system (410) may display content on a screen display unit based on the received screen information.

[0088] In one embodiment, the content system (410) can detect the movement of a user while the content is being displayed on the screen display. For example, the content system (410) can detect the movement of a user and / or an external object using a first sensor (476-1) (or the first sensor (476-1) and a third sensor (476-3)). For example, the content system (410) can detect a value regarding the posture of the electronic device (401) using the first sensor (476-1) and determine whether the posture of the electronic device (401) has changed based on the detected value. In this case, the content system (410) can determine that there is movement of the user (or the electronic device (401) worn by the user). In one embodiment, when movement of a user and / or an external object is detected while the content is being displayed on the screen display, screen information received from the external electronic device (401) can be corrected based on the detected movement information (e.g., fourth information). The content system (410) can output the corrected screen information to the screen display. In another embodiment, the movement information of the user and / or an external object may be acquired by a vision system (412). In this case, the vision system (412) may send a message to the external electronic device (403) or server (408) requesting rendering so that the screen information is updated based on the movement information. The external electronic device (403) or server (408) may update and render the content in response to the message received from the electronic device (401). The external electronic device (403) or server (408) may acquire the updated screen information. The vision system (412) can receive the updated screen information from an external electronic device (403) or a server (408) and, based on this, can display the updated content on the screen display.

[0089] A wearable electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1a) comprises: a frame (e.g., the frame (197) of FIG. 1a); a first arm (e.g., the first arm (193-1) of FIG. 1a) and a second arm (e.g., the second arm (193-2) of FIG. 1a) each extending from the frame; a screen display unit disposed in the frame and including a display (e.g., at least one of an input optical member (162), a transparent member (196), and a display (161) of FIG. 1a); at least one camera disposed in the frame (e.g., the camera module (380) of FIG. 3); and a first processor (e.g., the first processor (420-1) of FIG. 4) located within the first arm and operably connected to the display. and a second processor (e.g., the second processor (420-2) of FIG. 4) located within the second arm and operatively connected to the at least one camera, wherein the first processor is configured to display content on the screen display unit using the display, and the second processor may be configured to perform an operation associated with the at least one camera.

[0090] The second processor may be configured to acquire at least one of: first information regarding the direction of the user's gaze; second information regarding the direction of the user's head movement; or third information regarding the location of an external object using the at least one camera.

[0091] The wearable electronic device is electrically connected to the second processor and includes a second communication circuit (e.g., the second communication circuit (492-2) of FIG. 4) located within the second arm, and the second processor may be configured to transmit at least one of the acquired first information, the second information, or the third information to an external electronic device (e.g., the external device (403) of FIG. 5) using the second communication circuit.

[0092] The wearable electronic device is electrically connected to the first processor and includes a first communication circuit (e.g., the first communication circuit (492-1) of FIG. 4) located within the first arm, and the first processor is configured to receive screen information from the external electronic device using the first communication circuit, and the screen information may be information obtained by rendering data associated with the content based on at least one of the first information, the second information, or the third information received from the wearable electronic device.

[0093] The above screen information can be obtained by the external electronic device or a server that has established wireless communication with the external electronic device (e.g., the server (408) of FIG. 5).

[0094] The first processor above may be configured to display the content on the screen display unit based on the received screen information.

[0095] The wearable electronic device includes at least one sensor (e.g., the first sensor (476-1) of FIG. 4) that is electrically connected to the first processor, and the first processor may be configured to detect a value for the posture of the wearable electronic device using the at least one sensor, and to determine whether the posture of the wearable electronic device has changed based on the detected value while displaying the content on the screen display.

[0096] The first processor may be configured to obtain fourth information regarding the changed posture of the wearable electronic device based on a value detected by at least one sensor when the posture of the wearable electronic device changes while the content is displayed on the screen display, and to correct the content being displayed on the screen display based on the obtained fourth information.

[0097] The first processor may be configured to, when the posture of the wearable electronic device changes while displaying the content on the screen display unit: acquire a fourth information regarding the changed posture of the wearable electronic device based on a value detected by at least one sensor, transmit the fourth information to the external electronic device using the first communication circuit, and transmit a message requesting an update of the screen information based on the fourth information to the external electronic device using the first communication circuit.

[0098] The wearable electronic device comprises a first PCB disposed within the first arm (e.g., the first PCB (187-1) of FIG. 1b); and a second PCB disposed within the second arm (e.g., the second PCB (187-2) of FIG. 1b), wherein the first processor may be disposed in the first PCB and the second processor may be disposed in the second PCB.

[0099] A third PCB (e.g., the third PCB (487) of FIG. 1b) extending from the first arm along the edge of the frame to the second arm, including at least a partially flexible portion, and electrically connected to the first PCB and the second PCB, wherein the display is electrically connected to the first processor through the third PCB, and the at least one camera can be electrically connected to the second processor through the third PCB.

[0100] A wearable electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1a) comprises a frame (e.g., the frame (197) of FIG. 1a); A first arm extending from the edge portion of the first direction of the frame (e.g., the first arm (187-1) of FIG. 1a); a second arm extending from the edge portion of the frame opposite to the first direction (e.g., the second arm (187-2) of FIG. 1a); a screen display unit disposed in the frame and including a display (e.g., at least one of the input optical member (162), transparent member (196), and display (161) of FIG. 1a); a sensor unit disposed in the frame and including a head tracking sensor (e.g., the first camera (180-1) and the second camera (180-2) of FIG. 1a) and a gaze tracking sensor (the first gaze tracking sensor (176-1) and the second gaze tracking sensor (176-2) of FIG. 1a)) (at least one camera (480) and / or at least one sensor (476) of FIG. 5); a first for processing content displayed on the screen display unit It includes a system (e.g., content system (410) of FIG. 4); and a second system (e.g., vision system (412) of FIG. 4) for processing values ​​detected by the sensor unit, wherein the first system is positioned on the first arm and the second system can be positioned on the second arm.

[0101] The first system may include: a receiver configured to receive a wireless signal (e.g., the first communication circuit (492-1) of FIG. 4); and a first processor (e.g., the first processor (420-1) of FIG. 4) operatively connected to the receiver and the display, and the second system may include: a transmitter configured to transmit a wireless signal (e.g., the second communication circuit (492-2) of FIG. 4); and a second processor (e.g., the second processor (420-2) of FIG. 4) operatively connected to the transmitter and the sensor unit.

[0102] The second processor above may be configured to acquire first information regarding the direction of the user's gaze and second information regarding the direction of the user's head movement based on a value detected using the sensor unit.

[0103] The second processor may be configured to provide the first information and the second information to an external electronic device (e.g., the external device (403) of FIG. 5) using the transmitter, and to provide third information about the content to be provided through the display to the external electronic device using the transmitter.

[0104] The first processor is configured to receive screen information from the external electronic device using the receiver, and the screen information may be information obtained by rendering data associated with the content based on the first information, the second information, and the third information received from the wearable electronic device.

[0105] The first processor above may be configured to display the content on the screen display unit based on the received screen information.

[0106] It includes at least one sensor (e.g., the first sensor (476-1) of FIG. 4) that is distinct from the sensor unit and electrically connected to the first processor, and the first processor may be configured to detect whether the posture of the wearable electronic device has changed using the at least one sensor while displaying the content on the screen display unit.

[0107] The first processor may be configured to acquire fourth information regarding the changed posture of the wearable electronic device using at least one sensor when the posture of the wearable electronic device changes while the content is being displayed on the screen display, and to correct the content being displayed on the screen display based on the acquired fourth information.

[0108] The first processor may be configured to, when the posture of the wearable electronic device changes while displaying the content on the screen display unit: acquire fourth information regarding the changed posture of the wearable electronic device using the at least one sensor, transmit the fourth information to the external electronic device using the transmitter, and transmit a message requesting an update of the screen information based on the fourth information to the external electronic device using the transmitter.

[0109] FIG. 6 shows the heat generation temperature of an electronic device in which a distributed system is implemented according to one embodiment. FIG. 7 shows the heat generation temperature of an electronic device according to a comparative embodiment.

[0110] Referring to FIG. 6, the content system (410) and the vision system (412) are distributed in the first arm (193-1) and the second arm (193-2), respectively, so that the heat generated by the electronic device (401) according to one embodiment can be evenly distributed overall. This may be because the current consumption of the content system (410) and the current consumption of the vision system (412) are similar. For example, the power consumption of the first processor (420-1) of the content system (410) may be about 300 mW, and the power consumption of the first communication circuit (492-1) may be about 250 mW. The power consumption of the second processor (420-2) of the vision system (412) on the opposite side may be about 550 mW. In addition, since the signal lines required for the content system (410) and the vision system (412) are distributed in the first arm (193-1) and the second arm (193-2), design constraints regarding the volume and mounting space of the electronic device (401) can be reduced, and signal interference that may be caused by high-density signal lines can be reduced.

[0111] Referring to FIG. 7, in the comparative embodiment, the electronic device (701) may have both the content system (410) and the vision system (412) placed on the first arm (193-1). In this case, the temperature of the first arm (193-1) may be relatively higher than that of the second arm (193-2). Therefore, in order to reduce the heat dissipation deviation of the electronic device (701) according to the comparative embodiment, a separate heat dissipation structure is required on the first arm (193-1), which may result in increased weight and reduced comfort due to the deviation. Additionally, in order to resolve the weight deviation caused by the heat dissipation structure, the electronic device (701) according to the comparative embodiment may require an additional structure on the second arm (193-2) to balance the weight.

[0112] The electronic device (401) according to one embodiment does not require the heat dissipation structure required in the comparative embodiment described above, so unnecessary weight increase can be prevented. In addition, the electronic device (401) according to one embodiment maintains weight balance even without the additional structure required to balance weight in the comparative embodiment described above, so the user's wearing comfort can be improved without unnecessary weight increase.

[0113] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, each of phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another component and do not limit the components in any other aspect (e.g., importance or order). Where any (e.g., first) component is referred to as “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicationally,” it means that said component may be connected to said other component directly (e.g., wired), wirelessly, or through a third component.

[0114] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0115] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0116] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0117] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations among the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

Claim 1 A wearable electronic device comprising: a housing including a frame and a first arm and a second arm each extending from the frame; a transparent member disposed in the frame and including a screen output area; a display disposed in the frame and configured to output content to the screen output area; at least one camera disposed in the frame; a communication circuit disposed within the housing; a first processor located within the first arm; and a second processor located within the second arm, wherein the first processor is configured to display content received through the communication circuit using the display, and the second processor is configured to acquire at least one of first information related to a user's gaze or second information related to a user's head movement based on an image acquired through the at least one camera. Claim 2 A wearable electronic device according to claim 1, wherein the second processor is configured to acquire at least one of third information regarding the position of an external object using the at least one camera. Claim 3 A wearable electronic device according to claim 2, wherein the communication circuit comprises a first communication circuit and a second communication circuit, and the second processor is configured to transmit at least one of the acquired first information, the second information, or the third information to an external electronic device using the second communication circuit. Claim 4 A wearable electronic device according to claim 2, wherein the communication circuit comprises a first communication circuit and a second communication circuit, and the first processor is configured to receive screen information from an external electronic device using the first communication circuit, and the screen information is information obtained by rendering data associated with the content based on at least one of the first information, the second information, or the third information received from the wearable electronic device. Claim 5 delete Claim 6 A wearable electronic device according to claim 4, wherein the first processor is configured to control the display to display the content in the screen output area based on the received screen information. Claim 7 A wearable electronic device according to claim 6, comprising at least one sensor electrically connected to the first processor, wherein the first processor is configured to detect a value for the posture of the wearable electronic device using the at least one sensor, and to determine whether the posture of the wearable electronic device has changed based on the detected value while displaying the content in the screen output area. Claim 8 A wearable electronic device according to claim 7, wherein the first processor is configured to correct the content being displayed in the screen output area based on the acquired fourth information regarding the changed posture of the wearable electronic device when the posture of the wearable electronic device is changed while the content is being displayed in the screen output area. Claim 9 A wearable electronic device according to claim 7, wherein, when the posture of the wearable electronic device is changed while displaying the content in the screen output area: the first processor is configured to acquire fourth information regarding the changed posture of the wearable electronic device based on a value detected by the at least one sensor, transmit the fourth information to the external electronic device using the first communication circuit, and transmit a message requesting an update of the screen information based on the fourth information to the external electronic device using the first communication circuit. Claim 10 A wearable electronic device according to any one of claims 1 to 4 and claims 6 to 9, comprising: a first printed circuit board (PCB) disposed within a first arm; and a second PCB disposed within a second arm, wherein the first processor is disposed on the first PCB and the second processor is disposed on the second PCB. Claim 11 A wearable electronic device according to claim 10, comprising a third PCB extending from the first arm to the second arm along the edge of the frame, comprising at least a partially flexible portion, and electrically connected to the first PCB and the second PCB, wherein the display is electrically connected to the first processor through the third PCB, and the at least one camera is electrically connected to the second processor through the third PCB. Claim 12 A method of operation of a wearable electronic device comprising: a housing including a frame and a first arm and a second arm each extending from the frame; a transparent member disposed in the frame and including a screen output area; a display disposed in the frame and configured to output content to the screen output area; at least one camera disposed in the frame; a communication circuit disposed within the housing; a first processor located within the first arm and a second processor located within the second arm, the method comprising: an operation by the first processor to display content received through the communication circuit using the display; and an operation by the second processor to acquire at least one of first information related to a user's gaze or second information related to a user's head movement based on an image acquired through the at least one camera. Claim 13 A method according to claim 12, comprising the operation of acquiring at least one of third information regarding the position of an external object through at least one camera by the second processor. Claim 14 A method according to claim 13, comprising the operation of transmitting at least one of the first information, the second information, or the third information to an external electronic device by the second processor. Claim 15 A method according to claim 13, comprising the operation of receiving screen information from an external electronic device by the first processor, wherein the screen information is information obtained by rendering data associated with the content based on at least one of the first information, the second information, or the third information received from the wearable electronic device. Claim 16 A method according to claim 15, comprising the operation of displaying the content in the screen output area based on the received screen information by the first processor. Claim 17 The method of claim 16, comprising the operation of detecting a value for the posture of the wearable electronic device using at least one sensor by the first processor, and determining whether the posture of the wearable electronic device has changed based on the detected value. Claim 18 In claim 17, when the posture of the wearable electronic device is changed while the content is displayed in the screen output area: the first processor acquires fourth information regarding the changed posture of the wearable electronic device based on a value detected by the at least one sensor, and corrects the content being displayed in the screen output area based on the acquired fourth information. Claim 19 A method according to claim 17, wherein when the posture of the wearable electronic device is changed while displaying the content in the screen output area: the first processor acquires fourth information regarding the changed posture of the wearable electronic device based on a value detected by the at least one sensor, transmits the fourth information to the external electronic device using the communication circuit, and transmits a message requesting an update of the screen information based on the fourth information to the external electronic device using the communication circuit. Claim 20 delete

Images