Wearable electronic device for displaying virtual object and control method therefor

Abstract

This wearable electronic device comprises: a camera; a display; and at least one processor operatively connected directly or indirectly to the camera and the display, wherein the at least one processor may: identify a displayable area corresponding to an external display included in an image of a space acquired through the camera; on the basis of at least a part of the external display being excluded from the displayable area by the movement of the external display, identify a first area of the displayable area in which a part of the external display is disposed and a second area of the displayable area that is a remaining area other than the first area; and control the display to display a virtual screen in the second area.

Description

Wearable electronic device displaying virtual objects and method for controlling the same

[0001] Certain embodiments relate to a wearable electronic device displaying a virtual object and a method for controlling the same.

[0002] The variety of services and additional features offered through electronic devices, such as smartphones, is steadily increasing. To enhance the utility of these devices and satisfy the diverse needs of users, telecommunications service providers and electronic device manufacturers are competitively developing electronic devices to offer a variety of features and differentiate themselves from competitors. Consequently, the various functions offered through electronic devices are also becoming increasingly sophisticated.

[0003] The variety of services and additional features offered through wearable electronic devices, such as augmented reality glasses (AR glasses), virtual reality glasses (VR glasses), and head-mounted displays (HMDs), is steadily increasing. To enhance the utility of these devices and satisfy the needs of diverse users, telecommunications service providers and electronic device manufacturers are competitively developing electronic devices that offer diverse features and differentiate themselves from competitors. Consequently, the various functions offered through wearable electronic devices are also becoming increasingly sophisticated.

[0004] AR glasses or VR glasses, worn on the user's head, can provide a realistic experience by displaying virtual images. AR glasses or VR glasses can replace smartphones in various fields, such as gaming, entertainment, education, and social networking services (SNS). Users can access smartphone content and / or content similar to reality through the AR glasses or VR glasses worn on their head.

[0005] According to one embodiment, a wearable electronic device may include a memory, a camera, a display, and at least one processor operatively connected directly or indirectly with the memory, the camera, and the display.

[0006] According to one embodiment, the memory may store instructions that, when executed by the processor(s), cause the wearable electronic device to identify a displayable area corresponding to an external display included in an image of a space acquired through the camera.

[0007] According to one embodiment, the memory may store instructions that, when executed by the processor(s), cause the wearable electronic device to identify a first region in which a portion of the external display is disposed among the displayable regions and a second region, which is an area remaining among the displayable regions excluding the first region, based on the fact that at least a portion of the external display is excluded from the displayable region due to movement of the external display.

[0008] In one embodiment, the memory may store instructions that, when executed by the processor(s), cause the wearable electronic device to control the display to display a virtual screen in the second area.

[0009] According to one embodiment, a method for controlling a wearable electronic device may include an operation of identifying a displayable area corresponding to an external display included in an image of a space acquired through a camera of the wearable electronic device.

[0010] According to one embodiment, a control method of a wearable electronic device may include an operation of identifying a first region in which a portion of the external display is disposed among the displayable regions and a second region, which is an area remaining among the displayable regions excluding the first region, based on the fact that at least a portion of the external display is excluded from the displayable region due to movement of the external display.

[0011] According to one embodiment, a method for controlling a wearable electronic device may include an action of displaying a virtual screen in the second area.

[0012] According to one embodiment, a non-transitory computer-readable recording medium storing one or more programs may store instructions that cause a wearable electronic device to identify a displayable area corresponding to an external display included in an image of a space acquired through a camera of the wearable electronic device.

[0013] According to one embodiment, the one or more programs may store instructions that cause the wearable electronic device to identify a first region in which a portion of the external display is disposed among the displayable regions and a second region, which is an area remaining among the displayable regions excluding the first region, based on movement of the external display excluding at least a portion of the external display from the displayable region.

[0014] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to control the display to display a virtual screen in the second area.

[0015] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0016] FIG. 2 is a perspective view illustrating the internal configuration of a wearable electronic device according to one embodiment.

[0017] FIG. 3 is a drawing showing the front of a wearable electronic device according to one embodiment.

[0018] FIG. 4 is a drawing showing the back of a wearable electronic device according to one embodiment.

[0019] FIG. 5 is another perspective view of a wearable electronic device according to one embodiment.

[0020] FIG. 6 is a drawing schematically illustrating an operation of displaying a virtual screen of an electronic device according to one embodiment.

[0021] FIG. 7A is a diagram illustrating an operation of identifying a displayable area based on an external display of an electronic device according to one embodiment.

[0022] FIG. 7b is a drawing for explaining an embodiment in which first content is displayed in a displayable area according to one embodiment.

[0023] FIG. 8A is a diagram illustrating an operation of identifying a displayable area as a first area and a second area based on movement of an external display device of an electronic device according to one embodiment.

[0024] FIG. 8B is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device according to one embodiment.

[0025] FIG. 8C is a diagram for explaining an operation of displaying a virtual screen in at least a portion of a displayable area based on tracking a user's gaze of an electronic device according to one embodiment.

[0026] FIG. 9A is a diagram illustrating an operation of displaying first content across the entire displayable area of ​​an electronic device according to one embodiment.

[0027] FIG. 9b is a diagram illustrating an operation of displaying content through a portion of a displayable area based on movement of an external display according to one embodiment.

[0028] FIG. 9c is a diagram illustrating an operation of displaying a virtual screen on at least a portion of a displayable area of ​​an electronic device based on movement of an external display, according to one embodiment.

[0029] FIG. 9D is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display according to one embodiment.

[0030] FIG. 10 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display according to one embodiment.

[0031] FIG. 11A is a diagram illustrating an operation of displaying first content in the entire displayable area of ​​an electronic device according to one embodiment.

[0032] FIG. 11B is a diagram illustrating an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display, according to one embodiment.

[0033] FIG. 11C is a diagram illustrating an operation of displaying a virtual screen on at least a portion of a displayable area of ​​an electronic device based on movement of an external display, according to one embodiment.

[0034] FIG. 12 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display according to one embodiment.

[0035] FIG. 13 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display according to one embodiment.

[0036] FIG. 14 is a flowchart illustrating an operation for obtaining information about movement of an external display of an electronic device according to one embodiment.

[0037] FIG. 15 is a flowchart illustrating a virtual screen display operation based on movement of an external display of an electronic device according to one embodiment.

[0038] FIG. 16 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display according to one embodiment.

[0039] FIG. 17 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of a sunroof when the external display is a sunroof, according to one embodiment.

[0040] FIG. 18 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of a door when an external display is a door, according to one embodiment.

[0041] FIG. 19 is a drawing for explaining an operation of displaying a virtual screen on at least a portion of a displayable area of ​​an electronic device based on rolling of a rollable display when the external display is a rollable display, according to one embodiment.

[0042] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100), according to one embodiment. Referring to FIG. 1 , in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0043] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0044] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0045] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0046] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0047] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0048] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0049] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0050] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0051] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0052] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0053] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0054] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0055] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0056] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0057] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0058] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0059] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0060] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0061] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0062] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0063] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0064] FIG. 2 is a perspective view illustrating the internal configuration of a wearable electronic device according to one embodiment.

[0065] Referring to FIG. 2, a wearable electronic device (200) according to one embodiment may include at least one of a light output module (211), a display member (201), and a camera module (250).

[0066] According to one embodiment, the light output module (211) may include a light source capable of outputting an image and a lens for guiding the image to the display member (201). According to one embodiment, the light output module (211) may include at least one of a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED).

[0067] According to one embodiment, the display member (201) may include an optical waveguide (e.g., a waveguide). According to one embodiment, an output image of the optical output module (211) incident on one end of the optical waveguide may be propagated inside the optical waveguide and provided to a user. According to one embodiment, the optical waveguide may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) or at least one reflective element (e.g., a reflective mirror). For example, the optical waveguide may guide the output image of the optical output module (211) to the user's eyes by using at least one diffractive element or reflective element.

[0068] According to one embodiment, the camera module (250) can capture still images and / or moving images. According to one embodiment, the camera module (250) is disposed within the lens frame and can be disposed around the display member (201).

[0069] According to one embodiment, the first camera module (251) can capture and / or recognize the trajectory of the user's eye (e.g., pupil, iris) or gaze. According to one embodiment, the first camera module (251) can periodically or aperiodically transmit information related to the trajectory of the user's eye or gaze (e.g., trajectory information) to a processor (e.g., processor (120) of FIG. 1).

[0070] Each "processor" of the present disclosure may include processing circuitry and / or may include multiple processors. For example, the term "processor" as used in this disclosure, including in the claims, may include various processing circuitry including at least one processor, one or more of which are configured to individually and / or collectively perform the various functions described herein in a distributed manner. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform various functions, these terms encompass, for example, without limitation, situations where a single processor performs some of the recited functions, other processor(s) perform other of the recited functions, and situations where a single processor can perform all of the recited functions. Additionally, at least one processor may include a combination of processors that perform the various functions enumerated / disclosed, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform the various functions.

[0071] According to one embodiment, the second camera module (253) can capture an external image.

[0072] According to one embodiment, the third camera module (255) can be used for hand detection and tracking, and user gesture (e.g., hand movement) recognition. According to one embodiment, the third camera module (255) can be used for 3 degrees of freedom (3DoF), 6DoF head tracking, position (spatial, environmental) recognition, and / or movement recognition. According to one embodiment, the second camera module (253) can also be used for hand detection and tracking, and user gesture recognition. According to one embodiment, at least one of the first camera module (251) to the third camera module (255) can be replaced with a sensor module (e.g., a LiDAR sensor). For example, the sensor module can include at least one of a vertical cavity surface emitting laser (VCSEL), an infrared sensor, and / or a photodiode.

[0073] FIG. 3 is a drawing showing the front of a wearable electronic device according to one embodiment.

[0074] FIG. 4 is a drawing showing the back of a wearable electronic device according to one embodiment.

[0075] Referring to FIGS. 3 and 4, in one embodiment, camera modules (311, 312, 313, 314, 315, 316) and / or depth sensors (317) for obtaining information related to the surrounding environment of the wearable electronic device (300) may be arranged on the first surface (310) of the housing.

[0076] In one embodiment, the camera modules (311, 312) can acquire images related to the environment surrounding the wearable electronic device.

[0077] In one embodiment, the camera modules (313, 314, 315, 316) can acquire images while the wearable electronic device is worn by a user. The camera modules (313, 314, 315, 316) can be used for hand detection, tracking, and recognition of user gestures (e.g., hand movements). The camera modules (313, 314, 315, 316) can be used for 3DoF, 6DoF head tracking, position (spatial, environmental) recognition, and / or movement recognition. In one embodiment, the camera modules (311, 312) can also be used for hand detection and tracking, and user gestures.

[0078] In one embodiment, a depth sensor (317) may be configured to transmit a signal and receive a signal reflected from a subject, and may be used for purposes such as time of flight (TOF) to determine the distance to an object. Instead of or in addition to the depth sensor (217), the camera modules (313, 314, 315, 316) may determine the distance to an object.

[0079] According to one embodiment, a camera module (325, 326) for facial recognition and / or a display (321) (and / or a lens) may be arranged on the second side (320) of the housing.

[0080] In one embodiment, a face recognition camera module (325, 326) adjacent to the display may be used to recognize a user's face, or may recognize and / or track both eyes of the user.

[0081] In one embodiment, the display (321) (and / or lens) may be disposed on the second side (320) of the wearable electronic device (300). In one embodiment, the wearable electronic device (300) may not include camera modules (315, 316) among the plurality of camera modules (313, 314, 315, 316). Although not illustrated in FIGS. 3 and 4, the wearable electronic device (300) may further include at least one of the configurations illustrated in FIG. 2.

[0082] As described above, according to one embodiment, the wearable electronic device (300) may have a form factor for being worn on a user's head. The wearable electronic device (300) may further include a strap and / or a wearing member for being secured on a body part of the user. The wearable electronic device (300) may provide a user experience based on augmented reality, virtual reality, and / or mixed reality while being worn on the user's head.

[0083] FIG. 5 is another perspective view of an electronic device according to one embodiment.

[0084] Referring to FIG. 5, the electronic device (400) may be a head-mounted device (HMD) capable of providing images in front of the user's eyes. The configuration of the electronic device (400) of FIG. 5 may be entirely or partially identical to the configuration of the electronic device (200) of FIG. 2.

[0085] According to one embodiment, the electronic device (400) may include a housing (410, 420, 430) that may form the exterior of the electronic device (400) and provide a space in which components of the electronic device (400) may be placed.

[0086] According to one embodiment, the electronic device (400) may include a first housing (410) capable of surrounding at least a portion of a user's head. According to one embodiment, the first housing (410) may include a first side (400a) facing the exterior of the electronic device (400) (e.g., in the +Z direction).

[0087] According to one embodiment, the first housing (410) can surround at least a portion of the internal space (I). For example, the first housing (410) can include a second side (400b) facing the internal space (I) of the electronic device (400) and a third side (400c) opposite the second side (400b). According to one embodiment, the first housing (410) can be combined with the third housing (430) to form a closed curve shape surrounding the internal space (I).

[0088] According to one embodiment, the first housing (410) can accommodate at least some of the components of the electronic device (400). For example, an optical output module and a circuit board can be placed within the first housing (410).

[0089] According to one embodiment, the electronic device (400) may include one display member (440) corresponding to the left and right eyes. The display member (440) may be disposed in the first housing (410). The configuration of the display member (440) of FIG. 5 may be all or part of the same as the configuration of the display member (201) of FIG. 2.

[0090] According to one embodiment, the electronic device (400) may include a second housing (420) that can be mounted on a user's face. According to one embodiment, the second housing (420) may include a fourth surface (400d) that can at least partially face the user's face. According to one embodiment, the fourth surface (400d) may be a surface facing the internal space (I) of the electronic device (400) (e.g., -Z direction). According to one embodiment, the second housing (420) may be directly or indirectly coupled to the first housing (410).

[0091] According to one embodiment, the electronic device (400) may include a third housing (430) that can be mounted on the back of the user's head. According to one embodiment, the third housing (430) may be directly or indirectly coupled to the first housing (410). According to one embodiment, the third housing (430) may accommodate at least some of the components of the electronic device (400). For example, a battery (e.g., battery (189) of FIG. 1) may be placed within the third housing (430).

[0092] FIG. 6 is a drawing schematically illustrating an operation of displaying a virtual screen of an electronic device according to one embodiment.

[0093] Referring to FIG. 6, in operation 610, an electronic device (or a wearable electronic device) (e.g., the electronic device (101) of FIG. 1, the wearable electronic device (200) of FIG. 2, the wearable electronic device (300) of FIG. 3, the wearable electronic device (300) of FIG. 4, or the electronic device (400) of FIG. 5) can identify a displayable area corresponding to an external display included in an image of a space acquired through a camera (e.g., the camera module (160) of FIG. 1, the second camera module (253) of FIG. 2, the third camera module (255) of FIG. 2, the camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or the depth sensor (317) of FIG. 3).

[0094] According to one embodiment, the electronic device may display in real time an image captured of an actual space in which the electronic device is located on a display (e.g., a display module (160) of FIG. 1, a display member (201) of FIG. 2, a display (321) of FIG. 4, or a display member (440) of FIG. 5). For example, the actual space in which the electronic device is located may be the interior of a vehicle, a house, or a building such as an office.

[0095] In one embodiment, the external display may be movable through a wall or a housing. The external display may be engraved in at least a portion of the wall or housing as it moves. For example, the external display may include at least one of a window of a vehicle, a sunroof, or a window or door (e.g., a glass door or an automatic door) of a building such as a home or office. In one embodiment, an embodiment in which the external display is a window of a vehicle will be described with reference to FIGS. 7A to 16 below. In one embodiment, an embodiment in which the external display is a sunroof will be described with reference to FIG. 17 below. In one embodiment, an embodiment in which the external display is a window or door of a building such as a home or office will be described with reference to FIG. 18 below. In one embodiment, FIGS. 7A to 16 illustrate and describe the external display as a window of a vehicle, but this is not limited thereto, and the external display may also be applied to a sunroof of a vehicle, a window, or a door within a building.

[0096] According to one embodiment, the electronic device may communicate with a control device that controls an external display. For example, the electronic device may communicate with the external electronic device that controls the external display via a proximity communication method such as BT, BLE, UWB, or WiFi. For example, the external electronic device that controls the external display may be a vehicle control device that controls the vehicle's windows or sunroof.

[0097] In one embodiment, the external electronic device may be a terminal device, such as a smartphone, capable of communicating with the vehicle's control unit. In one embodiment, the vehicle's control unit may be controlled to control a window or sunroof.

[0098] According to one embodiment, the electronic device can determine that the electronic device is located within the vehicle based on communicating with a control device of the vehicle, which is an external electronic device, through a communication module (e.g., the communication module (190) of FIG. 1).

[0099] In one embodiment, an electronic device may receive spatial data (e.g., 3D data) from an external electronic device. For example, the electronic device may receive vehicle data (e.g., 3D data) from the vehicle's control unit. For example, the vehicle data may relate to the structure of the vehicle's interior, such as seat positions, seat configurations, window positions, and / or window configurations.

[0100] In one embodiment, the electronic device can identify a displayable area based on the received 3D data. The displayable area may be the maximum area that can display content on an external display. For example, if the external display is a vehicle window, the displayable area may be the area where the window is positioned when closed.

[0101] According to one embodiment, an electronic device may identify a displayable area based on an image captured by a camera. For example, the electronic device may recognize an image captured by the camera, recognize an external display (e.g., a window) included in the image, and identify an area corresponding to the external display as a displayable area.

[0102] In one embodiment, the electronic device may recognize an image acquired through a camera to identify that the electronic device is located inside a vehicle.

[0103] According to one embodiment, in operation 620, the electronic device can identify a first region in which the external display is arranged among the displayable regions and a second region, which is the remaining region excluding the first region, based on the fact that at least a portion of the external display is excluded from the displayable region due to movement of the external display.

[0104] In one embodiment, the external display may have areas excluded from the displayable area, even if no areas are obscured by the movement of the external display, such as a building window. In one embodiment, the external display may be at least partially obscured by a surrounding vehicle body or housing, such as a vehicle window or a rollable display, as the external display moves.

[0105] According to one embodiment, the electronic device can identify movement of the external display based on at least one of sensor data acquired through at least one sensor (176) (e.g., the sensor module (176) of FIG. 1, the camera module (180) of FIG. 1, the second camera module (253) of FIG. 2, the third camera module (255) of FIG. 2, the camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, and the depth sensor (317) of FIG. 3) or an image acquired through a camera.

[0106] In one embodiment, when an external display (e.g., a window) is moved (e.g., lowered or opened) and a portion of the external display is excluded from the displayable area while the external display is completely closed, the electronic device may identify the portion of the displayable area where the external display is located as a first portion. The electronic device may identify the portion of the displayable area where the external display is not located as a second portion. The second portion may be the remaining portion of the displayable area excluding the first portion.

[0107] In one embodiment, when an external display (e.g., a window) moves (e.g., moves down) and only a portion of the external display is exposed, while the remainder is obscured by a housing, chassis, or wall, the electronic device may identify the area of ​​the displayable area where the external display is exposed as a first area. The electronic device may identify the area of ​​the displayable area where the external display is not positioned as the second area. The second area may be the remaining area of ​​the displayable area excluding the first area.

[0108] According to one embodiment, an electronic device may receive information related to movement of an external display from an external electronic device connected via at least a communication module including a communication circuit. For example, the information related to movement of the external display may include at least one of information related to whether the external display has moved, information related to the movement distance of the external display, or information related to the size of a portion of the external display exposed by the movement of the external display.

[0109] According to one embodiment, the electronic device can identify a first area and a second area included in a displayable area based on information related to movement of an external display.

[0110] According to one embodiment, the electronic device can identify (or calculate) the position and direction of the electronic device within the space using at least one of data of the received space, distance information with respect to an external display, and information recognized from an image acquired through a camera, and based on this, display content and information in real time in a first area and / or a second area of ​​the displayable area that matches the external display of the actual space.

[0111] According to one embodiment, the operation of receiving information related to movement of an external display from an external electronic device is described in more detail below with reference to FIG. 14.

[0112] According to one embodiment, the electronic device may display an indicator for at least one of the displayable area, the first area, or the second area. For example, the electronic device may display the boundary of the displayable area, the boundary of the first area, and / or the boundary of the second area. For example, the electronic device may display the boundary of the displayable area, the boundary of the first area, and / or the boundary of the second area in different colors.

[0113] According to one embodiment, the electronic device may display the displayable area, the first area, and / or the second area in different colors.

[0114] In one embodiment, in operation 630, the electronic device may display a virtual screen in the second area.

[0115] In one embodiment, the external display may be displaying first content.

[0116] According to one embodiment, an electronic device may receive information associated with first content displayed on an external display from an external electronic device that controls the external display via a communication module including a communication circuit. According to one embodiment, the electronic device may display the information associated with the received first content on at least a portion of a displayable area.

[0117] According to one embodiment, an electronic device may receive first content displayed on an external display from an external electronic device that controls the external display connected at least through a communication module. The electronic device may display a portion of the received first content corresponding to a second area as a virtual screen in the second area. For example, based on movement of the external display, the first content may be displayed by the external display in a first area of ​​the displayable area, and the first content may be displayed as a virtual screen by the electronic device in the second area.

[0118] According to one embodiment, the electronic device may output a virtual screen (or light source, image) to a second area via a light output module (e.g., a projector) (e.g., the light output module (211) of FIG. 2) disposed in a housing (e.g., a glasses frame). According to one embodiment, the virtual screen may be reflected and / or diffracted by the second area and guided to the user's eyes.

[0119] This allows the electronic device to provide a continuous content viewing experience before and after movement of the external display.

[0120] According to one embodiment, the operation of displaying one content through an external display and an electronic device is described in more detail with reference to FIGS. 9b, 10, and 11b below.

[0121] According to one embodiment, the electronic device can display first content received from an external electronic device as a virtual screen in the displayable area. For example, the display operation of the external display may be stopped, and the electronic device may display the first content as a virtual screen in the entire displayable area. This allows the electronic device to provide a continuous content viewing experience before and after the external display is moved.

[0122] According to one embodiment, the electronic device may display content as a virtual screen in the displayable area before the external display is moved. According to one embodiment, when the external display is moved, the electronic device may maintain the operation of displaying content in the displayable area.

[0123] In one embodiment, when displaying one content through the first area and the second area after moving the external display, the brightness of the first area and the second area where the external display is not placed may be different. In one embodiment, the electronic device may control the brightness of the virtual screen of the second area based on the brightness of the first area. For example, the electronic device may obtain brightness information of the first area through a sensor and / or a camera. In one embodiment, the brightness information of the first area may be a reflection of the brightness value of the content displayed in the first area and a brightness value due to the external environment (e.g., weather, illuminance).

[0124] According to one embodiment, the electronic device can control a brightness value of a virtual screen displayed in a second area to correspond to a brightness value of the first area based on brightness information of the first area.

[0125] According to one embodiment, the electronic device may display information related to the external environment in the second area based on the movement of the external display displaying the first content. For example, as the external environment is exposed to the second area by the movement of the external display, the electronic device may display information related to the external environment as a virtual object (e.g., a virtual screen) in the second area. For example, the information related to the external environment may include at least one of temperature information, weather information (e.g., weather, rainfall, humidity, wind speed), fine dust information, information related to a building visible through the second area, or an image capturing the outside. According to one embodiment, the electronic device may display information related to the external environment and environmental information inside the vehicle (e.g., temperature, humidity) together, and may further include location-related surrounding information (e.g., local service information, road conditions (e.g., congestion level, speed limit, road name, navigation route)).

[0126] According to one embodiment, the electronic device may display the first content as a virtual screen in the first area based on the movement of the external display displaying the first content. For example, the electronic device may display information related to the external environment as a virtual screen in the second area, and display the entire first content as a virtual screen in the first area, resized to fit the size of the first area. According to one embodiment, when the electronic device displays the first content with the resized size as a virtual screen in the first area, the display operation of the external display device may be stopped.

[0127] In one embodiment, the electronic device may, based on the movement of the external display displaying the first content, display the entire first content on a virtual screen by adjusting the size of the first area to fit the size of the first area and transmit a command to the external display device to stop the display operation when the size of the first area falls below a set value. In one embodiment, the set value of the size of the first area may be set by the manufacturer or the user.

[0128] According to one embodiment, an operation of displaying information related to an external environment in a second area where an external display is not placed by moving the external display will be described below with reference to FIGS. 8b and 12.

[0129] According to one embodiment, an electronic device may display information related to an external environment as a virtual screen in a first area based on detecting a user's gaze toward a second area through at least one sensor. For example, information related to an external environment may be displayed as a virtual screen in a second area where an external display is not positioned due to movement of the external display, and when content is displayed by an external display device or content is displayed as a virtual screen by the electronic device in the first area and the user's gaze toward the second area is detected, the electronic device may display information related to the external environment as a virtual screen in the first area. By changing the virtual screen in this way based on the user's gaze, the electronic device may provide content of interest to the user.

[0130] According to one embodiment, the electronic device may display an indicator indicating that the user's gaze is directed at an area where the user's gaze is detected.

[0131] According to one embodiment, the operation of controlling the display of a virtual screen based on the user's gaze will be described in more detail with reference to FIG. 8c below.

[0132] According to one embodiment, the electronic device may display content different from content previously displayed on the external display as a virtual screen on the first area based on the size of the first area where the external display is placed among the displayable areas being smaller than a set size.

[0133] According to one embodiment, an external electronic device controlling an external display may stop outputting content when the size of a first area of ​​the external display becomes smaller than a set size. According to one embodiment, when it is determined that content output through the external display has stopped when the size of the first area where the external display is placed among the displayable areas becomes smaller than the set size, the electronic device may display content different from the content previously displayed on the external display as a virtual screen on the first area.

[0134] In one embodiment, the electronic device may transmit a command to a control device controlling the external display to stop outputting content when the size of a first area of ​​the displayable area where the external display is placed becomes smaller than a set size. In one embodiment, when it is confirmed that content output through the external display has stopped, the electronic device may display content different from the content previously displayed on the external display as a virtual screen on the first area.

[0135] According to one embodiment, when the size of a first area in which an external display is arranged among the displayable areas becomes smaller than a set size, the electronic device may display content different from content previously displayed on the external display as a virtual screen on the first area, regardless of whether the content is displayed on the external display.

[0136] For example, the electronic device may display a user interface (UI) for controlling a music playback application running on the electronic device or an external electronic device as a virtual screen on the first area if the size of the first area is less than a set value (e.g., about 20% of the displayable area).

[0137] This allows for the utilization of the area of ​​small external displays where content such as videos would otherwise be difficult to display.

[0138] According to one embodiment, an embodiment in which a virtual screen including different functions is displayed based on the size of the first area is described in more detail below with reference to FIG. 13.

[0139] In this way, when viewing content on a moving structure using a wearable device, the content can be provided in real time in accordance with the movement of the structure.

[0140] FIG. 7A is a diagram illustrating an operation of identifying a displayable area based on an external display of an electronic device according to one embodiment.

[0141] Referring to FIG. 7A, an electronic device (or wearable electronic device) (e.g., the electronic device (101) of FIG. 1, the wearable electronic device (200) of FIG. 2, the wearable electronic device (300) of FIG. 3, the wearable electronic device (300) of FIG. 4, or the electronic device (400) of FIG. 5) can identify a displayable area (710). For example, the displayable area (710) may be greater than or equal to the maximum area in which content can be displayed by an external display. For example, if the external display is a window of a vehicle, it may be an area in which the window is positioned when the window is closed.

[0142] According to one embodiment, the electronic device may identify a displayable area based on spatial data (e.g., 3D data) received from an external electronic device (e.g., the electronic device (104) of FIG. 1). For example, the spatial data may be related to the structure of the vehicle interior, such as seat positions, seat configurations, window positions, and / or window configurations.

[0143] According to one embodiment, the electronic device can identify a displayable area (710) based on an image acquired through a camera (e.g., the camera module (160) of FIG. 1, the second camera module (253) of FIG. 2, the third camera module (255) of FIG. 2, the camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or the depth sensor (317) of FIG. 3). For example, the electronic device can recognize an image acquired through the camera, recognize an external display (e.g., a window) included in the image, and identify an area corresponding to the external display as a displayable area. According to one embodiment, the electronic device can acquire spatial data (e.g., 3D data) based on an image acquired through the camera, and determine an external display area identified based on the spatial data as a displayable area.

[0144] FIG. 7b is a drawing for explaining an embodiment in which first content is displayed in a displayable area according to one embodiment.

[0145] Referring to FIG. 7B, the displayable area may display first content (711). For example, the first content (711) may be displayed on an external display under the control of an external electronic device.

[0146] According to one embodiment, the first content (711) may include image data.

[0147] According to one embodiment, the first content (711) may be a virtual screen displayed on a display (e.g., the display module (160) of FIG. 1, the display member (201) of FIG. 2, the display (321) of FIG. 4, or the display member (440) of FIG. 5) by the electronic device. For example, the electronic device may display the first content (711) as a virtual screen on a displayable area that is an area that matches an external display.

[0148] FIG. 8A is a diagram illustrating an operation of identifying a displayable area as a first area and a second area based on movement of an external display device of an electronic device according to one embodiment.

[0149] Referring to FIG. 8A, an electronic device (or a wearable electronic device) (e.g., an electronic device (101) of FIG. 1, a wearable electronic device (200) of FIG. 2, a wearable electronic device (300) of FIG. 3, a wearable electronic device (300) of FIG. 4, or an electronic device (400) of FIG. 5) identifies a displayable area (810) corresponding to an external display by recognizing data (e.g., 3D data) received from an external electronic device (e.g., an electronic device (104) of FIG. 1) and / or an image acquired by a camera (e.g., a camera module (160) of FIG. 1, a second camera module (253) of FIG. 2, a third camera module (255) of FIG. 2, camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or a depth sensor (317) of FIG. 3). Can be.

[0150] In one embodiment, as the external display moves, some areas of the external display may be obscured by the housing, chassis, and / or wall.

[0151] According to one embodiment, as the external display moves, the displayable area (810) may include a first area (811) where a portion of the external display is positioned and a second area (812) which is the remaining area excluding the first area (811). The second area (812) may be an area where the external display is not positioned and is exposed to the external environment.

[0152] According to one embodiment, a first region (811) and a second region (812) may be distinguished based on the edge of the external display. According to one embodiment, based on the edge of the external display, an area where image output is possible through the external display may be distinguished as the first region (811), and the remaining area may be distinguished as the second region (812).

[0153] FIG. 8B is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device according to one embodiment.

[0154] Referring to FIG. 8B, the electronic device may display a virtual screen in a second area (822) where no external display is placed. For example, the electronic device may display information related to the external environment on the virtual screen. According to one embodiment, the external environment may include an outside space other than the space inside the external display where the electronic device is placed. For example, the information related to the external environment may include at least one of temperature information, weather information (e.g., weather, rainfall, humidity, wind speed), or fine dust information of an area including the area where the electronic device is located, information related to a building visible through the second area (822), or an image captured from the outside. According to one embodiment, information related to the external environment may be displayed together with information related to the external environment and environmental information inside the vehicle (e.g., temperature, humidity), and may further include location-related surrounding information (e.g., local service information, road conditions (e.g., congestion, speed limit, road name, navigation route). According to one embodiment, information related to the external environment may be received from a server (e.g., server (108) of FIG. 1).

[0155] In one embodiment, the second region (822) is an area exposed to the external environment because an external display is not arranged therein. Instead of displaying an image of the outside as a virtual screen on the second region (822), the external environment may be visible through the virtual screen. For example, the electronic device may output a highly transparent virtual screen through the second region (822), thereby allowing the external environment to be visible through the virtual screen.

[0156] According to one embodiment, the electronic device may output a virtual screen (or light source, image) to the second area (822) via a light output module (e.g., a projector) (e.g., the light output module (211) of FIG. 2) disposed in a housing (e.g., a glasses frame). According to one embodiment, the virtual screen may be reflected and / or diffracted by the second area (822) and guided to the user's eyes.

[0157] According to one embodiment, content may be displayed on the first area (821) where the external display is positioned. For example, the content may be resized based on the size of the first area (821). According to one embodiment, an area of ​​the first area (821) where no content is displayed is made of transparent glass, allowing the external environment to be seen through it.

[0158] According to one embodiment, the electronic device may also display virtual information in the first area (821). For example, the electronic device may receive information related to content from an external electronic device and display virtual information based on the received information in the first area. According to one embodiment, the electronic device may generate virtual information (e.g., a virtual object) to be displayed in the first area based on the received information.

[0159] According to one embodiment, the electronic device may generate a virtual object (e.g., a virtual screen) based on content received from an external electronic device and display the generated virtual object in a first area (821).

[0160] FIG. 8C is a diagram for explaining an operation of displaying a virtual screen in at least a portion of a displayable area based on tracking a user's gaze of an electronic device according to one embodiment.

[0161] Referring to FIG. 8C, when the electronic device detects that the user's gaze is directed toward a second area exposed to the external environment through at least one sensor (e.g., the sensor module (176) of FIG. 1 or the first camera module (251) of FIG. 2), the electronic device can reduce the size of a virtual screen (823) containing information related to the external environment. Based on the reduction in the size of the virtual screen (823), the external environment can be exposed through the second area.

[0162] According to one embodiment, when the electronic device detects that the user's gaze is directed toward the second area, the electronic device may receive an image capturing the external environment from an external electronic device and display the received image as a virtual screen in the second area. According to one embodiment, the electronic device may receive an image capturing the external environment from the external electronic device, along with information related to the movement of the external display. Based on the information related to the user's gaze and the movement of the external display, the electronic device may crop a portion of the image and display it as a virtual screen in the second area.

[0163] FIG. 9A is a diagram illustrating an operation of displaying first content across the entire displayable area of ​​an electronic device according to one embodiment.

[0164] Referring to FIG. 9A, an electronic device (or a wearable electronic device) (e.g., an electronic device (101) of FIG. 1, a wearable electronic device (200) of FIG. 2, a wearable electronic device (300) of FIG. 3, a wearable electronic device (300) of FIG. 4, or an electronic device (400) of FIG. 5) identifies a displayable area (910) corresponding to an external display by recognizing data (e.g., 3D data) received from an external electronic device (e.g., an electronic device (104) of FIG. 1) and / or an image acquired by a camera (e.g., a camera module (160) of FIG. 1, a second camera module (253) of FIG. 2, a third camera module (255) of FIG. 2, camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or a depth sensor (317) of FIG. 3). Can be.

[0165] According to one embodiment, the displayable area (910) may display first content (901). For example, the first content (901) may be displayed on an external display under the control of an external electronic device.

[0166] According to one embodiment, when the display of the electronic device (e.g., the display module (160) of FIG. 1, the display member (201) of FIG. 2, the display (321) of FIG. 4, or the display member (440) of FIG. 5) is a display that transmits light in real space, the external display and the first content (901) displayed on the external display can be shown to the user by transmitting through the display of the electronic device.

[0167] In one embodiment, if the display of the electronic device does not transmit light from the real space, the electronic device may display an image of the real space acquired by the camera on the display. For example, the image displayed by the display may include first content (901) being displayed by an external display.

[0168] FIG. 9b is a diagram illustrating an operation of displaying content through a portion of a displayable area based on movement of an external display according to one embodiment.

[0169] Referring to FIG. 9B, a portion of the external display may be obscured by movement of the external display (e.g., by lowering a window). In one embodiment, the displayable area due to movement of the external display may include a first area (911) where the external display is positioned and a second area (920) where the external display is not positioned.

[0170] According to one embodiment, the first area (911) may display content (903) whose size is adjusted to fit the size of the first area (911) by an external display. According to one embodiment, when a part of the displayable area is excluded as the external display moves, the external electronic device may adjust the size of the content (e.g., the first content (901) of FIG. 9A) to fit the size of the first area (911) capable of displaying an image among the displayable areas, and display the adjusted content (903) on the first area (911) of the external display.

[0171] FIG. 9c is a diagram illustrating an operation of displaying a virtual screen on at least a portion of a displayable area of ​​an electronic device based on movement of an external display, according to one embodiment.

[0172] Referring to FIG. 9C, a portion of the external display may be obscured by movement of the external display (e.g., by lowering a window). In one embodiment, the displayable area due to movement of the external display may include a first area (911) where the external display is positioned and a second area (920) where the external display is not positioned.

[0173] According to one embodiment, a portion (901-1) corresponding to the first area (911) of the content (901) illustrated in FIG. 9A may be displayed by an external display in the first area (911). According to one embodiment, when a portion of the displayable area is excluded as the external display moves, the external electronic device may display a portion (901-1) corresponding to the first area (911) of the content (e.g., the first content (901) illustrated in FIG. 9A) in the first area (911) capable of displaying an image among the displayable areas.

[0174] According to one embodiment, the electronic device may display a portion (902) corresponding to a second area (920) among content (e.g., first content (901) of FIG. 9A) in the second area (920). For example, the electronic device may generate virtual information (e.g., a virtual screen) based on the portion (902) corresponding to the second area (920) among the content, and display the generated virtual information in the second area (920).

[0175] According to one embodiment, the electronic device may output a virtual screen (or light source, image) to the second area (920) via a light output module (e.g., a projector) (e.g., the light output module (211) of FIG. 2) disposed in a housing (e.g., a glasses frame). According to one embodiment, the virtual screen may be reflected and / or diffracted by the second area (920) and guided to the user's eyes.

[0176] According to one embodiment, the electronic device may obtain brightness information of the first area (911) via a sensor and / or a camera. According to one embodiment, the brightness information of the first area (911) may be a reflection of the brightness value of the content (901-1) displayed in the first area (911) and a brightness value due to the external environment (e.g., weather, illuminance).

[0177] According to one embodiment, the electronic device can control the brightness value of a virtual screen displayed in the second area (920) to correspond to the brightness value of the first area (911) based on the brightness information of the first area (911).

[0178] This allows the electronic device to provide a continuous content viewing experience before and after movement of the external display.

[0179] In one embodiment, content may be displayed with its size adjusted based on the movement of the external display, as in 9b, or one content may be displayed through the external display and the virtual screen of the electronic device, as in 9c.

[0180] According to one embodiment, if the size of the first area is greater than or equal to the set size, the content may be displayed in a resized manner as in FIG. 9b, and if the size of the first area is less than the set size, one content may be displayed through an external display and a virtual screen of an electronic device as in FIG. 9c.

[0181] FIG. 9D is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display according to one embodiment.

[0182] Referring to FIG. 9D, if the size of the first area becomes smaller than the set size based on the movement of the external display, the output of the external display may be stopped. According to one embodiment, the electronic device may display a portion (902-1) corresponding to the second area (921) among the content (901) illustrated in FIG. 9A in the second area (921).

[0183] FIG. 10 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display according to one embodiment.

[0184] Referring to FIG. 10, based on the movement of the external display on which the first content is being displayed, a portion (1011) corresponding to a portion (1010) of the first content may be displayed in an exposed portion (1010) of the external display while a portion of the external display is covered. For example, the portion (1010) of the external display may be an area in which an image can be displayed among the displayable areas (1012).

[0185] According to one embodiment, an electronic device (or wearable electronic device) (101) (e.g., the electronic device (101) of FIG. 1, the wearable electronic device (200) of FIG. 2, the wearable electronic device (300) of FIG. 3, the wearable electronic device (300) of FIG. 4, or the electronic device (400) of FIG. 5) may identify a first area (1010) where an external display is arranged and a second area (1020) where an external display is not arranged based on movement information of an external display received from a camera (e.g., the camera module (160) of FIG. 1, the second camera module (253) of FIG. 2, the third camera module (255) of FIG. 2, the camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or the depth sensor (317) of FIG. 3) or an external electronic device.

[0186] According to one embodiment, the electronic device (101) may control the display (160) (e.g., the display module (160) of FIG. 1, the display member (201) of FIG. 2, the display (321) of FIG. 4, or the display member (440) of FIG. 5) to display a portion (1021) corresponding to the second area (1020) of the first content as a virtual screen in the second area (1020).

[0187] According to one embodiment, when the display (160) of the electronic device is a display that transmits light in an actual space, a portion (1011) corresponding to a part (1010) of the external display and the first content displayed on the external display may be shown to the user by transmitting through the display (160) of the electronic device. According to one embodiment, the electronic device may display only a virtual screen of a portion (1021) corresponding to the second area (1020) through the display (106).

[0188] According to one embodiment, when the display (160) of the electronic device does not transmit light of the real space, the electronic device may display an image of the real space acquired by the camera on the display. For example, the image displayed by the display may include an external display and a portion (1011) corresponding to a portion (1010) of the first content displayed on the external display. According to one embodiment, the electronic device may display a virtual screen of a portion (1011) corresponding to a portion (1010) of the first content displayed on the external display and a portion (1021) corresponding to a second area (1020) through the display (106). According to one embodiment, the electronic device may output a virtual screen (or light source, image) to the second area (1020) through a light output module (e.g., a projector) (e.g., the light output module (211) of FIG. 2) disposed in a housing (e.g., a glasses frame). In one embodiment, the virtual screen may be guided to the user's eyes by reflection and / or diffraction by the second area (1020) of the display (160).

[0189] In this way, by displaying one content through the display of the external display and the virtual screen of the electronic device, the electronic device can provide a continuous content viewing experience before and after movement of the external display.

[0190] FIG. 11A is a diagram illustrating an operation of displaying first content in the entire displayable area of ​​an electronic device according to one embodiment.

[0191] Referring to FIG. 11A, an electronic device (or a wearable electronic device) (e.g., an electronic device (101) of FIG. 1, a wearable electronic device (200) of FIG. 2, a wearable electronic device (300) of FIG. 3, a wearable electronic device (300) of FIG. 4, or an electronic device (400) of FIG. 5) recognizes data (e.g., 3D data) received from an external electronic device (e.g., an electronic device (104) of FIG. 1) and / or an image acquired by a camera (e.g., a camera module (160) of FIG. 1, a second camera module (253) of FIG. 2, a third camera module (255) of FIG. 2, camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or a depth sensor (317) of FIG. 3) to display a displayable area (1010) corresponding to an external display. Can be identified.

[0192] According to one embodiment, the displayable area (1010) may display an external environment (1101) and first content (1102) (e.g., information about the external environment) that are visible through the external display by an external display. For example, the external environment (1101) may be a building, and the first content (1102) may include information associated with the building. For example, the information associated with the building may include at least one of building information (e.g., building name, building size (height)), restaurant information within the building (e.g., restaurant name, restaurant type, phone number), time information required to get to the building, and route information (e.g., route map).

[0193] According to one embodiment, the first content (1102) may include advertising information related to a building and / or a restaurant within the building. For example, advertising information related to the building may be displayed around the building or partially overlapping the building. According to one embodiment, advertising information related to the restaurant may be displayed around the building or partially overlapping the building, based on the location of the restaurant within the building.

[0194] In one embodiment, the first content (1102) may include advertising information unrelated to the building (e.g., public service announcements or outdoor advertisements). In one embodiment, the advertising information unrelated to the building may be displayed around the building or partially overlapping the building.

[0195] In one embodiment, if a portion of an exterior wall of a building includes a space for advertising, the electronic device may display advertising information by overlapping the space for advertising.

[0196] According to one embodiment, the electronic device may display an image acquired by a camera on a display (e.g., a display module (160) of FIG. 1, a display member (201) of FIG. 2, a display (321) of FIG. 4, or a display member (440) of FIG. 5). For example, the image displayed by the display may include an external environment (1101) and first content (1102) that are visible through an external display.

[0197] FIG. 11B is a diagram illustrating an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display, according to one embodiment.

[0198] Referring to FIG. 11B, a portion of the external display may be obscured by movement of the external display (e.g., by lowering a window). In one embodiment, the displayable area due to movement of the external display may include a first area (1111) where the external display is positioned and a second area (1120) where the external display is not positioned.

[0199] According to one embodiment, the electronic device may display information (1121) about the external environment (1101) as a virtual screen in the second area (1120). According to one embodiment, the electronic device may output the virtual screen (or light source, image) to the second area (1120) through a light output module (e.g., a projector) (e.g., the light output module (211) of FIG. 2) disposed in a housing (e.g., a glasses frame). According to one embodiment, the virtual screen may be reflected and / or diffracted by the second area (1120) and guided to the user's eyes.

[0200] For example, information (1121) about the external environment may be received from a server (e.g., server (108) of FIG. 1), received from an external electronic device controlling an external display (e.g., electronic device (104) of FIG. 1), or captured information displayed by an external display as illustrated in FIG. 11A.

[0201] FIG. 11C is a diagram illustrating an operation of displaying a virtual screen on at least a portion of a displayable area of ​​an electronic device based on movement of an external display, according to one embodiment.

[0202] Referring to FIG. 11C, an electronic device may not display a virtual screen in an area where an external display is not positioned as the external display moves. According to one embodiment, the external electronic device may only display information (1102) about the external environment included in an area (1112) where a portion of the external display is positioned as the external display moves, based on the location of the information (1102) about the external environment shown in FIG. 11A. For example, no information may be displayed in an area where an external display is not positioned as the external display moves.

[0203] According to one embodiment, the external electronic device may not display information (1102) about the external environment shown in FIG. 11A that was positioned in an area where the external display is not positioned due to movement of the external display in an area (1112) where a portion of the external display is positioned. According to one embodiment, the external electronic device may move the display position of the information (1102) about the external environment within the external display in a direction opposite to the movement direction of the external display so that the position of the information (1102) about the external environment included in the area (1112) where a portion of the external display is positioned is maintained due to movement of the external display.

[0204] FIG. 12 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display according to one embodiment.

[0205] Referring to FIG. 12, an electronic device (or a wearable electronic device) (e.g., an electronic device (101) of FIG. 1, a wearable electronic device (200) of FIG. 2, a wearable electronic device (300) of FIG. 3, a wearable electronic device (300) of FIG. 4, or an electronic device (400) of FIG. 5) recognizes data (e.g., 3D data) received from an external electronic device (e.g., an electronic device (104) of FIG. 1) and / or an image acquired by a camera (e.g., a camera module (160) of FIG. 1, a second camera module (253) of FIG. 2, a third camera module (255) of FIG. 2, camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or a depth sensor (317) of FIG. 3) to thereby display a displayable area (1210) corresponding to an external display. Can be identified.

[0206] According to one embodiment, the displayable area (1210) may display first content (1201). For example, the first content (1201) may be displayed on an external display under the control of an external electronic device.

[0207] According to one embodiment, when the display of the electronic device (e.g., the display module (160) of FIG. 1, the display member (201) of FIG. 2, the display (321) of FIG. 4, or the display member (440) of FIG. 5) is a display that transmits light in real space, the external display and the first content (1201) displayed on the external display can be shown to the user by transmitting through the display of the electronic device.

[0208] In one embodiment, if the display of the electronic device does not transmit light from the real space, the electronic device may display an image of the real space acquired by the camera on the display. For example, the image displayed by the display may include first content (1201) being displayed by an external display.

[0209] In one embodiment, a portion of the external display may be obscured by movement of the external display (e.g., by lowering a window). In one embodiment, the displayable area (1210) due to movement of the external display may include a first area (1211) where the external display is positioned and a second area (1220) where the external display is not positioned.

[0210] According to one embodiment, the first area (1211) may display content (1202) that is sized to fit the size of the first area (1211) by an external display.

[0211] In one embodiment, the externally exposed second area (1220) may be visible to the external environment (1202) (e.g., a building).

[0212] According to one embodiment, if the movement of the external display continues and the size of the first area (1212) is less than a set size (e.g., about 20% of the displayable area (1210)), the external display may stop outputting content. According to one embodiment, if the size of the first area (1212) is less than the set size, the external display may stop outputting content under the control of an external electronic device controlling the external display. According to one embodiment, if the size of the first area (1212) is less than the set size, the electronic device may transmit a command to stop outputting content to the external electronic device controlling the external display, and the external electronic device controlling the external display may stop outputting content based on the received command.

[0213] According to one embodiment, if the size of the first area (1212) is less than a set size (e.g., about 20% of the displayable area (1210)), the electronic device may display content other than the content (1201, 1202) output through the external display as a virtual object.

[0214] According to one embodiment, when the display of the electronic device is a display that transmits light of an actual space, the external environment (1202) visible through the second area (1221) whose size is greater than a set value may be transmitted through the display of the electronic device and shown to the user. According to one embodiment, the electronic device may display only information (1203) related to the external environment (1202) as a virtual object on the display. According to one embodiment, the electronic device may output a virtual object (or a light source, an image) to the second area (1221) through a light output module (e.g., a projector) (e.g., the light output module (211) of FIG. 2) disposed in a housing (e.g., a glasses frame). According to one embodiment, the virtual object may be reflected and / or diffracted by the second area (1221) and guided to the user's eyes.

[0215] In one embodiment, when the display of the electronic device does not transmit light of the actual space, the electronic device may display an image including an external environment (1202) visible through a second area (1221) whose size is greater than a set value, obtained through a camera, on the display, and display information (1203) related to the external environment (1202) as a virtual object.

[0216] According to one embodiment, the electronic device may display content in a virtual screen in which the size of the first content (1201) is adjusted based on the size of the second area (1221) whose size is greater than a set value.

[0217] FIG. 13 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display according to one embodiment.

[0218] Referring to FIG. 13, an electronic device (or a wearable electronic device) (e.g., an electronic device (101) of FIG. 1, a wearable electronic device (200) of FIG. 2, a wearable electronic device (300) of FIG. 3, a wearable electronic device (300) of FIG. 4, or an electronic device (400) of FIG. 5) may display content different from content previously displayed on the external display as a virtual screen in the first area when the size of the first area (1310) where the external display is placed is less than a set value due to movement of the external display.

[0219] According to one embodiment, an external electronic device controlling an external display may stop outputting content when the size of a first area of ​​the external display becomes smaller than a set size. According to one embodiment, when it is determined that content output through the external display has stopped when the size of the first area where the external display is placed among the displayable areas becomes smaller than the set size, the electronic device may display content different from the content previously displayed on the external display as a virtual screen on the first area.

[0220] In one embodiment, the electronic device may transmit a command to a control device controlling the external display to stop outputting content when the size of a first area of ​​the displayable area where the external display is placed becomes smaller than a set size. In one embodiment, when it is confirmed that content output through the external display has stopped, the electronic device may display content different from the content previously displayed on the external display as a virtual screen on the first area.

[0221] According to one embodiment, when the size of a first area in which an external display is arranged among the displayable areas becomes smaller than a set size, the electronic device may display content different from content previously displayed on the external display as a virtual screen on the first area, regardless of whether the content is displayed on the external display.

[0222] For example, a user interface (UI) (1301) for controlling a music playback application running on an electronic device or an external electronic device can be displayed as a virtual screen in the first area (1310).

[0223] In one embodiment, the electronic device may display information about the external environment as a virtual screen in a second area where no external display is placed.

[0224] According to one embodiment, when the size of the first area (1310) where the external display is placed is less than a set value due to movement of the external display, a UI (1301) for controlling a music playback application on the external display may be displayed by the external electronic device.

[0225] FIG. 14 is a flowchart illustrating an operation for obtaining information about movement of an external display of an electronic device according to one embodiment.

[0226] Referring to FIG. 14, in operation 1401, an external electronic device (104) (e.g., the electronic device (104) of FIG. 1) may display first information on a display of the external electronic device (104). For example, the first information may be content (e.g., an image).

[0227] In one embodiment, in operation 1402, the external electronic device (104) may detect movement of the display. For example, the movement of the display may be detected based on a physical force, such as an input of a control command to move the display or manual control by a user. For example, the external electronic device (104) may detect movement of the display when a control command to move the display is input based on a user input. In one embodiment, the distance the display has moved may be detected based on the time for which the user input is maintained.

[0228] In one embodiment, the external electronic device (104) may detect whether the display is moving through a sensor. In one embodiment, the external electronic device (104) may detect the movement distance of the display through a sensor.

[0229] According to one embodiment, in operation 1403, an electronic device (or wearable electronic device) (101) (e.g., the electronic device (101) of FIG. 1 , the wearable electronic device (200) of FIG. 2 , the wearable electronic device (300) of FIG. 3 , the wearable electronic device (300) of FIG. 4 , or the electronic device (400) of FIG. 5 ) can detect movement of a display of an external electronic device (104).

[0230] For example, the electronic device (101) can recognize an image acquired by a camera (e.g., the camera module (160) of FIG. 1, the second camera module (253) of FIG. 2, the third camera module (255) of FIG. 2, the camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or the depth sensor (317) of FIG. 3) to confirm the movement of the display of the external electronic device (104).

[0231] According to one embodiment, in operation 1404, the electronic device (101) may request information associated with movement of the display of the external electronic device (104) from the external electronic device (104). For example, the information associated with movement of the display of the external electronic device (104) may include movement distance information and / or movement direction information of the display of the external electronic device (104).

[0232] In one embodiment, in operation 1405, the external electronic device (104) may transmit information associated with movement of the display of the external electronic device (104) to the electronic device (101).

[0233] According to one embodiment, in operation 1406, the electronic device (101) may check a change between a first area where the display of the external electronic device (104) is exposed and a second area where the display is not exposed. For example, the first area may be a portion of the display of the external electronic device (104) that is not covered by the movement of the display of the external electronic device (104). According to one embodiment, the second area may be an area where the display of the external electronic device (104) was placed or an area where the display of the external electronic device (104) is not placed due to the movement of the display of the external electronic device (104).

[0234] According to one embodiment, the external electronic device (104) can perform operation 1405 when movement of the display of the external electronic device (104) is detected even without operation 1404.

[0235] According to one embodiment, the electronic device (101) may determine movement of the display of the external electronic device (104) through at least one sensor and / or camera of the electronic device (101) without receiving information related to movement of the display of the external electronic device (104) from the external electronic device (104).

[0236] FIG. 15 is a flowchart illustrating a virtual screen display operation based on movement of an external display of an electronic device according to one embodiment.

[0237] Referring to FIG. 15, in operation 1501, an electronic device (or wearable electronic device) (101) (e.g., the electronic device (101) of FIG. 1, the wearable electronic device (200) of FIG. 2, the wearable electronic device (300) of FIG. 3, the wearable electronic device (300) of FIG. 4, or the electronic device (400) of FIG. 5) may request indoor information from an external electronic device (104) (e.g., the electronic device (104) of FIG. 1). For example, the indoor information may be related to a structure inside a space, such as a seat position, a seat shape, a window (or door) position, and / or a window shape.

[0238] According to one embodiment, when the electronic device (101) approaches or enters the interior of an external electronic device (104), it may request indoor information from the external electronic device (104) through a short-range communication method such as BT, BLE, UWB, or Wi-Fi.

[0239] According to one embodiment, the electronic device (101) can confirm proximity with an external electronic device (104) through BLE and confirm an accurate distance using UWB communication.

[0240] According to one embodiment, the electronic device (101) may also establish a communication connection with a second external electronic device (e.g., a smartphone) that is in communication with the external electronic device (104) without a direct communication connection with the external electronic device (104).

[0241] In one embodiment, in operation 1502, the external electronic device (104) may transmit indoor information to the electronic device (101).

[0242] According to one embodiment, the electronic device (101) can be connected to an external electronic device (104) via Wi-Fi, BT, UWB, or 5G networks, and the location of the electronic device (101) within the external electronic device (104) (e.g., a vehicle) can be identified using BT, UWB, DMS (driver monitoring system), or OMS (occupant monitoring system).

[0243] According to one embodiment, when requesting indoor information while wearing an electronic device (101), an external electronic device (104) can determine the location of the electronic device (101) and transmit data (e.g., 3D data) based on the location of the electronic device (101) to the electronic device (101).

[0244] According to one embodiment, the electronic device (101) can calculate the position of the electronic device (101) in space through received data, an image acquired through a camera (e.g., RGB Camera), and at least one sensor (e.g., depth sensor, radar).

[0245] According to one embodiment, in operation 1503, the electronic device (101) may request display control information from the external electronic device (104). For example, the display control information may include at least one of information related to whether the display of the external electronic device (104) is moved, information about the movement distance of the display of the external electronic device (104), or information about the size of a portion of the display of the external electronic device (104) exposed by the movement of the display of the external electronic device (104).

[0246] In one embodiment, in operation 1504, the external electronic device (104) may transmit display control information to the electronic device (101).

[0247] According to one embodiment, in operation 1505, the electronic device (101) may request external environmental information from a server (108) (e.g., server (108) of FIG. 1 ). For example, the external environmental information may include at least one of temperature information, weather information, fine dust information, or information related to a building that is transmitted through the display of the external electronic device (104) or is visible by movement of the display of the external electronic device (104).

[0248] In one embodiment, in operation 1506, the server (108) may transmit external environmental information to the electronic device (101).

[0249] In one embodiment, in operation 1507, the electronic device (101) may request an external camera image from an external electronic device (104). For example, the electronic device (101) may request an external camera image from the external electronic device (104) when the user's gaze is directed toward the external environment.

[0250] In one embodiment, in operation 1508, the external electronic device (104) may transmit an external camera image to the electronic device (101).

[0251] According to one embodiment, in operation 1509, the electronic device (101) may display external images and external environment information. For example, the electronic device (101) may display external images and external environment information as virtual screens in a first area and / or a second area of ​​a displayable area where a display of the external electronic device (104) is arranged.

[0252] FIG. 16 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of an external display according to one embodiment.

[0253] Referring to FIG. 16, an electronic device (or, wearable electronic device) (101) (e.g., the electronic device (101) of FIG. 1, the wearable electronic device (200) of FIG. 2, the wearable electronic device (300) of FIG. 3, the wearable electronic device (300) of FIG. 4, or the electronic device (400) of FIG. 5) can determine a mixing level of an external display and a virtual screen displayed by the electronic device based on a user input.

[0254] For example, if the blending level can be selected from 1 to 5, the type and amount of information contained on the virtual screen can increase as the level increases. At the highest level, the virtual screen can display not only the displayable area but also the surrounding area.

[0255] As an example, Figure 16 explains the contents from mixing level 1 to level 3.

[0256] According to one embodiment, the electronic device can divide the displayable area into a first area where the external display is placed and a second area where the external display is not placed by movement of the external display.

[0257] In one embodiment, in the first stage of the mixed level, the electronic device can display images captured by the camera on the display without a virtual screen display. For example, the images displayed on the display of the electronic device may include content (1610) displayed in a first area by an external display and an external environment (1620) visible through a second area without an external display.

[0258] In one embodiment, in the mixed level 2 step, the electronic device can display a virtual screen so that it can continuously provide content before and after the external display is moved, even if the external display is moved.

[0259] For example, a portion (1611) corresponding to the first area of ​​the first content displayed before moving the external display may be displayed in the first area by the external display.

[0260] According to one embodiment, the electronic device may display a portion (1621) corresponding to the second area of ​​the first content as a virtual screen in the second area.

[0261] In one embodiment, output from the external display may be stopped, and the electronic device may display the first content as a virtual screen in the first area and the second area.

[0262] In this way, by displaying one content through the external display and the virtual screen even after the external display is moved, a continuous content viewing experience can be provided before and after the external display is moved.

[0263] In one embodiment, in the mixed level 3 step, information about the external environment can be output together with the content output.

[0264] For example, content (1612) that was displayed before the external display was moved and whose size is adjusted to correspond to the size of the first area may be displayed on a part of the first area by the external display.

[0265] According to one embodiment, the electronic device may display a virtual screen (1622) in an area of ​​the second area and the first area, excluding the area where content (1612) is displayed. For example, the virtual screen (1622) may include information about the external environment. For example, the information about the external environment may include at least one of temperature information, weather information, fine dust information, information related to a building visible through the second area, or an image capturing the exterior.

[0266] FIG. 17 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device based on movement of a sunroof when the external display is a sunroof, according to one embodiment.

[0267] Referring to FIG. 17, an electronic device (or a wearable electronic device) (e.g., an electronic device (101) of FIG. 1, a wearable electronic device (200) of FIG. 2, a wearable electronic device (300) of FIG. 3, a wearable electronic device (300) of FIG. 4, or an electronic device (400) of FIG. 5) can display corresponding data (e.g., 3D data) received from an external electronic device (e.g., an electronic device (104) of FIG. 1) and / or an image acquired by a camera (e.g., a camera module (160) of FIG. 1, a second camera module (253) of FIG. 2, a third camera module (255) of FIG. 2, camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or a depth sensor (317) of FIG. 3) on an external display (e.g., a sunroof) by recognizing the image. The area (1710) can be identified.

[0268] In one embodiment, a portion of the external display may be obscured by movement of the external display (e.g., opening a sunroof). In one embodiment, the displayable area (1710) due to movement of the external display may include a first area (1711) where the external display is positioned and a second area (1712) where the external display is not positioned.

[0269] According to one embodiment, the first area (1711) may display content (1720) that is sized to fit the size of the first area (1711) by an external display.

[0270] According to one embodiment, the electronic device may display information (1721) related to the external environment as a virtual screen on a second area (1712) exposed to the outside. For example, the electronic device may display information (1721) related to the external environment as a 3D virtual object. According to one embodiment, the electronic device may display information (e.g., a 3D map) related to an area of ​​interest to the user as a virtual object on the second area (1712). According to one embodiment, the electronic device may output a virtual object (or light source, image) to the second area (1712) through a light output module (e.g., a projector) (e.g., the light output module (211) of FIG. 2) disposed on a housing (e.g., a glasses frame). According to one embodiment, the virtual object may be reflected and / or diffracted by the second area (1712) and guided to the user's eyes.

[0271] FIG. 18 is a drawing for explaining an operation of displaying a virtual screen in at least a portion of a displayable area of ​​an electronic device according to one embodiment.

[0272] Referring to FIG. 18, an electronic device (or a wearable electronic device) (e.g., an electronic device (101) of FIG. 1, a wearable electronic device (200) of FIG. 2, a wearable electronic device (300) of FIG. 3, a wearable electronic device (300) of FIG. 4, or an electronic device (400) of FIG. 5) can display a first display corresponding to a first external display (e.g., a door) by recognizing data (e.g., 3D data) received from an external electronic device (e.g., an electronic device (104) of FIG. 1) and / or an image acquired by a camera (e.g., a camera module (160) of FIG. 1, a second camera module (253) of FIG. 2, a third camera module (255) of FIG. 2, camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or a depth sensor (317) of FIG. 3) A second displayable area (1820) corresponding to the area (1810) and a second external display (e.g., a door) can be identified.

[0273] According to one embodiment, the displayable area (1810) may display content (1811). For example, the content (1811) may be displayed on an external display under the control of an external electronic device. For example, if the first external display is a door inside a building, the content (1811) may include external weather information.

[0274] According to one embodiment, the second displayable area (1820) may display content (1821). For example, the content (1821) may be displayed on an external display under the control of an external electronic device. For example, if the first external display is a door inside a building, the content (1821) may include external environmental information (e.g., an outside landscape).

[0275] According to one embodiment, when the display of the electronic device (e.g., the display module (160) of FIG. 1, the display member (201) of FIG. 2, the display (321) of FIG. 4, or the display member (440) of FIG. 5) is a display that transmits light in real space, the external display and the content (1811, 1821) displayed on the external display can be shown to the user by transmitting through the display of the electronic device.

[0276] In one embodiment, if the display of the electronic device does not transmit light from the real space, the electronic device may display an image captured by the camera on the display. For example, the image displayed by the display may include content (1811) being displayed by an external display.

[0277] In one embodiment, when the second external display is moved (e.g., the door is opened) and a portion of the second external display is covered by the first external display, the second displayable area (1820) may include a first area (1820-1) where the second external display is arranged and a second area (1830) where the second external display is not arranged. In one embodiment, a portion of the content (1821) corresponding to the first area (1820-1) may be displayed in the first area (1820-1) where the second external display is arranged under the control of an external electronic device. In one embodiment, the area (1820-2) of the second external display covered by the first external display may have its content display stopped under the control of the second external display.

[0278] According to one embodiment, the electronic device may generate virtual information for a portion of the content (1821) corresponding to a second area (1830) where an external display is not arranged, and display the generated virtual information in the second area (1830). According to one embodiment, the electronic device may output the virtual information (or light source, image) to the second area through a light output module (e.g., a projector) (e.g., the light output module (211) of FIG. 2) arranged in a housing (e.g., a glasses frame). According to one embodiment, the virtual information may be reflected and / or diffracted by the second area and guided to the user's eyes.

[0279] In one embodiment, when the second external display continues to move so that the second external display is completely excluded from the second displayable area (1820) (e.g., the second external display is completely covered by the movement), the electronic device may generate virtual information corresponding to the entire content (1821) and display the generated virtual information in the second displayable area (1820).

[0280] In one embodiment, the door may be equipped with a door opening sensor, and the electronic device may obtain information related to the movement of the door through the door opening sensor that is connected to the communication.

[0281] According to one embodiment, the operations of FIGS. 6 to 17 can also be performed for doors placed inside a building.

[0282] Although the present disclosure illustrates the external display moving downward, this is not limited thereto, and the present disclosure may also be applied to cases where the external display moves upward, left, right, diagonally, or in other scenarios.

[0283] While the present disclosure illustrates a single external display moving in a single direction, this is not limited to the above. The external display may include two or more pieces, and may be capable of moving left and right, up and down, or in three or more directions, or in other scenarios. In one embodiment, the first and second regions may each be multiple, and if multiple, they may be spaced apart from each other, or the present disclosure may be applied to other scenarios.

[0284] FIG. 19 is a drawing for explaining an operation of displaying a virtual screen on at least a portion of a displayable area of ​​an electronic device based on rolling of a rollable display when the external display is a rollable display, according to one embodiment.

[0285] Referring to FIG. 19, an electronic device (or a wearable electronic device) (e.g., an electronic device (101) of FIG. 1, a wearable electronic device (200) of FIG. 2, a wearable electronic device (300) of FIG. 3, a wearable electronic device (300) of FIG. 4, or an electronic device (400) of FIG. 5) can display corresponding data (e.g., 3D data) received from an external electronic device (e.g., an electronic device (104) of FIG. 1) and / or an image acquired by a camera (e.g., a camera module (160) of FIG. 1, a second camera module (253) of FIG. 2, a third camera module (255) of FIG. 2, camera modules (311, 312, 313, 314, 315, 316) of FIG. 3, or a depth sensor (317) of FIG. 3) on an external display (e.g., a rollable display) by recognizing the image. An area (1910) can be identified. According to one embodiment, the displayable area (1910) may be an area of ​​the rollable display in which an image can be displayed when the rollable display is fully expanded.

[0286] According to one embodiment, content (1911) may be displayed in the displayable area (1910). For example, the content (1911) may be a screen displayed on an external display under the control of an external electronic device. For example, if the external display is a rollable display, the content (1911) may include a screen (e.g., a home screen, an application execution screen) of the external electronic device including the rollable display.

[0287] In one embodiment, if the display of the electronic device is a display that transmits light in real space, the external display and the content (1911) displayed on the external display can be shown to the user by transmitting through the display of the electronic device.

[0288] According to one embodiment, when the display (160) of the electronic device does not transmit light in real space, the electronic device may display an image acquired by a camera on a display (e.g., the display module (160) of FIG. 1, the display member (201) of FIG. 2, the display (321) of FIG. 4, or the display member (440) of FIG. 5). For example, the image displayed by the display may include content (1911) being displayed by an external display.

[0289] According to one embodiment, when the external display, which is a rollable display, moves (e.g., when a door is opened), the displayable area (1910) may include a first area (1920) where the external display is arranged and a second area (1921) where the external display is not arranged. According to one embodiment, in the first area (1920) where the external display is arranged, a portion of the content (1911) corresponding to the first area (1920) may be displayed under the control of an external electronic device. According to one embodiment, the electronic device may generate virtual information for a portion of the content (1911) corresponding to the second area (1921) where the external display is not arranged, and display the generated virtual information in the second area (1921). According to one embodiment, the electronic device may output virtual information (or a light source, an image) to the second area (1921) via a light output module (e.g., a projector) (e.g., the light output module (211) of FIG. 2) disposed in a housing (e.g., a glasses frame). According to one embodiment, the virtual information may be reflected and / or diffracted by the second area (1921) and guided to the user's eyes.

[0290] According to one embodiment, if the display of the electronic device is a display that transmits light in a real space, the external display and a portion corresponding to the first area (1920) of the content (1911) displayed on the external display may be shown to the user by transmitting through the display of the electronic device. According to one embodiment, the electronic device may display only a virtual screen of a portion corresponding to the second area (1921) of the content (1911) where the external display is not arranged through the display.

[0291] According to one embodiment, when the display of the electronic device does not transmit light of the real space, the electronic device may display an image of the real space acquired by the camera on the display. For example, the image displayed by the display may include an external display and a portion corresponding to a first area (1920) among the contents (1911) displayed on the external display. According to one embodiment, the electronic device may display a virtual screen of a portion corresponding to the first area (1920) among the contents (1911) displayed on the external display and a portion corresponding to a second area (1921) among the contents (1911) where the external display is not arranged, through the display.

[0292] According to one embodiment, a wearable electronic device may include a memory, a camera, a display, and at least one processor operatively connected directly or indirectly with the memory, the camera, and the display.

[0293] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to identify a displayable area corresponding to an external display included in an image of a space acquired through the camera.

[0294] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to identify a first region in which a portion of the external display is disposed among the displayable regions and a second region, which is an area remaining among the displayable regions excluding the first region, based on the fact that at least a portion of the external display is excluded from the displayable region due to movement of the external display.

[0295] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to control the display to display a virtual screen in the second area.

[0296] According to one embodiment, the wearable electronic device may further include at least one sensor.

[0297] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to determine movement of the external display based on at least one of sensor data acquired through the at least one sensor or an image acquired through the camera.

[0298] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to identify the first area and the second area based on movement of the external display.

[0299] According to one embodiment, the wearable electronic device may further include a communication module.

[0300] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to receive information related to movement of the external display from an external electronic device controlling the external display connected via the communication module.

[0301] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to identify the first area and the second area based on information related to movement of the external display.

[0302] In one embodiment, the external display may be at least one of a vehicle window, a sunroof, a window, or a rollable display.

[0303] According to one embodiment, the external electronic device may be at least one of a control device of the vehicle that controls a window or the sunroof of the vehicle, an external electronic device that controls the window, or an external electronic device that controls the rollable display.

[0304] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to determine, via the communication module, that the wearable electronic device is located within the vehicle.

[0305] In one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to receive 3D data of the vehicle from the control device.

[0306] In one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to further utilize the 3D data to identify the displayable area.

[0307] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to receive first content displayed on the external display from an external electronic device controlling the external display connected via the communication module.

[0308] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to display a portion of the received first content corresponding to the second area in the second area.

[0309] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to receive first content displayed on the external display from an external electronic device controlling the external display connected via the communication module.

[0310] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to display the received first content in the displayable area.

[0311] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to display information related to an external environment in the second area based on movement of the external display displaying first content.

[0312] According to one embodiment, the wearable electronic device may further include at least one sensor.

[0313] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to display virtual information related to the external environment in the second area based on detecting a user's gaze toward the second area through the at least one sensor.

[0314] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to control the brightness of the virtual screen of the second area based on the brightness of the first area.

[0315] According to one embodiment, the memory may store instructions that, when executed by the processor, cause the wearable electronic device to display content different from content previously displayed on the external display as a virtual screen on at least a portion of the displayable area including the first area, based on a size of the first area being less than a set size.

[0316] According to one embodiment, a method for controlling a wearable electronic device may include an operation of identifying a displayable area corresponding to an external display included in an image of a space acquired through a camera of the wearable electronic device.

[0317] According to one embodiment, a control method of a wearable electronic device may include an operation of identifying a first region in which a portion of the external display is disposed among the displayable regions and a second region, which is an area remaining among the displayable regions excluding the first region, based on the fact that at least a portion of the external display is excluded from the displayable region due to movement of the external display.

[0318] According to one embodiment, a method for controlling a wearable electronic device may include an action of displaying a virtual screen in the second area.

[0319] According to one embodiment, the operation of identifying the second area, which is an area remaining from the first area among the first area and the displayable area, may confirm movement of the external display based on at least one of sensor data acquired through at least one sensor of the wearable electronic device or an image acquired through the camera.

[0320] According to one embodiment, the operation of identifying the second area, which is an area remaining from the first area among the first area and the displayable area, may identify the first area and the second area based on movement of the external display.

[0321] According to one embodiment, the operation of identifying the second area, which is an area remaining from the first area among the first area and the displayable area, may receive information related to movement of the external display from an external electronic device that controls the external display.

[0322] According to one embodiment, the operation of identifying the second area, which is an area remaining from the first area among the first area and the displayable area, may identify the first area and the second area based on information related to movement of the external display.

[0323] In one embodiment, the external display may be at least one of a window, a sunroof, a window, or a sunroof of the vehicle.

[0324] According to one embodiment, the external electronic device may be at least one of a control device of the vehicle that controls a window or the sunroof of the vehicle, an external electronic device that controls the window, or an external electronic device that controls the rollable display.

[0325] According to one embodiment, the act of identifying the displayable area may include confirming, via the communication module, that the wearable electronic device is located within the vehicle.

[0326] According to one embodiment, the operation of identifying the displayable area may receive 3D data of the vehicle from the control device.

[0327] According to one embodiment, the operation of identifying the displayable area may further use the 3D data to identify the displayable area.

[0328] According to one embodiment, the operation of identifying the displayable area, the operation of displaying the virtual screen in the second area, may receive first content being displayed on the external display from an external electronic device controlling the external display.

[0329] According to one embodiment, the operation of identifying the displayable area may include the operation of displaying the virtual screen in the second area, and the operation of displaying a portion of the received first content corresponding to the second area may be displayed in the second area.

[0330] According to one embodiment, the operation of displaying the virtual screen in the second area may display information related to an external environment in the second area based on movement of the external display displaying the first content.

[0331] According to one embodiment, the control method of the wearable electronic device may further include an operation of displaying virtual information related to the external environment in the first area based on detecting a user's gaze toward the second area through at least one sensor of the wearable electronic device.

[0332] According to one embodiment, the operation of displaying the virtual screen in the second area may control the brightness of the virtual screen in the second area based on the brightness of the first area.

[0333] According to one embodiment, a non-transitory computer-readable recording medium storing one or more programs may store instructions that cause a wearable electronic device to identify a displayable area corresponding to an external display included in an image of a space acquired through a camera of the wearable electronic device.

[0334] According to one embodiment, the one or more programs may store instructions that cause the wearable electronic device to identify a first region in which a portion of the external display is disposed among the displayable regions and a second region, which is an area remaining among the displayable regions excluding the first region, based on movement of the external display excluding at least a portion of the external display from the displayable region.

[0335] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to control the display to display a virtual screen in the second area.

[0336] According to one embodiment, the wearable electronic device may further include at least one sensor.

[0337] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to identify movement of the external display based on at least one of sensor data acquired through the at least one sensor or an image acquired through the camera.

[0338] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to identify the first area and the second area based on movement of the external display.

[0339] According to one embodiment, the wearable electronic device may further include a communication module.

[0340] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to receive information related to movement of the external display from an external electronic device that controls the external display connected via the communication module.

[0341] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to identify the first area and the second area based on information related to movement of the external display.

[0342] As used herein, “based on” includes “at least based on.”

[0343] In one embodiment, the external display may be at least one of a vehicle window, a sunroof, a window, or a rollable display.

[0344] According to one embodiment, the external electronic device may be at least one of a control device of the vehicle that controls a window or the sunroof of the vehicle, an external electronic device that controls the window, or an external electronic device that controls the rollable display.

[0345] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to confirm, via the communication module, that the wearable electronic device is located within the vehicle.

[0346] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to receive 3D data of the vehicle from the control device.

[0347] In one embodiment, the one or more programs may store instructions that cause the wearable electronic device to further utilize the 3D data to identify the displayable area.

[0348] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to receive first content displayed on the external display from an external electronic device that controls the external display connected via the communication module.

[0349] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to display a portion of the received first content corresponding to the second area in the second area.

[0350] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to receive first content displayed on the external display from an external electronic device that controls the external display connected via the communication module.

[0351] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to display the received first content in the displayable area.

[0352] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to display information related to an external environment in the second area based on movement of the external display displaying the first content.

[0353] According to one embodiment, the wearable electronic device may further include at least one sensor.

[0354] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to display virtual information related to the external environment in the second area based on detecting a user's gaze toward the second area through the at least one sensor.

[0355] According to one embodiment, the one or more programs may store commands that cause the wearable electronic device to control the brightness of the virtual screen of the second area based on the brightness of the first area.

[0356] According to one embodiment, the one or more programs may store instructions that cause the wearable electronic device to display content different from content previously displayed on the external display as a virtual screen on at least a portion of the displayable area including the first area, based on a size of the first area being less than a set size.

[0357] Electronic devices according to the embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0358] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through at least third components.

[0359] The term "module" used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC). Accordingly, each "module" of the present disclosure may include a circuit.

[0360] One embodiment of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0361] According to one embodiment, the method according to one embodiment disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0362] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0363] While the present disclosure has been illustrated and described with reference to various embodiments, it is to be understood that the various embodiments are intended to be illustrative and not limiting. Those skilled in the art will appreciate that various changes in form and detail may be made without departing from the true spirit and scope of the present disclosure, including the appended claims and their equivalents. It is also to be understood that any embodiment(s) described in this disclosure may be used in conjunction with any other embodiment(s) described herein.

Claims

1. In wearable electronic devices, camera; display; At least one processor comprising a processing circuit; and When individually or collectively executed by at least one processor, said wearable electronic device causes: Identify a displayable area corresponding to an external display included in an image of a space acquired through the above camera, Based on the fact that at least a part of the external display is excluded from the displayable area due to movement of the external display, a first area in which a part of the external display is placed among the displayable areas and a second area which is the remaining area excluding the first area among the displayable areas are identified, A wearable electronic device comprising a memory storing commands for controlling the display to display a virtual screen in the second area.

2. In paragraph 1, further comprising at least one sensor; The above instructions, when individually or collectively executed by the at least one processor, cause the wearable electronic device to: Detecting movement of the external display based on at least one of the sensor data acquired through the at least one sensor and / or the image acquired through the camera, A wearable electronic device configured to identify the first area and the second area based on movement of the external display.

3. In paragraph 1 or 2, A communication module including a communication circuit; further comprising: The above instructions, when individually or collectively executed by the at least one processor, cause the wearable electronic device to: At least through the communication module, the wearable electronic device is connected and receives information related to movement of the external display from an external electronic device controlling the external display, A wearable electronic device configured to identify the first area and the second area based on information related to movement of the external display.

4. In paragraph 3, The above external display is, At least one of the vehicle's windows, sunroof, windows, or rollable displays, The above external electronic device, A wearable electronic device, at least one of a control device of the vehicle including a processing circuit for controlling a window or a sunroof of the vehicle, an external electronic device including a processing circuit for controlling the window, or an external electronic device including a processing circuit for controlling the rollable display.

5. In paragraph 4, The above instructions, when individually or collectively executed by the at least one processor, cause the wearable electronic device to: Through the above communication module, it is confirmed that the wearable electronic device is located within the vehicle, Receive 3D data of the vehicle from the above control device, A wearable electronic device further utilizing said 3D data to identify said displayable area.

6. In any one of paragraphs 1 to 5, A communication module including a communication circuit; further comprising: The above instructions, when individually or collectively executed by the at least one processor, cause the wearable electronic device to: At least, a first content displayed on the external display is received from an external electronic device connected to the wearable electronic device through the communication module and controlling the external display, A wearable electronic device that displays a portion of the received first content corresponding to the second area in the second area.

7. In any one of paragraphs 1 to 5, A communication module including a communication circuit; further comprising: The above instructions, when individually or collectively executed by the at least one processor, cause the wearable electronic device to: At least, a first content displayed on the external display is received from an external electronic device connected to the wearable electronic device through the communication module and controlling the external display, A wearable electronic device that displays the received first content in the displayable area.

8. In any one of paragraphs 1 to 5, The above instructions, when individually or collectively executed by the at least one processor, cause the wearable electronic device to: A wearable electronic device that displays information related to an external environment in a second area based on movement of the external display displaying first content.

9. In paragraph 8, further comprising at least one sensor; The above instructions, when individually or collectively executed by the at least one processor, cause the wearable electronic device to: A wearable electronic device that displays virtual information related to the external environment in the second area based on detecting the user's gaze toward the second area through at least one sensor.

10. In any one of paragraphs 1 to 9, The above instructions, when individually or collectively executed by the at least one processor, cause the wearable electronic device to: A wearable electronic device that controls the brightness of the virtual screen of the second area based on the brightness of the first area.

11. In any one of paragraphs 1 to 10, The above instructions, when individually or collectively executed by the at least one processor, cause the wearable electronic device to: A wearable electronic device that displays content different from content previously displayed on the external display as a virtual screen on at least a portion of the displayable area including the first area, based on the size of the first area being less than a set size.

12. A method for controlling a wearable electronic device, An action of identifying a displayable area corresponding to an external display included in an image of a space acquired through a camera of the wearable electronic device; An operation of identifying a first region in which a part of the external display is placed among the displayable regions and a second region, which is an area remaining from the first region among the displayable regions, based on the fact that at least a part of the external display is excluded from the displayable region due to movement of the external display; and A method for controlling a wearable electronic device, comprising: an operation of displaying a virtual screen in the second area; 13. In paragraph 12, The operation of identifying the second area, which is the remaining area excluding the first area among the first area and the displayable area, is as follows: Detecting movement of the external display based on at least one of sensor data acquired through at least one sensor of the wearable electronic device or the image acquired through the camera, A control method of a wearable electronic device for identifying the first area and the second area based on movement of the external display.

14. In paragraph 12 or 13, The operation of identifying the second area, which is the remaining area excluding the first area among the first area and the displayable area, is as follows: Receive information related to movement of the external display from an external electronic device controlling the external display, A control method of a wearable electronic device for identifying the first area and the second area based on information related to movement of the external display.

15. In paragraph 14, The above external display is, At least one of a vehicle window, a vehicle sunroof, a window, or a rollable display, The above external electronic device, A method for controlling a wearable electronic device, wherein at least one of a control device of the vehicle controlling a window of the vehicle and / or the sunroof of the vehicle, an external electronic device controlling the window, or an external electronic device controlling the rollable display.

Images