Display module control method and electronic device for implementing the same

Abstract

A display module control method and an electronic device implementing the same are disclosed. According to various embodiments, an electronic device may include: a display module; a processor; and a memory electrically connected to the processor and storing instructions executable by the processor. When the instructions are executed, the processor may control the display module to display the first information on at least one first surface of the virtual object and the second information on at least one second surface of the virtual object at a determined location in at least one of the real world and the virtual world. The processor may determine a gaze direction of the user based on a location of the user in at least one of the real world and the virtual world. If the gaze direction of the user is a direction corresponding to the at least one second surface, the processor may control the display module to display the first information on the at least one second surface.

Description

Technical Field

[0001] The present disclosure relates to a method of controlling a display module that displays a virtual object in an augmented reality (AR) mode or a virtual reality (VR) mode, and an electronic device that performs the method. Background Art

[0002] An electronic device that displays a virtual object on a display module in an augmented reality (AR) mode or a virtual reality (VR) mode may employ spatial anchoring that anchors the position of the virtual object to a specific position.

[0003] A virtual object that is anchored at a specific location can have a window that sends information to the user's location when the virtual object is anchored.

[0004] When the user of the electronic device moves away from the location where the virtual object is anchored and the information provided by the virtual object through the window changes or a notification appears, the user can unanchor the virtual object or move to the location where the virtual object is anchored to identify the new information or notification. Summary of the invention

[0005] Solution to the problem

[0006] According to various embodiments, an electronic device may include: a display module; a processor; and a memory, the memory being electrically coupled to the processor and storing instructions executable by the processor. When the instructions are executed, the processor may be configured to: control the display module to display first information on at least one first surface of a virtual object and second information on at least one second surface of the virtual object at a determined position in at least one of the real world or the virtual world. The processor may be configured to determine a gaze direction of the user based on the user's position in the real world or the virtual world. In response to the user's gaze direction being a direction corresponding to the at least one second surface, the processor may be configured to control the display module to display the first information on the at least one second surface.

[0007] According to various embodiments, an electronic device may include: a display module; a processor; and a memory, the memory being electrically coupled to the processor and storing instructions executable by the processor. When the instructions are executed, the processor may be configured to: identify a first position of a user in at least one of a real world or a virtual world. The processor may be configured to: in response to the first position, control the display module to display first information on at least one first surface of a virtual object at a determined position in the real world or the virtual world, and display second information on at least one second surface of the virtual object. The processor may be configured to identify a second position of the user in the real world or the virtual world. In response to the gaze direction of the user being a direction corresponding to the at least one second surface, the processor may be configured to control the display module to display the first information on the at least one second surface based on the distance and direction between the determined position and the second position.

[0008] According to various embodiments, a display module control method may include: controlling the display module to display first information on at least one first surface of a virtual object and display second information on at least one second surface of the virtual object at a determined position in at least one of the real world or the virtual world; determining a gaze direction of a user based on the user's position in the real world or the virtual world; and in response to the user's gaze direction being a direction corresponding to the at least one second surface, controlling the display module to display the first information on the at least one second surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a block diagram illustrating example electronic devices in a network environment according to an embodiment.

[0010] Figure 2 is a diagram showing an example structure of a wearable electronic device according to an embodiment.

[0011] Figure 3 is a diagram illustrating example operations performed by an electronic device for anchoring a position of a virtual object according to an embodiment.

[0012] Figure 4 is a diagram illustrating an example first surface of a virtual object according to an embodiment.

[0013] Figure 5 is a diagram illustrating an example second surface of a virtual object according to an embodiment.

[0014] Figure 6 is a diagram illustrating an example operation performed by an electronic device for providing first information using a second surface of a virtual object according to an embodiment.

[0015] Figure 7 is a diagram illustrating an example operation performed by an electronic device for providing first information using a second surface of a virtual object having a changed position and size according to an embodiment.

[0016] Figure 8 is a flowchart illustrating an example method of controlling a display module according to an embodiment.

[0017] Fig. 9 is a diagram illustrating an example operation performed by an electronic device for providing first information using a second virtual object according to an embodiment.

[0018] Fig.10 is a flowchart illustrating an example method of controlling a display module that provides first information using a second virtual object according to an embodiment. DETAILED DESCRIPTION

[0019] The embodiments will be described in detail below with reference to the accompanying drawings. When describing the embodiments with reference to the accompanying drawings, like reference numerals refer to like elements, and the related description will not be repeated.

[0020] As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0021] Figure 1 is a block diagram illustrating an example electronic device 101 in a network environment 100 according to an embodiment.

[0022] Reference Figure 1, the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with at least one of the electronic device 104 and the server 108 via the second network 199 (e.g., a long-range wireless communication network). The electronic device 101 may communicate with the electronic device 104 via the server 108. The electronic device 101 may include a processor 120, a memory 130, an input module 150, at least one sound output module 155, a display module 160, an audio module 170, a sensor module 176, an interface 177, a connection terminal 178, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. In various embodiments, at least one of the above components (e.g., the connection terminal 178) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In various embodiments, some of the above-described components (eg, sensor module 176, camera module 180, or antenna module 197) may be integrated into a single component (eg, display module 160).

[0023] The processor 120 may run, for example, software (e.g., program 140) to control at least one other component (e.g., hardware component or software component) of the electronic device 101 connected to the processor 120, and may perform various data processing or calculations. According to an embodiment, as at least part of the data processing or calculation, the processor 120 may store a command or data received from another component (e.g., sensor module 176 or communication module 190) in the volatile memory 132, process the command or data stored in the volatile memory 132, and store the resultant data in the non-volatile memory 134. The processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is independent of or combined with the main processor 121 in operation. For example, when the electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or adapted to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121, or as part of the main processor 121.

[0024] When the main processor 121 is in an inactive (e.g., sleep) state, the auxiliary processor 123 (rather than the main processor 121) may control at least some of the functions or states related to at least one component among the components of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190), or when the main processor 121 is in an active state (e.g., running an application), the auxiliary processor 123 may control at least some of the functions or states related to at least one component among the components of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190) together with the main processor 121. The auxiliary processor 123 (e.g., ISP or CP) may be implemented as part of another component (e.g., camera module 180 or communication module 190) that is functionally related to the auxiliary processor 123. The auxiliary processor 123 (e.g., NPU) may include a hardware structure dedicated to artificial intelligence (AI) model processing. The AI ​​model may be generated through machine learning. For example, such learning may be performed by an electronic device 101 in which the AI ​​model is executed or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, for example. The AI ​​model may include multiple artificial neural network layers. The artificial neural network may include, for example, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q network or a combination of two or more thereof, but is not limited thereto. Alternatively or additionally, the artificial intelligence model may include a software structure in addition to a hardware structure.

[0025] The memory 130 may store various data segments used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data segments may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134.

[0026] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0027] The input module 150 may receive commands or data to be used by other components (e.g., the processor 120) of the electronic device 101 from outside (e.g., a user) of the electronic device 101. The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus).

[0028] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. The receiver can be implemented as a separate part from the speaker, or as part of the speaker.

[0029] The display module 160 may visually provide information to the outside of the electronic device 101 (e.g., a user). The display module 160 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. The display module 160 may include a touch sensor adapted to sense a touch or a pressure sensor adapted to measure the strength of the force of the touch.

[0030] The audio module 170 can convert sound into an electrical signal, or vice versa. The audio module 170 can obtain sound via the input module 150, or output sound via the sound output module 155 or an external electronic device (e.g., electronic device 102, such as a speaker or earphone) directly or wirelessly connected to the electronic device 101.

[0031] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a user's state) outside the electronic device 101, and then generate an electrical signal or data value corresponding to the detected state. The sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0032] The interface 177 may support one or more specific protocols to be used to connect the electronic device 101 directly (e.g., wired) or wirelessly with an external electronic device (e.g., the electronic device 102). The interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0033] The connection end 178 may include a connector, wherein the electronic device 101 can be physically connected to an external electronic device (e.g., the electronic device 102) via the connector. The connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0034] The haptic module 179 may convert the electric signal into mechanical stimulation (eg, vibration or motion) or electric stimulation that can be recognized by the user via his tactile sense or kinesthetic sense. The haptic module 179 may include, for example, a motor, a piezoelectric element, or an electric stimulator.

[0035] The camera module 180 may capture still images or moving images. The camera module 180 may include one or more lenses, an image sensor, an ISP, and a flash.

[0036] The power management module 188 may manage power supply to the electronic device 101. The power management module 188 may be implemented as, for example, at least a portion of a power management integrated circuit (PMIC).

[0037] The battery 189 may power at least one component of the electronic device 101. The battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0038] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more CPs that can operate independently from the processor 120 (e.g., AP) and support direct (e.g., wired) communication or wireless communication. The communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device (e.g., electronic device 104) via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, a fifth generation (5G) network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules may be implemented as a single component (e.g., a single chip), or may be implemented as multiple components (e.g., multiple chips) separated from each other. The wireless communication module 192 may identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the SIM 196.

[0039] The wireless communication module 192 can support 5G networks after the fourth generation (4G) network and next generation communication technologies (e.g., new radio (NR) access technology). NR access technology can support enhanced mobile broadband (eMBB), massive machine type communication (mMTC), or ultra-reliable low latency communication (URLLC). The wireless communication module 192 can support high frequency bands (e.g., millimeter wave bands) to achieve, for example, high data transmission rates. The wireless communication module 192 can support various technologies for ensuring performance on high frequency bands, such as, for example, beamforming, massive multiple input multiple output (MIMO), full dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. The wireless communication module 192 can support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., a second network 199). The wireless communication module 192 may support peak data rates for implementing eMBB (e.g., 20 Gbps or greater), loss coverage for implementing mMTC (e.g., 164 dB or less), or U-plane latency for implementing URLLC (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip).

[0040] The antenna module 197 may transmit a signal or power to the outside of the electronic device 101 (e.g., an external electronic device) or receive a signal or power from the outside of the electronic device 101 (e.g., an external electronic device). The antenna module 197 may include an antenna including a radiating element including a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). The antenna module 197 may include a plurality of antennas (e.g., an antenna array). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190. A signal or power may then be transmitted or received between the communication module 190 and the external electronic device via at least one selected antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiating element may be additionally formed as part of the antenna module 197.

[0041] The antenna module 197 may form a millimeter wave antenna module. The millimeter wave antenna module may include a PCB, an RFIC, and a plurality of antennas (e.g., an antenna array), wherein the RFIC is on a first surface (e.g., a bottom surface) of the PCB, or is adjacent to the first surface of the PCB and is capable of supporting a specified high frequency band (e.g., a millimeter wave band), and the plurality of antennas are disposed on a second surface (e.g., a top surface or a side surface) of the PCB, or is adjacent to the second surface of the PCB and is capable of transmitting or receiving signals of the specified high frequency band.

[0042] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and exchange signals (e.g., commands or data) therebetween.

[0043] According to an embodiment, a command or data may be sent or received between the electronic device 101 and an external electronic device (e.g., the electronic device 104) via a server 108 connected to the second network 199. Each of the external electronic devices (e.g., the electronic device 102 or the electronic device 104) may be a device of the same type as the electronic device 101, or a device of a different type from the electronic device 101. All or some operations to be executed by the electronic device 101 may be executed by one or more external electronic devices (e.g., the electronic devices 102 and 104 and the server 108). For example, if the electronic device 101 needs to automatically execute a function or service or should execute a function or service in response to a request from a user or another device, the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service instead of executing the function or service, or the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service in addition to executing the function or service. The one or more external electronic devices receiving the request may execute at least part of the requested function or service, or execute another function or another service related to the request, and may transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a partial reply to the request in the case of further processing the result or in the case of not further processing the result. To this end, cloud computing technology, distributed computing technology, mobile edge computing (MEC) technology or client-server computing technology, for example, may be used. The electronic device 101 may use, for example, distributed computing or MEC to provide ultra-low latency services. According to an embodiment, an external electronic device (e.g., electronic device 104) may include an Internet of Things (IoT) device. The server 108 may be an intelligent server using machine learning and / or neural networks. The external electronic device (e.g., electronic device 104) or server 108 may be included in the second network 199. The electronic device 101 may be applied to intelligent services (e.g., smart homes, smart cities, smart cars or health care) based on 5G communication technology or IoT-related technologies.

[0044] Figure 2 is a diagram showing an example structure of a wearable electronic device 200 according to an embodiment.

[0045] Reference Figure 2, the wearable electronic device 200 (eg, Figure 1 The electronic device 101) can be worn on the user's face to provide the user with images associated with augmented reality (AR) services and / or virtual reality (VR) services.

[0046] In an embodiment, the wearable electronic device 200 may include a first display 205, a second display 210, a screen display portion 215, an input optical member 220, a first transparent member 225a, a second transparent member 225b, lighting units 230a and 230b, a first printed circuit board (PCB) 235a, a second PCB 235b, a first hinge 240a, a second hinge 240b, a first camera 245a and 245b, multiple microphones (e.g., a first microphone 250a, a second microphone 250b, and a third microphone 250c), multiple speakers (e.g., a first speaker 255a and a second speaker 255b), a battery 260, a second camera 275a and 275b, a third camera 265, and sunshades 270a and 270b.

[0047] In an embodiment, the display (e.g., the first display 205 and the second display 210) may include, for example, a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), a micro light emitting diode (micro-LED), etc. Although not shown, when the display is one of LCD, DMD and LCoS, the wearable electronic device 200 may include a light source configured to emit light to the screen output area of ​​the display. In other embodiments, when the display is capable of self-luminescence, for example, when the display is OLED or micro-LED, the wearable electronic device 200 can provide a relatively high-quality virtual image to the user even if a separate light source is not included. For example, when the display is implemented as an OLED or micro-LED, a light source that illuminates the wearable electronic device 200 may not be required. Hereinafter, a display capable of self-luminescence may be referred to as a "self-luminous display", and will be described assuming a self-luminous display.

[0048] The display (e.g., the first display 205 and the second display 210) according to various embodiments may include at least one micro-light-emitting LED. For example, the micro-light-emitting LED can express red (R), green (G), and blue (B) by emitting light by itself, and since the size of the micro-light-emitting LED is relatively small (e.g., 100 μm or less), a single chip can implement a single pixel (e.g., one of the R, G, and B pixels). Therefore, when the display is composed of micro-light-emitting LEDs, the display can provide high resolution without a backlight unit (BLU).

[0049] However, the embodiment is not limited thereto. A single pixel may include R, G, and B, and a single chip may be implemented by a plurality of pixels including R, G, and B pixels.

[0050] In an embodiment, a display (e.g., a first display 205 and a second display 210) may include a display area including pixels for displaying a virtual image and light receiving pixels (e.g., light sensor pixels) disposed among the pixels, which receive light reflected from the eye, convert the reflected light into electrical energy, and output the electrical energy.

[0051] In an embodiment, the wearable electronic device 200 (eg, Figure 2 The processor 120 of the wearable electronic device 200 can detect the gaze direction of the user (e.g., pupil movement) through the light receiving pixels. For example, the wearable electronic device 200 can detect and track the gaze direction of the user's right eye and the gaze direction of the user's left eye through one or more light receiving pixels of the first display 205 and one or more light receiving pixels of the second display 210. The wearable electronic device 200 can determine the center position of the virtual image based on the gaze direction detected through one or more light receiving pixels (e.g., the direction in which the pupils of the right eye and the left eye of the user are looking).

[0052] In an embodiment, light emitted from a display (e.g., the first display 205 and the second display 210) may reach the picture display portion 215 formed on the first transparent member 225a facing the right eye of the user and the picture display portion 215 formed on the second transparent member 225b facing the left eye of the user by passing through a lens (not shown) and a waveguide. For example, light emitted from a display (e.g., the first display 205 and the second display 210) may be reflected from the grating area formed on the input optical member 220 and the picture display portion 215 by passing through a waveguide to be transmitted to the user's eyes. The first transparent member 225a and / or the second transparent member 225b may be formed, for example, as a glass plate, a plastic plate, or a polymer, and may be formed transparently or translucently.

[0053] In an embodiment, a lens (not shown) may be disposed on a front surface of a display (e.g., the first display 205 and the second display 210). The lens (not shown) may include a concave lens and / or a convex lens. For example, the lens (not shown) may include a projection lens or a collimating lens.

[0054] In an embodiment, the screen display portion 215 or the transparent members (eg, the first transparent member 225a and the second transparent member 225b) may include a lens including a waveguide and a reflective lens.

[0055] In an embodiment, the waveguide may be made of glass, plastic, or polymer, and may have a nano pattern formed on one surface of the inner or outer side thereof, such as a grating structure of a polygonal or curved shape. According to an embodiment, light incident to one end of the waveguide may be propagated within the display waveguide through the nano pattern to be provided to the user. In an embodiment, a waveguide formed as a free-form prism may provide incident light to the user via a reflector. The waveguide may include at least one diffraction element (e.g., a diffraction optical element (DOE) or a hologram optical element (HOE)) or at least one reflective element (e.g., a reflector). In an embodiment, the waveguide may use at least one diffraction element or reflective element included in the waveguide to guide light emitted from a display (e.g., the first display 205 and the second display 210) to the user's eyes.

[0056] According to various embodiments, the diffraction element may include an input optical member 220 and / or an output optical member (not shown). For example, the input optical member 220 may refer to an input grating region, and the output optical member (not shown) may refer to an output grating region. The input grating region may be used as an input end for diffracting (or reflecting) light output from a display (e.g., the first display 205 and the second display 210) (e.g., a micro-luminescent LED) to transmit the light to a transparent member (e.g., a first transparent member 225a and a second transparent member 225b) of the screen display portion 215. The output grating region may be used as an exit for diffracting (or reflecting) light transmitted to the transparent member (e.g., the first transparent member 225a and the second transparent member 225b) of the waveguide toward the user's eyes.

[0057] According to various embodiments, the reflective element may include a total reflection optical element or a total reflection waveguide for total internal reflection (TIR). For example, TIR as one of the schemes for inducing light may form an incident angle such that light (e.g., a virtual image) input through the input grating region is completely (e.g., 100%) reflected from one surface (e.g., a specific surface) of the waveguide to completely transmit the light to the output grating region.

[0058] In an embodiment, the optical path of light emitted from a display (e.g., the first display 205 and the second display 210) can be guided by a waveguide through the input optical member 220. The light propagating in the waveguide can be directed to the user's eyes through the output optical member. The screen display portion 215 can be determined based on the light emitted toward the eyes.

[0059] In an embodiment, the first cameras 245a and 245b may include cameras for three degrees of freedom (3DoF) and six degrees of freedom (6DoF) head tracking, hand detection and tracking, and gesture and / or spatial recognition. For example, the first cameras 245a and 245b may include global shutter (GS) cameras to detect movement of the head or hand and track the movement.

[0060] For example, the first cameras 245a and 245b may use stereo cameras for head tracking and spatial recognition, and cameras with the same specifications and performance may be applied to the first cameras 245a and 245b. The first cameras 245a and 245b may use GS cameras with excellent performance (e.g., image dragging) to detect fine movements (e.g., fast hand and finger movements) and track these movements.

[0061] For example, the first cameras 245a and 245b may use a rolling shutter (RS) camera. The first cameras 245a and 245b may perform a simultaneous localization and mapping (SLAM) function for 6DoF spatial recognition and depth imaging. The first cameras 245a and 245b may also perform a user gesture recognition function.

[0062] In an embodiment, the second cameras 275a and 275b may be used to detect and track pupils. The second cameras 275a and 275b may also be referred to as eye tracking (ET) cameras. The second cameras 275a and 275b may track the gaze direction of the user. Based on the gaze direction of the user, the wearable electronic device 200 may allow the center of the virtual image projected on the screen display portion 215 to be set according to the gaze position of the user.

[0063] The second cameras 275a and 275b can use GS cameras to detect pupils and track rapid pupil movement. The second cameras 275a and 275b can be installed for the left eye or the right eye, respectively, and cameras with the same specifications and performance can be used for the second cameras 275a and 275b for the left eye and the right eye.

[0064] In an embodiment, the second cameras 275a and 275b may include a gaze tracking sensor. For example, the wearable electronic device 200 may further include a lighting unit, and the gaze tracking sensor may detect reflected light of infrared light projected from the lighting unit onto the user's eyes. For example, the gaze tracking sensor may use the reflected light to track the user's gaze direction.

[0065] In an embodiment, the third camera 265 may also be referred to as a "high resolution (HR)" or "video recording (PV)" camera, and may include an HR camera. The third camera 265 may include a color camera having functions for obtaining high-quality images, such as an auto focus (AF) function and an optical image stabilizer (OIS). However, the example is not limited thereto, and the third camera 265 may include a GS camera or an RS camera.

[0066] In an embodiment, at least one sensor (e.g., a gyroscope sensor, an acceleration sensor, a geomagnetic sensor, a touch sensor, an illumination sensor, and / or a gesture sensor) and the first cameras 245a and 245b may perform at least one of functions such as 6DoF head tracking, pose estimation and prediction, gesture and / or spatial recognition, and SLAM through depth imaging.

[0067] In another embodiment, the first cameras 245a and 245b may use separate cameras, ie, a camera for head tracking and a camera for hand tracking.

[0068] In an embodiment, the lighting units 230a and 230b may be used differently depending on the locations to which the lighting units 230a and 230b are attached. For example, the lighting units 230a and 230b may be attached together with the first cameras 245a and 245b, which are disposed around hinges (e.g., first hinges 240a and second hinges 240b) connecting the frame and the temples or around a bridge connecting the frame. For example, when the GS camera is used to capture an image, the lighting units 230a and 230b may be used to supplement the brightness of the surrounding environment. For example, the lighting units 230a and 230b may be used in a dark environment, or the lighting units 230a and 230b may be used when the object to be captured is not easily detected due to a mixture of reflected light and various light sources.

[0069] In an embodiment, the lighting units 230a and 230b attached to the periphery of the frame of the wearable electronic device 200 may be auxiliary devices for assisting the detection of the eye gaze direction when the pupil is captured using the second cameras 275a and 275b. When the lighting units 230a and 230b are used as auxiliary devices for detecting the gaze direction, they may include IR LEDs of IR wavelengths.

[0070] In an embodiment, the components included in the wearable electronic device 200 (eg, Figure 1 The processor 120 and the memory 130 of the wearable electronic device 200 may be disposed on a PCB (eg, a first PCB 235a and a second PCB 235b). The PCB may transmit electrical signals to components included in the wearable electronic device 200.

[0071] In an embodiment, a plurality of microphones (eg, first microphone 250a, second microphone 250b, and third microphone 250c) may process external acoustic signals into electrical audio data. Depending on the function being performed (or the application being run) by the wearable electronic device 200, the electrical audio data may be used in various ways.

[0072] In an embodiment, a plurality of speakers (eg, a first speaker 255a and a second speaker 255b) may output audio from a communication circuit (eg, Figure 2 The audio data received by the communication circuit 210) or the memory (for example, Figure 1 The audio data stored in the memory 130).

[0073] In an embodiment, the battery 260 may be provided as one or more batteries and may supply power to components included in the wearable electronic device 200 .

[0074] In an embodiment, the visors 270a and 270b can adjust the amount of transmission of external light incident on the user's eyes according to the transmittance. The visors 270a and 270b can be set before or after the screen display part 215. The front side of the screen display part 215 can indicate the direction opposite to the user wearing the wearable electronic device 200, and the back side of the screen display part 215 can indicate the direction of the user side of the user wearing the electronic device 200. The visors 270a and 270b can protect the screen display part 215 and adjust the amount of transmission of external light.

[0075] For example, the light shielding plates 270a and 270b may include a control module and an electrochromic element. The control module may control the electrochromic element to adjust the transmittance of the electrochromic element.

[0076] Figure 3 is a diagram showing a method of performing a task by an electronic device 300 (eg, Figure 1 The electronic device 101 and Figure 2 The electronic device 200) executes a method for controlling a display module (eg, Figure 1 The display module 160, Figure 2 FIG. 1 is a diagram of an example operation of displaying a virtual object (eg, 310 - 1 , 310 - 2 , and 310 - 3 ) at a determined location in at least one of a real world or a virtual world using a first display 205 and a second display 210 .

[0077] Reference Figure 3 , the electronic device 300 may display at least one virtual object (eg, 310-1, 310-2, and 310-3) on the display modules (eg, 205 and 210). Figure 3As shown, the electronic device 300 can control the display module (e.g., 205 and 210) to display the virtual objects (e.g., 310-1, 310-2, and 310-3) so that the user can recognize that the virtual objects (e.g., 310-3, 310-1, and 310-2) are displayed as if in the real world or the virtual world. The electronic device 300 can control the display module (e.g., 205 and 210) to display the virtual objects (e.g., 310-1, 310-2, and 310-3) in AR mode or VR mode.

[0078] For example, the electronic device 300 set to AR mode can control the display module (e.g., 205 and 210) to display the virtual object (e.g., 310-1, 310-2, and 310-3) at a determined position in the real world. For example, the electronic device 300 set to VR mode can control the display module (e.g., 205 and 210) to display the virtual object (e.g., 310-1, 310-2, and 310-3) at a determined position in the virtual world.

[0079] For example, the electronic device 300 may change from an AR mode to a VR mode, or vice versa from a VR mode to an AR mode. In this example, in response to such a mode change, the electronic device 300 may control the display modules (e.g., 205 and 210) so as to change the position at which the virtual objects (e.g., 310-1, 310-2, and 310-3) are to be displayed. For example, in response to a change from a VR mode to an AR mode, the electronic device 300 may control the display modules (e.g., 205 and 210) to display the virtual objects (e.g., 310-1, 310-2, and 310-3) at a determined position in the real world corresponding to a determined position in the virtual world. For example, in contrast to the aforementioned example, in response to a change from an AR mode to a VR mode, the electronic device 300 may operate substantially the same as described above.

[0080] According to an embodiment, the electronic device 300 anchors the position of at least one virtual object (e.g., 310-1, 310-2, and 310-3). For example, the electronic device 300 may perform spatial anchoring to anchor the position of the virtual object (e.g., 310-1, 310-2, and 310-3) so that the virtual object (e.g., 310-3, 310-3, and 310-1) is displayed at a specific position.

[0081] For example, when the anchor position of the virtual objects (e.g., 310-1, 310-2, and 310-3) is within the user's field of view (FOV), the electronic device 300 can control the display modules (e.g., 205 and 210) to display the virtual objects (e.g., 310-1, 310-2, and 310-3).

[0082] For example, when the anchor position of the virtual object (e.g., 310-1, 310-2, and 310-3) is not within the user's FOV, the electronic device 300 can control the display module (e.g., 205 and 210) not to display the virtual object (e.g., 310-3, 310-1, and 310-2).

[0083] For example, the electronic device 300 may anchor the positions of the virtual objects (e.g., 310-1, 310-2, and 310-3) based on the position of the user when anchoring the positions of the virtual objects (e.g., 310-1, 310-2, and 310-3). For example, the electronic device 300 may anchor the positions of the virtual objects (e.g., 310-1, 310-2, and 310-3) based on the position of the user in the real world. For example, the electronic device 300 may anchor the positions of the virtual objects (e.g., 310-1, 310-2, and 310-3) based on the positions in the virtual world corresponding to the position of the user.

[0084] For example, in AR mode, the electronic device 300 may anchor the position of the virtual objects (e.g., 310-1, 310-2, and 310-3) based on the position of the user in the real world. In this example, when the user views the anchor position of the virtual objects (e.g., 310-1, 310-2, and 310-3), the electronic device 300 may control the display module (e.g., 205 and 210) to display the virtual objects (e.g., 310-1, 310-2, and 310-3). In this example, when the user does not view the anchor position of the virtual objects (e.g., 310-1, 310-2, and 310-3), the electronic device 300 may control the display module (e.g., 205 and 210) not to display the virtual objects (e.g., 310-3, 310-3, and 310-1).

[0085] Based on the user's gaze direction and the user's field of view (e.g., FOV), the electronic device 300 may control the display modules (e.g., 205 and 210) to partially or completely display the virtual objects (e.g., 310-1, 310-2, and 310-3) whose positions are anchored. For example, the electronic device 300 may use a camera module (e.g., Figure 1 The camera module 180 and Figure 2 The first cameras 245a and 245b) are used to perform head tracking and identify the user's gaze direction.

[0086] For example, in VR mode, the electronic device 300 may anchor the position of the virtual object (e.g., 310-1, 310-2, and 310-3) based on the position in the virtual world corresponding to the position of the user. In this example, when the user views the anchor position of the virtual object (e.g., 310-1, 310-2, and 310-3) in the virtual world, the electronic device 300 may control the display module (e.g., 205 and 210) to display the virtual object (e.g., 310-1, 310-2, and 310-3). In this example, when the user does not view the anchor position of the virtual object (e.g., 310-1, 310-2, and 310-3) in the virtual world, the electronic device 300 may control the display module (e.g., 205 and 210) not to display the virtual object (e.g., 310-1, 310-2, and 310-3). The electronic device 300 may control the display modules (eg, 205 and 210 ) to partially display virtual objects (eg, 310 - 1 , 310 - 2 , and 310 - 3 ) whose positions are anchored based on the user's gaze direction and the user's field of view in the virtual world.

[0087] In addition to the above-mentioned position anchoring method, the electronic device 300 may also use various known position anchoring methods to anchor the positions of the virtual objects (eg, 310 - 1 , 310 - 2 , and 310 - 3 ).

[0088] In the previous example, the anchor position of the virtual object (e.g., 310-1, 310-2, and 310-3) in the real world or the virtual world can represent the determined position of the virtual object (e.g., 310-1, 310-2, and 310-3) in the real world or the virtual world. For example, the electronic device 300 can anchor the position of the virtual object (e.g., 310-1, 310-2, and 310-3) in the real world or the virtual world, and control the display module (e.g., 205 and 210) to display the virtual object (e.g., 310-3, 310-3, and 310-1) at the anchor position.

[0089] According to an embodiment, the processor 120 of the electronic device 300 may determine or anchor the position of the virtual objects (e.g., 310-1, 310-2, and 310-3) to be displayed by the display modules (e.g., 205 and 210) in the real world or the virtual world. According to an embodiment, an application (e.g., application 146) running in association with the virtual objects (e.g., 310-1, 310-2, and 310-3) may determine or anchor the position of the virtual objects (e.g., 310-1, 310-2, and 310-3) to be displayed by the display modules (e.g., 205 and 210) in the real world or the virtual world.

[0090] For example, in the case where the first surface is included in the user's FOV, the electronic device 300 may control the display modules (e.g., 205 and 210) to display first information on the first surface at the anchor position of the virtual object (e.g., 310-1, 310-2, and 310-3). For example, in the case where the second surface is included in the user's FOV, the electronic device 300 may control the display modules (e.g., 205 and 210) to display second information on the second surface at the anchor position of the virtual object (e.g., 310-1, 310-2, and 310-3).

[0091] For example, the electronic device 300 may execute a program stored in a memory (eg, Figure 1 130) in an application 146 (e.g., Figure 1 For example, the electronic device 300 may control the display modules (eg, 205 and 210) to display virtual objects (eg, 310-1, 310-2, and 310-3) according to the running application 146.

[0092] The virtual objects (eg, 310-1, 310-2, and 310-3) may include multiple faces. For example, the virtual objects (eg, 310-1, 310-2, and 310-3) may be displayed on the display modules (eg, 205 and 210) in a shape having multiple faces according to their form.

[0093] For example, multiple faces of a virtual object (e.g., 310-1, 310-2, and 310-3) may include at least one first surface for providing information to a user. For example, when the running application 146 is a messaging application 146, the interface of the messaging application 146 may be provided through the first surface.

[0094] The first surface of the virtual object (eg, 310-1, 310-2, and 310-3) may refer to a surface of the virtual object (310-1, 310-2, and 310-3) for providing an interface according to the executed application 146. The first information may refer to information provided through the interface of the executed application 146.

[0095] For example, multiple faces of the virtual object (e.g., 310-1, 310-2, and 310-3) may include at least one second surface for providing a set video, image, icon, etc. For example, in a case where the running application 146 is a messaging application 146, at least one of the set video, image, or icon of the messaging application 146 may be provided through the second surface.

[0096] The second surface of the virtual object (eg, 310-1, 310-2, and 310-3) may refer to a surface of the virtual object (eg, 310-1, 310-2, and 310-3) represented as a set video, image, or icon. The first information may refer to information provided via an interface of the executed application 146.

[0097] The electronic device 300 may run various applications 146 other than the messaging application 146 of the aforementioned example, and control the display modules (e.g., 205 and 210) to display virtual objects (e.g., 310-1, 310-2, and 310-3) according to the running applications 146. The electronic device 300 may control the display modules (e.g., 205 and 210) to display virtual objects (e.g., 310-1, 310-2, and 310-3) according to the operating system (OS) (e.g., Figure 1 OS 142), middleware (e.g., Figure 1 The middleware 144) or the like displays virtual objects (e.g., 310-1, 310-2, and 310-3).

[0098] like Figure 3 As shown, at the user's location, the user can recognize the first surface of the virtual objects (e.g., 310-1, 310-2, and 310-3). The second surface of the virtual objects (e.g., 310-1, 310-2, and 310-3) may not be included in the user's FOV based on the user's location when the virtual objects (e.g., 310-1, 310-2, and 310-3) are anchored.

[0099] Figure 3 The electronic device 300 controls the display module (eg, 205 and 210) to display multiple virtual objects 310-1, 310-2, and 310-3 in the form of a screen or window. However, the form or shape of the virtual objects 310-1, 310-2, and 310-3 is not limited to Figure 3 , and the virtual objects 310 - 1 , 310 - 2 , and 310 - 3 may be provided in various shapes, such as an octahedron, a sphere, and the like.

[0100] Figure 4 is a diagram showing a virtual object (eg, Figure 3 FIG. 3 is a diagram of an example first surface 311 of virtual objects 310 - 1 , 310 - 2 , and 310 - 3 ).

[0101] In an embodiment, an electronic device (e.g., Figure 3 The electronic device 300) can control the display module (for example, Figure 1 The display module 160, Figure 2The first display 205 and the second display 210 ) provide first information through a first surface 311 of a virtual object (eg, 310 - 1 , 310 - 2 , and 310 - 3 ). Figure 4 A first surface 311 of a virtual object (eg, 310-1, 310-2, and 310-3) is shown, which provides a message transmission application (eg, Figure 1 The interface of the application 146). Figure 4 As shown, the electronic device 300 may provide first information including a message sender, a message, a message reception time, etc. through a first surface 311 of a virtual object (eg, 310 - 1 , 310 - 2 , and 310 - 3 ).

[0102] Figure 4 An example of providing an interface through the first surface 311 of the virtual object (e.g., 310-1, 310-2, and 310-3) is shown, and the interface is used to provide received messages of the running messaging application, but the first information provided by the electronic device 300 through the first surface 311 of the virtual object (e.g., 310-1, 310-2, and 310-3) is not limited to Figure 4 For example, the electronic device 300 may provide various interfaces provided by a messaging application, such as an interface for sending messages, an interface for managing received messages, etc., through the first surface 311 of the virtual objects (e.g., 310-1, 310-2, and 310-3). For example, the electronic device 300 may run various applications 146, such as an email application, a photo album application, etc., and provide the interface of the running application 146 through the first surface 311 of the virtual objects (e.g., 310-1, 310-2, and 310-3).

[0103] For example, the content of the first information provided by the first surface 311 of the virtual object (e.g., 310-1, 310-2, and 310-3) can change. For example, the first information can be changed based on the operation of the application 146 associated with the virtual object (e.g., 310-1, 310-2, and 310-3), for example, when a notification occurs on the running application 146, the information provided by the application 146 on the first surface 311 is updated, the interface provided by the application 146 on the first surface 311 is changed, etc.

[0104] For example, in the case where the running application 146 is a messaging application, in response to receiving a new message, the content of the first information may be changed so that the interface of the messaging application displays the new message, such as Figure 4 For example, when the content of the first information changes, the electronic device 300 may use a display module (eg, 205 and 210), an audio module (eg, Figure 1An audio module 170) and a haptic module (eg, Figure 1 The electronic device 300 may provide a notification by controlling at least one of the display modules (eg, 205 and 210) or a combination thereof. For example, the electronic device 300 may control the display modules (eg, 205 and 210) to provide a notification via a virtual object (eg, 310-1, 310-2, and 310-3).

[0105] Although an example in which the first information is changed in response to the operation of the application 146 has been described above, the first information may also be changed according to a program associated with the virtual object (eg, 310 - 1 , 310 - 2 , and 310 - 3 ), an OS, or the like.

[0106] Figure 5 is a diagram showing a virtual object (eg, Figure 3 FIG. 3 is a diagram of an example second surface 312 of virtual objects 310 - 1 , 310 - 2 , and 310 - 3 ).

[0107] In an embodiment, an electronic device (e.g., Figure 3 The electronic device 300) can control the display module (for example, Figure 1 The display module 160, Figure 2 The first display 205 and the second display 210 of the virtual object (eg, 310 - 1 , 310 - 2 , and 310 - 3 ) provide second information via a second surface 312 of the virtual object (eg, 310 - 1 , 310 - 2 , and 310 - 3 ). Figure 5 A second surface 312 of the virtual objects (eg, 310 - 1 , 310 - 2 , and 310 - 3 ) is shown on which the electronic device 300 provides a setup image.

[0108] The second information provided by the electronic device 300 via the second surface 312 of the virtual object (eg, 310-1, 310-2, and 310-3) is not limited to Figure 5 For example, the second information may include a semi-transparent or blurred image, a fixed color image, an outline of the second surface 312 of the virtual object (e.g., 310-1, 310-2, and 310-3), a user-specified color image, a primary color specified for an application (e.g., application 146), a user-specified image / video, an image of an icon of the running application 146, etc.

[0109] Figure 6 is a diagram showing a method of performing a process ... Figure 3 The electronic device 300) performs a process for using a virtual object (eg, Figure 3 The second surface (eg, Figure 5 The second surface 312) provides a diagram of example operations of the first information.

[0110] Figure 6 The example operation performed by the electronic device 300 in the following case is shown: when the user wearing the electronic device 300 is located at the first position 301, the display module (for example, 205 and 210) is controlled to display the virtual object (for example, Figure 3 After the virtual objects 310 - 1 , 310 - 2 , and 310 - 3 ) are displayed at the determined positions in the real world or the virtual world, the user wearing the electronic device 300 has moved to the second position 302 .

[0111] For example, the electronic device 300 may recognize the position of the user in the real world or the virtual world and / or the position of the electronic device 300. Since the electronic device 300 operates while being worn by the user, the position of the electronic device 300 may be substantially the same as the position of the user.

[0112] like Figure 6 As shown, the electronic device 300 can control the display module (e.g., 205 and 210) to display the virtual objects (e.g., 310-1, 310-2, and 310-3) at a determined position in the real world or the virtual world according to the user's first position 301. For example, the electronic device 300 can anchor the positions of the virtual objects (e.g., 310-1, 310-2, and 310-3) to positions within the user's FOV according to the first position 301.

[0113] For example, the electronic device 300 may control the display module (eg, Figure 1 The display module 160 and Figure 2 The electronic device 300 may control the display modules (eg, 205 and 210) so that the first surface 311 is set to face the first position 301. The electronic device 300 may control the display modules (eg, 205 and 210) so that the second surface 312 is set to face the direction opposite to the first position 301.

[0114] For example, the electronic device 300 may set the virtual objects (e.g., 310-1, 310-2, and 310-3) at the anchor position of the virtual objects (e.g., 310-1, 310-2, and 310-3) so that the first surface 311 faces the first position 301. For example, the electronic device 300 may set the first surfaces 311 of the virtual objects (e.g., 310-1, 310-2, and 310-3) so that the first surfaces 311 are located within the FOV of the user at the first position 301, for example, the first surfaces 311 of the virtual objects (e.g., 310-1, 310-2, and 310-3) face the direction of 12 o'clock, as shown in FIG. Figure 6 shown.

[0115] For example, the electronic device 300 may place the virtual objects (e.g., 310-1, 310-2, and 310-3) at the positions of the anchored virtual objects (e.g., 310-1, 310-2, and 310-3) so that the second surface 312 faces the direction opposite to the first position 301, for example, as Figure 6 As shown, the second surface 312 is set to face the 6 o'clock direction. For example, the electronic device 300 can set the second surface 312 of the virtual object (e.g., 310-1, 310-2, and 310-3) so that the second surface 312 is not included in the field of view of the user at the first position 301, for example, the second surface 312 is arranged to face the 6 o'clock direction, as shown in FIG. Figure 6 shown.

[0116] For example, at the first position 301 , the electronic device 300 may control the display modules (eg, 205 and 210 ) to display first information on a first surface 311 included in the user's FOV.

[0117] For example, at the first position 301 , the electronic device 300 may control the display modules (eg, 205 and 210 ) to display second information on a second surface 312 included in the user's FOV.

[0118] For example, the electronic device 300 may determine the user's gaze direction based on the user's position in the real world or the virtual world. For example, the electronic device 300 may use a camera module (e.g., Figure 1 The camera module 180 and Figure 2 The second camera 275a and 275b of the electronic device 300 can determine the user's gaze direction. For example, the electronic device 300 can use the camera module (e.g., 275a and 275b) to detect the pupil of the user's eye and track the user's gaze direction.

[0119] For example, in response to the user's gaze direction being a direction corresponding to the second surface 312 , the electronic device 300 may control the display modules (eg, 205 and 210 ) to display the first information on the second surface 312 .

[0120] For example, in the case where the user has moved from the first position 301 to the second position 302, the electronic device 300 may determine the gaze direction of the user at the second position 302. In this example, the gaze direction of the user at the second position 302 may be a direction corresponding to the second surface 312 (e.g., Figure 6 The electronic device 300 may control the display modules (eg, 205 and 210 ) to provide the first information on the second surface 312 .

[0121] For example, in the case where the gaze direction of the user is a direction corresponding to the second surface 312 and the content of the first information has changed, the electronic device 300 may control the display module (e.g., 205 and 210) to provide the first information on the second surface 312. For example, the first information provided on the second surface 312 may include the changed content. For example, in the case where the gaze direction of the user is a direction corresponding to the second surface 312 and the content of the first information has changed, the electronic device 300 may control the display module (e.g., 205 and 210) to provide the third information on the second surface 312. For example, the third information may include the content that has changed from the first information.

[0122] For example, if the user's gaze direction at the second position 302 is a direction corresponding to the second surface 312 and the content provided by the first information has not changed, the electronic device 300 can control the display module (e.g., 205 and 210) to provide second information on the second surface 312.

[0123] For example, in the case where the content provided by the first information has changed, the electronic device 300 can control the display module (e.g., 205 and 210) to display an icon or the like on the second surface 312, which indicates changes, updates, etc. made in the first information. For example, in the case where the content provided by the first information has changed, the electronic device 300 can provide a sound signal, a vibration signal, etc.

[0124] For example, the first information provided through the second surface 312 may include content that has been changed from content included in the first message provided through the first surface 311. For example, the first information provided through the second surface 312 may include an interface associated with the content included in the first message provided through the first surface 311.

[0125] For example, in a running application (e.g. Figure 1 Application 146) is Figure 6 In the case of a messaging application in the example of FIG. 3 and a user being located at a first position 301, the electronic device 300 may control the display modules (eg, 205 and 210) to display first information (eg, Figure 4 The electronic device 300 may also control the display modules (eg, 205 and 210) to display second information (eg, Figure 5 The second information shown on the second surface 312).

[0126] For example, the running application 146 is Figure 6In the case of a messaging application in the example of FIG. 3 and the user is located at the second position 302, the electronic device 300 can control the display modules (e.g., 205 and 210) to display the first information on the second surface 312. When the electronic device 300 displays the first information on the second surface 312, the electronic device 300 can provide substantially the same first information or second information on the first surface 311.

[0127] For example, the electronic device 300 may use various methods to indicate a change or update of the content of the first information, or a notification from the running application 146. In this example, the electronic device 300 may control the display modules (e.g., 205 and 210) to flash the second surface 312 of the virtual object (e.g., 310-1, 310-2, and 310-3). For example, the electronic device 300 may use a sound output module (e.g., Figure 1 The sound output module 155 and Figure 2 For example, when the second information including the icon of the running application 146 is provided on the second surface 312 of the virtual object (e.g., 310-1, 310-2, and 310-3), the electronic device 300 can control the display module (e.g., 205 and 210) to use the icon of the application 146 to indicate that a notification has occurred.

[0128] For example, the electronic device 300 may determine whether the second position 302 of the user is in the direction of the second surface 312 of the virtual object (e.g., 310-1, 310-2, and 310-3). When the second position 302 of the user is in the direction of the second surface 312 of the virtual object (e.g., 310-1, 310-2, and 310-3), the electronic device 300 may control the display module (e.g., 205 and 210) to display the first information on the second surface 312 of the virtual object (e.g., 310-1, 310-2, and 310-3).

[0129] Figure 7 is a diagram showing an electronic device (eg, Figure 3 FIG. 3 is a diagram of example operations performed by an electronic device 300 of the present invention for providing first information using a second surface 312 of a virtual object 310-5 having a changed position and size. Figure 7 An example situation where a user moves from a first position 301 to a second position 302 is shown.

[0130] exist Figure 7 In the example of , when the user is located at the first position 301, the electronic device 300 can control the display modules (eg, 205 and 210) to display the virtual object 310-4 at a determined position in the real world or the virtual world.

[0131] When the user is located at the first position 301 , the electronic device 300 may control the display module (eg, 205 and 210 ) to display first information on a first surface 311 of the virtual object 310 - 4 , and to display second information on a second surface 312 of the virtual object 310 - 4 .

[0132] Reference Figure 7 , the electronic device 300 may control the display modules (eg, 205 and 210 ) to display the virtual object 310 - 5 at a position determined based on the second position 302 of the user in the real world or the virtual world.

[0133] The electronic device 300 may control the display modules (eg, 205 and 210 ) to display the first information on the second surface 314 of the virtual object 310 - 5 located at the changed position.

[0134] For example, the electronic device 300 can determine whether to control the display module (e.g., 205 and 210) to display the virtual object 310-5 at the position determined based on the user's second position 302 based on the user's first position 301 and second position 302 and the direction corresponding to the second surface 312 of the virtual object 310-4.

[0135] For example, Figure 7 As shown, when the user views the second surface 312 from the second position 302, it may not be easy to accurately identify the first information or the second information provided on the second surface 312, so the electronic device 300 can control the display module (e.g., 205 and 210) to display the virtual object 310-5 at a position determined based on the user's second position 302.

[0136] The electronic device 300 may control the display module (eg, 205 and 210 ) to display the virtual object 310 - 5 at a position determined based on the second position 302 of the user, based on the direction of the second surface 312 and the FOV when the user views the second surface 312 .

[0137] For example, the electronic device 300 may control the display modules (eg, 205 and 210) to display the virtual object 310-5 between the first position 301 and the second position 302, such as Figure 7 For example, the electronic device 300 may change the positions of the virtual objects (eg, 310 - 1 , 310 - 2 , and 310 - 3 ) to match the direction in which the first information or the second information is provided on the second surface 314 with the direction from the first position 301 toward the second position 302 .

[0138] For example, the electronic device 300 may change the size of the virtual object 310-5. For example, the electronic device 300 may change the size of at least one second surface 312 of the virtual object 310-5 based on the position of the user. For example, in response to the distance between the first position 301 and the second position 302 exceeding a preset range, the electronic device 300 may increase the size of the second surface 314 of the virtual object 310-5, such as Figure 7 shown.

[0139] For example, in the running application (e.g. Figure 1 In the case where the application 146 of the electronic device 300 is a messaging application and a new message is received, the electronic device 300 may provide a notification about the reception of the message to the second surface 314 of the virtual object 310-5. The electronic device 300 may receive input from the user via the second surface 314 of the virtual object 310-5, and may provide an interface provided by the messaging application (e.g., an interface for replying to a message, an interface for managing messages, etc.) through the second surface 314 of the virtual object 310-5.

[0140] For example, the electronic device 300 may control the display module (e.g., 205 and 210) to display the virtual object 310-5 at the determined position based on the user's input. For example, when the user checks the first information provided by the second surface 314 of the virtual object 310-5, the electronic device 300 may control the display module (e.g., 205 and 210) to display the virtual object 310-4 at the determined position.

[0141] For example, when the running application 146 is a messaging application and a new message is received, the electronic device 300 can provide the first information to the user at the second position 302 through the second surface 314 of the virtual object 310-5. The electronic device 300 can then receive input from the user using the second surface 314 of the virtual object 310-5. In this case, when an input indicating that the message has been checked is received from the user, the electronic device 300 can control the display module (e.g., 205 and 210) to display the virtual object 310-4 at a determined position. When a reply in response to the received message is sent based on the input from the user, the electronic device 300 can control the display module (e.g., 205 and 210) to display the virtual object 310-4 at a determined position.

[0142] Figure 7An example case is shown in which the direction corresponding to the second surface 314 of the virtual object 310-5 faces the second position 302. According to an embodiment, the electronic device 300 may control the display modules (e.g., 205 and 210) so that the direction corresponding to the first surface 313 of the virtual object 310-5 at the changed position faces the second position 302. The electronic device 300 may control the display modules (e.g., 205 and 210) to provide the first information on the first surface 313 of the virtual object 310-5.

[0143] Figure 8 is a diagram showing a control display module (eg, Figure 1 The display module 160 and Figure 2 205 and the second display 210).

[0144] In an embodiment, at operation 810, an electronic device (eg, Figure 3 The electronic device 300 may control the display modules (eg, 205 and 210) to display the first information on the virtual object (eg, Figure 3 The first surface of the virtual objects 310-1, 310-2, and 310-3) (eg, Figure 6 The second information is displayed on the first surface 311 of the virtual object (eg, Figure 3 The second surface (eg, Figure 6 on the second surface 312).

[0145] For example, the first information provided on the first surface 311 may include the running application (eg, Figure 1 For example, the content included in the first information may be changed or updated.

[0146] For example, the second information provided on the second surface 312 may include a preset video, an image, an icon, and the like.

[0147] In an embodiment, at operation 820, the electronic device 300 may determine the gaze direction of the user based on the position of the user in the real world or the virtual world. For example, the electronic device 300 may use a camera module (e.g., Figure 1 The camera module 180 and Figure 2 The second camera 275a and 275b) tracks and / or identifies the user's gaze direction.

[0148] In an embodiment, at operation 830 , the electronic device 300 may determine whether the content provided by the first information has changed, and at operation 840 , the electronic device 200 may determine whether the gaze direction of the user is a direction corresponding to the second surface 312 .

[0149] In an embodiment, in operation 850, when the content provided by the first information in operation 830 has changed and the user's gaze direction in operation 840 is a direction corresponding to the second surface 312, the electronic device 300 can control the display modules (e.g., 205 and 210) to provide the first information on the second surface 312 of the virtual object (e.g., 310-1, 310-2, and 310-3).

[0150] In an embodiment, in operation 840, when the content provided by the first information in operation 830 has not changed and / or the user's gaze direction in operation 840 is not a direction corresponding to the second surface 312, the electronic device 300 may control the display module (e.g., 205 and 210) to display the second information on the second surface 312 of the virtual object (e.g., 310-1, 310-2, and 310-3).

[0151] Fig. 9 is a diagram showing a method of performing a process ... Figure 3 The electronic device 300) is used to use the second virtual object 310-7 (for example, Figure 3 A diagram of an example operation of providing first information with virtual objects 310-1, 310-2, and 310-3). Fig. 9 An example is shown in which the electronic device 300 can control the display module (e.g., 205 and 210) to display a virtual object 310-6 when the user wearing the electronic device 300 is located at the first position 301. In this example, the electronic device 300 can control the display module (e.g., 205 and 210) to display first information on a first surface 311 of the virtual object 310-6 and display second information on a second surface 312 of the virtual object 310-6.

[0152] In this case, when the user wearing the electronic device 300 moves to the second position 302, the electronic device 300 can control the display module (e.g., 205 and 210) to display the second virtual object 310-7. For example, when the content included in the first information has changed and the distance between the first position 301 and the second position 302 exceeds a preset range, the electronic device 300 can control the display module (e.g., 205 and 210) to display the second virtual object 310-7 that is substantially the same as the virtual object 310-6.

[0153] For example, the first surface 315 of the second virtual object 310-7 may be substantially the same as the first surface 311 of the virtual object 310-6, and the second surface 316 of the second virtual object 310-7 may be substantially the same as the second surface 312 of the virtual object 310-6. For example, the electronic device 300 may control the display module (e.g., 205 and 210) to display the first information on the second surface 316 of the second virtual object 310-7.

[0154] For example, when the distance between the first position 301 and the second position 302 exceeds a preset range, the electronic device 300 can control the display module (e.g., 205 and 210) to display the second virtual object 310-7 even when the second position 302 is located in the direction of the first surface 311 of the virtual object 310-6.

[0155] In an embodiment, the electronic device 300 may receive input from a user using the second virtual object 310-7. Based on the received input, the electronic device 300 may control the display module (e.g., 205 and 210) to display the third information using the second virtual object 310-7. For example, the third information may include information displayed by the running application (e.g., Figure 1 For example, the third information may refer to an interface provided by the application 146 included in the first information in response to a user input.

[0156] For example, when the running application 146 is a messaging application and a new message is received, the electronic device 300 can use the second virtual object 310-7 to provide a notification of the received message. When receiving an input for replying to a message from a user, the electronic device 300 can control the display module (e.g., 205 and 210) to display third information including an interface for writing a reply to the message. For example, the electronic device 300 can control the display module (e.g., 205 and 210) to display the third information on the first surface 315 or the second surface 316 of the second virtual object 310-7.

[0157] For example, the electronic device 300 may control the display module (e.g., 205 and 210) to provide the first information to the first surface 315 or the second surface 316 of the second virtual object 310-7 without displaying the virtual object 310-6. For example, when the first information is provided using the second virtual object 310-7, the electronic device 300 may control the display module (e.g., 205 and 210) so that the virtual object 310-6 is not displayed even when the user looks at the direction of the virtual object 310-6.

[0158] For example, the electronic device 300 may control the display module (e.g., 205 and 210) to provide the first information to the first surface 315 or the second surface 316 of the second virtual object 310-7 and display the virtual object 310-6. For example, even when the first information is provided using the second virtual object 310-7, the electronic device 300 may control the display module (e.g., 205 and 210) to display the virtual object 310-6 when the user looks at the direction of the virtual object 310-6.

[0159] The electronic device 300 may control the display modules (eg, 205 and 210 ) to terminate outputting the second virtual object 310 - 7 based on the received input.

[0160] Fig.10 is a diagram showing a control display module (eg, Figure 1 The display module 160 and Figure 2 A flowchart of an example method of a first display 205 and a second display 210 of the display module using a second virtual object (e.g., Fig. 9 The second virtual object 310-7) provides the first information.

[0161] In an embodiment, at operation 1010, an electronic device (eg, Figure 3 The electronic device 300 can transfer the user's location (e.g., Fig. 9 The second position 302) and the position of the virtual object (eg, Fig. 9 The distance between the virtual objects 310-6) is compared with a preset first range.

[0162] For example, at operation 1020, in response to the distance between the user's position 302 and the position of the virtual object 310-6 exceeding a preset first range, the electronic device 300 may control the display module (e.g., 205 and 210) to display a second virtual object 310-7 that is substantially the same as the virtual object 310-6.

[0163] exist Figure 8 or Fig.10 In the embodiment of the present invention, the operations can be performed in sequence, but not necessarily in sequence. For example, the order of the operations can be changed, and at least two operations can be performed in parallel.

[0164] According to the embodiment, the above reference Figure 8 or Fig.10 The described operations may be interpreted as being performed by the processor 120 (eg, Figure 1 Executed by processor 120).

[0165] According to various embodiments, an electronic device (eg, Figure 1 electronic device 101, Figure 2 The electronic device 200 and Figure 3 The electronic device 300 may include: a display module (eg, Figure 1 The display module 160 and Figure 2 a first display 205 and a second display 210); a processor (eg, Figure 1 processor 120); and memory (e.g., Figure 1 The memory 130 is electrically coupled to the processor 120 and stores instructions executable by the processor 120. When the instructions are executed, the processor 120 may be configured to control the display modules (e.g., 205 and 210) to display the first information on the virtual object (e.g., Figure 3 At least one first surface (eg, Figure 6 310 - 1 , 310 - 2 , and 310 - 3 ), and displays the second information on at least one second surface (eg, Figure 6 The processor 120 may be configured to determine the gaze direction of the user based on the user's position in the real world or the virtual world. In response to the gaze direction of the user being a direction corresponding to at least one second surface 312, the processor 120 may be configured to control the display module (e.g., 205 and 210) to display the first information on at least one second surface 312.

[0166] In response to the distance between the user's position and the determined position exceeding a preset first range, the processor 120 can be configured to control the display module (e.g., 205 and 210) to display the first information using a second virtual object substantially identical to the virtual object (e.g., 310-1, 310-2, and 310-3).

[0167] The processor 120 may be configured to receive input from the user using the second virtual object. In response to the input, the processor 120 may be configured to control the display module (eg, 205 and 210) to display the third information using the second virtual object or to terminate outputting the second virtual object.

[0168] The processor 120 may be configured to control the display modules (eg, 205 and 210 ) to display virtual objects (eg, 310 - 1 , 310 - 2 , and 310 - 3 ) at locations determined based on a user's location in at least one of the real world or the virtual world.

[0169] The processor 120 can be configured to control the display modules (e.g., 205 and 210) so that the direction in which at least one second surface 312 is displayed is changed based on the user's position, or the size in which at least one second surface 312 is displayed is changed based on the distance between the user's position and the determined position.

[0170] When the content included in the first information changes, the processor 120 may be configured to control the display modules (eg, 205 and 210 ) to display the changed first information on the at least one second surface 312 .

[0171] According to various embodiments, an electronic device (eg, Figure 1 electronic device 101, Figure 2 The electronic device 200 and Figure 3 The electronic device 300 may include: a display module (eg, Figure 1 The display module 160, Figure 2 a first display 205 and a second display 210); a processor (eg, Figure 1 processor 120); and memory (e.g., Figure 1 130), which is electrically coupled to the processor 120 and stores instructions executable by the processor 120. When the instructions are executed, the processor 120 may be configured to identify a first location of the user in at least one of the real world or the virtual world (e.g., Figure 6 The processor 120 may be configured to, in response to the first position 301, control the display modules (e.g., 205 and 210) to display the first information on the virtual object (e.g., Figure 3 At least one first surface (eg, Figure 6 The first surface 311 of the virtual object is displayed on the second surface, and the second surface is displayed on at least one second surface (eg, 310-1, 310-2, 310-3) of the virtual object (eg, 310-1, 310-2, 310-3). Figure 6 The processor 120 may be configured to identify a second location of the user in the real world or the virtual world (eg, Figure 6 In response to the user's gaze direction being a direction corresponding to at least one second surface 312, the processor 120 may be configured to control the display modules (e.g., 205 and 210) to display the first information on at least one second surface 312 based on the distance and direction between the determined position and the second position 302.

[0172] In response to the distance between the second position 302 and the determined position exceeding a preset first range, the processor 120 can be configured to control the display module (e.g., 205 and 210) to display the first information using a second virtual object substantially identical to the virtual objects (e.g., 310-1, 310-2, and 310-3).

[0173] The processor 120 may be configured to receive input from the user using the second virtual object. In response to the input, the processor 120 may be configured to control the display module (eg, 205 and 210) to display the third information using the second virtual object or to terminate outputting the second virtual object.

[0174] The processor 120 may be configured to control the display modules (eg, 205 and 210 ) to display virtual objects (eg, 310 - 1 , 310 - 2 , and 310 - 3 ) at locations determined based on a user's location in at least one of the real world or the virtual world.

[0175] The processor 120 can be configured to control the display modules (e.g., 205 and 210) so that the direction in which at least one second surface 312 is displayed is changed based on the second position 302, or the size in which at least one second surface 312 is displayed is changed based on the distance between the user's position and the determined position.

[0176] According to various embodiments, a display module control method may include: controlling a display module (eg, Figure 1 The display module 160, Figure 2 The first display 205 and the second display 210 of the embodiment of the present invention are used to display the first information on the virtual object (for example, Figure 3 At least one first surface (eg, Figure 6 310 - 1 , 310 - 2 , and 310 - 3 ), and displays the second information on at least one second surface (eg, Figure 6 determining a gaze direction of the user based on the user's position in the real world or the virtual world; and in response to the user's gaze direction being a direction corresponding to at least one second surface 312, controlling the display modules (e.g., 205 and 210) to display first information on at least one second surface 312.

[0177] Controlling the display module (e.g., 205 and 210) to display first information on at least one second surface 312 may include: in response to the distance between the user's position and the determined position exceeding a preset first range, controlling the display module (e.g., 205 and 210) to display the first information using a second virtual object substantially identical to the virtual object (e.g., 310-1, 310-2, 310-3).

[0178] The display module control method may further include: receiving input from a user using the second virtual object; and in response to the input, controlling the display module (eg, 205 and 210) to display third information using the second virtual object or to terminate outputting the second virtual object.

[0179] Controlling the display module (e.g., 205 and 210) to display first information on at least one second surface 312 may include: controlling the display module to display virtual objects (e.g., 310-1, 310-2, and 310-3) at a position determined based on the position of at least one of the user in the real world or the virtual world.

[0180] Controlling the display module (e.g., 205 and 210) to display first information on at least one second surface 312 may include: controlling the display module (e.g., 205 and 210) to change the direction in which at least one second surface 312 is displayed based on the user's position, or changing the size in which at least one second surface 312 is displayed based on the distance between the user's position and the determined position.

[0181] Controlling the display module (e.g., 205 and 210) to display the first information on at least one second surface 312 may include: when the content included in the first information changes, controlling the display module (e.g., 205 and 210) to display the changed first information on at least one second surface 312.

[0182] According to an example embodiment of the present disclosure, the electronic device may be one of various types of electronic devices. The electronic device may include, as non-limiting examples, a portable communication device (e.g., a smart phone, etc.), a computing device, a portable multimedia device, a portable medical device, a camera, a wearable device, a home appliance, etc. However, the electronic device is not limited to the aforementioned examples.

[0183] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalents or alternative forms for the corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the items listed together with the corresponding one of the multiple phrases. As used herein, terms such as "first" or "second" may only be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “combined with another element (e.g., the second element)”, “combined to another element (e.g., the second element)”, “connected with another element (e.g., the second element)”, or “connected to another element (e.g., the second element)” when the terms “operably” or “communicatively” are used or when the terms “operably” or “communicatively” are not used, it means that the element may be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0184] As used in connection with certain embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, or a combination thereof, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or a minimum unit or portion of the single integrated component. For example, depending on the embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0185] The various embodiments described herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). For example, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. A machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" only means that the storage medium is a tangible device and does not include a signal (e.g., an electromagnetic wave), but the term does not distinguish between data being semi-permanently stored in a storage medium and data being temporarily stored in a storage medium.

[0186] According to various embodiments of the present disclosure, the method described herein may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be released in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., Play Store TM ) The computer program product may be published (e.g., downloaded or uploaded) online, or the computer program product may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If published online, at least part of the computer program product may be temporarily generated, or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, an application store's server, or a forwarding server).

[0187] According to various embodiments, each component (e.g., module or program) in the above-mentioned components may include a single entity or multiple entities, and some entities in the multiple entities may be separately arranged in different components. According to various embodiments, one or more components in the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each component in the multiple components in the same or similar manner as the corresponding one component in the multiple components before integration. According to various embodiments, the operations performed by a module, program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more operations in the operations may be run or omitted in different orders, or one or more other operations may be added.

Claims

1. An electronic device (300), comprising: Display module (205, 210); Processor (120); as well as a memory electrically coupled to the processor (120) and storing instructions executable by the processor (120), Wherein, when the instruction is executed, the processor (120) is configured to: Controlling the display module (205, 210) to display first information on at least one first surface (311) of a virtual object (310-1, 310-2, 310-3) at a determined position in at least one of the real world or the virtual world, and displaying second information on at least one second surface (312) of the virtual object (310-1, 310-2, 310-3); Determining a gaze direction of the user based on the user's position in the real world or the virtual world; and In response to the user's gaze direction being a direction corresponding to the at least one second surface (312), the display module (205, 210) is controlled to display the first information on the at least one second surface (312).

2. The electronic device (300) according to claim 1, wherein: The processor (120) is configured to: In response to the distance between the user's position and the determined position exceeding a preset first range, the display module (205, 210) is controlled to display the first information using a second virtual object that is substantially the same as the virtual object (310-1, 310-2, 310-3).

3. The electronic device (300) according to claim 2, wherein: The processor (120) is configured to: receiving input from the user using the second virtual object; and In response to the input, the display module (205, 210) is controlled to display third information using the second virtual object or to terminate outputting the second virtual object.

4. The electronic device (300) according to any one of claims 1 to 3, wherein: The processor (120) is configured to: The display module (205, 210) is controlled to display the virtual object (310-1, 310-2, 310-3) at a position determined based on the position of the user in at least one of the real world or the virtual world.

5. The electronic device (300) according to any one of claims 1 to 4, wherein: The processor (120) is configured to: The display module (205, 210) is controlled so that the direction in which the at least one second surface (312) is displayed is changed based on the position of the user, or the size in which the at least one second surface (312) is displayed is changed based on the distance between the position of the user and the determined position.

6. The electronic device (300) according to claim 1, wherein: The processor (120) is configured to: When the content included in the first information changes, the display module (205, 210) is controlled to display the changed first information on the at least one second surface (312).

7. An electronic device (300), comprising: Display module (205, 210); Processor (120); as well as a memory electrically coupled to the processor (120) and storing instructions executable by the processor (120), Wherein, when the instruction is executed, the processor (120) is configured to: identifying a first location of a user in at least one of a real world or a virtual world (301); In response to the first position (301), controlling the display module (205, 210) to display first information on at least one first surface (311) of a virtual object (310-1, 310-2, 310-3) at a determined position in the real world or the virtual world, and displaying second information on at least one second surface (312) of the virtual object (310-1, 310-2, 310-3); identifying a second location of the user in the real world or the virtual world (302); and In response to the user's gaze direction being a direction corresponding to the at least one second surface (312), based on the distance and direction between the determined position and the second position (302), the display module (205, 210) is controlled to display the first information on the at least one second surface (312).

8. The electronic device (300) according to claim 7, wherein: The processor (120) is configured to: In response to the distance between the second position (302) and the determined position exceeding a preset first range, the display module (205, 210) is controlled to display the first information using a second virtual object that is substantially the same as the virtual object (310-1, 310-2, 310-3).

9. The electronic device (300) according to claim 8, wherein: The processor (120) is configured to: receiving input from the user using the second virtual object; and In response to the input, the display module (205, 210) is controlled to display third information using the second virtual object or to terminate outputting the second virtual object.

10. The electronic device (300) according to any one of claims 7 to 9, wherein: The processor (120) is configured to: The display module (205, 210) is controlled to display the virtual object (310-1, 310-2, 310-3) at a position determined based on the position of the user in at least one of the real world or the virtual world.

11. A method for controlling a display module, the method comprising: Controlling a display module (205, 210) to display first information on at least one first surface (311) of a virtual object (310-1, 310-2, 310-3) at a determined position in at least one of the real world or the virtual world, and to display second information on at least one second surface (312) of the virtual object (310-1, 310-2, 310-3); Determining a gaze direction of the user based on a position of the user in the real world or the virtual world; as well as In response to the user's gaze direction being a direction corresponding to the at least one second surface (312), the display module (205, 210) is controlled to display the first information on the at least one second surface (312).

12. The method according to claim 11, wherein: Controlling the display module (205, 210) to display the first information on the at least one second surface (312) comprises: In response to the distance between the user's position and the determined position exceeding a preset first range, the display module (205, 210) is controlled to display the first information using a second virtual object that is substantially the same as the virtual object (310-1, 310-2, 310-3).

13. The method according to claim 12, further comprising: receiving input from the user using the second virtual object; as well as In response to the input, the display module (205, 210) is controlled to display third information using the second virtual object or to terminate outputting the second virtual object.

14. The method according to any one of claims 11 to 13, wherein: Controlling the display module (205, 210) to display the first information on the at least one second surface (312) comprises: The display module (205, 210) is controlled to display the virtual object (310-1, 310-2, 310-3) at a position determined based on the position of the user in at least one of the real world or the virtual world.

15. The method according to any one of claims 11 to 14, wherein: Controlling the display module (205, 210) to display the first information on the at least one second surface (312) comprises: The display module (205, 210) is controlled so that the direction in which the at least one second surface (312) is displayed is changed based on the position of the user, or the size in which the at least one second surface (312) is displayed is changed based on the distance between the position of the user and the determined position.

Images