Wearable device, method, and storage medium for displaying content

Abstract

Provided are a wearable device, a method, and a storage medium for displaying content. The wearable device comprises: a display; at least one processor including a processing circuit; and a memory storing instructions that are individually or collectively executed by the at least one processor. The at least one processor may be configured to individually or collectively execute the stored instructions and cause the wearable device to determine the usage environment of the wearable device displaying content. The at least one processor is configured to cause the wearable device to generate a boundary on the basis of at least one of the number of pieces of content or the usage environment. The at least one processor is configured to cause the wearable device to display, through the display, content arranged on the basis of the shape of the boundary. The boundary includes an area set in advance on the basis of the user wearing the wearable device or the wearable device, displays the content on the surface of the boundary, and includes a virtual space in which content including a lock attribute is moved on the surface according to the posture of the user.

Description

Wearable device, method, and storage medium for displaying content

[0001] The present disclosure relates to a wearable device, a method, and a storage medium, and, for example, to a wearable device, a method, and a storage medium for displaying content.

[0002] A wearable device is a device worn on a user's body and performs a function. For example, a wearable device may take the form of a head-mounted display (HMD), glasses, a watch, a bracelet, or a ring. An HMD includes a display and can display content on the display. The HMD can provide the user with a three-dimensional and realistic experience by moving the content in response to the user's changes in direction or movement. A glasses-type device may include a transparent display. The glasses-type device can recognize external objects visible through the transparent display and display additional information about the recognized objects on the transparent display. As an example, a glasses-type device can display a generated virtual object on the transparent display.

[0003] The above information may be provided merely as background information to aid in understanding the present disclosure. None of the above is claimed as prior art related to the present disclosure or can be used in determining prior art.

[0004] A wearable device according to various embodiments of the present document may include a display, at least one processor including a processing circuit, and a memory storing instructions to be individually or collectively executed by the at least one processor. The at least one processor may be configured to individually or collectively execute the stored instructions and determine a usage environment of the wearable device for displaying content. The at least one processor may be configured to cause the wearable device to generate a boundary based on at least one of the number of contents and the usage environment. The at least one processor may be configured to cause the wearable device to display the contents arranged based on the shape of the boundary through the display. The boundary may include a virtual space including a preset area based on a user wearing the wearable device or the wearable device, displaying the contents on the surface of the boundary, and moving the contents including a lock attribute on the surface according to the posture of the user.

[0005] A method for displaying content on a wearable device according to various embodiments of the present document may include an operation for determining a usage environment of the wearable device displaying the content. The method may include an operation for generating a boundary based on at least one of the number of contents and the usage environment. The method may include an operation for displaying the content arranged based on the shape of the boundary. The boundary may include a preset area surrounding a user wearing the wearable device or at least a portion of the user or the wearable device based on the wearable device, and may include a virtual space for displaying the content on a surface of the boundary and moving the content including a lock attribute on the surface according to the posture of the user.

[0006] Instructions of a non-transitory computer-readable storage medium having recorded thereon a program for performing a method for displaying content on a wearable device according to various embodiments of the present document may perform an operation of determining a usage environment of the wearable device displaying the content. The instructions may perform an operation of generating a boundary based on at least one of the number of contents and the usage environment. The instructions may perform an operation of displaying the content arranged based on the shape of the boundary. The boundary may include a preset area surrounding a user wearing the wearable device or at least a part of the user or the wearable device based on the wearable device, and may include a virtual space for displaying the content on the surface of the boundary and moving the content including a lock attribute on the surface according to the posture of the user.

[0007] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0008] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0009] Figure 2 is a block diagram illustrating the configuration of a wearable device according to various embodiments.

[0010] FIG. 3 is a flowchart illustrating an operation of adjusting content according to a user's location according to various embodiments.

[0011] Figures 4a and 4b are flowcharts illustrating operations for generating boundaries according to various embodiments.

[0012] Figures 5a, 5b and 5c are drawings illustrating boundaries according to various embodiments.

[0013] FIG. 6 is a flowchart illustrating an operation of arranging content within content according to various embodiments.

[0014] FIG. 7 is a drawing illustrating a protection area according to various embodiments.

[0015] FIG. 8 is a drawing illustrating a screen on which content is arranged according to various embodiments.

[0016] FIG. 9 is a diagram illustrating lock properties of content according to various embodiments.

[0017] FIGS. 10A and 10B are diagrams illustrating content displayed based on properties in response to a user's change of direction according to various embodiments.

[0018] FIGS. 11a, 11b, and 11c are diagrams illustrating content displayed according to a user's posture and gaze direction according to various embodiments.

[0019] FIG. 12 is a drawing illustrating content displayed according to a user's posture according to various embodiments.

[0020] FIG. 13 is a diagram illustrating content displayed according to repeated user postures according to various embodiments.

[0021] FIGS. 14a, 14b, and 14c are drawings illustrating content displayed on a boundary according to various embodiments.

[0022] FIG. 15 is a drawing illustrating content displayed on a rectangular solid boundary according to various embodiments.

[0023] FIGS. 16a, 16b, and 16c are drawings illustrating content displayed according to changes in boundaries according to various embodiments.

[0024] FIG. 17a and FIG. 17b are drawings explaining the offset of a virtual space according to various embodiments.

[0025] FIGS. 18A, 18B, and 18C are diagrams illustrating the operation of a wearable device when a user's position and / or gaze direction is moved according to various embodiments.

[0026] Figure 19 is a flowchart illustrating a method of displaying content according to various embodiments.

[0027] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0028] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0029] The processor (120) may include various processing circuits and / or multiple processors. For example, the term "processor" as used herein, including in the claims, may include various processing circuits, including at least one processor, one or more of which may be individually and / or collectively configured to perform various functions described herein. When the terms "processor," "at least one processor," and "one or more processors" as used herein are described as being configured to perform a number of functions, these terms may include, for example, without limitation, a situation where one processor performs some of the recited functions and other processor(s) perform other recited functions, and a situation where a single processor may perform all of the recited functions. Furthermore, the at least one processor may include a combination of processors that perform various recited / disclosed functions, for example, in a distributed manner. The at least one processor may execute program instructions to achieve or perform various functions. The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134).According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0030] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0031] The memory (130) can store various data used by at least one component (e.g., the processor (120) or the sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., the program (140)) and input data or output data for commands related thereto. The memory (130) can include a volatile memory (132) or a non-volatile memory (134). The non-volatile memory (134) can include at least one internal memory (136) and an external memory (138).

[0032] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0033] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0034] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0035] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0036] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0037] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0038] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0039] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0040] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0041] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0042] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0043] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0044] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0045] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0046] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0047] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0048] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0049] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0050] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0051] Figure 2 is a block diagram illustrating the configuration of a wearable device according to various embodiments.

[0052] Referring to FIG. 2, a wearable device (200) may include a display (210), a memory (220), and a processor (230) (e.g., including a processing circuit).

[0053] The display (210) (e.g., the display module (160) of FIG. 1) can display content. For example, the display (210) can be implemented as a general display (e.g., a liquid crystal display (LCD), an organic light emitting diode (OLED), etc.) or a transparent display. As an example, the wearable device (200) (e.g., the electronic device (101) of FIG. 1) can include a transparent display and a camera (e.g., the camera module (180) of FIG. 1). The wearable device (200) can recognize actual content (e.g., a desk, furniture, picture frames, etc. in a room, which is a real environment) shown to the user through the transparent display using the camera. The wearable device (200) can implement an augmented reality (AR) environment by displaying virtual content on the transparent display in consideration of the recognized actual content. As an example, the wearable device (200) can include a general display. The wearable device (200) can implement a virtual reality (VR) environment by displaying generated content on a general display. As an example, the wearable device (200) may further include a camera. The wearable device (200) can capture images of the surrounding environment using the camera. The wearable device (200) can recognize actual content from the captured image and display the captured image on the general display. In addition, the wearable device (200) can implement an AR environment by displaying virtual content on a transparent display based on the recognized actual content.

[0054] The memory (220) (e.g., the memory (130) of FIG. 1) can store data, algorithms, programs, commands, etc. that perform the functions of the wearable device (200). The commands, etc. stored in the memory (220) can be loaded into the processor (230) and executed by the processor (230).

[0055] The processor (230) (e.g., the processor (120) of FIG. 1) includes various processing circuits and can control each component of the wearable device (200). The wearable device (200) may include one or more processors (230). For example, the processor (230) may correspond to a plurality of processors that collectively perform a plurality of functions by dividing them among the processors. The processor (120) may include various processing circuits and / or multiple processors. For example, the term “processor” as used in this document, including the claims, may include various processing circuits, including at least one processor, and at least one or more of the processors may be configured to individually and / or collectively perform the various functions described in this document in a distributed manner. When the terms "processor," "at least one processor," and "one or more processors" are used herein to describe a processor configured to perform a number of functions, these terms may include, for example, without limitation, situations where one processor performs some of the recited functions and other processor(s) perform other recited functions, and situations where a single processor can perform all of the recited functions. Furthermore, the at least one processor may include a combination of processors that perform various recited / disclosed functions, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0056] For example, the processor (230) may determine the usage environment of the wearable device (200) that displays content. For example, the usage environment may include a VR environment, an AR environment, an XR (extended reality) environment, and / or MR (mixed reality). For example, the XR environment and the MR environment may be similar to a VR environment if implemented based on virtual content, and may be similar to an AR environment if implemented based on a real environment (or, real image) and virtual content. As an example, if the wearable device (200) displays only virtual content, the usage environment of the wearable device (200) may be a VR environment (or, XR environment, MR environment). If the wearable device (200) displays virtual content and real content, the usage environment of the wearable device (200) may be an AR environment (or, XR environment, MR environment). In the present disclosure, a description will be given focusing on the VR environment and the AR environment.

[0057] The processor (230) may generate boundaries based on at least one of the number of contents and the usage environment. For example, the usage environment of the wearable device (200) may include various contents. For example, the contents may include contents that do not actually exist in the surrounding environment but are generated as images and / or contents that exist in the surrounding environment. Content that does not actually exist but is generated as images may include general contents (e.g., picture frames, windows, desks, etc.) and / or task-related contents (e.g., task windows, task-related bars, pop-ups, etc.). The processor (230) may set environment-related properties (or background properties) or task-related properties to the contents based on their relevance to the task. For example, the processor (230) may set the properties of the contents based on the characteristics of the contents, the user's usage patterns, and / or the user's commands. For example, the processor (230) may set picture frames and windows, which have low relevance to the task, as environment-related properties, and may set desks, which have high relevance to the task, as task-related properties. For example, the processor (230) may display virtual content by avoiding content that includes environment-related attributes. The processor (230) may display virtual content at a location other than a location where content that includes environment-related attributes is displayed. For example, the processor (230) may display virtual content by anchoring it to content that includes task-related attributes. The processor (230) may display virtual content by relating it to content that includes task-related attributes within or around content that includes task-related attributes. The content may be referred to as an object, a window, a task, and / or a component.

[0058] For example, content may include a lock attribute. The lock attribute may include a user lock (body lock) attribute or an environment lock (or world lock) attribute related to the object being fixed. For example, the processor (230) may fix content including a user lock attribute based on the user, and fix content including an environment lock attribute based on the environment (or background). For example, the environment (or background, screen) displayed to the user may change depending on the movement of the user (e.g., movement of the face or neck of a user wearing a wearable device). As an example, even if the displayed environment changes, the processor (230) may maintain the position of the content including a user lock attribute based on the user. The content including a user lock attribute may be fixed to the user (e.g., the direction of the user's gaze or viewing angle). The direction of the user's gaze and / or viewing angle while viewing the content including a user lock attribute may be maintained regardless of changes in the environment displayed to the user. In other words, the processor (230) may move the position of the content including a user lock attribute based on the environment. For example, when the displayed environment changes, the processor (230) can change the position of the content including the environment lock attribute displayed on the display (210) based on the user according to the change in the displayed environment. In other words, the content including the environment lock attribute can be fixed to the environment. The direction of the user's gaze and / or viewing angle while viewing the content including the environment lock attribute can be changed according to the change in the environment (or background, screen) displayed to the user. In other words, the processor (230) can maintain the position of the content including the environment lock attribute based on the environment.

[0059] For example, a lock attribute (e.g., a user lock attribute or an environment lock attribute) may be set by a user or automatically set by the processor (230). The processor (230) may automatically set the lock attribute of content based on the distance from the user (or the wearable device (200)), frequency of use, recent use, user preference, etc., among the contents used (or activated, executed) in the wearable device (200). As an example, the processor (230) may set control content, content being worked on, frequently used content determined based on the user's usage pattern, content set by the user, and / or notification-related content as user lock attributes. In addition, the processor (230) may set the remaining contents as environment lock attributes. As an example, the processor (230) may set recently used (or activated) content (or content used within a preset time) as the user lock attribute. As an example, the processor (230) may set content that includes a recent usage history among contents located within a certain distance (e.g., close range) from the wearable device (200) as a user lock attribute. As an example, the processor (230) may set frequently used content (or content used / activated more than a preset number of times or content used / activated more than a preset number of times within a preset time) as a user lock attribute. As an example, the processor (230) may set content with a recently and / or frequently used environment lock attribute as an environment lock attribute.

[0060] For example, a boundary may be a virtual space including a preset (or specific) shape and area. As an example, the boundary may be in the shape of a rectangular parallelepiped, a cylinder, a capsule, a sphere, and / or a hemisphere. The boundary includes a preset area surrounding a user wearing the wearable device (200) or at least a portion of the wearable device (200), and content (or an object, a window, a task, or an element) may be displayed in a certain area based on the surface of the boundary. As an example, when the surface of the boundary is located at a distance of about 1 m in radius from a user wearing the wearable device (200) (or the wearable device (200)), the processor (230) may display content in an area of ​​about 0.9 m to about 1.1 m in radius from the user wearing the wearable device (200).

[0061] For example, if the environment in which the wearable device (200) is used is an AR environment and a plane is detected in the real environment, the processor (230) may generate a rectangular parallelepiped boundary. As an example, if a user wearing the wearable device (200) uses the wearable device (200) in an AR environment in a room, the processor (230) may detect a wall in the room. Since the wall in the room is a plane, the processor (230) may generate a rectangular parallelepiped boundary. For example, if the environment in which the wearable device (200) is used is a VR environment and the number of contents (e.g., task-related contents) is equal to or less than a preset first number, the processor (230) may generate a cylindrical boundary, and if the number of contents exceeds the preset first number, the processor (230) may generate a capsule-shaped boundary. As an example, the preset first number may be 1. A user wearing a wearable device (200) can use the wearable device (200) in a VR environment and execute one content. The processor (200) can display the executed content in the form of a window on the display (210). Since the use environment of the wearable device (200) is a VR environment and the number of contents displayed on the display (210) is one, the processor (230) can create a cylindrical boundary. As an example, the wearable device (200) can display first content (e.g., a task window) and second content (e.g., a status bar) on the display (210). Since the use environment of the wearable device (200) is a VR environment and the number of contents displayed on the display (210) exceeds one, the processor (230) can create a capsule-shaped boundary.

[0062] The processor (230) can display content arranged based on the shape of the boundary through the display (210). For example, the processor (230) can display (or arrange) content based on the location and properties of other content. For example, if the usage environment of the wearable device (200) is a VR environment, the processor (230) can determine the properties of the content as task-related properties or environment-related properties. For example, the content can include general content (e.g., picture frames, windows, desks, etc.) and / or task-related content (e.g., task windows, task-related bars, pop-ups, etc.). The processor (230) can determine the environment-related properties (or background properties) or task-related properties based on the relevance of the general content to the task-related content.

[0063] For example, the processor (230) may display (or place) task-related content by avoiding general content determined to have (or including) environment-related attributes. The processor (230) may display (or place) task-related content by anchoring it to general content determined to have (or including) task-related attributes. As an example, general content may include a bed and a desk, and task-related content may include a window and a control bar. The bed may be general content related to the background, and the desk may be content related to the background and a task. The processor (230) may determine (or set) the bed as an environment-related attribute, and determine (or set) the desk as a task-related attribute. The processor (230) may display a window by avoiding the bed, and display a control bar by anchoring it to the desk. For example, if the generated boundary is a rectangular solid boundary, the processor (230) can change the size of the task-related content to display the task-related content in an area of ​​one side of the rectangular solid boundary. If general content is located on one side of the rectangular solid boundary, the processor (230) can change the size of the task-related content to display the task-related content in a part of one side of the rectangular solid boundary, avoiding the general content.

[0064] As an example, the processor (230) may display content in a predetermined area including the surface of the boundary, and move the content near (including the surface) the surface of the boundary based on the user's posture, movement of the gaze direction, and the lock property of the content. For example, the wearable device (200) may include a sensor (e.g., the sensor module (176) of FIG. 1). The processor (230) may determine the user's posture and / or gaze direction based on signals detected by the sensor. As an example, the sensor may include a gyro sensor, an acceleration sensor, and a camera.

[0065] The processor (230) may display content at a location corresponding to the user's posture. For example, if the user's posture is a standing posture, the processor (230) may display content at a first location corresponding to the standing posture, and if the user's posture is a sitting posture, the processor (230) may display content at a second location corresponding to the sitting posture. If the user's posture is a first posture and a second posture repeated a preset number of times and / or for a preset time, the processor (230) may display content at a location intermediate between the first location of the content corresponding to the first posture and the second location of the content corresponding to the second posture.

[0066] The wearable device (200) of this document can pre-generate boundaries for displaying content based on the usage environment and the number of contents, pre-identify fixed and movable contents based on lock properties, and move the contents in groups. The wearable device (200) can render contents and / or image frames containing the contents in advance by predicting the display area and movement target of the contents.

[0067] For example, the wearable device (200) can display content in a form corresponding to the surface of the generated boundary. The wearable device (200) can determine in advance the location and form in which the content is displayed by generating the boundary. The wearable device (200) can pre-render movable content as a group according to the lock attribute. As an example, the wearable device (200) can display first content and second content including the user lock attribute on the front of a cylindrical boundary, and display third content and fourth content including the environment lock attribute. When the user's gaze moves laterally, the wearable device (200) can display the first content and second content in a curved form by moving them based on the environment (or background). In other words, when the user's gaze moves laterally, the wearable device (200) can display the first content and second content in a curved form by maintaining the current position based on the user.

[0068] When the user's gaze moves in an upward or downward direction, the wearable device (200) can display the first content and the second content in a planar form by moving them based on the environment. The wearable device (200) can group the first content and the second content and pre-render the first content and the second content in a curved form. The wearable device (200) can pre-render the first content and the second content in a planar form. For example, when the user's gaze moves in a side direction, the wearable device (200) can display an image corresponding to the changed viewing angle and display the first content and the second content in a pre-rendered curved form on the displayed image. For example, when the user's gaze moves in an upward direction, the wearable device (200) can display an image corresponding to the changed viewing angle and display the first content and the second content in a planar form in a pre-rendered form on the displayed image. For example, the third content and the fourth content including the environment lock attribute can be fixed to the environment. The wearable device (200) can display at least a portion of the third content and the fourth content while maintaining the current position relative to the environment. In other words, the wearable device (200) can display at least a portion of the third content and the fourth content by moving them relative to the user according to the changed viewing angle.

[0069] Accordingly, the wearable device (200) of this document can reduce the load on the processor (230) and display an image that is naturally connected to a moved location according to the user's movement, gaze movement, and / or expanded use of the content.

[0070] If the number of contents in the AR environment exceeds a preset first number or the number of contents in the VR environment changes, the processor (230) can change the generated boundary based on the number of contents. For example, the preset first number may be 1. The processor (230) may generate a rectangular parallelepiped boundary in the AR environment, but if the number of contents is 2 or more, the rectangular parallelepiped boundary may be changed to a cylindrical boundary or a capsule-shaped boundary. For example, the processor (230) may generate a cylindrical boundary in the VR environment, but if the number of contents is changed to 2 or more, the cylindrical boundary may be changed to a capsule-shaped boundary. For example, the processor (230) may generate a capsule-shaped boundary in the VR environment, but if the number of contents is changed to 1, the capsule-shaped boundary may be changed to a cylindrical boundary.

[0071] The processor (230) can change the shape of content displayed near the surface of the boundary according to the change of the boundary. For example, the processor (230) can display content in a curved shape on the curved surface of a cylindrical boundary or a capsule-shaped boundary. When the cylindrical boundary or the capsule-shaped boundary is changed to a rectangular parallelepiped boundary or a cylindrical boundary, the processor (230) can change the shape of the content displayed in a curved shape to a planar shape on the plane of the rectangular parallelepiped boundary or the upper and lower surfaces of the cylindrical boundary and display it. As an example, the processor (230) can display content in a planar shape on the upper and lower surfaces of the rectangular parallelepiped boundary or the cylindrical boundary. When the shape of the content is changed from a rectangular or cylindrical boundary to a cylindrical boundary or a capsule-shaped boundary, the processor (230) may change the shape of the content displayed in a flat shape into a curved shape on the surface of the cylindrical or capsule-shaped boundary. As an example, when the processor (230) changes the shape of the content and displays it, the content may be temporarily displayed on the display (210) without a curve applied. In other words, when the processor (230) changes the shape of the content from a curved shape to a flat shape or from a flat shape to a curved shape, the processor (230) may temporarily display the content without a curve applied on the display (210), thereby allowing the user to perceive that the shape of the content has been naturally changed.

[0072] FIG. 3 is a flowchart illustrating an operation of adjusting content according to a user's location according to various embodiments, and FIGS. 4A and 4B are flowcharts illustrating an operation of creating a boundary according to various embodiments. FIGS. 5A, 5B, and 5C are diagrams illustrating boundaries according to various embodiments, and FIG. 6 is a flowchart illustrating an operation of arranging content according to various embodiments. In addition, FIG. 7 is a diagram illustrating a protection area according to various embodiments.

[0073] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0074] According to one embodiment, steps 310 to 350 may be understood to be performed in a processor (e.g., processor (230) of FIG. 2) of an electronic device (e.g., electronic device (200) of FIG. 2).

[0075] Referring to FIG. 3, the wearable device (200) can identify content (310). For example, the wearable device (200) can determine the size, location, properties, and / or number of content.

[0076] The wearable device (200) can determine the state of the wearable device (200) (320). For example, the wearable device (200) can determine the position and / or angle of the wearable device (200) using a sensor. The position, direction, and / or angle of the wearable device (200) may be the user's posture and / or gaze direction.

[0077] The wearable device (200) can determine the user's usage pattern (330). For example, the wearable device (200) can determine the number of previously used contents, frequently used contents, task-related contents, and / or usage posture corresponding to the identified contents.

[0078] The wearable device (200) can generate a boundary (340) and display content by adjusting it based on the generated boundary (350). For example, when displaying content on a flat boundary, the wearable device (200) can display the flat content on the surface (or near the surface) of the flat boundary. When displaying content on a curved boundary, the wearable device (200) can display the curved content on the surface of the curved boundary.

[0079] Referring to FIGS. 4a and 4b, the operation of generating a boundary is described.

[0080] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0081] According to one embodiment, steps 405 to 455 may be understood to be performed in a processor (e.g., processor (230) of FIG. 2) of an electronic device (e.g., electronic device (200) of FIG. 2).

[0082] Referring to FIG. 4A, the wearable device (405) can determine whether the usage environment is an AR environment or a VR environment (405). For example, the AR environment may be an environment that displays generated content together with actual content, and the VR environment may be an environment that displays only generated content. The wearable device (200) may be used in an XR environment and / or an MR environment. If the XR environment and / or the MR environment is an environment that displays generated content together with actual content, the wearable device (200) may perform the same operation as the operation of the AR environment. If the XR environment and / or the MR environment is an environment that displays only generated content, the wearable device (200) may perform the same operation as the operation of the VR environment.

[0083] If the usage environment is an AR environment (405-AR), the wearable device (200) can determine the actual environment using a camera (410). For example, the wearable device (200) can determine the surrounding environment (e.g., wall, floor), actual content, size, location, and / or distance of the actual content. The wearable device (200) can determine whether a plane is detected (415). As an example, if a user wearing the wearable device (200) is located in a room, the wearable device (200) can detect the plane of the ceiling, floor, and / or wall (plane detecting). If a plane is detected (415-YES), the wearable device (200) can generate a rectangular prism-shaped boundary (420).

[0084] If the usage environment is a VR environment (405-VR) or a plane is not detected (415-NO), the wearable device (200) can determine the user's posture (425). For example, the wearable device (200) can determine the state of the wearable device (200), including the position, direction, and / or angle of the wearable device (200), using a sensor, and the user can wear the wearable device (200). Accordingly, the user's posture can be the state of the wearable device (200).

[0085] Referring to FIG. 4B, the wearable device (200) can determine whether the user is in a standing or sitting position (430). If the wearable device (200) determines that the user is in a standing or sitting position (430-YES), the wearable device (200) can determine whether the number of contents (e.g., task-related contents) exceeds a first number (435). For example, the first number may be 1. If the number of contents exceeds the first number (435-YES), the wearable device (200) can generate a capsule-shaped boundary (440). If the number of contents is less than or equal to the first number (435-NO), the wearable device (200) can generate a cylindrical boundary (445).

[0086] If the wearable device (200) determines that the user is not standing or sitting (430-NO), it can determine that the user is lying down (450). If the wearable device (200) determines that the user is lying down, it can create a hemispherical / capsule-shaped boundary (455).

[0087] The wearable device (200) can change the generated boundary when the user's posture or the number of contents changes. For example, if the usage environment is an AR environment and a plane is detected in the actual environment, the wearable device (200) can create a rectangular parallelepiped boundary. In addition, the wearable device (200) can determine the number of contents. If the number of contents exceeds a first number, the wearable device (200) can determine the user's posture. If the user's posture is determined to be a standing or sitting posture, the wearable device (200) can change the rectangular parallelepiped boundary into a capsule-shaped boundary. If the user's posture is determined to be a lying posture, the wearable device (200) can change the rectangular parallelepiped boundary into a hemispherical / capsule-shaped boundary.

[0088] Referring to Fig. 5a, a rectangular parallelepiped boundary (1) is illustrated. For example, the boundary may be a virtual space that includes a preset area centered on a user wearing a wearable device (200) (or, wearable device (200)) and displays content near the surface of the boundary. Each face of the rectangular parallelepiped boundary (1) may be a plane. The wearable device (200) may display flat content (51) near the surface of the rectangular parallelepiped boundary (1) based on the user's gaze direction.

[0089] Referring to FIG. 5B, a cylindrical boundary (3) is illustrated. The upper and lower surfaces of the cylindrical boundary (3) may be flat, and the side surfaces may be curved. The wearable device (200) may display content (53) near the surface of the cylindrical boundary (3) based on the user's gaze direction. For example, if the user's gaze direction is directed toward the front (or, side, rear), the wearable device (200) may display curved content (53) on the side surface of the cylindrical boundary (3). As an example, if the user's gaze direction is directed toward the upper or lower surface, the wearable device (200) may display flat content (53) on the upper or lower surface of the cylindrical boundary (3).

[0090] Referring to Fig. 5c, a capsule-shaped boundary (5) is illustrated. The upper and lower surfaces of the capsule-shaped boundary (5) may be hemispherical, and the side surfaces may be cylindrical. Each surface of the capsule-shaped boundary (5) may be curved. The wearable device (200) may display curved content (55) near the surface of the capsule-shaped boundary (5) based on the user's gaze direction.

[0091] Referring to Figure 6, an operation for adjusting and displaying content is described.

[0092] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0093] According to one embodiment, steps 610 to 670 may be understood to be performed in a processor (e.g., processor (230) of FIG. 2) of an electronic device (e.g., electronic device (200) of FIG. 2).

[0094] Referring to FIG. 6, the wearable device (200) can determine whether the usage environment is an AR environment or a VR environment (610). If the usage environment is an AR environment (610-AR), the wearable device (200) can detect actual content (620). For example, the actual content may include furniture such as a desk placed in an actual room, a vase, a picture frame, and / or a tree or bench located in an actual outdoor environment. The wearable device (200) can anchor the content to the actual content based on the floor of the actual content and provide the content (630). For example, the wearable device (200) can place content such as a status bar on a desk placed in an actual room.

[0095] If the usage environment is a VR environment (610-VR), the wearable device (200) can check the protection area (640).

[0096] Referring to FIG. 7, a guardian area (9) and a boundary (7) are illustrated. For example, the guardian area may be a certain area within which a user can safely move. The VR environment may be an environment that only shows content generated for the user. Therefore, when the wearable device (200) is used in a VR environment, the user may not be aware of content placed around him / her. The wearable device (200) may set a certain area within which the user can safely move in the VR environment, taking into account the content placed around him / her. As an example, the boundary (7) may be created as an area smaller than or equal to the size of the guardian area. As an example, when the user moves into the boundary area of ​​the guardian area (9), the wearable device (200) may provide a notification to the user.

[0097] The wearable device (200) can determine whether the attribute of the content is a task-related attribute or an environment-related attribute (650). For example, the content may include general content (e.g., a picture frame, a window, a desk, etc.) and / or task-related content (e.g., a task window, a task-related bar, a pop-up, etc.). The wearable device (200) can determine the environment-related attribute (or background attribute) or the task-related attribute based on the relevance of the general content to the task-related content. As an example, the wearable device (200) can determine the attribute of the general content based on the characteristics of the general content, the user's usage pattern, and / or the user's command. As an example, if the content includes a picture frame and a desk, the wearable device (200) can determine the picture frame as an environment-related attribute and the desk as a task-related attribute. As an example, the wearable device (200) can determine the attribute of the content based on the attribute set according to the user's selection.

[0098] If the property of the content is a task-related property (650 - task-related property), the wearable device (200) can anchor the task-related content to the content including the task-related property and provide it (660). For example, the wearable device (200) can anchor the control window to a desk and display it on the desk. If the property of the content is an environment-related property (650 - environment-related property), the wearable device (200) can provide the task-related content by avoiding the content including the environment-related property (670). For example, the wearable device (200) can display the task window on the wall of the content by avoiding the picture frame.

[0099] Figure 8 is a drawing illustrating content arranged according to various embodiments.

[0100] Referring to FIG. 8, a screen of content (91) displayed on a display (210) is illustrated. The content (91) may include various contents. For example, the content (91) may include a bed (57), a desk (59), a tool bar (61), a keyboard (63), and / or a task window (65). The bed (57) and the desk (59) may be general contents, while the tool bar (61), the keyboard (63), and the task window (65) may be task-related contents.

[0101] For example, if the usage environment of the wearable device (200) is an AR environment, the bed (57) and the desk (59) may be actual contents. The wearable device (200) may anchor the tool bar (61) and / or the keyboard (63) to the desk (59) based on the bottom of the desk (59) and provide it. The wearable device (200) may anchor the task window (65) based on the bottom of the wall and provide it. At this time, since the bed (57) is located on the wall where the task window (65) is to be displayed, the wearable device (200) may provide the task window (65) by avoiding the bed (57).

[0102] For example, if the usage environment of the wearable device (200) is a VR environment, the wearable device (200) can determine the attributes of the bed (57) and the desk (59). For example, the wearable device (200) can determine the bed (57) having a low relevance to task-related content as an environment-related attribute, and the desk (59) having a high relevance to task-related content as a task-related attribute. For example, the wearable device (200) can set the bed (57) having no relevance to task performance as an environment-related attribute, and set the desk (59) having a high relevance to task performance as a task-related attribute. For example, the wearable device (200) can set the bed (57) as an environment-related attribute, and the desk (59) as a task-related attribute, based on the user's usage pattern. For example, the user can set the bed (57) as an environment-related attribute, and the desk (59) as a task-related attribute. The wearable device (200) can determine the properties of the bed (57) and the desk (59) based on the set properties. The wearable device (200) can avoid the bed (57) determined by the environment-related properties and provide a task window (65). The wearable device (200) can be fixed to the desk (59) determined by the task-related properties and provide a tool bar (61) and / or a keyboard (63).

[0103] FIG. 9 is a diagram illustrating lock properties of content according to various embodiments.

[0104] Referring to FIG. 9, a plurality of contents are illustrated. For example, the plurality of contents may include a first task window (61a), a second task window (61b), a status bar (63a), a notification pop-up (63b), and a task bar (63c). The plurality of contents may be displayed on the display (210). The wearable device (200) may set a lock attribute to the contents. For example, the lock attribute may include a user lock or an environment lock. As an example, the first task window (61a) and the second task window (61b) may be set to an environment lock, and the status bar (63a), the notification pop-up (63b), and the task bar (63c) may be set to a user lock. The first task window (61a) and the second task window (61b) set to an environment lock may be fixed based on the background (or environment). The status bar (63a), notification pop-up (63b) and task bar (63c) set to user lock can be fixed based on the user.

[0105] For example, the environment shown on the wearable device (200) and / or the environment displayed on the wearable device (200) may change in response to the user's movement (or gaze direction movement). The wearable device (200) may position the content including the user lock attribute so that the direction of the user's gaze and / or viewing angle while looking at the content including the user lock attribute displayed on the display (210) is maintained. In other words, the wearable device (200) may maintain the position of the content including the user lock attribute based on the user, and may change the position of the content including the user lock attribute based on the environment that was previously shown / displayed. The wearable device (200) may move the status bar (63a), the notification pop-up (63b), and the task bar (63c) including the user lock attribute based on the environment that was previously shown / displayed. The wearable device (200) can move the status bar (63a), the notification pop-up (63b), and the task bar (63c) while maintaining the viewing angle. The wearable device (200) can group the status bar (63a), the notification pop-up (63b), and the task bar (63c) into one group and move the group. The wearable device (200) can position content including an environment lock attribute in response to a change in the environment. The wearable device (200) can move the position of the content including an environment lock attribute based on the user. In other words, the wearable device (200) can maintain the position of the content including an environment lock attribute based on the environment that was previously shown / displayed. The wearable device (200) can maintain the positions of the first task window (61a) and the second task window (61b) including an environment lock attribute based on the environment that was previously shown / displayed.

[0106] FIGS. 10A and 10B are diagrams illustrating content displayed based on properties in response to a user's change of direction according to various embodiments.

[0107] Referring to FIG. 10A, an image (93a) including content displayed on a display (210) is illustrated. For example, the content may include a status bar (67a), a task window A (67b), a task bar (67c), and a task window B (65). As an example, the status bar (67a) and the task bar (67c) may be displayed as separate content, as illustrated in FIG. 10A. As an example, the status bar (67a) and the task bar (67c) may be displayed in a form that is separated within a single content. For example, content(s) including the status bar (67a), the task bar (67c), and / or various bars (e.g., a control bar) may be referred to as a system UI (user interface).

[0108] For example, the wearable device (200) can be used in an AR environment. The wearable device (200) can recognize an external environment (e.g., actual content). As an example, the wearable device (200) can capture an external environment using a camera and recognize the external environment from the captured image. If the wearable device (200) includes a transparent display, the recognized external environment may be the external environment shown through the transparent display. If the wearable device (200) includes a general display, the recognized external environment may be the external environment of an image displayed on the display. The wearable device (200) can display content by taking into account the recognized external environment.

[0109] For example, the wearable device (200) can determine the properties of actual content (e.g., a tree) in the external environment. If the actual content in the external environment includes task-related properties, the wearable device (200) can display content (e.g., task-related content) by fixing it to the actual content including the task-related properties. If the actual content in the external environment includes environment-related properties, the wearable device (200) can display the content by avoiding the actual content including the environment-related properties. As an example, as illustrated in FIG. 10A, the wearable device (200) can recognize a tree. The tree can include environment-related properties. The wearable device (200) can display a status bar (67a), a task window A (67b), a task bar (67c), and a task window B (65) by avoiding the tree.

[0110] For example, the wearable device (200) can be used in a VR environment. The wearable device (200) can display content by considering the general content of the content displayed on the display (210). For example, the wearable device (200) can determine the attributes of the general content (e.g., a tree). If the general content includes task-related attributes, the wearable device (200) can display the content (e.g., the task-related content) by fixing it to the general content including the task-related attributes. If the general content includes environment-related attributes, the wearable device (200) can display the content by avoiding the general content including the environment-related attributes. As an example, as illustrated in FIG. 10A, the wearable device (200) can recognize a tree. The tree may include environment-related attributes. The wearable device (200) can avoid trees and display a status bar (67a), a task window A (67b), a task bar (67c), and a task window B (65).

[0111] Referring to FIG. 10b, an image (93b) that changes according to a change in the user's gaze direction is illustrated. For example, if the user's gaze direction changes to the right, the wearable device (200) may display an image (93b) corresponding to the changed viewing angle. The content may include a lock attribute. For example, the status bar (67a), task window A (67b), and task bar (67c) may include a user lock attribute, and task window B (65) may include an environment lock attribute.

[0112] The wearable device (200) can move content including user lock attributes while maintaining the user's gaze relative to the environment (or background) according to a change in the user's gaze direction, and can maintain the position of content including environment lock attributes relative to the environment. As an example, the wearable device (200) can move a status bar (67a), a task window A (67b), and a task bar (67c) including user lock attributes relative to the environment in response to a change in the user's gaze direction, and can maintain the position of a task window B (65) including environment lock attributes relative to the environment. For example, the wearable device (200) can group the status bar (67a), the task window A (67b), and the task bar (67c) into one group, and move the group. For example, since task window B (65) is fixed to the environment (or background), when the user's gaze direction changes to the right, it may be displayed to the left of the position indicated in the image (93a) illustrated in FIG. 10a on the display (210) according to the movement of the environment. Since the status bar (67a), task window A (67b), and task bar (67c) are fixed to the user, they may be moved while maintaining the viewing angle based on the environment in response to the change in the user's gaze direction. The status bar (67a), task window A (67b), and task bar (67c) may be displayed at positions similar to the positions indicated in the image (93a) illustrated in FIG. 10a on the display (210).

[0113] FIGS. 11a, 11b, and 11c are diagrams illustrating content displayed according to a user's posture and gaze direction according to various embodiments.

[0114] Referring to FIG. 11A, content (e.g., task-related content) displayed in a standing posture is illustrated. For example, the side of the user may be in the x-axis direction, the front may be in the y-axis direction, and the vertical direction may be in the z-axis direction. The user may wear a wearable device (200). For example, when the user stands and looks straight ahead, the wearable device (200) may display first content (69a) and second content (69b) in the front direction (e.g., +y-axis direction). As an example, the content may include a window, a bar, and / or a pop-up. The gaze direction of the user wearing the wearable device (200) may be directed upward or downward. In other words, the wearable device (200) may be rotated about the x-axis.

[0115] Content may include a lock attribute. For example, the lock attribute may include a user lock or an environment lock. For example, if the first content (69a) and the second content (69b) include an environment lock attribute, the first content (69a) and the second content (69b) may be displayed fixed to the environment (or background) regardless of the movement of the user's gaze direction. If the first content (69a) includes a user lock attribute and the second content (69b) includes an environment lock attribute, the first content (69a) may move while maintaining a constant viewing angle relative to the environment depending on the movement of the user's gaze direction, and the second content (69b) may be displayed fixed to the environment.

[0116] For example, as illustrated in FIG. 11a, if the first content (69a) and the second content (69b) include user lock attributes, the first content (69a) and the second content (69b) can be moved while maintaining a constant viewing angle based on the environment according to the movement of the user's gaze direction. The first content (69a) and the second content (69b) can be grouped, and the first content (69a) and the second content (69b) can be moved in a grouped state.

[0117] Referring to FIG. 11B, content displayed in a sitting position is illustrated. When a user is seated and looking straight ahead, the wearable device (200) can display first content (69a) and second content (69b) in a frontal direction (e.g., along the +y axis). For example, when a user is seated, the locations where the first content (69a) and second content (69b) are displayed may be shifted in the -z axis direction compared to the locations where they are displayed when the user is standing.

[0118] The gaze direction of a user wearing a wearable device (200) may be directed upward or downward. In other words, the wearable device (200) may be rotated around the x-axis.

[0119] For example, if the first content (69a) and the second content (69b) include an environment lock attribute, the first content (69a) and the second content (69b) can be displayed in a state fixed to the environment (or background) regardless of the movement of the user's gaze direction. If the first content (69a) includes a user lock attribute and the second content (69b) includes an environment lock attribute, the first content (69a) can move while maintaining a constant viewing angle based on the environment according to the movement of the user's gaze direction, and the second content (69b) can be displayed in a state fixed to the environment.

[0120] For example, as illustrated in FIG. 11b, if the first content (69a) and the second content (69b) include user lock attributes, the first content (69a) and the second content (69b) can be moved while maintaining a constant viewing angle based on the environment according to the movement of the user's gaze direction. The first content (69a) and the second content (69b) can be grouped, and the first content (69a) and the second content (69b) can be moved in a grouped state.

[0121] Referring to FIG. 11C, content displayed in a lying position is illustrated. When the user lies down and looks straight ahead, the wearable device (200) can display first content (69a) and second content (69b) in a frontal direction (e.g., along the +z axis). For example, when the user lies down, the location where the first content (69a) and second content (69b) are displayed may be the upper surface of the boundary.

[0122] The gaze direction of a user wearing a wearable device (200) may be directed upward or downward. In other words, the wearable device (200) may be rotated around the x-axis.

[0123] For example, if the first content (69a) and the second content (69b) include an environment lock attribute, the first content (69a) and the second content (69b) can be displayed in a state fixed to the environment (or background) regardless of the movement of the user's gaze direction. If the first content (69a) includes a user lock attribute and the second content (69b) includes an environment lock attribute, the first content (69a) can move while maintaining a constant viewing angle based on the environment according to the movement of the user's gaze direction, and the second content (69b) can be displayed in a state fixed to the environment.

[0124] For example, as illustrated in FIG. 11c, if the first content (69a) and the second content (69b) include user lock attributes, the first content (69a) and the second content (69b) can be moved while maintaining a constant viewing angle based on the environment according to the movement of the user's gaze direction. The first content (69a) and the second content (69b) can be grouped, and the first content (69a) and the second content (69b) can be moved in a grouped state.

[0125] FIG. 12 is a drawing illustrating content displayed according to a user's posture according to various embodiments.

[0126] Referring to FIG. 12, first content (69a) and second content (69b) displayed in a standing and sitting posture are illustrated. A user can wear a wearable device (200). For example, if a user is standing and looking straight ahead, the wearable device (200) can display the first content (69a) and second content (69b) in a frontal direction (e.g., +y-axis direction). A standing user can change their posture to a sitting posture. In other words, the wearable device (200) can move vertically in the -z-axis direction.

[0127] For example, if the first content (69a) and the second content (69b) include an environment lock attribute, the first content (69a) and the second content (69b) can be displayed in a fixed state to the environment (or background) regardless of a change in the user's posture. If the first content (69a) includes a user lock attribute and the second content (69b) includes an environment lock attribute, the first content (69a) can move while maintaining a constant viewing angle based on the environment depending on a change in the user's posture, and the second content (69b) can be displayed in a fixed state to the environment.

[0128] For example, as illustrated in FIG. 12, if the first content (69a) and the second content (69b) include a user lock attribute, the first content (69a) and the second content (69b) can be moved in the z-axis direction while maintaining a constant viewing angle based on the environment according to a change in the user's posture. The first content (69a) and the second content (69b) can be grouped, and the first content (69a) and the second content (69b) can be moved in a grouped state.

[0129] FIG. 13 is a diagram illustrating content displayed according to repeated user postures according to various embodiments.

[0130] Referring to FIG. 13, the first content (71a) and the second content (71b) displayed in a standing posture and a sitting posture are illustrated. As an example, as described in FIG. 12, if the first content (69a) and the second content (69b) include a user lock attribute, the first content (69a) and the second content (69b) can be moved in the -z-axis direction while maintaining a constant viewing angle based on the environment according to a change in the user's posture. The first content (69a) and the second content (69b) can be grouped, and the first content (69a) and the second content (69b) can be moved in a grouped state. As an example, if the first content (69a) and the second content (69b) include an environment lock attribute, the first content (69a) and the second content (69b) can be displayed in a fixed state to the environment (or background) regardless of a change in the user's posture.

[0131] For example, if the first content (69a) includes an environment lock attribute and the second content (69b) includes a user lock attribute, the first content (69a) is displayed in a fixed state relative to the environment, and the second content (69b) can move while maintaining a constant viewing angle relative to the environment, depending on changes in the user's posture. As an example, as illustrated in FIG. 13, when the user is in a standing posture, the first content (71a) can be displayed vertically (almost vertically) relative to the ground, and the second content (71b) can be displayed at a constant angle relative to the ground. When the user's posture changes to a sitting posture, the display position of the first content (71a) including the environment lock attribute can be maintained relative to the environment, and can be displayed by changing to an angle corresponding to the user's viewing angle according to the sitting posture. The second content (71b) can be moved in the -z-axis direction while maintaining the user's viewing angle relative to the environment. The wearable device (200) can adjust the angle of the second content (71b) based on the moved position of the second content (71b) and the shape of the generated boundary.

[0132] For example, a user may repeat a standing posture and a sitting posture. As an example, the wearable device (200) may determine that the user's posture is repeated if the same posture is repeated a preset number of times or more within a preset time. The wearable device (200) may display the first content (71a) including the environment lock attribute by fixing the angle at an intermediate range between a first angle displayed when standing and a second angle displayed when sitting. The wearable device (200) may display the second content (71b) including the user lock attribute by fixing the position at an intermediate range between a first position displayed when standing and a second position displayed when sitting and / or an intermediate range between a third angle displayed when standing and a fourth angle displayed when sitting.

[0133] FIGS. 14a, 14b, and 14c are drawings illustrating content displayed on a boundary according to various embodiments.

[0134] Referring to FIG. 14A, the shape of a boundary that changes according to a change in a user's posture is illustrated. For example, when the user is in a standing posture, the wearable device (200) can generate a capsule-shaped boundary (11a). The wearable device (200) can display content (73) (e.g., task-related content) in front of the user (or on the side of the boundary) when the user is in a standing posture. When the user is in a standing posture, the side of the capsule-shaped boundary (11a) can include a first height. When the user's posture changes to a sitting posture, the wearable device (200) can change the shape of the boundary to a shape corresponding to the user's posture. When the user is in a sitting posture, the wearable device (200) can change the shape of the boundary to a capsule-shaped boundary (11b) of a second height lower than the first height, and display the content (73) based on the boundary (11b) of the changed second height. As an example, the wearable device (200) can display content (73) in front of the user (or on the side of the boundary) even when the user is in a sitting position. As an example, since the height of the boundary is lowered when the user is in a sitting position, when the user's gaze angle (or direction) changes, the wearable device (200) can display content (73) in the upper and lower hemispherical areas of the capsule-shaped boundary (11b) of the second height.

[0135] Referring to FIGS. 14b and 14c, as an example, the shape and position of the content displayed on the capsule-shaped boundary (13) are illustrated.

[0136] Referring to FIG. 14b, the wearable device (200) can display content (74) (e.g., task-related content) on the side of the capsule-shaped boundary (13). As described in FIG. 14a, the content can be displayed on the side of the boundary corresponding to the user's gaze direction movement. For example, the content (74) can include task-related content, and the content can include a user lock attribute. The content including the user lock attribute can be moved while maintaining the field of view relative to the environment according to the user's gaze direction movement.

[0137] For example, when the user's gaze direction moves upward, the wearable device (200) may display content including a user lock attribute in the side area (74a) of the upper hemisphere. As an example, the content (74) may include content including an environment lock attribute displayed in the side area (74a) of the upper hemisphere. When the user's gaze direction is directed forward, the content including an environment lock attribute displayed in the side area (74a) of the upper hemisphere may not be displayed in the user's field of view (or screen area). When the user's gaze direction moves upward, the content including an environment lock attribute located in the side area (74a) of the upper hemisphere may be displayed in the screen area.

[0138] For example, when the user's gaze direction moves downward, the wearable device (200) may display content including a user lock attribute in the side area (74b) of the lower hemisphere. As an example, the content (74) may include content including an environment lock attribute displayed in the side area (74b) of the lower hemisphere. When the user's gaze direction is directed forward, the content including an environment lock attribute displayed in the side area (74b) of the lower hemisphere may not be displayed in the user's field of view (or screen area). When the user's gaze direction moves downward, the content including an environment lock attribute located in the side area (74b) of the lower hemisphere may be displayed in the screen area.

[0139] Referring to FIG. 14c, a drawing of a capsule-shaped boundary (13) viewed from above is illustrated. For example, the side surface of the capsule-shaped boundary (13) may be curved. When the wearable device (200) displays content (74) on the side surface of the capsule-shaped boundary (13), the content (74) may be displayed in a curved shape. The upper and / or lower hemispheres of the capsule-shaped boundary (13) may also be curved. When the wearable device (200) displays content (74) on the upper and / or lower hemispheres of the capsule-shaped boundary (13), the content (74) may be displayed in a curved shape.

[0140] FIG. 15 is a drawing illustrating content displayed on a rectangular solid boundary according to various embodiments.

[0141] Referring to Fig. 15, a rectangular solid-shaped boundary (14) is illustrated. Each face of the rectangular solid-shaped boundary (14) may be a plane. When the wearable device (200) displays content (75) (e.g., task-related content) on the face of the rectangular solid-shaped boundary (14), the content (75) may be displayed in a flat shape.

[0142] For example, the wearable device (200) can change the size of the content (75) to display the content (75) on one side of a rectangular parallelepiped boundary (14). As an example, as illustrated in FIG. 15, the wearable device (200) can change the size of the content (75) so that it is displayed on the entire area of ​​one side of the rectangular parallelepiped boundary (14). As an example, when at least one content is positioned on one side of the content (75) displaying the content, the wearable device (200) can change the size of the content (75) so that the content (75) is displayed on a partial area of ​​one side of the rectangular parallelepiped boundary (14) by avoiding the at least one content.

[0143] FIGS. 16a, 16b, and 16c are drawings illustrating content displayed according to changes in boundaries according to various embodiments.

[0144] Referring to FIG. 16A, an example of changing between a cylindrical boundary (15) and a capsule-shaped boundary (16) is illustrated. For example, the wearable device (200) can change the shape of the boundary based on the usage environment, the number of contents, and / or the user's posture. For example, when the number of contents is less than or equal to a first number (e.g., 1) when the user is in a standing or sitting posture, the wearable device (200) can generate a cylindrical boundary (15). When the number of contents exceeds the first number, the wearable device (200) can generate a capsule-shaped boundary (16). When the number of contents changes, the wearable device (200) can change the shape of the boundary.

[0145] The top view of the cylindrical boundary (15) and the top view of the capsule-shaped boundary (16) may be circular. The content (77a) displayed on the side of the cylindrical boundary (15) and the content (77b) displayed on the side of the capsule-shaped boundary (16) may have the same curved shape. The side view of the cylindrical boundary (15) may have a square shape, and the side view of the capsule-shaped boundary (16) may have a shape including upper and lower hemispherical regions and a central region of a plane. The content (78a, 78c) displayed on the upper and lower surfaces of the cylindrical boundary (15) may have a flat shape, and the content (79a, 79c) displayed on the upper and lower hemispherical regions of the capsule-shaped boundary (16) may have a curved shape.

[0146] For example, the form of the displayed content may also change depending on the change in the boundary. For example, when a cylindrical boundary (15) is changed to a capsule-shaped boundary (16), the contents (78a, 78c) displayed in a flat form on the upper and lower surfaces of the cylindrical boundary (15) may be changed to contents (79a, 79c) in which curves are applied to the upper and lower hemispheres of the capsule-shaped boundary (16). For example, when a capsule-shaped boundary (16) is changed to a cylindrical boundary (15), the contents (79a, 79c) in which curves are applied to the upper and lower hemispheres of the capsule-shaped boundary (16) may be changed to contents (78a, 78c) displayed in a flat form on the upper and lower surfaces of the cylindrical boundary (15). As an example, when content (78a, 78c) in a flat shape (or content (79a, 79c) with a curve applied) is changed to content (79a, 79c) with a curve applied (or content (78a, 78c) in a flat shape), the wearable device (200) can temporarily display content (31) without a curve applied so that the user recognizes that the shape of the content is naturally changed.

[0147] Referring to FIG. 16B, an example of changing between a cylindrical boundary (18) and a rectangular parallelepiped boundary (19) is illustrated. For example, when the number of contents is less than or equal to a first number (e.g., 1) while the user is standing or sitting, the wearable device (200) may generate a cylindrical boundary (18). When a plane is detected in the AR environment and the number of contents is less than or equal to the first number, the wearable device (200) may generate a rectangular parallelepiped boundary (19). When the usage environment changes or a plane is not detected, the wearable device (200) may change the shape of the boundary.

[0148] A top view of a cylindrical boundary (18) may be circular, and a top view of a rectangular parallelepiped boundary (19) may be square. Contents (81a, 81b, 81c) displayed on the side of the cylindrical boundary (18) may have a curved shape, and contents (82a, 82b, 82c) displayed on the side of the rectangular parallelepiped boundary (19) may be flat. A side view of the cylindrical boundary (18) and the rectangular parallelepiped boundary (19) may be square. Contents (84a, 84c) displayed on the upper and lower surfaces of the cylindrical boundary (18) and contents (85a, 85c) displayed on the upper and lower surfaces of the rectangular parallelepiped boundary (19) may be flat.

[0149] For example, the form of the displayed content may also change depending on the change in the boundary. For example, if a cylindrical boundary (18) is changed to a rectangular parallelepiped boundary (19), the content (81a, 81c) displayed in a form in which a curve is applied to the side of the cylindrical boundary (18) may be changed to a flat content (82a, 82c) displayed on the left and right sides of the rectangular parallelepiped boundary (19). For example, if a rectangular parallelepiped boundary (19) is changed to a cylindrical boundary (18), the flat content (82a, 82c) displayed in a form in which a curve is applied to the side of the cylindrical boundary (15). As an example, when content (81a, 81c) with a curve applied (or content (82a, 82c) in a flat shape) is changed to content (82a, 82c) in a flat shape (or content (81a, 81c) with a curve applied), the wearable device (200) can temporarily display content (33) with no curve applied so that the user recognizes that the shape of the content is naturally changed.

[0150] Referring to FIG. 16c, an example of changing between a capsule-shaped boundary (21) and a rectangular parallelepiped boundary (22) is illustrated. For example, if the user is in a lying position or the number of contents exceeds a first number (e.g., 1), the wearable device (200) may generate a capsule-shaped boundary (21). If a plane is detected in the AR environment and the number of contents is equal to or less than the first number, the wearable device (200) may generate a rectangular parallelepiped boundary (22). If the usage environment, posture, and / or the number of contents change, the wearable device (200) may change the shape of the boundary.

[0151] The shape of the capsule-shaped boundary (21) when viewed from above (top view) may be circular, and the shape of the rectangular parallelepiped boundary (22) when viewed from above may be square. The contents (87a, 87b, 87c) displayed on the side of the capsule-shaped boundary (21) may have a curved shape, and the contents (88a, 88b, 88c) displayed on the side of the rectangular parallelepiped boundary (22) may have a flat shape. The shape of the capsule-shaped boundary (21) when viewed from the side (side view) may include the upper and lower hemispherical regions and the central region of the plane, and the shape of the rectangular parallelepiped boundary (22) when viewed from the side may be square. The contents (90a, 90c) displayed on the upper and lower hemispherical areas of the capsule-shaped boundary (21) may have a curved shape, and the contents (91a, 81c) displayed on the upper and lower surfaces of the rectangular solid-shaped boundary (22) may have a flat shape.

[0152] For example, the form of the displayed content may also change depending on the change in the boundary. For example, if a capsule-shaped boundary (21) is changed to a rectangular parallelepiped boundary (22), contents (87a, 87c) displayed in a form with curves applied to the sides of the capsule-shaped boundary (21) may be changed to contents (88a, 88c) in a flat form on the left and right sides of the rectangular parallelepiped boundary (22). Contents (90a, 90c) in a form with curves applied to the upper and lower hemispherical regions of the capsule-shaped boundary (21) may be changed to contents (91a, 91c) displayed in a flat form on the upper and lower sides of the rectangular parallelepiped boundary (22). As an example, when a rectangular solid boundary (22) is changed to a capsule-shaped boundary (21), flat contents (88a, 88c) on the left and right sides of the rectangular solid boundary (22) can be changed to contents (87a, 87c) displayed in a form in which a curve is applied to the side of the capsule-shaped boundary (21). Flat contents (91a, 91c) displayed in a form on the upper and lower sides of the rectangular solid boundary (22) can be changed to contents (90a, 90c) displayed in a form in which a curve is applied to the upper and lower hemispherical regions of the capsule-shaped boundary (21). As an example, when content (87a, 87c, 90a, 90c) with a curve applied (or content (88a, 88c, 91a, 91c) in a flat shape) is changed to content (88a, 88c, 91a, 91c) with a flat shape (or content (87a, 87c, 90a, 90c) with a curve applied), the wearable device (200) can temporarily display content (35, 37) without a curve applied so that the user recognizes that the shape of the content is naturally changed.

[0153] FIG. 17a and FIG. 17b are drawings explaining the offset of a virtual space according to various embodiments.

[0154] Referring to FIG. 17A, a boundary (24) generated centering on a user wearing a wearable device (200) is illustrated. As an example, the boundary (24) may be in the shape of a capsule. The size of the capsule-shaped boundary (24) may be set in consideration of the user (e.g., the height of the user). As an example, the height of the upper and lower hemispheres of the capsule-shaped boundary (24) may be set to approximately 1.8 m. The height of the cylindrical portion between the upper and lower hemispheres may be set to approximately 1 m. The total height of the capsule-shaped boundary (24) including the upper hemisphere, the lower hemisphere, and the cylindrical portion may be set to approximately 4.6 m.

[0155] Content (e.g., task-related content) may be displayed in a predetermined area including the surface of the boundary (24). The content displayed near the surface of the boundary (24) may be displayed at different distances from the user depending on the type. For example, task windows and / or applications may be displayed at a distance of about 1.3 m to about 2 m from the user. System-related content may be displayed at a distance of about 0.7 m from the user. When generating the boundary (24), the wearable device (200) may generate the boundary (24) by taking into account an offset for the user's visual convenience, operational convenience, and / or natural (or smooth) angle arrangement between contents (e.g., applications). As an example, as illustrated in FIG. 17A, the wearable device (200) may generate a boundary (24) such that the center point (24a) of the boundary is formed a certain distance behind the center point (24b) of the user's face (or the worn wearable device (200)). The distance between the center point (24a) of the boundary (24) and the center point (24b) of the user's face may be an offset of the boundary. As an example, the offset of the boundary may be set to approximately 0.5 m.

[0156] Referring to FIG. 17B, the offset of content displayed on the surface area of ​​the boundary (24) is illustrated. For example, the gaze of a user wearing a wearable device (200) may be directed to a point (26b) slightly below the horizontal point (26a). When the central region of the content displayed on the surface area of ​​the boundary (24) is displayed at a point (26b) slightly below the horizontal point (26a) of the user's face, the user can comfortably view the content. The wearable device (200) may display the content such that the central region of the content is located at a point (26b) slightly below the horizontal point (26a) of the user's gaze. The angle between the horizontal point (26a) of the user's gaze and the point (246) where the central region of the content is located may be the offset of the content. As an example, the offset of the content may be set to approximately -10 degrees.

[0157] FIGS. 18A, 18B, and 18C are diagrams illustrating the operation of a wearable device when a user's position and / or gaze direction is moved according to various embodiments.

[0158] Referring to FIG. 18A, a wearable device (200) can create a boundary centered on a user. The user can be positioned within the boundary area (41). The wearable device (200) can display content (93) near the surface of the boundary. The wearable device (200) can set a lock attribute to the content (93). For example, the wearable device (200) can set a user lock attribute to the content (93). The content (93) set as a user lock can be fixed based on the user. The user can move outside the boundary area (41) or change the direction of his / her gaze.

[0159] Referring to FIG. 18B, an example in which a user has moved a first distance from a current location is illustrated. The area (42) to which the user has moved may be an area including an existing boundary area (41) and an overlapping area (41a). For example, if the radius of the boundary (or the distance to the boundary of the boundary) is approximately 1 m and the user has moved less than approximately 2 m, the area (42) to which the user has moved and the existing boundary area (41) may include an overlapping area (41a). If the area (42) to which the user has moved and the existing boundary area (41) include an overlapping area (41a), the wearable device (200) may maintain the position of the content (93) for a certain period of time.

[0160] As described above, the wearable device (200) can move the content (93) including the user lock attribute while maintaining the field of view based on the environment according to the movement of the user. However, if the movement distance of the user is less than a certain distance, the user may have temporarily left the boundary area (41). If the user has temporarily left the boundary area (41), the user may return within the boundary area (41). Considering the possibility that the user may return within the boundary area (41), the wearable device (200) can maintain the current position of the content (93) including the user lock attribute for a certain period of time. If the user does not return within the boundary area (41) even after the certain period of time, the wearable device (200) can create a boundary based on the moved position or move the position of the content (93) including the user lock attribute based on the environment.

[0161] As an example, a user can maintain a position and move the gaze direction. Since the user can temporarily move the gaze direction, the wearable device (200) can maintain the current position of the content (93) including the user lock attribute for a certain period of time. If the user's gaze direction does not return even after a certain period of time, the wearable device (200) can move the position of the content (93) including the user lock attribute based on the user's gaze direction, based on the environment.

[0162] Referring to FIG. 18c, an example in which a user moves a second distance from a current location is illustrated. The area (43) to which the user has moved may be an area outside an existing boundary area (41). For example, if the radius of the boundary (or the distance to the boundary of the boundary) is about 1 m and the user moves about 3 m, the area (43) to which the user has moved may be outside an existing boundary area (41). If the area (44) to which the user has moved is outside an existing boundary area (41), the wearable device (200) may provide a notification to the user. For example, the wearable device (200) may display a notification pop-up and / or a notification message on the display (210), or output a notification sound and / or a notification voice through a speaker (e.g., the audio output module (155) of FIG. 1).

[0163] If the user returns to the existing boundary area (41), the wearable device (200) can maintain the current location of the content (93). If the user does not return to the existing boundary area (41), the wearable device (200) can create a boundary based on the moved location. For example, when the wearable device (200) is used in a VR environment, the wearable device (200) can set a protection area considering the user's safety. The wearable device (200) can create a boundary considering the protection area in the VR environment. The wearable device (200) can place the content (94) (e.g., content including a lock attribute) based on the created boundary. For example, the wearable device (200) can place the content (94) in an area designated by the user.

[0164] As an example, the wearable device (200) can store information about the generated boundary and information about the displayed content. For example, the boundary information can include information about the shape and / or size of the boundary. The content information can include information about the type and / or display location of the displayed content. The wearable device (200) can be turned off or switched to a standby state due to non-use by the user or for a preset period of time. Thereafter, when the wearable device (200) is re-enabled (e.g., turned on, switched to an active state), the wearable device (200) can determine the usage environment (e.g., AR environment, VR environment) and / or the surrounding environment (e.g., outdoors, indoors). If the re-enabled environment (e.g., usage environment and / or surrounding environment) is the same as (or similar to) the environment before the operation, the wearable device (200) can display the content based on the stored boundary information and the displayed content information.

[0165] Figure 19 is a flowchart illustrating a method of displaying content according to various embodiments.

[0166] In the following examples, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, and at least two operations may be performed in parallel.

[0167] According to one embodiment, steps 1910 to 1930 may be understood to be performed in a processor (e.g., processor (230) of FIG. 2) of an electronic device (e.g., electronic device (200) of FIG. 2).

[0168] Referring to FIG. 19, the wearable device (200) can determine the usage environment of the wearable device (200) displaying content (1910). For example, the usage environment may include an AR environment, a VR environment, an XR environment, and / or an MR environment.

[0169] The wearable device (200) can generate a boundary based on at least one of the number of contents and the usage environment (1920). For example, the boundary can include a preset area surrounding a user wearing the wearable device (200) or the wearable device (200). The wearable device (200) can display contents on the boundary surface. The wearable device (200) can move contents including a lock attribute on the boundary surface according to the user's posture. For example, the preset lock attribute can include a user lock attribute or an environment lock attribute.

[0170] For example, if the use environment is an AR environment and a plane is detected in the real environment, the wearable device (20) may generate a rectangular parallelepiped boundary. For example, if the use environment is a VR environment and the number of contents is less than or equal to a first preset number, the wearable device (200) may generate a cylindrical boundary. For example, the first number may be 1. As an example, if the wearable device (200) is a single-task environment, the wearable device (200) may generate a cylindrical boundary. For example, if the use environment is a VR environment and the number of contents exceeds the first preset number, the wearable device (200) may generate a capsule-shaped boundary. As an example, if the wearable device (200) is a multi-task environment, the wearable device (200) may generate a capsule-shaped boundary.

[0171] The wearable device (200) can display content arranged based on the shape of the boundary (1930). For example, if the usage environment is a VR environment, the wearable device (200) can determine the properties of the content included in the content as task-related properties or environment-related properties. The wearable device (200) can display the content by avoiding the content including the environment-related properties. As an example, the wearable device (200) can display the content by fixating it to the content including the task-related properties. For example, if the generated boundary is a rectangular solid-shaped boundary, the wearable device (200) can change the size of the content to display the content in the entire area of ​​one side of the rectangular solid-shaped boundary. As an example, the wearable device (200) can change the size of the content by considering the content so that the content can be displayed by avoiding the content located on one side of the boundary.

[0172] The wearable device (200) can change the boundary. For example, if the number of contents in an AR environment exceeds a preset first number or the number of contents changes in a VR environment, the wearable device (200) can change the boundary generated based on the number of contents. For example, if the first boundary changes to a second boundary, the wearable device (200) can change the shape of the first content displayed in a curved shape on the curved surface of the first boundary to a flat shape on the plane of the second boundary. As an example, the wearable device (200) can change the shape of the first content displayed in a flat shape on the plane of the first boundary to a curved shape on the curved surface of the second boundary and display it. As an example, when changing the shape of the first content and displaying it, the wearable device (200) can temporarily display the first content without a curve applied.

[0173] For example, the wearable device (200) can set control content, content being worked on, frequently used content determined based on the user's usage pattern, task-related content set by the user, and / or notification-related content as attributes of the user lock. The wearable device (200) can set content other than content including the attribute of the user lock as an attribute of the environment lock. The wearable device can fix the content including the attribute of the user lock with respect to the user. The wearable device (200) can move the content including the attribute of the user lock in response to the user's movement and display it while maintaining the viewing angle with respect to the environment. For example, when there are multiple contents including the attribute of the user lock, the wearable device (200) can group the multiple contents into one group. The wearable device (200) can move the group with respect to the environment according to the user's movement.

[0174] As an example, a wearable device may include a display, at least one processor including a processing circuit, and a memory storing instructions to be executed by the at least one processor, individually or collectively. The at least one processor may be configured to individually or collectively execute the stored instructions and determine a usage environment of the wearable device for displaying content. The at least one processor may be configured, individually or collectively, to cause the wearable device to generate a boundary based on at least one of the number of contents and the usage environment. The at least one processor may be configured, individually or collectively, to cause the wearable device to display the task content arranged based on the shape of the boundary through the display. The above boundary may include a virtual space that includes a preset area surrounding a user wearing the wearable device or at least a part of the user or the wearable device based on the wearable device, displays the content on the surface of the boundary, and moves the content including a lock attribute on the surface according to the posture of the user.

[0175] As an example, the at least one processor may be individually or collectively configured to cause the wearable device to generate a rectangular parallelepiped boundary when the use environment is an augmented reality (AR) environment and a plane is detected in the actual environment. The at least one processor may be individually or collectively configured to cause the wearable device to generate a cylindrical boundary when the use environment is a virtual reality (VR) environment and the number of contents is equal to or less than the first preset number. The at least one processor may be individually or collectively configured to cause the wearable device to generate a capsule-shaped boundary when the use environment is a virtual reality (VR) environment and the number of contents exceeds the first preset number.

[0176] As an example, the at least one processor may be configured, individually or collectively, to cause the wearable device to change a boundary generated based on the number of contents when the number of contents in the AR environment exceeds the first preset number or when the number of contents in the VR environment changes.

[0177] As an example, the at least one processor may be individually or collectively configured to cause the wearable device to, when the first boundary is changed to the second boundary, change the shape of the first content displayed in a curved shape on the curved surface of the first boundary to a planar shape on the plane of the second boundary, or change the shape of the first content displayed in a planar shape on the plane of the first boundary to a curved shape on the curved surface of the second boundary and display it on the display. The at least one processor may be individually or collectively configured to cause the wearable device to, when changing the shape of the first content and displaying it, temporarily display the first content without a curve applied on the display.

[0178] As an example, the at least one processor may be individually or collectively configured to cause the wearable device to change the size of the content and display the content in an area of ​​one side of the rectangular parallelepiped boundary, if the generated boundary is a rectangular parallelepiped boundary.

[0179] As an example, the at least one processor may be individually or collectively configured to cause the wearable device to determine, if the usage environment is a VR environment, an attribute of the second content as a task-related attribute or an environment-related attribute. The at least one processor may be individually or collectively configured to cause the wearable device to avoid second content including the environment-related attribute and display the content at a location other than a location where the second content is displayed, or to fixate the content to the second content including the task-related attribute and display the content within or around the second content.

[0180] As an example, the lock attribute may include an attribute of a user lock. The at least one processor may be individually or collectively configured to cause the wearable device to set at least one of control content, content being worked on, frequently used content determined based on the user's usage pattern, content set by the user, and notification-related content as an attribute of the user lock.

[0181] As an example, the at least one processor may be individually or collectively configured to cause the wearable device to fix content including attributes of the user lock relative to the user. The at least one processor may be individually or collectively configured to cause the wearable device to move content including attributes of the user lock relative to the environment in response to movement of the user and display it on the display while maintaining a viewing angle.

[0182] As an example, the at least one processor may be individually or collectively configured to cause the wearable device to group a plurality of contents, if multiple contents include the attribute of the user lock, into a single group. The at least one processor may be individually or collectively configured to cause the wearable device to move the group based on the environment according to the movement of the user.

[0183] As an example, the wearable device may further include a sensor. The at least one processor may be individually or collectively configured to cause the wearable device to determine the user's posture based on a signal detected by the sensor. The at least one processor may be individually or collectively configured to cause the wearable device to, when the user's posture is repeated between a first posture and a second posture, display the content at an intermediate position between a first position of the content corresponding to the first posture and a second position of the content corresponding to the second posture.

[0184] As an example, a method for displaying content on a wearable device may include an operation for determining a usage environment of the wearable device displaying the content. The method may include an operation for generating a boundary based on at least one of the number of contents and the usage environment. The method may include an operation for displaying the content arranged based on the shape of the boundary. The boundary may include a virtual space that includes a preset area based on a user wearing the wearable device or the wearable device, displays the content on a surface of the boundary, and moves the content including a lock attribute on the surface according to the posture of the user.

[0185] As an example, the operation of generating the boundary may generate a rectangular parallelepiped boundary when the use environment is an augmented reality (AR) environment and a plane is detected in the actual environment, generate a cylindrical boundary when the use environment is a virtual reality (VR) environment and the number of contents is equal to or less than the first preset number, and generate a capsule-shaped boundary when the use environment is a virtual reality (VR) environment and the number of contents exceeds the first preset number.

[0186] As an example, the method may further include an operation of changing a boundary generated based on the number of contents when the number of contents in the AR environment exceeds the first preset number or the number of contents in the VR environment changes.

[0187] As an example, the method may further include an operation of changing the shape of the first content displayed in a curved shape on the surface of the first boundary to a planar shape on the plane of the second boundary when the first boundary is changed to a second boundary, or changing the shape of the first content displayed in a planar shape on the plane of the first boundary to a curved shape on the surface of the second boundary and displaying it. The operation of changing and displaying may temporarily display the first content without a curve applied when changing and displaying the shape of the first content.

[0188] As an example, the operation of displaying the content may further include an operation of changing the size of the content and displaying the content in an area of ​​one side of the rectangular parallelepiped boundary, if the generated boundary is a rectangular parallelepiped boundary.

[0189] As an example, the method may further include an operation of determining the properties of the second content as task-related properties or environment-related properties if the usage environment is a VR environment. The operation of displaying the content may display the content in a location other than the location where the second content is displayed by avoiding the second content including the environment-related properties, or may display the content by fixing it to the second content including the task-related properties.

[0190] As an example, the lock property may include a property of a user lock. The method may further include an operation of setting at least one of control content, content being worked on, frequently used content determined based on the user's usage pattern, content set by the user, and notification-related content as a property of the user lock.

[0191] As an example, the method may further include an operation of fixing content including the attributes of the user lock relative to the user, and displaying the content including the attributes of the user lock while maintaining the viewing angle by moving the content including the attributes of the user lock relative to the environment in response to the movement of the user.

[0192] As an example, if there are multiple contents including the attribute of the user lock, the method may further include an operation of grouping the multiple contents into one group and moving the group based on the environment according to the movement of the user.

[0193] As an example, instructions of a non-transitory computer-readable storage medium having recorded thereon a program for performing a method of displaying content on a wearable device may perform an operation of determining a usage environment of the wearable device displaying the content. The instructions may perform an operation of generating a boundary based on at least one of the number of contents and the usage environment. The instructions may perform an operation of displaying the content arranged based on the shape of the boundary. The boundary may include a preset area surrounding a user wearing the wearable device or at least a portion of the user or the wearable device based on the wearable device, and may include a virtual space for displaying the content on a surface of the boundary and moving the content including a lock attribute on the surface according to the posture of the user.

[0194] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0195] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0196] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0197] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0198] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0199] While this disclosure has been described with reference to various embodiments, it should be understood that these various embodiments are illustrative and not limiting. Those skilled in the art will appreciate that various changes in form and detail may be made within the spirit and scope of this disclosure, including the appended claims and their equivalents. Furthermore, it should be understood that any of the embodiments described in this disclosure may be combined with other embodiments.

Claims

1. In wearable devices, display; At least one processor comprising a processing circuit; and A memory storing instructions that are individually or collectively executed by at least one processor; The at least one processor, individually or collectively, executes the stored instructions and causes the wearable device to: Determine the usage environment of the wearable device displaying the content, Generate a boundary based on at least one of the number of contents and the usage environment, The content is set to be displayed through the display based on the shape of the above boundary, The above boundary is, A wearable device comprising a virtual space including a preset area surrounding at least a portion of a user or the wearable device based on a user wearing the wearable device or the wearable device, displaying the content on the surface of the boundary, and moving the content including a lock attribute on the surface according to the posture of the user.

2. In paragraph 1, The at least one processor, individually or collectively, causes the wearable device to: If the above usage environment is an AR (augmented reality) environment and a plane is detected in the actual environment, a rectangular prism-shaped boundary is created, If the above usage environment is a VR (virtual reality) environment and the number of contents is less than or equal to the first preset number, a cylindrical boundary is created. A wearable device configured to generate a capsule-shaped boundary when the above-mentioned usage environment is a VR (virtual reality) environment and the number of contents exceeds a preset first number.

3. In paragraph 2, The at least one processor, individually or collectively, causes the wearable device to: A wearable device configured to change a boundary generated based on the number of contents when the number of contents in the AR environment exceeds the first preset number or the number of contents in the VR environment changes.

4. In paragraph 3, The at least one processor, individually or collectively, causes the wearable device to: When the first boundary is changed to the second boundary, the shape of the first content displayed in a curved shape on the surface of the first boundary is changed to a flat shape on the plane of the second boundary, or the shape of the first content displayed in a flat shape on the plane of the first boundary is changed to a curved shape on the surface of the second boundary and displayed on the display. A wearable device configured to display the first content with the curve temporarily unapplied on the display when changing the form of the first content.

5. In paragraph 1, The at least one processor, individually or collectively, causes the wearable device to: A wearable device configured to display the content in an area of one side of the rectangular parallelepiped boundary by changing the size of the content, if the generated boundary is a rectangular parallelepiped boundary.

6. In paragraph 1, The at least one processor, individually or collectively, causes the wearable device to: If the above usage environment is a VR environment, the properties of the second content are judged as task-related properties or environment-related properties, A wearable device configured to display the content in a location other than a location where the second content is displayed by avoiding the second content including the environment-related attribute, or to display the content within the second content or around the second content by fixing the second content including the task-related attribute.

7. In paragraph 1, The above lock properties include the properties of the user lock, The at least one processor, individually or collectively, causes the wearable device to: A wearable device configured to set at least one of control content, content being worked on, frequently used content determined based on the user's usage pattern, content set by the user, and notification-related content as an attribute of the user lock.

8. In paragraph 7, The at least one processor, individually or collectively, causes the wearable device to: A wearable device configured to fix content including the attributes of the user lock relative to the user, and to display the content including the attributes of the user lock on the display while maintaining the viewing angle by moving the content relative to the environment in response to the movement of the user.

9. In paragraph 8, The at least one processor, individually or collectively, causes the wearable device to: A wearable device configured to group multiple contents including the attributes of the user lock into one group and move the group based on the environment according to the movement of the user.

10. In paragraph 1, including sensors; The at least one processor, individually or collectively, causes the wearable device to: The user's posture is determined based on the signal detected through the above sensor, A wearable device configured to display the content at a middle position between the first position of the content corresponding to the first posture and the second position of the content corresponding to the second posture when the user's posture is repeated between the first posture and the second posture.

11. In a method of displaying content on a wearable device, An action for determining the usage environment of the wearable device displaying the content; An operation of generating a boundary based on at least one of the number of contents and the usage environment; and An operation of displaying the content arranged based on the shape of the boundary; The above boundary is, A method comprising a virtual space including a preset area surrounding a user wearing the wearable device or at least a part of the wearable device based on the wearable device, displaying the content on the surface of the boundary, and moving the content including a lock attribute on the surface according to the posture of the user.

12. In paragraph 11, The action of creating the above boundary is: If the above usage environment is an AR (augmented reality) environment and a plane is detected in the actual environment, a rectangular prism-shaped boundary is created, If the above usage environment is a VR (virtual reality) environment and the number of contents is less than or equal to the first preset number, a cylindrical boundary is created. A method for generating a capsule-shaped boundary when the above-mentioned usage environment is a VR (virtual reality) environment and the number of contents exceeds a preset first number.

13. In paragraph 12, A method further comprising: an operation of changing a boundary generated based on the number of contents when the number of contents in the AR environment exceeds the first preset number or the number of contents in the VR environment changes.

14. In paragraph 13, When the first boundary is changed to the second boundary, the operation of changing the shape of the first content displayed in a curved shape on the surface of the first boundary to a planar shape on the plane of the second boundary, or changing the shape of the first content displayed in a planar shape on the plane of the first boundary to a curved shape on the surface of the second boundary and displaying it; further includes; The action of changing and displaying the above is, A method for displaying the first content by changing the shape of the first content, wherein the first content is temporarily displayed without a curve applied.

15. In paragraph 11, The action of displaying the above content is: A method further comprising: if the generated boundary is a rectangular solid-shaped boundary, an operation of changing the size of the content and displaying the content in an area of one side of the rectangular solid-shaped boundary;

Images