Wearable electronic device and method for providing spatial sound of virtual space in wearable electronic device

Abstract

The present disclosure relates to a wearable electronic device comprising: a display; at least one sensor; at least one camera; a plurality of speakers; at least one processor; and a memory for storing instructions, wherein the instructions, when executed individually or collectively by the at least one processor, may instruct the wearable electronic device to: display a virtual object that generates audio in a virtual space through the display while the wearable electronic device is worn by a user; identify a first audio channel designated for providing spatial sound for the audio from among a plurality of audio channels; identify a center position designated for providing the spatial sound for the audio; identify a distance between the designated center position and a first position of the wearable electronic device in the virtual space; output the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers if the distance between the designated center position and the first position does not exceed a designated first distance; identify a second audio channel having fewer channels than the first audio channel if the distance between the designated center position and the first position exceeds the designated first distance; and output the audio as a second audio signal of the second audio channel through at least some of the plurality of speakers. Various other embodiments are possible.

Description

A wearable electronic device and a method for providing spatial acoustics of a virtual space in a wearable electronic device

[0001] The present disclosure relates to a wearable electronic device and a method for providing spatial acoustics of a virtual space in a wearable electronic device.

[0002] With the development of digital technology, electronic devices are being provided in various forms such as smartphones, tablet PCs, or PDAs. Electronic devices are also being developed as wearable electronic devices that can be worn by users to improve portability and user accessibility. Wearable electronic devices can be configured in various forms to be worn on parts of the user's body, and as technology advances, wearable electronic devices can provide services related to virtual reality technology that offers virtual environments (e.g., virtual reality (VR) space, augmented reality (AR) space, mixed reality (MR), or extended reality (XR)).

[0003] Wearable electronic devices can provide VR, AR, MR, or XR experiences to users by utilizing VR content, AR content, MR content, or XR content. With the advancement of technology, the boundaries between VR, AR, MR, or XR content are becoming blurred, and XR can be referred to as a concept that encompasses VR, AR, and MR. Various types of wearable electronic devices capable of providing XR content may be provided, and for example, wearable electronic devices may include AR glasses, VR headsets, HMD (head mounted display) devices, VST (video see-through) devices, or other wearable electronic devices.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0005] A wearable electronic device can display a virtual reality space (or virtual space) and a virtual object that generates audio in the virtual reality space using XR (or VR, or AR, or MR) content (or virtual object (e.g., virtual TV, virtual speaker, virtual laptop, application object, or other virtual object that generates audio)). The wearable electronic device can output an audio signal for the audio generated through the virtual object in the virtual reality space through the speaker of the wearable electronic device so that the user can listen to it.

[0006] A wearable electronic device can provide spatial acoustics for audio generated through virtual objects. Spatial acoustics may refer to outputting audio signals through multiple audio channels so that the user perceives the sound as originating from different locations in the virtual space. A wearable electronic device can provide spatial acoustics for audio generated through virtual objects in a virtual reality space, centered on the user's location. When a wearable electronic device provides spatial acoustics for audio generated through virtual objects in a virtual reality space centered on the user's location, the device provides spatial acoustics regardless of the distance between the virtual object and the user's location, which may result in a mismatch between the visual information about the virtual object that the user can see through the wearable electronic device's display and the spatial acoustics that can be heard through the wearable electronic device's speaker.

[0007] For example, if a user moves while listening to spatial audio generated by a virtual object with an HMD (or VST) device on their head, the spatial audio may need to change due to the change in distance between the user and the virtual object. If a wearable electronic device provides spatial audio centered on the user's location, the same spatial audio centered on the user's location will be provided regardless of whether the user moves away from or closer to the virtual object, potentially fixing the spatial sense of the sound. If a wearable electronic device can provide different spatial audio depending on the distance between the user and the virtual object generating audio in the virtual reality space, it would allow the user to hear the audio generated by the virtual object in the virtual reality space more realistically.

[0008] Furthermore, when a user listens to audio while wearing an HMD (or VST) device on their head and viewing the display screen of a virtual object generating audio in a virtual reality space, spatial sound can be provided regardless of the size of the virtual object's display screen. If the number of audio channels for spatial sound is increased to maximize the effect when the virtual object's display screen is large, and the number of audio channels is decreased to reduce the effect when the virtual object's display screen is small, it will be possible to make the user feel the audio emanating from the virtual object in the virtual reality space more realistically.

[0009] A wearable electronic device according to one embodiment of the present disclosure may include a display, at least one sensor, at least one camera, a plurality of speakers, at least one processor, and a memory for storing commands. When the commands are executed individually or collectively by the at least one processor, the wearable electronic device may display a virtual object that generates audio in a virtual space through the display while the wearable electronic device is worn by a user. When the commands are executed individually or collectively by the at least one processor, the wearable electronic device may identify a first audio channel designated for providing spatial acoustics for the audio among a plurality of audio channels. When the commands are executed individually or collectively by the at least one processor, the wearable electronic device may identify a central location designated for providing spatial acoustics for the audio. When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device may identify the distance between the designated center position and the first position of the wearable electronic device in the virtual space. When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device may output the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers if the distance between the designated center position and the first position does not exceed a designated first distance.When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device may identify a second audio channel different from the first audio channel when the distance between the designated center position and the first position exceeds the designated first distance, and output the audio as a second audio signal of the second audio channel through at least some of the plurality of speakers.

[0010] A method for providing spatial acoustics in a virtual space in a wearable electronic device according to one embodiment of the present disclosure may include an operation of displaying a virtual object that generates audio in a virtual space through a display of the wearable electronic device while the wearable electronic device is worn by a user. The method may include an operation of identifying a first audio channel designated for providing spatial acoustics for the audio among a plurality of audio channels. The method may include an operation of identifying a center position designated for providing spatial acoustics for the audio. The method may include an operation of identifying a distance between the designated center position and a first position of the wearable electronic device in the virtual space. The method may include an operation of outputting the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the distance between the designated center position and the first position does not exceed a designated first distance. The above method may include the operation of identifying a second audio channel different from the first audio channel when the distance between the designated center position and the first position exceeds the designated first distance, and outputting the audio as a second audio signal of the second audio channel through at least some of the plurality of speakers.

[0011] In a non-transient storage medium storing commands according to one embodiment of the present disclosure, the commands are configured to cause the electronic device to perform at least one operation when executed by the electronic device, wherein the at least one operation may include an operation of displaying a virtual object that generates audio in a virtual space through a display of the wearable electronic device while the wearable electronic device is worn by a user. The at least one operation may include an operation of identifying a first audio channel designated for providing spatial sound for the audio among a plurality of audio channels. The at least one operation may include an operation of identifying a center position designated for providing spatial sound for the audio. The at least one operation may include an operation of identifying a distance between the designated center position and a first position of the wearable electronic device in the virtual space. The method may include an operation of outputting the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the distance between the designated center position and the first position does not exceed a designated first distance. The above at least one operation may include identifying a second audio channel different from the first audio channel when the distance between the designated center position and the first position exceeds the designated first distance, and outputting the audio as a second audio signal of the second audio channel through at least some of the plurality of speakers.

[0012] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0013] FIG. 2a is a perspective view showing the structure of an electronic device according to one embodiment.

[0014] FIG. 2b is a diagram showing the structure of a display and an eye-tracking camera of an electronic device according to one embodiment.

[0015] FIG. 3a is a perspective view showing a different structure of a wearable electronic device according to one embodiment.

[0016] FIG. 3b is a perspective view showing the front structure of a wearable electronic device according to one embodiment.

[0017] FIG. 3c is a perspective view showing the rear structure of a wearable electronic device according to one embodiment.

[0018] FIG. 4 is a configuration diagram of a wearable electronic device according to one embodiment.

[0019] FIG. 5 is a diagram illustrating an application for providing spatial acoustics for audio in a virtual space in a wearable electronic device according to one embodiment.

[0020] FIG. 6 is a flowchart illustrating the operation of providing spatial sound in a virtual space in a wearable electronic device according to one embodiment.

[0021] FIG. 7 is a flowchart illustrating an operation of providing spatial sound according to the distance between a designated center position and a first position of the wearable electronic device, the distance between the designated center position and a position of a virtual object, and the size of the display screen of the virtual object in a wearable electronic device according to one embodiment.

[0022] FIG. 8 is a diagram illustrating a designated center position and a distance from a wearable electronic device to provide spatial acoustics for audio generated through a virtual object in a virtual space according to one embodiment.

[0023] FIG. 9a is a diagram showing a case where a user of a wearable electronic device according to one embodiment is located within a first distance range based on a center position for spatial acoustics.

[0024] FIG. 9b is a diagram showing a case where a user of a wearable electronic device according to one embodiment is located in a second distance range based on a center position for spatial acoustics.

[0025] FIG. 9c is a diagram showing a case where a user of a wearable electronic device according to one embodiment is located in a third distance range based on a center position for spatial acoustics.

[0026] FIG. 10a is a diagram showing a case where a wearable electronic device according to one embodiment is located at a first distance based on a center position for spatial acoustics.

[0027] FIG. 10b is a diagram showing a case where a wearable electronic device according to one embodiment is located at a second position based on a center position for spatial acoustics worn by the device.

[0028] FIG. 10c is a diagram showing a case where a wearable electronic device according to one embodiment is located at a third distance based on a center position for spatial acoustics.

[0029] FIG. 11 is a diagram showing audio channels according to the size of the display screen of a virtual object according to one embodiment.

[0030] FIG. 12a is a drawing showing a case in which the display size of a virtual object in a virtual space according to one embodiment is not smaller than the first size.

[0031] FIG. 12b is a drawing showing a case in which the display size of a virtual object in a virtual space according to one embodiment is smaller than the first size and not smaller than the second size.

[0032] FIG. 12c is a drawing showing a case where the display size of a virtual object in a virtual space according to one embodiment is a second size smaller.

[0033] FIG. 13 is a diagram illustrating an example in which a user of a wearable electronic device according to one embodiment is at a center position for spatial acoustics, and a first to third distance range is displayed on a display based on the center position.

[0034] FIG. 14 is a diagram showing an example in which an audio channel that changes according to the distance between a center position for spatial acoustics and a virtual object is displayed on a display of a wearable electronic device according to one embodiment.

[0035] FIG. 15 is a diagram showing an example in which audio channels according to the display screen size of a virtual object in a virtual space according to one embodiment are displayed on the display of a wearable electronic device.

[0036] FIG. 16 is a diagram illustrating an example of specifying multiple spatial acoustic ranges based on multiple virtual objects when multiple virtual objects exist in a virtual space mapped to a real space according to one embodiment.

[0037] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0038] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, identical or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity. The term "user" as used in the embodiments of the present disclosure may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0039] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0040] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0041] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0042] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0043] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0044] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0045] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0046] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0047] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0048] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0049] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0050] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0051] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0052] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0053] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0054] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0055] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0056] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0057] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0058] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0059] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0060] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0061] FIG. 2a is a perspective view showing the structure of a wearable electronic device according to one embodiment. FIG. 2b is a drawing showing the structure of a display and an eye-tracking camera of a wearable electronic device according to one embodiment.

[0062] Referring to FIGS. 2a and 2b, a wearable electronic device (200) according to one embodiment (e.g., electronic device (101) of FIG. 1, an electronic device (102 or 104) communicating with the electronic device (101) of FIG. 1, or the electronic device (101) of FIG. 1) may provide a service (or function) related to virtual reality technology that provides a virtual environment. The service related to virtual reality technology may include a virtual reality (VR) service, an augmented reality (AR) service, a mixed reality (MR) service, and / or an extended reality (XR) service.

[0063] A wearable electronic device (200) according to one embodiment may be implemented as AR glasses, a VR headset, a head-mounted display (HMD) device, a video see-through (VST) device, or other wearable electronic devices configured to be worn on a user's body, as shown in FIG. 2a. For example, the wearable electronic device (200) may be configured to combine with an external electronic device, such as a mobile device, and may utilize components of the external electronic device (e.g., a display module, a camera module, an audio output module, or other components). Not limited thereto, the wearable electronic device (200) may be implemented in various forms that can be worn on a user's body.

[0064] According to one embodiment, the wearable electronic device (200) may configure a virtual space (e.g., virtual reality space, augmented reality space, mixed reality space, or extended reality space) that displays a real image corresponding to the actual external environment captured in the surrounding environment where the user is located and / or a virtual image provided (e.g., 2D or 3D image). According to one embodiment, the wearable electronic device (200) may output an audio signal corresponding to the audio through an audio output module (155) while displaying an object that generates audio (or a virtual object (e.g., a virtual TV, a virtual speaker, a virtual laptop, an application object, or other virtual object that generates audio)) in the virtual space.

[0065] According to one embodiment, the wearable electronic device (200) may include a processor (120), memory (130), display module (160), sensor module (176), camera module (180), charging module (e.g., battery (189) of FIG. 1) and communication module (190) as shown in FIG. 1. The wearable electronic device (200) may further include an acoustic output device (155), an input module (150) as shown in FIG. 1, or other components as shown in FIG. 1. In addition, the wearable electronic device (200) may be configured to include other components necessary to provide a virtual reality service (or function), an augmented reality service (or function), a mixed reality service (or function), and / or an extended reality service (or function).

[0066] According to one embodiment, the processor (120) is electrically connected to other components and can control other components. The processor (120) can perform various data processing or operations in accordance with the execution of various functions (e.g., actions, services, or programs) provided by the wearable electronic device (200). The processor (120) can perform various data processing or operations to display at least one virtual object related to real objects included in an image captured in real space and / or a virtual object corresponding to a user (e.g., an avatar) in a virtual reality space. The processor (120) can perform various data processing or operations to express user interaction or movement of the virtual object displayed in the virtual reality space.

[0067] Again, with reference to FIG. 2a, an electronic device (200) according to one embodiment will be described. As described above, the electronic device (200) is not limited to a glasses-type (e.g., AR glasses) augmented reality device, and can be implemented as various devices capable of providing immersive content (e.g., content based on XR technology) to the user's eyes (e.g., AR head-mounted display type, 2D / 3D head-mounted display device or VR head-mounted display device).

[0068] According to one embodiment, a camera module of a wearable electronic device (200) (e.g., camera module (180) or camera circuit of FIG. 1) can capture still images and / or videos. According to one embodiment, the camera module may be placed within a lens frame and around a first display (251) and a second display (252). According to one embodiment, the camera module may include one or more first cameras (211-1, 211-2), one or more second cameras (212-1, 212-2), and one or more third cameras (213). According to one embodiment, images acquired through one or more first cameras (211-1, 211-2) may be used for detecting hand gestures by a user, tracking the user's head, and / or spatial recognition. One or more first cameras (211-1, 211-2) may be a GS (global shutter) camera or an RS (rolling shutter) camera. One or more first cameras (211-1, 211-2) can perform simultaneous localization and mapping (SLAM) operations through depth imaging. One or more first cameras (211-1, 211-2) can perform spatial recognition and / or motion recognition for 3DoF (depth of field) and / or 6DoF. According to one embodiment, the first cameras (211-1, 211-2) can periodically or non-periodically transmit information (e.g., trajectory information) related to the user's eyes (e.g., left eye and right eye) or the trajectory of the gaze (e.g., eye tracking) to a processor (e.g., processor (120) of FIG. 1).The first camera (211-1, 211-2) can be used to position the center of a virtual image projected onto a wearable electronic device (200) (e.g., AR glasses) according to the direction in which the user's pupil gazes, and a GS camera can be primarily used to detect the pupil and track rapid pupil movements. The first camera (211-1, 211-2) can be configured for the left eye and the right eye, respectively, and the first camera (211-1, 211-2) configured for the left eye and the right eye, respectively, may have the same performance and specifications.

[0069] According to one embodiment, the wearable electronic device (200) may use another camera (e.g., a third camera (213)) for hand detection and tracking and user gesture recognition. According to one embodiment, at least one of the first camera (211-1, 211-2) to the third camera (213) may be replaced with a sensor module (e.g., a LiDAR sensor). For example, the sensor module may include at least one of a vertical cavity surface emitting laser (VCSEL), an infrared sensor, and / or a photodiode.

[0070] According to one embodiment, images acquired through one or more second cameras (212-1, 212-2) may be used to detect and track the user's pupils. One or more second cameras (212-1, 212-2) may be eye tracking (ET) cameras, as shown in FIG. 2b. One or more second cameras (212-1, 212-2) may be GS cameras. One or more second cameras (212-1, 212-2) may correspond to the left eye and the right eye, respectively, and the performance of one or more second cameras (212-1, 212-2) may be substantially identical. One or more third cameras (213) may be relatively high-resolution cameras. One or more third cameras (213) may perform auto-focusing (AF) and optical image stabilization (OIS) functions. One or more third cameras (213) may be GS (global shutter) cameras or RS (rolling shutter) cameras. One or more third cameras (213) may be color cameras. One or more third cameras (213) may be high resolution cameras, referred to as HR (high resolution) or PV (photo video). Color cameras equipped with AF functions and optical image stabilization (OIS) functions for obtaining high-quality images may be primarily used. The first camera (211-1, 211-2) or one or more fourth cameras (not shown) may be FT (face tracking) cameras and may be used to detect and track a user's facial expressions. A depth sensor may be used to determine the distance to an object, such as with TOF. TOF (time of flight) is a technology that measures the distance of an object using a signal (near-infrared, ultrasound, or laser). TOF technology emits a signal from a transmitter and measures the signal at a receiver, and can measure the flight time of the signal.

[0071] According to one embodiment, the electronic device (200) may include one or more light-emitting elements (214-1, 214-2) (illumination). The light-emitting elements (214-1, 214-2) are different from the light source described below, which irradiates light onto a screen output area of ​​a display. According to one embodiment, the light-emitting elements (214-1, 214-2) may irradiate light to facilitate pupil detection in detecting and tracking a user's pupil through one or more second cameras (212-1, 212-2). According to one embodiment, the light-emitting elements (214-1, 214-2) may each include an LED. According to one embodiment, the light-emitting elements (214-1, 214-2) may irradiate light in the infrared region. According to various embodiments, a light-emitting element (214-1, 214-2) may be attached around the frame of a wearable electronic device (200). According to one embodiment, the light-emitting element (214-1, 214-2) may be positioned around one or more first cameras (211-1, 211-2) and may assist gesture detection, head tracking, and spatial recognition by one or more first cameras (211-1, 211-2) when the wearable electronic device (200) is used in a dark environment. According to one embodiment, the light-emitting element (214-1, 214-2) may be positioned around one or more third cameras (213) and may assist image acquisition by one or more third cameras (213) when the augmented reality device (200) is used in a dark environment.

[0072] According to one embodiment, the wearable electronic device (200) may include a battery (235-1, 235-2) (e.g., the battery (189) of FIG. 1). The battery (235-1, 235-2) may store power to operate the remaining components of the wearable electronic device (200).

[0073] According to one embodiment, a display of a wearable electronic device (200) (e.g., a display module (160) of FIG. 1) may include a first display (251), a second display (252), one or more input optical members (253-1, 253-2), one or more transparent members (290-1, 290-2), and one or more screen display portions (254-1, 254-2). According to one embodiment, the first display (251) and the second display (252) may be light output modules and may include, for example, a liquid crystal display (LCD), a digital mirror device (DMD), a liquid crystal on silicon (LCoS), an organic light emitting diode (OLED), or a micro light emitting diode (micro LED). According to one embodiment, if the first display (251) and the second display (252) are made of one of a liquid crystal display device, a digital mirror display device, or a silicon liquid crystal display device, the wearable electronic device (200) may include a light source that irradiates light onto the screen output area of ​​the display. According to one embodiment, if the first display (251) and the second display (252) can generate light themselves, for example, if they are made of one of an organic light-emitting diode or a micro LED, the wearable electronic device (200) may provide a user with a good quality virtual image (e.g., an image of a virtual reality space) without including a separate light source. In one embodiment, if the display is implemented as an organic light-emitting diode or a micro LED, a light source is unnecessary, so the electronic device may be made lighter.

[0074] According to one embodiment, one or more transparent members (290-1, 290-2) included in the wearable electronic device (200) may be positioned to face the user's eyes (e.g., left and right eyes) when the user wears the wearable electronic device (200). The one or more transparent members (290-1, 290-2) may include at least one of a glass plate, a plastic plate, or a polymer. When the user wears the wearable electronic device (200), the user can see the external environment through the one or more transparent members (290-1, 290-2).

[0075] According to one embodiment, one or more input optical members (253-1, 253-2) included in a wearable electronic device (200) can guide light generated from a first display (251) and a second display (252) to the user's eye. An image based on the light generated from the first display (251) and the second display (252) is formed on one or more screen display portions (254-1, 254-2) on one or more transparent members (290-1, 290-2), and the user can see the image formed on the one or more screen display portions (254-1, 254-2).

[0076] According to one embodiment, the wearable electronic device (200) may include one or more optical waveguides (not shown). The optical waveguides may transmit light generated from the first display (251) and the second display (252) to the user's eyes. The wearable electronic device (200) may include one optical waveguide each corresponding to the left eye and the right eye. According to one embodiment, the optical waveguides may include at least one of glass, plastic, or polymer. The optical waveguides may include a nano-pattern formed on an inner or outer surface, for example, a grating structure in the shape of a polygon or a curve. The optical waveguides may include a free-form prism, in which case the optical waveguides may provide incident light to the user through a reflective mirror. According to one embodiment, the optical waveguide includes at least one of a diffractive element (e.g., a DOE (diffractive optical element), a HOE (holographic optical element)) or a reflective element (e.g., a reflective mirror), and can guide display light emitted from a light source to the user's eye using at least one diffractive element or reflective element included in the optical waveguide. According to one embodiment, the diffractive element may include an input / output optical member. According to one embodiment, the reflective element may include a member that causes total internal reflection (TIR) ​​(e.g., a total internal reflection optical element or a total internal reflection waveguide). For example, total internal reflection is a method of guiding light, which may mean creating an angle of incidence such that light (e.g., a virtual image) input through an input grating area is 100% reflected from one surface (e.g., a specific surface) of the waveguide and is transmitted 100% to an output grating area.

[0077] In one embodiment, light emitted from a display (e.g., a first display (251) and a second display (252)) may be guided along a light path to a waveguide through an input optical member (e.g., an optical waveguide). Light traveling within the waveguide may be guided toward the user's eye through an output optical member. A screen display may be determined based on the light emitted toward the eye. The waveguide may include an input optical member, an output optical member, and / or an extension optical member (not shown).

[0078] According to one embodiment, the wearable electronic device (200) may include one or more audio input devices (262-1, 262-2, 262-3) and one or more audio output devices (263-1, 263-2).

[0079] According to one embodiment, the wearable electronic device (200) may include a first PCB (270-1) and a second PCB (270-2). The first PCB (270-1) and the second PCB (270-2) may transmit electrical signals to components included in the wearable electronic device (200), such as a first camera (211-1, 211-2), a second camera (212-1, 212-2), a third camera (213), a display (251, 252), an audio module (e.g., the audio module (170) of FIG. 1), and a sensor module (e.g., the sensor module (176) of FIG. 1). According to one embodiment, the first PCB (270-1) and the second PCB (270-2) may be flexible printed circuit boards (FPCB). According to one embodiment, the first PCB (270-1) and the second PCB (270-2) may each include a first substrate, a second substrate, and an interposer disposed between the first substrate and the second substrate. The first PCB (270-1) and the second PCB (270-2) may be disposed in the temple portion of the glasses or in the center portion of the set. According to one embodiment, the wearable electronic device (200) may further include a microphone, an SPK, an antenna, and a sensor (an accelerometer, a gyroscope, and / or a touch sensor).

[0080] With reference to FIG. 2b, the structure of a display and an eye-tracking camera according to one embodiment will be described. A wearable electronic device (200) may include a display (a first display (251) or a second display (252)), a projection lens (225), an input optical member (253-1, 253-2), a display optical waveguide (256), an output optical member (257), an eye-tracking camera (212-1, 212-2), an eye-tracking optical waveguide (258), a first splitter (259-1), and / or a second splitter (259-2).

[0081] Light output from the display (251, 252) can be refracted by the projection lens (225) and converge into a smaller aperture area. The light refracted by the projection lens (225) passes through the input optical member (253-1, 253-2) and is incident on the display optical waveguide (256), and can be output through the output optical member (257) after passing through the display optical waveguide (256). The light output from the output optical member (257) can be seen by the user's eye (201). In the following specification, the expression "displaying an object on the display" may mean that light output from the display (252) is output through the output optical member (257), and the shape of the object is seen by the user's eye (201) by the light output through the output optical member (257). Additionally, the expression “control the display to display an object” may mean that the light output from the display (251, 252) is output through the output optical member (257), and the display (251, 252) is controlled so that the shape of the object is visible to the user’s eye (201) by the light output through the output optical member (257).

[0082] Light (203) reflected from the user's eye (201) passes through the first splitter (259-1) and is incident on the eye-tracking optical waveguide (442), and can be output to the eye-tracking camera (410) through the second splitter (259-2) after passing through the eye-tracking optical waveguide (442). According to one embodiment, the light (203) reflected from the user's eye (201) may be light output from the light-emitting elements (214-1, 214-2) of FIG. 1 and reflected from the user's eye (201).

[0083] If the wearable electronic device (200) of FIG. 2a described above is in the form of AR glasses capable of displaying virtual objects, the smart glasses may include a camera for verifying external objects (e.g., including at least one of RGB and IR cameras), a camera (IR) for recognizing the wearer's gaze, a microphone, a speaker, and a display for displaying virtual objects (e.g., a display placed in both eyes or in one eye).

[0084] FIG. 3a is a perspective view showing a different structure of a wearable electronic device according to one embodiment.

[0085] Referring to FIG. 3a, a wearable electronic device (300) according to one embodiment (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2a and FIG. 2b) may be a wearable device such as a head-mounted device (HMD) that can be worn on a user's head to provide an image (e.g., a virtual reality space image) in front of the eyes. The configuration of the wearable electronic device (300) of FIG. 3a may be all or partly the same as the configuration of the wearable electronic device (200) of FIG. 2a and FIG. 2b.

[0086] According to one embodiment, the wearable electronic device (300) may include a housing (310, 320, 330) that can form an exterior and provide a space in which components of the wearable electronic device (300) can be placed.

[0087] According to one embodiment, the wearable electronic device (300) may include a first housing (310) that can surround at least a portion of the user's head. According to one embodiment, the first housing (310) may include a first surface (300a) facing the outside of the wearable electronic device (300) (e.g., in the +X direction).

[0088] According to one embodiment, the first housing (310) may surround at least a portion of the internal space (I). For example, the first housing (310) may include a second surface (300b) facing the internal space (I) of the wearable electronic device (300) and a third surface (300c) opposite to the second surface (300b). According to one embodiment, the first housing (310) may be combined with a third housing (330) to form a closed curve shape surrounding the internal space (I).

[0089] According to one embodiment, the first housing (310) can accommodate at least some of the components of the wearable electronic device (300). For example, a light output module, a circuit board, and a speaker module can be placed within the first housing (310).

[0090] According to one embodiment, a display member (340) corresponding to the left and right eyes of a wearable electronic device (300) may be included. The display member (340) may be disposed in a first housing (310). The configuration of the display member (340) of FIG. 3a may be all or partly the same as the configuration of the screen display portion (254-1, 254-2) of FIG. 2a.

[0091] According to one embodiment, the wearable electronic device (300) may include a second housing (320) that can be placed on the face of a user. According to one embodiment, the second housing (320) may include a fourth surface (300d) that can face at least partially the face of a user. According to one embodiment, the fourth surface (300d) may be a surface facing the internal space (I) of the wearable electronic device (300) (e.g., -X direction). According to one embodiment, the second housing (320) may be combined with the first housing (310).

[0092] According to one embodiment, the wearable electronic device (300) may include a third housing (330) that can be seated on the back of a user's head. According to one embodiment, the third housing (330) may be combined with the first housing (310). According to one embodiment, the third housing (330) may accommodate at least some of the components of the electronic device (300). For example, a battery (e.g., the battery (235-1, 235-2) of FIG. 2a) may be placed within the third housing (330).

[0093] In order to enhance the user's overall user experience, usage environment, and usability of a head-mounted wearable electronic device (300), it may be necessary for the sensations felt and experienced by the user in a virtual reality space, an augmented reality space, a mixed reality space, and / or an extended reality space to be as similar as possible to the sensations of the real world.

[0094] FIG. 3b is a perspective view showing the front structure of a wearable electronic device according to one embodiment. FIG. 3c is a perspective view showing the rear structure of a wearable electronic device according to one embodiment.

[0095] Referring to FIG. 3b and FIG. 3c, in one embodiment, camera modules (311, 312, 313, 314, 315, 316) and / or a depth sensor (317) for acquiring information related to the surrounding environment of the wearable electronic device (300) may be disposed on the first surface (310) of the housing.

[0096] According to one embodiment, camera modules (311, 312) can acquire images related to the surrounding environment of the wearable electronic device (300). In one embodiment, camera modules (313, 314, 315, 316) can acquire images while the wearable electronic device (300) is worn by a user. Camera modules (313, 314, 315, 316) can be used for hand detection, tracking, and user gesture (e.g., hand movements) recognition. Camera modules (313, 314, 315, 316) can be used for 3DoF, 6DoF head tracking, location (space, environment) recognition, and / or movement recognition. In one embodiment, camera modules (311, 312) may be used for hand detection and tracking, and user gestures.

[0097] In one embodiment, the depth sensor (317) may be configured to transmit a signal and receive a signal reflected from a subject, and may be used for determining the distance to an object, such as time of flight (TOF). In place of or additionally to the depth sensor (317), camera modules (313, 314, 315, 316) may determine the distance to an object.

[0098] According to one embodiment, a face recognition camera module (325, 326) (e.g., FT (Face Tracking) camera) and / or a display (321) (and / or a lens) may be disposed on the second surface (320) of the housing.

[0099] In one embodiment, a face recognition camera module (325, 326) adjacent to the display may be used to recognize the user's face or to recognize and / or track both of the user's eyes. In one embodiment, the lens may serve to adjust the focus so that the screen output to the display (321) can be seen by the user's eyes, and may be composed of, for example, a Fresnel lens, a Pancake lens, or a multi-channel lens.

[0100] In one embodiment, the display (321) (and / or lens) may be disposed on a second surface (320) of the wearable electronic device (300). In one embodiment, the wearable electronic device (300) may not include camera modules (315, 316) among a plurality of camera modules (313, 314, 315, 316). Although not illustrated in FIG. 3b and FIG. 3c, the wearable electronic device (300) may further include at least one of the configurations illustrated in FIG. 2a and FIG. 2b.

[0101] As described above, according to one embodiment, the wearable electronic device (300) may have a form factor for being worn on a user's head. The wearable electronic device (300) may further include a strap and / or a wearing member for being secured on a part of the user's body. The wearable electronic device (300) may provide a user experience based on augmented reality, virtual reality, and / or mixed reality while being worn on the user's head.

[0102] In the case where the wearable electronic device (300) according to one embodiment is in the form of a VST, it may include a camera for external object verification (e.g., including at least one of RGB and IR), a camera for recognizing the wearer's gaze (IR), a microphone, a speaker, a display for displaying virtual objects (e.g., a display placed in both eyes), and an external display (e.g., an external display of VisionPro).

[0103] Hereinafter, the wearable electronic device described in the present disclosure may be an electronic device that can be worn by a user on the body (e.g., head), such as an HMD device, AR glasses, and / or a VST device, as described with reference to FIGS. 2a, 2b, and FIGS. 3a to 3c.

[0104] FIG. 4 is a configuration diagram of a wearable electronic device according to one embodiment.

[0105] Referring to FIG. 4, a wearable device (401) according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and FIG. 2b, or wearable electronic device (300) of FIG. 3a to FIG. 3c) may include a processor (410) (e.g., processor (120) of FIG. 1), a memory (415) (e.g., memory (130) of FIG. 1), a display (420) (e.g., display module (160) of FIG. 1), a camera (425) (e.g., camera module (180) of FIG. 1), a sensor (430) (e.g., sensor module (176) of FIG. 1), a communication circuit (435) (e.g., communication module (190) of FIG. 1), and a speaker (455) (e.g., sound output module (155) of FIG. 1). The processor (410), memory (415), display (420), camera (425), sensor (430), communication circuit (435), and speaker (455) may be electrically and / or operably coupled with each other by an electronic component such as a communication bus (402). The type and / or number of hardware components included in the wearable electronic device (401) are not limited to those shown in FIG. 4. For example, the wearable electronic device (401) may include only some of the hardware components shown in FIG. 4. The elements within the memory described below (e.g., layers and / or modules) may be logically separated. The elements within the memory (415) may be included within a hardware component that is separate from the memory (415). The operation performed by the processor (410) using each of the elements in the memory (415) is one example, and the processor (410) may perform a different operation different from the above operation through at least one of the elements in the memory (415).

[0106] A processor (410) of a wearable electronic device (401) according to one embodiment may include a hardware component for processing data based on one or more instructions. The hardware component for processing data may include, for example, an arithmetic and logic unit (ALU), a field programmable gate array (FPGA), and / or a central processing unit (CPU). The number of processors (410) may be one or more. For example, the processor (410) may include at least one processor having the structure of a multi-core processor, such as a dual core, a quad core, or a hexa core.

[0107] A memory (415) of a wearable electronic device (401) according to one embodiment may include a hardware component for storing data and / or instructions (or instructions) that are input to and / or output to a processor (410). The memory (415) may include, for example, volatile memory such as random-access memory (RAM) and / or non-volatile memory such as read-only memory (ROM). Volatile memory may include, for example, at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM). Non-volatile memory may include, for example, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disk, and embedded multi-media card (eMMC).

[0108] In one embodiment, a display (420) of a wearable electronic device (401) can output visualized information to a user of the wearable electronic device (401). For example, the display (420) can be controlled by a processor (410) including a circuit such as a GPU (graphic processing unit) to output visualized information to a user. The display (420) may include a flat panel display (FPD) and / or electronic paper. The FPD may include a liquid crystal display (LCD), a plasma display panel (PDP), and / or one or more light emitting diodes (LEDs). The LED may include an organic LED (OLED).

[0109] In one embodiment, the camera (425) of the wearable electronic device (401) may include one or more light sensors (e.g., a CCD (charged coupled device) sensor, a CMOS (complementary metal oxide semiconductor) sensor) that generate an electrical signal representing the color and / or brightness of light. The plurality of light sensors included in the camera (425) may be arranged in the form of a two-dimensional grid (2 dimensional array). The camera (425) may acquire the electrical signals of each of the plurality of light sensors substantially simultaneously to generate two-dimensional frame data corresponding to the light reaching the light sensors of the two-dimensional grid. For example, photo data captured using the camera (425) may refer to one (a) two-dimensional frame data acquired from the camera (425). For example, video data captured using the camera (425) may refer to a sequence of multiple two-dimensional frame data acquired from the camera (425) along a frame rate. The camera (425) may further include a flash light for outputting light in the direction in which the camera (425) receives light.

[0110] According to one embodiment, the wearable electronic device (401) may include a plurality of cameras positioned facing different directions as an example of a camera (425). Among the plurality of cameras, the first camera may be referred to as a motion recognition camera (e.g., motion recognition camera (260-2, 260-3) of FIG. 2b), and the second camera may be referred to as an eye-tracking camera (e.g., eye-tracking camera (260-1) of FIG. 2b). The wearable electronic device (401) may identify the position, shape, and / or gesture of a hand using an image acquired using the first camera. The wearable electronic device (401) may identify the direction of gaze of a user wearing the wearable electronic device (401) using an image acquired using the second camera. For example, the direction in which the first camera faces and the direction in which the second camera faces may be opposite.

[0111] According to one embodiment, a sensor (430) of a wearable electronic device (401) can generate electrical information that can be processed by a processor (410) and / or memory (415) of the wearable electronic device (401) from non-electronic information related to the wearable electronic device (401). The information may be referred to as sensor data. The sensor (430) may include a global positioning system (GPS) sensor for detecting the geographic location of the wearable electronic device (401), an image sensor, an illuminance sensor and / or a time-of-flight (ToF) sensor, and an inertial measurement unit (IMU) for detecting the physical motion of the wearable device (401).

[0112] In one embodiment, the communication circuit (435) of the wearable electronic device (401) may include hardware components to support the transmission and / or reception of electrical signals between the wearable electronic device (401) and an external electronic device. The communication circuit (435) may include, for example, at least one of a modem, an antenna, and an optic / electronic converter. The communication circuit (435) may support the transmission and / or reception of electrical signals based on various types of protocols such as Ethernet, LAN (local area network), WAN (wide area network), WiFi (wireless fidelity), Bluetooth, BLE (Bluetooth low energy), ZigBee, LTE (long term evolution), 5G NR (new radio) and / or 6G.

[0113] In one embodiment, the speaker (455) of the wearable electronic device (401) can output an audio signal to the outside of the electronic device (401). The speaker (455) may include a plurality of speakers, and at least one speaker among the plurality of speakers may be determined to be used to output an audio signal according to the number of audio channels.

[0114] According to one embodiment, the memory (415) of the wearable electronic device (401) may store one or more instructions (or instructions or instructions) representing operations and / or operations to be performed on the data by the processor (410) of the wearable device (401). A set of one or more instructions may be referred to as firmware, an operating system, a process, a routine, a sub-routine, and / or an application. For example, the wearable electronic device (401), and / or the processor (410) may perform at least one of the operations of FIG. 6 and FIG. 7 of the present disclosure when a set of a plurality of instructions distributed in the form of an operating system, firmware, a driver, and / or an application is executed.

[0115] In the following, the statement that an application is installed within a wearable electronic device (401) may mean that one or more instructions provided in the form of an application are stored in memory (415), and that the one or more applications are stored in an executable format (e.g., a file having an extension specified by the operating system of the wearable device (401)) by the processor (410). For example, an application may include a program and / or library related to a service provided to a user.

[0116] Referring to FIG. 4, programs installed on a wearable electronic device (401) may be classified into any one of different layers based on the target, including an application layer (440), a framework layer (450), and / or a hardware abstraction layer (HAL) (480). For example, within the hardware abstraction layer (480), programs (e.g., modules, or drivers) designed to target the hardware of the wearable device (401) (e.g., a display (420), a camera (420), and / or a sensor (430)) may be classified. The framework layer (450) may be referred to as an XR framework layer in that it includes one or more programs for providing virtual reality services, augmented reality services, mixed reality services, or extended reality services. For example, FIG. 4 illustrates the layers separated within memory (415), but the layers may be logically separated. However, it is not limited thereto. According to an embodiment, the layers may be stored in a designated area within memory (415).

[0117] For example, within the framework layer (450), programs designed to target at least one of the hardware abstraction layer (480) and / or the application layer (440) (e.g., location tracker (471), spatial recognizer (472), gesture tracker (473), and / or eye tracker (474), face tracker (475)) may be classified. Programs classified into the framework layer (450) may provide an application programming interface (API) that is executable based on other programs.

[0118] For example, within the application layer (440), programs designed to target a user controlling a wearable electronic device (401) may be classified. Examples of programs classified into the application layer (440) include an XR system UI (user interface) and / or an XR application (442), but embodiments are not limited thereto. For example, programs classified into the application layer (440) (e.g., software applications) may call an API (application programming interface) to cause the execution of functions supported by programs classified into the framework layer (450).

[0119] For example, a wearable electronic device (401) may display one or more visual objects on a display (420) to perform interaction with a user for using a virtual space based on the execution of an XR system UI (441). A visual object may mean an object that can be deployed on a screen for the transmission of information and / or interaction, such as text, images, icons, videos, buttons, checkboxes, radio buttons, text boxes, sliders, and / or tables. A visual object may be referred to as a visual guide, a virtual object, a visual element, a UI element, a view object, and / or a view element. Based on the execution of an XR system UI (441), the wearable electronic device (401) may provide a service to the user to control functions available in the virtual space.

[0120] Referring to FIG. 4, a lightweight renderer (443) and / or an XR plugin (444) are depicted within the XR system UI (441), but are not limited thereto. For example, the XR system UI (441) may cause the execution of supported functions in the lightweight renderer (443) and / or the XR plugin (444) included within the application layer (440).

[0121] For example, a wearable device (401) may acquire resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute a rendering pipeline, which is permitted to be partially modified, based on the execution of a lightweight renderer (443). The lightweight renderer (443) may be referred to as a lightweight render pipeline in terms of defining a rendering pipeline, which is permitted to be partially modified. The lightweight renderer (443) may include a renderer built prior to the execution of a software application (e.g., a pre-built renderer). For example, a wearable electronic device (401) may acquire resources (e.g., APIs, system processes, and / or libraries) used to define, create, and / or execute the entire rendering pipeline based on the execution of an XR plugin (444). The XR plugin (444) may be referred to as an open XR native client in terms of defining (or setting) the entire rendering pipeline.

[0122] For example, the wearable electronic device (401) may display a screen representing at least a portion of a virtual space on the display (420) based on the execution of the XR application (442). The XR plugin (444-1) included in the XR application (442) may be referenced by the XR plugin (444) of the XR system UI (441). Descriptions of the XR plugin (444-1) that overlap with descriptions of the XR plugin (444) may be omitted. The wearable electronic device (401) may trigger the execution of a screen composition manager (451) based on the execution of the XR application (442).

[0123] According to one embodiment, a wearable electronic device (101) may provide a virtual space service based on the execution of a screen composition manager (451). For example, the screen composition manager (451) may include a platform (e.g., an Android platform) for supporting the virtual space service. Based on the execution of the screen composition manager (451), the wearable electronic device (401) may display on a display the posture of a virtual object representing a rendered user's posture using data acquired through a sensor (430). The screen composition manager (451) may be referred to as a composition presentation manager (CPM).

[0124] For example, the screen configuration manager (451) may include a runtime service (452). In one example, the runtime service (452) may be referenced as an OpenXR runtime module. A wearable electronic device (401) may be used to provide at least one of a user pose prediction function, a frame timing function, and / or a spatial input function through the wearable electronic device (401) based on the execution of the runtime service (452). In one example, the wearable electronic device (401) may be used to perform rendering for a virtual space service for the user based on the execution of the runtime service (452). For example, an application (e.g., Unity or OpenXR native application) may be implemented based on the execution of the runtime service (452).

[0125] For example, the screen configuration manager (451) may include a pass-through library (453). The wearable electronic device (401) may, based on the execution of the pass-through library (453), display another screen representing real space acquired through a camera (425) superimposed on at least a portion of the screen while displaying a screen representing virtual space on the display (420).

[0126] For example, the screen composition manager (451) may include a renderer. The wearable device (101) can render a screen to be displayed on a display by compositing virtual layers (or virtual nodes) rendered based on sensor data (e.g., sensing data obtained through a camera (425) or sensor (430)) and pass-through layers (or pass-through nodes) obtained through a pass-through library (453) through the screen composition manager (451) using the renderer. The virtual layers may be referred to as virtual nodes and / or virtual surfaces. The wearable electronic device (101) can render each of the virtual layers or render all of the virtual layers through the screen composition manager (451).

[0127] For example, the screen configuration manager (451) may include an input manager (454). The wearable electronic device (401) may identify acquired data (e.g., sensor data) by executing one or more programs included in the recognition service layer (470) based on the execution of the input manager (454). The wearable electronic device (401) may initiate the execution of at least one of the functions of the wearable electronic device (401) using the acquired data.

[0128] For example, the perception abstract layer (460) may be used for data exchange between the screen configuration manager (451) and the perception service layer (470). In terms of being used for data exchange between the screen configuration manager (451) and the perception service layer (470), the perception abstract layer (460) may be referred to as an interface. As an example, the perception abstract layer (460) may be referred to as OpenPX and / or PPAL (perception platform abstract layer). The perception abstract layer (460) may be used for a perception client and a perception service.

[0129] According to one embodiment, the recognition service layer (470) may include one or more programs for processing data obtained from a sensor (430) (or a camera (425)). The one or more programs may include at least one of a location tracker (471), a spatial recognizer (472), a gesture tracker (473), an eye tracker (474), and / or a face tracker (475). The type and / or number of the one or more programs included in the recognition service layer (470) are not limited to those shown in FIG. 4.

[0130] For example, the wearable electronic device (401) can identify the posture of the wearable device (401) using the sensor (430) based on the operation of the position tracker (471). The wearable electronic device (401) can identify the 6 degrees of freedom pose (6 DOF pose) of the wearable electronic device (401) using data acquired using the camera (425) and the IMU based on the operation of the position tracker (471). The position tracker (471) may be referred to as a head tracking (HeT) module.

[0131] For example, the wearable electronic device (401) may be used to construct the surrounding environment of the wearable electronic device (401) (or the user of the wearable electronic device (401)) into a three-dimensional virtual space based on the execution of the space recognizer (472). The wearable electronic device (401) may reconstruct the surrounding environment of the wearable electronic device (401) in three dimensions using data acquired using a camera (425) based on the execution of the space recognizer (472). The wearable electronic device (401) may identify at least one of a plane, an incline, or a staircase based on the surrounding environment of the wearable device (401) reconstructed in three dimensions based on the execution of the space recognizer (472). The space recognizer (472) may be referred to as a scene understanding (SU) module.

[0132] For example, the wearable electronic device (401) may be used to identify (or recognize) the pose and / or gesture of the user's hand of the wearable electronic device (401) based on the execution of the gesture tracker (473). For example, the wearable electronic device (401) may identify the pose and / or gesture of the user's hand using data acquired from the sensor (430) based on the execution of the gesture tracker (473). For example, the wearable electronic device (401) may identify the pose and / or gesture of the user's hand based on data (or images) acquired using the camera (425) based on the execution of the gesture tracker (473). The gesture tracker (473) may be referred to as a hand tracking (HaT) module and / or a gesture tracking module.

[0133] For example, the wearable electronic device (401) can identify (or track) the movement of the user's eyes of the wearable electronic device (401) based on the execution of the eye tracker (474). For example, the wearable electronic device (401) can identify the movement of the user's eyes using data obtained from at least one sensor based on the execution of the eye tracker (474). For example, the wearable electronic device (401) can identify the movement of the user's eyes based on data obtained using a camera (425) (e.g., the eye tracking camera (260-1) of FIG. 2a and FIG. 2b) and / or an IR LED (infrared light emitting diode) based on the execution of the eye tracker (474). The eye tracker (474) may be referred to as an eye tracking (ET) module and / or a gaze tracking module.

[0134] For example, the recognition service layer (470) of the wearable electronic device (401) may further include a face tracker (475) for tracking the user's face. For example, the wearable electronic device (401) may identify (or track) the movement of the user's face and / or the user's facial expression based on the execution of the face tracker (475). The wearable electronic device (401) may estimate the user's facial expression based on the movement of the user's face based on the execution of the face tracker (475). For example, the wearable electronic device (401) may identify the movement of the user's face and / or the user's facial expression based on data (e.g., an image) acquired using a camera based on the execution of the face tracker (475).

[0135] FIG. 5 is a diagram illustrating an application for providing spatial acoustics for audio in a virtual space in a wearable electronic device according to one embodiment.

[0136] Referring to FIG. 5, a wearable electronic device (401) according to one embodiment may store an application (500) (e.g., the XR application (442) of FIG. 4) for providing spatial acoustics for audio in a virtual space in a memory (415).

[0137] An application (500) according to one embodiment may include an audio signal acquisition module (510), a first mixer (520), an audio channel identification module (530), a renderer and virtualizer (540), a second mixer (550), and / or a playback module (560). The decoder (510), the first mixer (520), the audio channel identification module (530), the renderer and virtualizer (540), the second mixer (550), and / or the playback module (560) according to one embodiment may each be a software module (or program), and may be described as separate software modules in the description of FIG. 5, but at least some of the software modules may be merged or all of the software modules may be composed of a single software module.

[0138] Referring to FIG. 5, a processor (410) of a wearable electronic device (401) according to one embodiment may acquire decoded audio data to output audio generated by an object through an audio signal acquisition module (510) when displaying an object (or virtual object (e.g., virtual TV, virtual speaker, virtual laptop, application object, or other virtual object that generates audio)) that outputs content (e.g., generates audio) in a virtual space. A processor (410) according to one embodiment may identify the audio channels of the acquired audio data. For example, the audio channels of the audio data may include a mono channel (mono), a stereo channel (2ch), a 5.1 channel (5.x ch), a 7.1 channel (7.x ch), a multi-channel including object metadata (e.g., 12 channels) (object metadata + multi ch or multi-channel + object), or a 3D channel (ambisonic).

[0139] A processor (410) according to one embodiment may identify a designated first audio channel (e.g., a maximum audio channel or 12 channels) to provide spatial acoustics for audio generated by a virtual object. A processor (410) according to one embodiment may identify a designated center location (hereinafter also referred to as the "spatial acoustic center location") to provide spatial acoustics for audio generated by a virtual object in a virtual space. According to one embodiment, the center location for spatial acoustics may be a location designated by a user (e.g., a place designated so that the user can clearly feel spatial acoustics while watching a video (e.g., a sofa or chair)). According to one embodiment, the center location for spatial acoustics may be the location of the virtual object generating the audio. A processor (410) according to one embodiment may perform upmixing to convert decoded audio data into an audio signal of the first audio channel to output audio generated by the virtual object through a first mixer (520).

[0140] A processor (410) according to one embodiment may transmit audio data of an upmixed first audio channel to a renderer and a virtualizer (540). A processor (410) according to one embodiment may bypass the first mixer (520) when it does not provide spatial acoustics for audio generated by a virtual object or when the audio channel of the audio data is the first audio channel.

[0141] A processor (410) according to one embodiment can identify whether the first audio channel is changed (e.g., whether the first audio channel is changed to an audio channel with a number of channels less than the number of channels of the first audio channel) through an audio channel change identification module (530).

[0142] According to one embodiment, the audio channel change identification module (530) may include a sensor data analysis module (532), a motion detection module (534), a rendering position map module (536), and a virtual object display screen size identification module (538). According to one embodiment, the processor (410) may identify the movement of the wearable electronic device (401) (or the user of the wearable electronic device (401)) by analyzing sensor data obtained from the sensor (430) through the sensor data analysis module (532). For example, the processor (410) may identify the movement of the wearable electronic device (401) (or the user of the wearable electronic device (401)) by analyzing sensor data obtained from the sensor (430) through the sensor data analysis module (532) while the position of the virtual object in the virtual space is fixed. A processor (410) according to one embodiment can identify and / or update the location of the wearable electronic device (401) (or the user of the wearable electronic device (401)) based on the identification of the movement of the wearable electronic device (401) (or the user of the wearable electronic device (401)).

[0143] A processor (410) according to one embodiment can identify the positional movement of a virtual object in a virtual space by detecting and analyzing the motion of a user through a motion detection module (532). For example, the processor (410) can identify the positional movement of a virtual object in a virtual space based on the detection of the user's motion through the motion detection module (532) while the position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in the virtual space is fixed. A processor (410) according to one embodiment can identify and / or update the position of a virtual object in a virtual space based on the identification of the positional movement of the virtual object in the virtual space.

[0144] A processor (410) according to one embodiment can render the wearable electronic device (401) (or user of the wearable electronic device (401)) and the virtual object at the location of the wearable electronic device (401) (or user of the wearable electronic device (401)) and the virtual object at the location of the virtual object in the virtual space through a location map rendering module (536). A processor (410) according to one embodiment can identify the distance between a center location for spatial acoustics and the location of the wearable electronic device (401) (or user of the wearable electronic device (401)) in the virtual space (e.g., a first distance) and / or the distance between a center location designated to provide spatial acoustics and the location of the virtual object in the virtual space (e.g., a second location) through a sensor data analysis module (532), a motion detection module (534), and a location map rendering module (536) of an audio channel change identification module (530). A processor (410) according to one embodiment can identify the size of the display screen of a virtual object generating audio in a virtual space through a virtual object display screen size identification module (536).

[0145] A processor (410) according to one embodiment can identify whether to change the first audio channel to a second audio channel different from the first audio channel through an audio channel change identification module (530) based on the distance between the center position for spatial sound and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space, the distance between the center position for spatial sound and the second position of the virtual object in virtual space, and / or the size of the display screen of the virtual object.

[0146] A processor (410) according to one embodiment can identify channel number information that changes according to the increase or decrease in distance between a center position for spatial acoustics and a first position of a wearable electronic device (401) (or a user of the wearable electronic device (401)) in virtual space through an audio channel change identification module (530). A processor (410) according to one embodiment can identify an audio channel having a smaller number of channels (e.g., 12 channels -> 5.1 channels -> 2 channels (or mono channels)) as the distance between a center position for spatial acoustics and a first position of a wearable electronic device (401) (or a user of the wearable electronic device (401)) in virtual space increases (e.g., 5 feet -> 10 feet -> 15 feet). A processor (410) according to one embodiment can identify a large number of audio channels (e.g., 2 channels (or mono channels) -> 5.1 channels -> 12 channels) as the distance between a central position for spatial acoustics and a first position of a wearable electronic device (401) (or a user of the wearable electronic device (401)) in virtual space becomes closer (e.g., 15 feet -> 10 feet -> 5 feet).

[0147] A processor (410) according to one embodiment can identify channel number information that changes according to the increase or decrease in distance between a center position for spatial sound and a second position of a virtual object in virtual space through an audio channel change identification module (530). A processor (410) according to one embodiment can identify an audio channel having a smaller number of channels as the distance between a center position for spatial sound and a second position of a virtual object in virtual space increases through an audio channel change identification module (530). A processor (410) according to one embodiment can identify an audio channel having a larger number of channels as the distance between a center position for spatial sound and a second position of a virtual object in virtual space decreases through an audio channel change identification module (530).

[0148] A processor (410) according to one embodiment can identify information on the number of channels that changes according to the increase or decrease in the size of the virtual object's display screen through an audio channel change identification module (530). A processor (410) according to one embodiment can identify audio channels having fewer channels (e.g., 12 channels -> 5.1 channels -> 2 channels (or mono channels)) as the size of the virtual object's display screen decreases (e.g., 300 cm width or height -> 150 cm width or height -> 50 cm width or height) through an audio channel change identification module (530). A processor (410) according to one embodiment can identify audio channels having more channels as the size of the virtual object's display screen increases through an audio channel change identification module (530).

[0149] According to one embodiment, the processor (410) can identify, through the audio channel change identification module (530), that the first audio channel (e.g., 12 channels) is not changed if the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space is less than or equal to a specified first distance (e.g., 5 feet). A processor (410) according to one embodiment can identify a second audio channel having fewer channels than the number of channels of the first audio channel (e.g., 12 channels -> 5.1 channels -> 2 channels (or mono channels)) according to an increase in distance (e.g., 5 feet -> 10 feet -> 15 feet) if the distance between a center position for spatial acoustics and a first position of a wearable electronic device (401) (or a user of the wearable electronic device (401)) in virtual space exceeds a specified first distance (e.g., 5 feet).

[0150] According to one embodiment, the processor (410) can identify, through an audio channel change identification module (530), whether the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space is less than or equal to a specified first distance (e.g., 10 feet) (e.g., when the user's position is fixed), whether the distance between the center position for spatial acoustics and the second position of the virtual object is less than or equal to a specified second distance.

[0151] According to one embodiment, the processor (410) can identify, through the audio channel change identification module (530), that the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space is less than or equal to a specified first distance (e.g., 5 feet) and the distance between the center position for spatial acoustics and the second position of the virtual object is less than or equal to a specified second distance, that the first audio channel (e.g., 12 channels) is not changed. A processor (410) according to one embodiment can identify a second audio channel having fewer channels than the number of channels of the first audio channel through an audio channel change identification module (530) if the distance between the center position for spatial sound and the first position of the wearable electronic device (401) (or user of the wearable electronic device (401)) in virtual space is less than or equal to a specified first distance and the distance between the center position for spatial sound and the second position of the virtual object is greater than a specified second distance.

[0152] According to one embodiment, the processor (410) can identify whether the size of the display screen of the virtual object is smaller than a specified first size if, through the audio channel change identification module (530), the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space is less than or equal to a specified first distance (e.g., 5 feet) and the distance between the center position for spatial acoustics and the second position of the virtual object is less than or equal to a specified second distance. According to one embodiment, the processor (410) can identify, through the audio channel change identification module (530), that the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space is less than or equal to a specified first distance (e.g., 5 feet), the distance between the center position for spatial acoustics and the second position of the virtual object is less than or equal to a specified second distance, and the size of the display screen of the virtual object is not smaller than a specified first size, thereby identifying that the first audio channel (e.g., 12 channels) is not changed. According to one embodiment, the processor (410) can identify a second audio channel having fewer channels than the number of channels of the first audio channel according to the degree of smallness of the size of the display screen of the virtual object, if the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space is less than or equal to a specified first distance (e.g., 5 feet) and the distance between the center position for spatial acoustics and the second position of the virtual object is less than or equal to a specified second distance and the size of the display screen of the virtual object is smaller than a specified first size, through an audio channel change identification module (530).

[0153] According to one embodiment, if it is identified that the first audio channel is not being changed, the processor (410) may bypass the first audio signal of the upmixed first audio channel to the output module (560) through the second mixer (550). According to one embodiment, if a second audio channel having a number of channels less than the number of channels of the first audio channel is identified, the processor (410) may downmix the first audio signal of the upmixed first audio channel into the second audio signal of the second audio channel through the second mixer (550) and transmit the downmixed second audio signal of the second audio channel to the output module (560).

[0154] A processor (410) according to one embodiment may output (or play) a first audio signal of a first audio channel through an output module (or playback module) (560) or output (or play) a second audio signal of a second audio channel having fewer channels than the first audio channel (or different from the first channel) and output through at least one speaker corresponding to each channel among a plurality of speakers.

[0155] The wearable electronic device (101, 200, 300, 401) of the present disclosure may include a display (160, 251, 252, 340, 321, 420), at least one sensor (176, 317, 430), at least one camera (180, 211-1, 211-2, 213, 313, 314, 315, 316, 440), a plurality of speakers (155, 455), at least one processor (120, 410), and a memory (130, 415) for storing commands. According to one embodiment, when the commands are executed individually or collectively by the at least one processor, the wearable electronic device may cause the wearable electronic device to display a virtual object that generates audio in a virtual space through the display while the wearable electronic device is worn by a user. When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device may identify a first audio channel designated for providing spatial acoustics for the audio among a plurality of audio channels. When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device may identify a center position designated for providing spatial acoustics for the audio. When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device may identify the distance between the designated center position and a first position of the wearable electronic device in the virtual space. When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device may output the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers if the distance between the designated center position and the first position does not exceed a designated first distance.When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device may identify a second audio channel that is different from the first audio channel (or has fewer channels than the first audio channel) when the distance between the designated center position and the first position exceeds the designated first distance, and output the audio as a second audio signal of the second audio channel through at least some of the plurality of speakers.

[0156] According to one embodiment, if the distance between the designated center position and the first position exceeds the designated first distance, the number of channels of the second audio channel may decrease as the distance exceeds the limit.

[0157] According to one embodiment, when the commands are executed individually or collectively by the at least one processor, the wearable electronic device may identify whether the distance between the designated center position and the second position of the virtual object exceeds a designated second distance when the distance between the designated center position and the first position does not exceed a designated first distance. When the commands are executed individually or collectively by the at least one processor, the wearable electronic device may output the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the distance between the designated center position and the second position does not exceed a designated second distance. When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device may be configured to output the audio through at least some of the plurality of speakers as a second audio signal of the second audio channel that is different from (or has fewer channels than the first audio channel) the first audio channel when the distance between the designated center position and the second position exceeds the designated second distance. According to one embodiment, when the distance between the designated center position and the second position exceeds the designated second distance, the number of channels of the second audio channel may decrease as the distance exceeds the specified second distance.

[0158] The commands according to one embodiment, when executed individually or collectively by the at least one processor, may cause the wearable electronic device to identify whether the size of the display screen of the virtual object is smaller than a first size when the distance between the designated center position and the first position does not exceed a designated first distance and the distance between the designated center position and the second position does not exceed a designated second distance. The commands, when executed individually or collectively by the at least one processor, may cause the wearable electronic device to output the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the size of the display screen of the virtual object is not smaller than the first size. When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device may be configured to output the audio through at least some of the plurality of speakers as a second audio signal of the second audio channel that is different from (or has fewer channels than the first audio channel) the first audio channel when the size of the display screen of the virtual object is smaller than the first size. In one embodiment, the number of channels of the second audio channel may be reduced as the size of the display screen of the virtual object is reduced.

[0159] The commands according to one embodiment, when executed individually or collectively by the at least one processor, may cause the wearable electronic device to identify the movement of the wearable electronic device while the wearable electronic device is worn by the user. The commands, when executed individually or collectively by the at least one processor, may cause the wearable electronic device to update the first location of the wearable electronic device in the virtual space based on the identification of the movement of the wearable electronic device.

[0160] The commands according to one embodiment, when executed individually or collectively by the at least one processor, may enable the wearable electronic device to identify the movement of the virtual object in the virtual space. The commands, when executed individually or collectively by the at least one processor, may enable the wearable electronic device to update the second location of the virtual object based on the identification of the movement of the virtual object in the virtual space.

[0161] According to one embodiment, when the commands are executed individually or collectively by the at least one processor, the wearable electronic device may display first information representing the first audio channel on the display when the first audio signal of the first audio channel is output. When the commands are executed individually or collectively by the at least one processor, the wearable electronic device may display second information representing the second audio channel on the display when the second audio signal of the second audio channel is output. According to one embodiment, the first information and the second information may be displayed differently.

[0162] When the commands according to one embodiment are executed individually or collectively by the at least one processor, the wearable electronic device may display a first object representing the first audio signal of the first audio channel in the virtual space through the display when the first audio signal of the first audio channel is output. When the commands are executed individually or collectively by the at least one processor, the wearable electronic device may display a second object representing the second audio signal of the second audio channel in the virtual space through the display when the second audio signal of the second audio channel is output.

[0163] According to one embodiment, the plurality of audio channels may include a mono channel, a stereo channel, a 5.1 channel, a 7.1 channel, a 12 channel, a multi-channel+Object, or ambisonic.

[0164] According to one embodiment, the designated first audio channel may include 12 channels.

[0165] FIG. 6 is a flowchart illustrating the operation of providing spatial sound in a virtual space in a wearable electronic device according to one embodiment.

[0166] Referring to FIG. 6, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of an electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) can perform at least one of 610 to 660 operations.

[0167] In operation 610, a processor (410) according to one embodiment may display a virtual object that generates audio in a virtual space (e.g., a virtual TV, a virtual speaker, a virtual laptop, an application object, or other virtual object that generates audio). When the processor (410) according to one embodiment displays a virtual object that generates audio in a virtual space, it may acquire decoded audio data through an audio signal acquisition module (510) to output audio generated by the object. The processor (410) according to one embodiment may identify the audio channels of the acquired audio data. For example, the audio channels of the audio data may include a mono channel (mono), a stereo channel (2ch), a 5.1 channel (5.x ch), a 7.1 channel (7.x ch), a multi-channel including object metadata (e.g., 12 channels) (object metadata + multi ch or multi-channel + object), or a 3D channel (ambisonic).

[0168] In operation 620, the processor (410) according to one embodiment may identify a designated first audio channel (e.g., a maximum channel or 12 channels) to provide spatial acoustics for audio generated by a virtual object.

[0169] In operation 630, the processor (410) according to one embodiment can identify the distance between a center position for spatial acoustics and a first position of the wearable electronic device (401) (or user of the wearable electronic device (401)) in virtual space.

[0170] In operation 640, the processor (410) according to one embodiment can identify whether the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space exceeds (or is less than) a specified first distance (e.g., 5 feet).

[0171] In operation 650, a processor (410) according to one embodiment may output audio as a first audio signal of a first audio channel through at least some of a plurality of speakers if the distance between a center position for spatial acoustics and a first position of a wearable electronic device (401) (or a user of the wearable electronic device (401)) in virtual space does not exceed (or is less than) a specified first distance (e.g., 5 feet). A processor (410) according to one embodiment may identify, through an audio channel change identification module (530), that the first audio channel (e.g., 12 channels) is not changed if the distance between a center position for spatial acoustics and a first position of a wearable electronic device (401) (or a user of the wearable electronic device (401)) in virtual space does not exceed (or is less than) a specified first distance (e.g., 5 feet), and may output audio as a first audio signal of a first audio channel through at least some of a plurality of speakers.

[0172] In operation 660, a processor (410) according to one embodiment may identify a second audio channel having a number of channels less than the number of channels of a first audio channel when the distance between a center position for spatial acoustics and a first position of a wearable electronic device (401) (or a user of the wearable electronic device (401)) in virtual space exceeds a specified first distance (e.g., 5 feet) and output audio as a second audio signal of the second audio channel through at least some of a plurality of speakers. A processor (410) according to one embodiment can identify a second audio channel having fewer channels than the number of channels of a first audio channel through an audio channel change identification module (530) when the distance between a center position for spatial acoustics and a first position of a wearable electronic device (401) (or a user of the wearable electronic device (401)) in virtual space exceeds a specified first distance (e.g., 5 feet), and can identify a second audio channel having fewer channels than the number of channels of a first audio channel (e.g., 12 channels -> 5.1 channels -> 2 channels (or mono channels)) as the distance increases (e.g., 5 feet -> 10 feet -> 15 feet).

[0173] FIG. 7 is a flowchart illustrating an operation of providing spatial sound according to the distance between a designated center position and a first position of the wearable electronic device, the distance between the designated center position and a position of a virtual object, and the size of the display screen of the virtual object in a wearable electronic device according to one embodiment.

[0174] Referring to FIG. 7, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of a wearable electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) can perform at least one of 712 to 724 operations.

[0175] In operation 712, a processor (410) according to one embodiment may display a virtual object that generates audio in a virtual space (e.g., a virtual TV, a virtual speaker, a virtual laptop, an application object, or other virtual object that generates audio). When the processor (410) according to one embodiment displays a virtual object that generates audio in a virtual space, it may acquire decoded audio data through an audio signal acquisition module (510) to output audio generated by the object. The processor (410) according to one embodiment may identify the audio channels of the acquired audio data. For example, the audio channels of the audio data may include a mono channel (mono), a stereo channel (2ch), a 5.1 channel (5.x ch), a 7.1 channel (7.x ch), a multi-channel including object metadata (e.g., 12 channels) (object metadata + multi ch or multi-channel + object), or a 3D channel (ambisonic).

[0176] In operation 712, the processor (410) according to one embodiment may identify a designated first audio channel (e.g., a maximum channel or 12 channels) to provide spatial acoustics for audio generated by a virtual object.

[0177] In operation 716, the processor (410) according to one embodiment can identify whether the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space exceeds (or is less than) a specified first distance (e.g., 5 feet).

[0178] In operation 718, the processor (410) according to one embodiment can identify whether the distance between the center position for spatial acoustics and the virtual object in virtual space exceeds a specified second distance if the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space does not exceed a specified first distance (e.g., 5 feet).

[0179] In operation 720, the processor (410) according to one embodiment can identify whether the size of the display screen of an object is smaller than a specified first size if the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space does not exceed a specified first distance (e.g., 5 feet) and the distance between the center position for spatial acoustics and the virtual object in virtual space does not exceed a specified second distance.

[0180] In operation 722, the processor (410) according to one embodiment may output audio as a first audio signal of a first audio channel through at least some of the plurality of speakers (455) if the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space does not exceed a specified first distance (e.g., 5 feet), the distance between the center position for spatial acoustics and the virtual object in virtual space does not exceed a specified second distance, and the size of the object's display screen is not smaller than a specified first size.

[0181] In operation 724, a processor (410) according to one embodiment can identify a second audio channel having fewer channels than the number of channels of a first audio channel if the distance between a center position for spatial acoustics and a first position of a wearable electronic device (401) (or a user of the wearable electronic device (401)) in virtual space exceeds a specified first distance (e.g., 5 feet), the distance between a center position for spatial acoustics and a virtual object in virtual space exceeds a specified second distance, or the size of the object's display screen is smaller than a specified first size. For example, the processor (410) can identify a second audio channel having fewer channels than the number of channels of the first audio channel (e.g., 12 channels -> 5.1 channels -> 2 channels (or mono channels)) according to an increase in distance (e.g., 5 feet -> 10 feet -> 15 feet) if the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or user of the wearable electronic device (401)) in virtual space exceeds a specified first distance (e.g., 5 feet). For example, the processor (410) can identify a second audio channel having fewer channels than the number of channels of the first audio channel through an audio channel change identification module (530) if the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space is less than or equal to a specified first distance and the distance between the center position for spatial acoustics and the second position of the virtual object is greater than a specified second distance.For example, the processor (410) can identify a second audio channel having fewer channels than the number of channels of the first audio channel according to the degree of smallness of the size of the display screen of the virtual object, if the distance between the center position for spatial acoustics and the first position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space is less than or equal to a specified first distance (e.g., 5 feet) and the distance between the center position for spatial acoustics and the second position of the virtual object is less than or equal to a specified second distance and the size of the display screen of the virtual object is smaller than a specified first size, through an audio channel change identification module (530).

[0182] In operation 726, the processor (410) according to one embodiment can output audio as a second audio signal of a second audio channel through at least some of the plurality of speakers.

[0183] A method for providing spatial sound in virtual reality in a wearable electronic device (101, 200, 300, 401) according to one embodiment of the present disclosure may include an operation of displaying a virtual object that generates audio in a virtual space through a display of the wearable electronic device while the wearable electronic device is worn by a user. The method may include an operation of identifying a first audio channel designated for providing spatial sound for the audio among a plurality of audio channels. The method may include an operation of identifying a center position designated for providing spatial sound for the audio. The method may include an operation of identifying a distance between the designated center position and a first position of the wearable electronic device in the virtual space. The method may include an operation of outputting the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the distance between the designated center position and the first position does not exceed a designated first distance. The above method may include the operation of identifying a second audio channel different from the first audio channel (or having fewer channels than the first audio channel) when the distance between the designated center position and the first position exceeds the designated first distance, and outputting the audio as a second audio signal of the second audio channel through at least some of the plurality of speakers.

[0184] In the method according to one embodiment, if the distance between the designated center position and the first position exceeds the designated first distance, the number of channels of the second audio channel may decrease as the distance exceeds the limit.

[0185] According to one embodiment, the method may include an operation of identifying whether the distance between the designated center position and the virtual object exceeds a designated second distance when the distance between the designated center position and the first position does not exceed a designated first distance. The method may include an operation of outputting the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the distance between the designated center position and the virtual object does not exceed a designated second distance. The method may include an operation of outputting the audio as a second audio signal of the second audio channel, which is different from the first audio channel (or has fewer channels than the first audio channel), through at least some of the plurality of speakers when the distance between the designated center position and the virtual object exceeds a designated second distance. In the method, when the distance between the designated center position and the second position exceeds the designated second distance, the number of channels of the second audio channel may decrease as the distance exceeds the designated second distance.

[0186] According to one embodiment, the method may include an operation of identifying whether the size of the display screen of the virtual object is smaller than a first size when the distance between the designated center position and the first position does not exceed a designated first distance and the distance between the designated center position and the virtual object does not exceed a designated second distance. The method may include an operation of outputting the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the size of the display screen of the virtual object is not smaller than the first size. The method may include an operation of outputting the audio as a second audio signal of the second audio channel, which is different from the first audio channel (or has fewer channels than the first audio channel), through at least some of the plurality of speakers when the size of the display screen of the virtual object is smaller than the first size. In the method, the number of channels of the second audio channel may decrease as the size of the display screen of the virtual object decreases.

[0187] The method according to one embodiment may include an operation of identifying the movement of the wearable electronic device while the wearable electronic device is worn by the user. The method may include an operation of updating the first position of the wearable electronic device in the virtual space based on the identification of the movement of the wearable electronic device.

[0188] The method according to one embodiment may include an operation of identifying the movement of the virtual object in the virtual space. The method may include an operation of updating the second position of the virtual object based on the identification of the movement of the virtual object in the virtual space.

[0189] The method according to one embodiment may include an operation of displaying first information representing the first audio channel on the display when the first audio signal of the first audio channel is output. The method may include an operation of displaying second information representing the second audio channel on the display when the second audio signal of the second audio channel is output. In the method, the first information and the second information may be displayed differently.

[0190] The method according to one embodiment may include an operation of displaying an object representing the first audio signal of the first audio channel in the virtual space through the display when the first audio signal of the first audio channel is output.

[0191] According to one embodiment, the plurality of audio channels include a mono channel, a stereo channel, a 5.1 channel, a 7.1 channel, a 12 channel, a multi-channel+Object, or ambisonic, and the designated first audio channel may include 12 channels.

[0192] FIG. 8 is a diagram illustrating a designated center position and a distance from a wearable electronic device to provide spatial acoustics for audio generated through a virtual object in a virtual space according to one embodiment.

[0193] Referring to FIG. 8, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of a wearable electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) can display a virtual object (850) (e.g., virtual TV) that generates audio in a virtual space (801) through a display while being worn by a user.

[0194] A processor (410) according to one embodiment may identify a designated center location (801) to provide spatial sound for audio generated by a virtual object (850) in a virtual space (801). According to one embodiment, the center location for spatial sound may be a location designated by a user (e.g., a place designated so that the user can clearly feel the spatial sound while watching a video (e.g., a sofa or chair)). According to one embodiment, although not shown in FIG. 8, the center location for spatial sound may be a center location of the virtual object (801) generating the audio.

[0195] A processor (410) according to one embodiment can set audio channels with fewer channels as the distance increases for each range (810, 820, 830) according to different distances between a central position for spatial acoustics and a first position of a wearable electronic device (401) (or a user of the wearable electronic device (401)) in virtual space. For example, a first distance range (810) within a first designated distance (e.g., 5 feet) may be set to a first audio channel (e.g., 12 channels), an audio channel having fewer channels than the first audio channel (e.g., 5.1 channels) may be set for a second distance range (820) between the first designated distance (e.g., 5 feet) and a distance further from the first designated distance (e.g., 10 feet), and an audio channel having fewer channels than the first audio channel (e.g., 2 channels) may be set for a third distance range (8230) between the first designated distance (e.g., 5 feet) and a distance further from the first designated distance (e.g., 15 feet). The distance and number of channels according to one embodiment may be set to different values.

[0196] FIG. 9a is a diagram illustrating a case where a user of a wearable electronic device according to one embodiment is located within a first distance range based on a center position for spatial acoustics. FIG. 9b is a diagram illustrating a case where a user of a wearable electronic device according to one embodiment is located within a second distance range based on a center position for spatial acoustics. FIG. 9c is a diagram illustrating a case where a user of a wearable electronic device according to one embodiment is located within a third distance range based on a center position for spatial acoustics.

[0197] Referring to FIG. 9a, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of a wearable electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) may output audio as a first audio signal of a first audio channel (e.g., 12 channels) designated for a spatial acoustic function when the virtual object (905) (e.g., virtual TV) that generates audio in a virtual space (900) through a display is located within a first distance range (910) relative to a center position (901) for spatial acoustics.

[0198] Referring to FIG. 9b, the processor (410) can output audio as an audio signal of an audio channel (e.g., 5.1 channels) with fewer channels than the first audio channel when a wearable electronic device (401) (or user) moves from a central position (901) to a second distance range (920) that is further away from a first distance range (910) while a virtual object (905) (e.g., virtual TV) generating audio in a virtual space (900) through a display.

[0199] Referring to FIG. 9c, the processor (410) can output audio as an audio signal of an audio channel (e.g., 2 channels) that has fewer channels than the first audio channel when a wearable electronic device (401) (or user) moves from a central position (901) to a third distance range (920) that is further away than the second distance range (920) while displaying a virtual object (905) (e.g., virtual TV) that generates audio in a virtual space (900) through a display.

[0200] As shown in FIGS. 9a to 9c, a processor (410) according to one embodiment can output audio as an audio signal of audio channels having fewer channels as the distance between the central position (901) for spatial sound and the position of the wearable electronic device (401) (or the user of the wearable electronic device (401)) in virtual space increases, thereby allowing the user to feel the distance and spatial sense of the audio more realistically.

[0201] FIG. 10a is a drawing showing a case where a wearable electronic device according to one embodiment is located at a first distance relative to a center position for spatial acoustics. FIG. 10b is a drawing showing a case where a wearable electronic device according to one embodiment is located at a second distance relative to a center position for spatial acoustics worn by the device. FIG. 10c is a drawing showing a case where a wearable electronic device according to one embodiment is located at a third distance relative to a center position for spatial acoustics.

[0202] Referring to FIG. 10a, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of a wearable electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) can output audio as a first audio signal of a first audio channel (e.g., 12 channels) designated for a spatial acoustic function when the wearable electronic device (905) is located at a first distance (1011) relative to a center position (901) for spatial acoustics while displaying a virtual object (905) (e.g., virtual TV) that generates audio of a virtual space (900) through a display (420).

[0203] Referring to FIG. 10b, the processor (410) can output audio as an audio signal of an audio channel (e.g., 5.1 channels) with fewer channels than the first audio channel when the wearable electronic device (401) moves from a central position (901) to a second distance (1021) further than the first distance (e.g., according to user gesture input) while displaying a virtual object (905) (e.g., virtual TV) that generates audio in a virtual space (900) through a display (420).

[0204] Referring to FIG. 10c, the processor (410) can output audio as an audio signal of a smaller number of audio channels (e.g., 2 channels) when a wearable electronic device (401) moves from a central position (901) to a third distance (1031) further than a second distance (e.g., according to user gesture input) while displaying a virtual object (905) (e.g., virtual TV) that generates audio in a virtual space (900) through a display (420).

[0205] As shown in FIGS. 10a to 10c, a processor (410) according to one embodiment can output audio as an audio signal of audio channels having a smaller number of channels as the distance between the center position (901) for spatial sound and the position of a virtual object (905) in virtual space increases, thereby allowing the user to feel the distance and spatial sense of the audio more realistically.

[0206] FIG. 11 is a diagram showing audio channels according to the size of the display screen of a virtual object according to one embodiment.

[0207] Referring to FIG. 11, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of a wearable electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) can identify audio channels (2ch, 5.1ch, 7.1ch+object) of different channel numbers depending on the size (1110, 1120, or 1130) of the display screen (screen) of a virtual object (1105) that generates audio in a virtual space (1100). A processor (410) according to one embodiment can identify audio channels having fewer channels (e.g., 7.1ch+object -> 5.1 channels -> 2 channels) as the size of the display screen of a virtual object (1105) decreases (e.g., 1130 -> 1120 -> 1110) through an audio channel change identification module (530). According to one embodiment, the size of the display screen may not be limited to a specific size, and the number of audio channels may not be limited to a specific number of channels.

[0208] FIG. 12a is a diagram showing a case in which the display size of a virtual object in a virtual space according to one embodiment is not smaller than a first size. FIG. 12b is a diagram showing a case in which the display size of a virtual object in a virtual space according to one embodiment is smaller than a first size and not smaller than a second size. FIG. 12c is a diagram showing a case in which the display size of a virtual object in a virtual space according to one embodiment is smaller than a second size.

[0209] Referring to FIG. 12a, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of a wearable electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) may output a first audio signal of a designated first audio channel (e.g., maximum audio channel or 12 channels) to provide spatial sound if the size of the display screen of a virtual object (1205) generating audio in a virtual space (900) is not smaller than a designated first size.

[0210] Referring to FIG. 12b, a processor (410) according to one embodiment may output audio signals of audio channels with a number of channels less than a specified first audio channel (e.g., maximum audio channel or 12 channels) if the size of the display screen of a virtual object (1206) generating audio in a virtual space (900) is smaller than a specified first size and not smaller than a second size.

[0211] Referring to FIG. 12c, a processor (410) according to one embodiment can output audio as an audio signal of fewer audio channels if the size of the display screen of a virtual object (1206) generating audio in a virtual space (900) is smaller than a specified second size.

[0212] FIG. 13 is a diagram illustrating an example in which a user of a wearable electronic device according to one embodiment is at a center position for spatial acoustics, and a first to third distance range is displayed on a display based on the center position.

[0213] Referring to FIG. 13, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of a wearable electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) <1301> As shown above, when a user of a wearable electronic device (401) is at a central position for spatial sound, a first object (1212) representing a first audio channel corresponding to a first distance range (1211) of a rectangle from the central position is displayed, a second object (1214) representing an audio channel corresponding to a second distance range (1213) from the first distance range (1211) and having fewer channels than the first audio channel (1212) is displayed, and a third object (1216) representing an audio channel corresponding to a third distance range (1215) and having fewer channels is displayed.

[0214] A processor (410) according to one embodiment is <1302> As shown above, when a user of a wearable electronic device (401) is at a central position for spatial sound, a first distance range (1222) in the form of beads laid out from the central position and a first object (1221) representing a first audio channel corresponding to the first distance range (1222) may be displayed, a second distance range (1224) from the first distance range (1222) and a second object (1223) representing an audio channel corresponding to the second distance range (1224) and having a smaller number of channels than the first audio channel (1221) may be displayed, and a third object (1225) representing an audio channel corresponding to the third distance range (1226) and having a smaller number of channels may be displayed.

[0215] A processor (410) according to one embodiment is <1303> As shown above, when a user of a wearable electronic device (401) is at a central position for spatial sound, a first object (1231) representing a first audio channel corresponding to a first distance range (1232) is displayed in a circular first distance range (1232) from the central position, a second object (1233) representing an audio channel corresponding to a second distance range (1224) from the first distance range (1232) and having fewer channels than the first audio channel (1231) is displayed, and a third object (1235) representing an audio channel corresponding to a third distance range (1236) and having fewer channels is displayed.

[0216] FIG. 14 is a diagram showing an example in which an audio channel that changes according to the distance between a center position for spatial acoustics and a virtual object is displayed on a display of a wearable electronic device according to one embodiment.

[0217] Referring to FIG. 14, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of a wearable electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) <1401> When the distance between the center position for spatial sound and the virtual object (1410) does not exceed a specified second distance, a first audio signal of a specified first audio channel can be output to provide spatial sound, and information indicating the first audio channel (e.g., 12ch) (1412) can be displayed on the display (420).

[0218] A processor (410) according to one embodiment is <1402> As such, when the distance between the center position for spatial sound and the virtual object (1420) exceeds a specified second distance, the first audio channel designated to provide spatial sound can be changed to an audio channel with fewer channels (e.g., 5.1ch) and information indicating the changed audio channel (e.g., 5.1ch) (1422) can be displayed on the display (420).

[0219] A processor (410) according to one embodiment is <1403> As such, the distance between the center position for spatial sound and the virtual object (1420) <1402> When exceeding a certain distance from, the audio channel (e.g., 5.1ch) can be changed to an audio channel with fewer channels (e.g., 2ch) and information indicating the changed audio channel (e.g., 2ch) (1432) can be displayed on the display (420).

[0220] FIG. 15 is a diagram showing an example in which audio channels according to the display screen size of a virtual object in a virtual space according to one embodiment are displayed on the display of a wearable electronic device.

[0221] Referring to FIG. 15, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of a wearable electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) <1501> As shown above, when the display screen size of a virtual object (1510) in a virtual space (1500) is large (e.g., 300 cm in width or height), an audio signal of an audio channel having a large number of channels (e.g., 12 channels) can be output, and information indicating an audio channel having a large number of channels can be displayed on the display using a border guide (1512) of a first color (or first thickness).

[0222] A processor (410) according to one embodiment is <1502> As such, when the display screen size of a virtual object (1520) in a virtual space (1500) is of a medium size (e.g., width or height 150 cm), an audio signal of an audio channel having a medium number of channels (e.g., 5.1 channels) is output, and information indicating an audio channel having a medium number of channels on the display can be displayed on the display using a border guide (1522) of a second color (or second thickness).

[0223] A processor (410) according to one embodiment is <1503> As such, when the display screen size of a virtual object (1530) in a virtual space (1500) is small (e.g., 50 cm in width or height), an audio signal of an audio channel having a small number of channels (e.g., 2 channels) is output, and information indicating an audio channel having a small number of channels on the display can be displayed using a border guide (1522) of a third color (or third thickness).

[0224] FIG. 16 is a diagram illustrating an example of specifying multiple spatial acoustic ranges based on multiple virtual objects when multiple virtual objects exist in a virtual space mapped to a real space according to one embodiment.

[0225] Referring to FIG. 16, a processor (e.g., processor (120) of FIG. 1 or processor (410) of FIG. 4) of a wearable electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1, wearable electronic device (200) of FIG. 2a and 2b, wearable electronic device (300) of FIG. 3a to 3c, or wearable electronic device (401) of FIG. 4) can set spatial acoustic regions (1610, 1620) for providing spatial acoustics based on the locations of a plurality of virtual objects (e.g., 1601, 1602, 1603, 1604, 1605) when a plurality of virtual objects exist in a virtual space (1600) mapped to a real space.

[0226] For example, the processor (410) may set a first spatial acoustic region (1610) for the first virtual objects (1601, 1602) and output audio signals of different audio channels for each distance range (1611, 1612, 1613) based on a first center position for spatial acoustics within the first spatial acoustic region (1610). For example, the processor (410) may set a second spatial acoustic region (1620) for the second virtual objects (1603, 1604, 1605) and output audio signals of different audio channels for each distance range (1621, 1622, 1623) based on a second center position for spatial acoustics within the second spatial acoustic region (1620).

[0227] The wearable electronic device of the present disclosure (e.g., the electronic device (101) of FIG. 1, the wearable electronic device (200) of FIG. 2a and FIG. 2b, the wearable electronic device (300) of FIG. 3a to FIG. 3c, or the wearable electronic device (401) of FIG. 4) can enable the user to hear audio generated from a virtual object in a virtual reality space more realistically by allowing the user to feel different senses of distance, different senses of direction, and / or different senses of depth as the user moves away from or closer to the virtual object relative to a central position for providing spatial sound based on a virtual object.

[0228] The wearable electronic device of the present disclosure (e.g., the electronic device (101) of FIG. 1, the wearable electronic device (200) of FIG. 2a and FIG. 2b, the wearable electronic device (300) of FIG. 3a to FIG. 3c, or the wearable electronic device (401) of FIG. 4) can provide an experience that allows the user to feel the audio generated by the virtual object in the virtual reality space more realistically by increasing the number of audio channels for spatial sound when the size of the virtual object's display screen is large and decreasing the number of audio channels when the size of the virtual object's display screen is small, thereby maximizing the effect of spatial sound when the user listens to audio while looking at the display screen of the virtual object generating audio in the virtual reality space.

[0229] In addition, various effects that are directly or indirectly identified through the present disclosure may be provided. The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0230] Furthermore, the embodiments disclosed in this disclosure are presented for the purpose of explaining and understanding the disclosed technical content and are not intended to limit the scope of the technology described in this disclosure. Accordingly, the scope of this disclosure should be interpreted to include all modifications or various other embodiments based on the technical concept of this disclosure.

[0231] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0232] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0233] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0234] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0235] A non-transient storage medium storing the commands of the present disclosure is configured such that when the commands are executed by a wearable electronic device (101, 200, 300, 401), the wearable electronic device is configured to perform at least one operation, wherein the at least one operation may include displaying a virtual object that generates audio in a virtual space through a display of the wearable electronic device while the wearable electronic device is worn by a user. The at least one operation may include identifying a first audio channel designated for providing spatial acoustics for the audio among a plurality of audio channels. The at least one operation may include identifying a center position designated for providing spatial acoustics for the audio. The at least one operation may include identifying a distance between the designated center position and a first position of the wearable electronic device in the virtual space. The above at least one operation may include an operation of outputting the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the distance between the specified center position and the first position does not exceed a specified first distance. The above at least one operation may include an operation of identifying a second audio channel having fewer channels than the first audio channel and outputting the audio as a second audio signal of the second audio channel through at least some of the plurality of speakers when the distance between the specified center position and the first position exceeds the specified first distance.

[0236] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0237] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a wearable electronic device (101, 200, 300, 401), Display(160, 251, 252, 340, 321, 420); At least one sensor (176, 317, 430); At least one camera (180, 211-1, 211-2, 213, 313, 314, 315, 316, 440); Multiple speakers (155, 455); At least one processor (120, 410); and It includes memory (130, 415) for storing instructions, When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device, While the above-mentioned wearable electronic device is worn by a user, a virtual object that generates audio in a virtual space is displayed through the above-mentioned display, and Identifying a first audio channel designated for providing spatial sound for the audio among a plurality of audio channels, and Identifying a designated center location for providing the spatial acoustics for the above audio, and Identifying the distance between the designated center position and the first position of the wearable electronic device in the virtual space, and If the distance between the designated center position and the first position does not exceed the designated first distance, the audio is output as a first audio signal of the first audio channel through at least some of the plurality of speakers, and A wearable electronic device that identifies a second audio channel different from the first audio channel when the distance between the designated center position and the first position exceeds the designated first distance, and outputs the audio as a second audio signal of the second audio channel through at least some of the plurality of speakers.

2. In Paragraph 1, A wearable electronic device in which, when the distance between the designated center position and the first position exceeds the designated first distance, the number of channels of the second audio channel decreases as the distance exceeds the limit.

3. In Paragraph 1 or 2, When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device, If the distance between the designated center position and the first position does not exceed the designated first distance, identify whether the distance between the designated center position and the second position of the virtual object exceeds the designated second distance, and If the distance between the designated center position and the second position does not exceed the designated second distance, the audio is output as a first audio signal of the first audio channel through at least some of the plurality of speakers, and If the distance between the designated center position and the second position exceeds the designated second distance, the audio is output through at least some of the plurality of speakers as the second audio signal of the second audio channel having fewer channels than the first audio channel, and A wearable electronic device in which, when the distance between the aforementioned designated center position and the aforementioned second position exceeds the aforementioned designated second distance, the number of the aforementioned second audio channels decreases as the distance exceeds the specified second distance.

4. In any one of paragraphs 1 through 3, When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device, If the distance between the designated center position and the first position does not exceed the designated first distance and the distance between the designated center position and the second position does not exceed the designated second distance, determine whether the size of the display screen of the virtual object is smaller than the first size, and If the size of the display screen of the virtual object is not smaller than the first size, the audio is output as the first audio signal of the first audio channel through at least some of the plurality of speakers, and When the size of the display screen of the virtual object is smaller than the first size, the audio is output through at least some of the plurality of speakers as the second audio signal of the second audio channel, which has fewer channels than the first audio channel, and A wearable electronic device in which the number of channels of the second audio channel decreases as the size of the display screen of the virtual object decreases.

5. In any one of paragraphs 1 through 4, When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device, Identifying the movement of the wearable electronic device while the wearable electronic device is worn by the user, and A wearable electronic device that updates the first position of the wearable electronic device in the virtual space based on the identification of the movement of the wearable electronic device.

6. In any one of paragraphs 1 through 5, When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device, Identifying the movement of the virtual object in the virtual space, and A wearable electronic device that updates the second position of the virtual object based on the identification of the movement of the virtual object in the virtual space.

7. In any one of paragraphs 1 through 6, When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device, A wearable electronic device that displays first information representing the first audio channel on the display when the first audio signal of the first audio channel is output, and displays second information representing the second audio channel on the display when the second audio signal of the second audio channel is output, wherein the first information and the second information are displayed differently.

8. In any one of paragraphs 1 through 7, When the above commands are executed individually or collectively by the at least one processor, the wearable electronic device, When the first audio signal of the first audio channel is output, a first object representing the first audio signal of the first audio channel is displayed in the virtual space through the display, and A wearable electronic device that displays a second object representing the second audio signal of the second audio channel in the virtual space through the display when the second audio signal of the second audio channel is output.

9. In any one of paragraphs 1 through 8, The above plurality of audio channels is a wearable electronic device including a mono channel, a stereo channel, a 5.1 channel, a 7.1 channel, a 12 channel, a multi-channel+Object, or ambisonic.

10. In any one of paragraphs 1 through 9, The above-mentioned first audio channel is a wearable electronic device comprising 12 channels.

11. A method for providing spatial acoustics of a virtual space in a wearable electronic device (101, 200, 300, 401), An operation of displaying a virtual object that generates audio in a virtual space through the display of the wearable electronic device while the wearable electronic device is worn by a user; An operation to identify a first audio channel designated for providing spatial sound for the audio among a plurality of audio channels; An action of identifying a designated center position for providing the spatial acoustics for the above audio; An operation to identify the distance between the designated center position and the first position of the wearable electronic device in the virtual space; An operation of outputting the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the distance between the designated center position and the first position does not exceed a designated first distance; A method comprising the operation of identifying a second audio channel different from the first audio channel when the distance between the designated center position and the first position exceeds the designated first distance, and outputting the audio as a second audio signal of the second audio channel through at least some of the plurality of speakers.

12. In Paragraph 11, A method in which the number of channels of the second audio channel decreases as the distance between the above-specified center position and the above-specified first position increases when the distance between the above-specified center position and the above-specified first position exceeds the above-specified first distance.

13. In Paragraph 11 or 12, An operation to identify whether the distance between the specified center position and the virtual object exceeds a specified second distance when the distance between the specified center position and the first position does not exceed a specified first distance; An operation of outputting the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the distance between the above-mentioned designated center position and the above-mentioned virtual object does not exceed a designated second distance; and The operation includes, when the distance between the designated center position and the virtual object exceeds a designated second distance, outputting the audio through at least some of the plurality of speakers as the second audio signal of the second audio channel having fewer channels than the first audio channel. A method in which the number of channels of the second audio channel decreases as the distance between the above-mentioned center position and the above-mentioned second position exceeds the above-mentioned second distance.

14. In any one of paragraphs 11 through 13, An operation to identify whether the size of the display screen of the virtual object is smaller than the first size when the distance between the designated center position and the first position does not exceed the designated first distance and the distance between the designated center position and the virtual object does not exceed the designated second distance; An operation of outputting the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the size of the display screen of the virtual object is not smaller than the first size; and When the size of the display screen of the virtual object is smaller than the first size, the operation includes outputting the audio through at least some of the plurality of speakers as the second audio signal of the second audio channel, which has fewer channels than the first audio channel. A method in which the number of channels of the second audio channel decreases as the size of the display screen of the virtual object decreases.

15. In a non-transient storage medium storing instructions, The above commands are configured to cause the wearable electronic device (101, 200, 300, 401) to perform at least one operation when executed by the wearable electronic device, wherein the at least one operation is, An operation of displaying a virtual object that generates audio in a virtual space through the display of the wearable electronic device while the wearable electronic device is worn by a user; An operation to identify a first audio channel designated for providing spatial sound for the audio among a plurality of audio channels; An action of identifying a designated center position for providing the spatial acoustics for the above audio; An operation to identify the distance between the designated center position and the first position of the wearable electronic device in the virtual space; An operation of outputting the audio as a first audio signal of the first audio channel through at least some of the plurality of speakers when the distance between the designated center position and the first position does not exceed a designated first distance; A storage medium comprising the operation of identifying a second audio channel different from the first audio channel when the distance between the specified center position and the first position exceeds the specified first distance, and outputting the audio as a second audio signal of the second audio channel through at least some of the plurality of speakers.

Images