Wearable electronic device for supporting camera activation notification and operation method thereof

Abstract

An electronic device and a method for operating the electronic device are provided. For example, a method of an electronic device may comprise the operations of: identifying whether there is an abnormality in a light emitter used to provide a notification notifying that a first camera is activated, on the basis that an execution request of the first camera is acquired; controlling the light emitter to output the notification, on the basis that it is identified that there is no abnormality in the light emitter; identifying whether an alternative notification method is available to provide an alternative notification notifying that the first camera is activated, on the basis that it is identified that there is an abnormality in the light emitter; and outputting the alternative notification by using the alternative notification method, on the basis that it is identified that the alternative notification method is available.

Description

Wearable electronic device supporting camera activation notification and method of operation thereof

[0001] The present disclosure relates to a wearable electronic device supporting camera activation notification and a method of operating the same.

[0002] With the advancement of digital technology, electronic devices are now available in various forms, such as smartphones, tablet personal computers (PCs), and personal digital assistants (PDAs). Furthermore, electronic devices are also being developed into wearable forms to enhance portability and accessibility.

[0003] For example, wearable electronic devices, such as augmented reality glasses (AR glasses), video see-through (VST) devices, and head-mounted displays (HMDs), are being developed to provide virtual spaces in virtual environments. The variety of services and additional functions offered by wearable electronic devices is steadily increasing. To enhance the utility of these electronic devices and satisfy the needs of diverse users, telecommunication service providers and electronic device manufacturers are competing to develop electronic devices that offer diverse functions and differentiate themselves from other companies. Accordingly, the various functions offered by wearable electronic devices are also becoming increasingly sophisticated.

[0004] Meanwhile, wearable electronic devices can capture images of the external environment through their cameras. In this case, for example, to protect the privacy of others, the wearable electronic device needs to provide an external notification indicating that the camera is activated.

[0005] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0006] According to one embodiment, a head-mounted display (HMD) device may be provided. The HMD device may include a first camera, a first display, a light emitter, at least one processor including a processing circuit, and a memory including at least one storage medium storing instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the HMD device to perform at least one operation. The at least one operation may include identifying, based on an acquisition of an execution request of the first camera, whether there is an abnormality in a light emitter configured to provide a notification indicating that the first camera is activated. The at least one operation may include controlling the light emitter to output the notification based on an identification that there is no abnormality in the light emitter. The at least one operation may include identifying, based on an identification that there is an abnormality in the light emitter, whether an alternative notification method is available for providing an alternative notification indicating that the first camera is activated. The at least one action may include outputting the alternative notification using the alternative notification method based on identifying that the alternative notification method is available.

[0007] According to one embodiment, a method of operating a HMD device may be provided. The method of operating the HMD device may include at least one operation. The at least one operation may include an operation of identifying, based on an acquisition of an execution request of the first camera, whether there is an abnormality in a light emitter configured to provide a notification notifying that the first camera is activated. The at least one operation may include an operation of controlling the light emitter to output the notification based on an identification that there is no abnormality in the light emitter. The at least one operation may include an operation of identifying, based on an identification that there is an abnormality in the light emitter, whether an alternative notification method is available for providing an alternative notification notifying that the first camera is activated. The at least one operation may include an operation of outputting the alternative notification using the alternative notification method based on an identification that the alternative notification method is available.

[0008] According to one embodiment, a storage medium storing at least one computer-readable instruction may be provided. The at least one instruction, when executed by at least a portion of at least one processor of an HMD device, may cause the HMD device to perform at least one operation. The at least one operation may include: identifying, based on an acquisition of an execution request of the first camera, whether an abnormality exists in a light emitter configured to provide a notification indicating that the first camera is activated. The at least one operation may include: controlling the light emitter to output the notification, based on an identification that there is no abnormality in the light emitter. The at least one operation may include: identifying, based on an identification that there is an abnormality in the light emitter, whether an alternative notification method is available for providing an alternative notification indicating that the first camera is activated. The at least one operation may include outputting the alternative notification using the alternative notification method, based on an identification that the alternative notification method is available.

[0009] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0010] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments of the present disclosure.

[0011] FIG. 2 is a diagram showing the configuration of a wearable electronic device according to an embodiment of the present disclosure.

[0012] FIGS. 3A through 3C illustrate front and rear views of a wearable electronic device according to one embodiment of the present disclosure.

[0013] FIG. 4 is a flowchart illustrating an operation for a wearable electronic device to provide a notification or alternative notification indicating that a camera is activated, according to one embodiment of the present disclosure.

[0014] FIG. 5A is a flowchart illustrating an operation of a wearable electronic device to determine whether to disable a camera based on a captured image, according to one embodiment of the present disclosure.

[0015] FIG. 5B is a flowchart illustrating an operation of a wearable electronic device to determine whether to disable a camera based on a captured image and user movement identification, according to one embodiment of the present disclosure.

[0016] FIG. 6 is a diagram illustrating an operation and camera status of a wearable electronic device for identifying an abnormality of an LED based on the current of the LED, according to one embodiment of the present disclosure.

[0017] FIG. 7 is a diagram for explaining an operation and camera state of a wearable electronic device that identifies an abnormality of an LED based on a connector connection state of the LED according to one embodiment of the present disclosure.

[0018] FIG. 8 is a diagram illustrating an operation and camera status of a wearable electronic device for identifying an abnormality of an LED based on light from the LED, according to one embodiment of the present disclosure.

[0019] FIG. 9A is a diagram illustrating an operation and camera state for determining whether to deactivate a camera based on a captured image by a wearable electronic device according to one embodiment of the present disclosure.

[0020] FIG. 9B is a diagram illustrating an operation and camera state for determining whether to deactivate a camera based on a captured image and user movement of a wearable electronic device according to one embodiment of the present disclosure.

[0021] FIG. 10A is a flowchart illustrating a method for a wearable electronic device to process a video upon detection of a video capture failure event, according to one embodiment of the present disclosure.

[0022] FIG. 10b is a flowchart illustrating a method for a wearable electronic device to process a video upon detection of a video recording resumption event, according to one embodiment of the present disclosure.

[0023] FIG. 11 shows an example of a screen including abnormal information according to one embodiment of the present disclosure.

[0024] FIG. 12 shows an example of a screen including video storage selection information according to one embodiment of the present disclosure.

[0025] FIG. 13 is a diagram illustrating a camera state according to one embodiment of the present disclosure.

[0026] FIG. 14 is a diagram illustrating a configuration of an electronic device according to one embodiment of the present disclosure.

[0027] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0028] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments of the present disclosure.

[0029] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0030] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0031] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where artificial intelligence is performed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0032] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0033] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0034] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0035] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0036] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0037] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0038] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0039] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0040] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0041] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0042] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0043] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0044] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0045] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a communication module (192) (e.g., a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0046] The communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The communication module (192) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The communication module (192) may support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0047] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0048] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0049] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface).

[0050] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0051] FIG. 2 is a diagram showing the configuration of a wearable electronic device according to an embodiment of the present disclosure.

[0052] The components and operations of the components described with reference to FIG. 2 may be partially or entirely identical to the components and operations of the components described with reference to FIG. 1. The components and operations of the components described with reference to FIG. 2 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 3A to 14, which will be described later.

[0053] Referring to FIG. 2, according to one embodiment, a wearable electronic device (200) (e.g., electronic device (101) of FIG. 1) may include a light output module (211), a display member (201), a camera module (250), a speaker (261), and / or a light emitter (270). However, the present disclosure is not limited thereto, and for example, according to another embodiment, the wearable electronic device (200) may include one or more additional components.

[0054] According to one embodiment, the light output module (211) (e.g., the display module (160) of FIG. 1) may include a light source capable of outputting an image and a lens for guiding the image to the display member (201). The light output module (211) may include, but is not limited to, a liquid crystal display, a digital mirror device, a liquid crystal on silicon, an organic light emitting diode, and / or a micro light emitting diode (micro LED).

[0055] According to one embodiment, the display member (201) (e.g., the display module (160) of FIG. 1) may include an optical waveguide (e.g., a waveguide). According to one embodiment, an output image of the optical output module (211) incident on one end of the optical waveguide may be propagated inside the optical waveguide and provided to a user. According to one embodiment, the optical waveguide may include at least one diffractive element (e.g., a diffractive optical element (DOE), a holographic optical element (HOE)) and / or a reflective element (e.g., a reflective mirror). For example, the optical waveguide may guide (e.g., direct) the output image of the optical output module (211) to the user's eyes by using at least one diffractive element or reflective element.

[0056] According to one embodiment, the camera module (250) (e.g., the camera module (180) of FIG. 1) can capture images (e.g., still images and / or moving images). According to one embodiment, the camera module (250) may be provided (or arranged) within a lens frame and provided (or arranged) around the display member (201). In the present disclosure, an image may be interpreted to include not only a still image but also a moving image. According to one embodiment, the camera module may include, but is not limited to, a first camera module (251), a second camera module (253), and a third camera module (255).

[0057] According to one embodiment, the first camera module (251) can capture and / or recognize the trajectory of the user's eye (e.g., pupil, iris) or gaze. According to one embodiment, the first camera module (251) can periodically or aperiodically transmit information related to the trajectory of the user's eye or gaze (e.g., trajectory information) to a processor (e.g., processor (120) of FIG. 1).

[0058] According to one embodiment, the second camera module (253) can capture an image of the outside. For example, the second camera module (253) can capture an image of the external environment in the front direction of the wearable electronic device (200). However, the present disclosure is not limited thereto, and for example, the second camera module (253) can be positioned at a position on the wearable electronic device (200) to capture an image in a direction other than the front direction of the wearable electronic device (200). For example, the second camera module (253) can capture an image in the side direction or the rear direction of the wearable electronic device (200).

[0059] According to one embodiment, the third camera module (255) can be used for hand detection and tracking, and recognition of user gestures. The user gestures can include, but are not limited to, hand motions. According to one embodiment, the third camera module (255) can be used for 3 degrees of freedom (3DoF), 6DoF head tracking, position (spatial, environmental) recognition, and / or movement recognition. According to one embodiment, the second camera module (253) can also be used for hand detection and tracking, and recognition of user gestures. According to one embodiment, at least one of the first camera module (251), the second camera module (253), and the third camera module (255) can be replaced with a sensor module (e.g., a LiDAR sensor). For example, the sensor module may include at least one of a vertical cavity surface emitting laser (VCSEL), a diode, an infrared (IR) sensor, an infrared (IR) diode, and / or a photodiode.

[0060] According to one embodiment, the speaker (261) (e.g., the audio output module (155) of FIG. 1) can output an audio signal (e.g., a sound and / or a virtual vibration sound). The speaker (261) is described as being configured in a member that is placed on the user's ear when the wearable electronic device (200) is worn in FIG. 2, but the present disclosure is not limited thereto, and the speaker may be configured in a different location depending on the implementation of the wearable electronic device (200).

[0061] In one embodiment, the light emitter (270) can emit light. For example, the light emitter (270) can emit light of at least one color (e.g., red, green, and / or blue). The light emitter (270) can include, but is not limited to, a light emitting diode (LED), a laser emitter, an electroluminescent device, a plasma emitter, and / or a chemiluminescent device. In one embodiment, the light emitter (270) can be used to provide a notification indicating an operating status of the second camera module (253). In one embodiment, the light emitter (270) can provide a first notification indicating that the second camera module (253) is activated by emitting light. For example, the light emitter (270) can provide a first notification indicating that the second camera module (253) is activated by emitting light of a first color (e.g., red). In one embodiment, the light emitter (270) may provide a second notification notifying that the second camera module (253) is deactivated by not emitting light. In one embodiment, the light emitter (270) may provide a second notification notifying that the second camera module (253) is deactivated by emitting light of a second color (e.g., blue) different from a first color (e.g., red) for the first notification. The light emitter (270) is described as being provided (or arranged) at a position adjacent to the second camera module (253) in FIG. 2, but the present disclosure is not limited thereto, and may be configured at a different position depending on the implementation of the wearable electronic device (200).

[0062] FIGS. 3A to 3C illustrate the front and back of a wearable electronic device according to one embodiment of the present disclosure.

[0063] The components and operations of the components described with reference to FIGS. 3A to 3C may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 and 2. The components and operations of the components described with reference to FIGS. 3A to 3C may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 4 to 14, which will be described later.

[0064] Referring to FIGS. 3A to 3C , according to one embodiment, the wearable electronic device (300) may include at least one first camera module (311, 312), at least one second camera module (313, 314, 315, 316), a depth sensor (317), a light emitter (340), and / or a second display (350). For example, on a first surface (or side) (310) of the housing, at least one first camera module (311, 312), at least one second camera module (313, 314, 315, 316), a depth sensor (317), a light emitter (340), and / or a second display (350) may be provided (or arranged) to obtain information related to the surrounding environment of the wearable electronic device (300) (e.g., the electronic device (101) of FIG. 1 ).

[0065] According to one embodiment, at least one first camera module (311, 312) can capture an image of the exterior of the wearable electronic device (300). For example, the first camera module (311, 312) can capture an image of the external environment in the front direction of the wearable electronic device (300).

[0066] According to one embodiment, at least one second camera module (313, 314, 315, 316) can acquire images while the wearable electronic device (300) is worn by a user. The second camera module (313, 314, 315, 316) can be used for hand detection, tracking, and recognition of user gestures (e.g., hand movements). The second camera module (313, 314, 315, 316) can be used for 3DoF, 6DoF head tracking, position (e.g., space, environment) recognition, and / or movement recognition. According to one embodiment, the first camera module (311, 312) can also be used for hand detection and tracking, and user gestures.

[0067] In one embodiment, the depth sensor (317) may be configured to transmit a signal and receive a signal reflected from a subject, and may be used for purposes such as time of flight (TOF) to determine the distance to an object. In one embodiment, one or more of the second camera modules (313, 314, 315, 316) may operate together with the depth sensor (317) to determine the distance to an object. For example, instead of or in addition to the depth sensor (317), the second camera modules (313, 314, 315, 316) may determine the distance to an object.

[0068] In one embodiment, the light emitter (340) can emit light. For example, the light emitter (340) can emit light of at least one color (e.g., red, green, and / or blue). The light emitter (340) can include, but is not limited to, a light emitting diode (LED), a laser emitter, an electroluminescent device, a plasma emitter, and / or a chemiluminescent device. In one embodiment, the light emitter (340) can be used to provide a notification indicating an operating status of the first camera module (311, 312). In one embodiment, the light emitter (340) can provide a first notification indicating that the first camera module (311, 312) is activated by emitting light. For example, the light emitter (340) can provide a first notification indicating that the first camera module (311, 312) is activated by emitting light of a first color (e.g., red). According to one embodiment, the light emitter (340) may provide a second notification notifying that the first camera module (311, 312) is deactivated by not emitting light. According to one embodiment, the light emitter (340) may provide a second notification notifying that the first camera module (311, 312) is deactivated by emitting light of a second color (e.g., blue) different from the first color (e.g., red) for the first notification. Although the light emitter (340) is described as being provided (or arranged) at a location adjacent to the first camera module (311, 312) in FIGS. 3A and 3C , the present disclosure is not limited thereto, and may be configured at other locations depending on the implementation of the wearable electronic device (300).

[0069] According to one embodiment, the second display (350) (and / or lens) may be provided (or disposed) on the first surface (or side) (310) of the wearable electronic device (300). According to one embodiment, the second display (350) may provide visual information to the outside of the wearable electronic device (300). For example, the second display (350) may be used to provide an alternative notification indicating the operating status of the first camera module (311, 312) instead of the light emitter (340).

[0070] According to one embodiment, face recognition camera modules (325, 326) and / or a first display (321) (and / or lens) may be arranged (or provided) on the second surface (or side) (320) of the housing.

[0071] According to one embodiment, the face recognition camera modules (325, 326) adjacent to the first display (321) may be used to recognize the user's face or to recognize and / or track both eyes of the user.

[0072] According to one embodiment, the first display (321) (and / or lens) may be provided (or disposed) on the second surface (or side) (320) of the wearable electronic device (300). According to one embodiment, the wearable electronic device (300) may not include the camera modules (315, 316) among the plurality of second camera modules (313, 314, 315, 316). According to one embodiment, the wearable electronic device (300) may further include at least one of the configurations illustrated in FIG. 2 in addition to the configurations illustrated in FIGS. 3A and 3B.

[0073] Referring to FIG. 3C, according to one embodiment, a wearable electronic device (300) may have a form factor (e.g., a head mounted display (HMD)) for being worn on a user's head. The wearable electronic device (300) may further include a strap for being fixed on a body part of the user, and / or a wearing member. The wearable electronic device (300) may include a volume button (331), a ventilation hole (333), a status indicator light (335), and a power button (337), and these configurations may be equally included in the wearable electronic device (300) illustrated in FIGS. 3A and 3B. According to one embodiment, the power button may include a fingerprint recognition sensor. In another example, the fingerprint recognition sensor may be provided separately from the power button. When worn on a user's head, the wearable electronic device (300) may provide a user experience based on AR, virtual reality (VR), and / or extended reality (or mixed reality). A wearable electronic device (300) configured in the form of an HMD may include components identical or similar to the components of FIGS. 3A and 3B described above.

[0074] According to one embodiment, the speaker (318) (e.g., the audio output module (155) of FIG. 1 or the speaker (261) of FIG. 2) may output an audio signal (e.g., a sound and / or a virtual vibration sound). The speaker (318) is described as being configured at a location adjacent to the ventilation hole (333) in FIGS. 3A to 3C , but the present disclosure is not limited thereto, and may be configured at other locations depending on the implementation of the wearable electronic device (200).

[0075] FIG. 4 is a flowchart illustrating an operation for a wearable electronic device to provide a notification or alternative notification indicating that a camera is activated, according to one embodiment of the present disclosure.

[0076] The components and operations of the components described with reference to FIG. 4 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 3. The components and operations of the components described with reference to FIG. 4 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 5 to 14, which will be described later.

[0077] Referring to FIG. 4, according to one embodiment, in operation 410, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A to 3C) (e.g., an HMD device) may obtain (or receive) a camera execution request of a camera (e.g., the second camera module (253) of FIG. 2 or the first camera modules (311, 312) of FIGS. 3A to 3C). The camera execution request may be referred to as a camera activation command or a camera activation request / command. According to one embodiment, the camera may obtain (e.g., capture) an image of the outside of the wearable electronic device. For example, the camera may capture an image of the external environment in the front direction of the wearable electronic device. In the present disclosure, a camera to be executed in response to the camera execution request may be referred to as a first camera.

[0078] According to one embodiment, the wearable electronic device may obtain a camera execution request based on receipt of a user input. For example, the wearable electronic device may obtain a camera execution request corresponding to the user input in response to receiving a user input requesting camera execution (e.g., a user gesture input). The user input may be obtained, for example, through a second camera (e.g., the third camera module (255) of FIG. 2 or the second camera modules (313, 314, 315, 316) of FIGS. 3A and 3C).

[0079] In one embodiment, the wearable electronic device may obtain a camera execution request based on the execution of an application. For example, the wearable electronic device may obtain a camera execution request corresponding to the application in response to the execution of an application utilizing a camera.

[0080] In one embodiment, the wearable electronic device may not activate the camera before performing operation 420 after a camera launch request is obtained. For example, the wearable electronic device may not activate the camera immediately even if a camera launch request is obtained. For example, the wearable electronic device may activate the camera based on the identification that there is no abnormality (e.g., a problem or issue) in the light emitting device (e.g., an LED) that provides a notification through operation 420, or based on the identification that an alternative notification method that provides an alternative notification is available through operation 440. In other words, the wearable electronic device may activate the camera based on the availability of a notification or an alternative notification, rather than immediately activating the camera even if a camera launch request is obtained. For example, the notification or alternative notification may indicate an operational state (e.g., an activated state) of the camera. Here, the term "abnormality" may be associated with and / or refer to a problem, issue, trouble, or the like, particularly as it relates to a light emitter and related functions (e.g., a non-functioning light emitter).

[0081] In one embodiment, the wearable electronic device may activate the camera before performing operation 420 after a camera launch request is obtained. For example, the wearable electronic device may activate the camera based on or in response to the acquisition of the camera launch request. In an exemplary case where it is identified through operation 420 that there is a problem with the light emitter that provides the notification and in an exemplary case where it is identified through operation 440 that an alternative notification method for providing an alternative notification is not available, the wearable electronic device may switch the activated camera to a disabled state. In other words, in an exemplary case where a camera launch request is obtained, the wearable electronic device may first activate the camera and then immediately deactivate the camera based on the identification that a notification indicating the operational state of the camera (e.g., an activated state) and / or an alternative notification is not available.

[0082] The following operations 420 to 460 may be performed when the camera is activated or deactivated. Each state of the camera may be, for example, as illustrated in FIG. 13.

[0083] According to one embodiment, in operation 420, the wearable electronic device may identify (or determine) whether there is a problem with a light emitter (e.g., light emitter 270 of FIG. 2 or light emitter 340 of FIG. 3c) (e.g., an LED) that provides a notification, based on the acquisition of a camera execution request. For example, the wearable electronic device may identify (or determine) whether there is a problem with the light emitter. In an exemplary case where it is determined that there is no problem with the light emitter, operation 430 may be performed. In an exemplary case where it is determined that there is a problem with the light emitter, operation 440 may be performed. In the present disclosure, the problem with the light emitter may include, but is not limited to, for example, that the light emitter does not operate normally (e.g., does not provide a notification normally). In an exemplary case where the light emitter fails to provide notifications normally due to at least one factor (e.g., light emitter failure, light emitter error, light emitter contact failure, light emitter fault, and / or light emitter bug), the wearable electronic device may identify that there is a problem with the light emitter.

[0084] In one embodiment, the light emitter may be used to provide a notification indicating the operational status of the camera. The operational status of the camera may include, but is not limited to, an active state indicating that the camera is active and / or a deactivated state indicating that the camera is deactivated. For example, the notification indicating the operational status of the camera may include a first notification indicating that the camera is active and / or a second notification indicating that the camera is deactivated. In this way, a person not wearing the wearable electronic device (e.g., a non-user or an outsider) can confirm that the camera is active through the first notification and that the camera is deactivated through the second notification. Through these notifications, an outsider can confirm whether the camera is capturing and / or storing captured images of themselves.

[0085] In one embodiment, the light emitter may provide a first notification indicating that the camera is activated by emitting light. For example, the light emitter may provide a first notification indicating that the camera is activated by emitting light of a first color (e.g., red). In one embodiment, the light emitter may provide a second notification indicating that the camera is deactivated by not emitting light. In one embodiment, the light emitter may provide a second notification indicating that the camera is deactivated by emitting light of a second color (e.g., blue) different from the first color (e.g., red) for the first notification.

[0086] According to one embodiment, a wearable electronic device can identify whether there is a problem with a light emitter based on a current of the light emitter acquired (or detected) through a sensor (e.g., a current sensor). For example, the wearable electronic device can identify that there is no problem with the light emitter based on the current flowing through the light emitter detected through the sensor being normal. For example, if the current flowing through the light emitter detected through the sensor is not normal, the wearable electronic device can identify that there is a problem with the light emitter. An example of an operation for determining a problem with a light emitter based on the current of the light emitter is described below with reference to FIG. 6.

[0087] According to one embodiment, the wearable electronic device can identify whether there is a problem in the light emitter based on a detection signal detected through at least one pin of a connector (e.g., the interface (177) or the connection terminal (178) of FIG. 1) for connecting (e.g., electrically connecting) at least one processor (or a printed circuit assembly (PBA) including at least one processor) of the wearable electronic device and the light emitter. If it is determined, based on the detection signal, that the light emitter and the at least one processor are normally connected through the connector, the wearable electronic device can identify that there is no problem in the light emitter. If it is determined, based on the detection signal, that the light emitter and the at least one processor are not normally connected through the connector, the wearable electronic device can identify that there is a problem in the light emitter. According to one embodiment, the connector can be connected to a printed circuit assembly including at least one processor. An example of an operation of determining a problem in the light emitter based on a detection signal of the connector is described below with reference to FIG. 7.

[0088] According to one embodiment, the wearable electronic device can identify whether there is a problem with the light emitter based on the light (light) from the light emitter. For example, the wearable electronic device can use a light-emitting detection device (e.g., a second camera (e.g., the second camera module (253) or the second camera modules (313, 314, 315, 316) of FIG. 2) or a sensor) to detect light emission, to determine whether light is emitted from the light emitter, and if it is confirmed that light is emitted from the light emitter, to identify that there is no problem with the light emitter, and if it is confirmed that light is not emitted from the light emitter, to identify that there is a problem with the light emitter. According to one embodiment, the wearable electronic device can use the light-emitting detection device to determine whether light is emitted from the light emitter at a time when the light emitter is activated (hereinafter, referred to as the activation time or on-timing) or at a time after the activation time in response to a camera execution request. The activation time of the light emitter can be, for example, a time after a specified period (e.g., 0.01 seconds) from a time when the camera execution request is acquired. A normal light emitter can begin emitting light at the time of activation. An example of an operation for determining an abnormality in a light emitter using a light detection device is described below with reference to FIG. 8.

[0089] According to one embodiment, in operation 430, the wearable electronic device may provide a notification notifying that the camera is activated using the light emitter based on the identification that there is no abnormality in the light emitter. According to one embodiment, the wearable electronic device may also display at least one first image acquired through the activated camera through a display (e.g., the display member (201) of FIG. 2 or the first display (321) of FIG. 3B) while providing the notification. For example, the wearable electronic device may display the at least one first image as a preview image through the display. The first image may include, but is not limited to, an image of an external environment in a front direction of the wearable electronic device captured through the camera.

[0090] In one embodiment, the light emitter may provide a notification that the camera is activated by emitting light. For example, the light emitter may provide a notification that the camera is activated by emitting light of a first color (e.g., red).

[0091] In one embodiment, the wearable electronic device may activate a camera that is in an inactive state based on the identification that there is no abnormality in the light emitter. For example, the wearable electronic device may switch the state of the camera from an inactive state to an activated state in response to the identification that there is no abnormality in the light emitter. In one embodiment, the wearable electronic device may maintain an activated camera in an activated state based on the identification that there is no abnormality in the light emitter.

[0092] In one embodiment, an activated camera can capture (e.g., capture) an image of the external environment of the wearable electronic device. For example, the activated camera can capture an image of the external environment facing the front of the wearable electronic device. The captured image can be displayed on a display. In the present disclosure, the display on which the image captured by the camera is displayed may be referred to as a first display.

[0093] According to one embodiment, at operation 440, based on the identification of a malfunction in the light emitter, the wearable electronic device may determine whether an alternative notification method for providing an alternative notification is available. If it is determined that an alternative notification method is available, operation 450 may be performed. If it is determined that an alternative notification method is not available, operation 460 may be performed.

[0094] According to one embodiment, the alternative notification may be a notification that notifies the camera's operating status (e.g., camera activation status) provided using an alternative notification method instead of a notification that notifies the camera's operating status provided using a light emitter. According to one embodiment, the alternative notification method may include at least one of a first alternative notification method that visually provides the alternative notification through a second display (e.g., the second display (350) of FIG. 3C), a second alternative notification method that audibly provides the alternative notification through a speaker (e.g., the speaker (261) of FIG. 2 or the speaker (318) of FIGS. 3A to 3C), or a third alternative notification method that visually provides the alternative notification through a second camera (e.g., the third camera module (255) of FIG. 2 or the second camera modules (313, 314, 315, 316 of FIGS. 3A and 3C)).

[0095] According to one embodiment, in operation 450, the wearable electronic device may provide an alternative notification indicating that the camera is activated using the alternative notification method based on the identification that an alternative notification method is available. According to one embodiment, the wearable electronic device may also display, through the display, at least one first image acquired through the activated camera, while providing the alternative notification. The first image may include, but is not limited to, an image of the external environment in the front direction of the wearable electronic device.

[0096] According to one embodiment, the wearable electronic device may display visual information (e.g., an indicator) corresponding to the replacement notification via the second display using the first replacement notification method. For example, the wearable electronic device may display the indicator corresponding to the replacement notification on a portion of the second display using the first replacement notification method. The color of the indicator corresponding to the replacement notification may be the same as the first color (e.g., red) corresponding to the notification of the light emitter, but is not limited thereto.

[0097] In one embodiment, the wearable electronic device may output a sound corresponding to the alternative notification through the speaker using the second alternative notification method. For example, the wearable electronic device may output a sound such as "The camera is active" or "The camera has been activated" using the second alternative notification method.

[0098] In one embodiment, the wearable electronic device may use the third alternative notification method to emit light corresponding to the alternative notification through an LED (e.g., an IR LED) included in the second camera. For example, the wearable electronic device may use the third alternative notification method to increase the power of the IR LED included in the second camera to emit light of a color corresponding to the alternative notification. The color corresponding to the alternative notification may be the same color as the first color (e.g., red) corresponding to the notification of the light emitter, but is not limited thereto.

[0099] In one embodiment, the wearable electronic device may activate a camera that is in an inactive state based on the identification that an alternative notification method is available. For example, the wearable electronic device may switch the state of the camera from an inactive state to an activated state in response to the identification that an alternative notification method is available. In one embodiment, the wearable electronic device may maintain an activated camera in an activated state based on the identification that an alternative notification method is available.

[0100] In one embodiment, the activated camera can capture images of the external environment (e.g., surroundings) of the wearable electronic device. For example, the activated camera can capture images of the external environment facing the front of the wearable electronic device. The captured images can be displayed on a display.

[0101] According to one embodiment, at operation 460, the wearable electronic device may perform a camera disable operation (operation 1) or a camera disable decision operation (operation 2) based on identifying that an alternative notification method is not available.

[0102] According to one embodiment, the camera deactivation operation (Operation 1) may include an operation of deactivating an activated camera or an operation of maintaining an inactive camera in a deactivated state. In an exemplary case where an alternative notification method is not available, the wearable electronic device may perform the camera deactivation operation to prevent images from being captured (e.g., photographed) through the camera. Through this camera deactivation operation, the wearable electronic device cannot capture images through the camera in a state where notifications and / or alternative notifications cannot be provided. For example, since other people may be present in the external environment facing the front of the wearable electronic device, there is a concern that taking images through the camera in an exemplary case where an image is taken without providing a notification or alternative notification indicating the activation status (photographing status) of the camera may invade the privacy of others (e.g., invasion of privacy due to illegal photography). Therefore, in a state where a notification or alternative notification cannot be provided, the camera can be deactivated through the above-described camera deactivation operation to prevent images from being captured.

[0103] However, even when notifications or alternative notifications cannot be provided, the camera may need to remain active for specific purposes (e.g., ensuring visibility for the safety of the wearable electronic device user). For this purpose, the camera deactivation decision operation described below may be performed instead of the camera deactivation operation described above.

[0104] According to one embodiment, the camera deactivation decision operation (Operation 2) may include, but is not limited to, an operation of determining whether to deactivate the activated camera based on a first image acquired through the activated camera. In an exemplary case where an alternative notification method is not available, the wearable electronic device may perform the camera deactivation decision operation to determine whether to deactivate the camera, and based on the determination to deactivate the camera, may disable the camera to prevent images from being captured through the camera, and based on the determination to activate the camera, may keep the camera activated to allow images to continue to be captured through the camera. Through this camera deactivation decision operation, the wearable electronic device may be able to capture images through the camera when necessary even when a notification and / or alternative notification cannot be provided. As described above, even when a notification or alternative notification cannot be provided, it may be necessary to keep the camera activated for a specific purpose while wearing the wearable electronic device. Therefore, even when a notification or alternative notification cannot be provided, the wearable electronic device may not immediately deactivate the camera, but may instead keep the camera activated or deactivate it according to the situation through the camera deactivation decision operation described above. An example of such a camera deactivation decision operation is described below with reference to FIGS. 5a and 5b.

[0105] FIG. 5A is a flowchart illustrating an operation of a wearable electronic device to determine whether to disable a camera based on a captured image, according to one embodiment of the present disclosure.

[0106] FIG. 5B is a flowchart illustrating an operation of a wearable electronic device to determine whether to disable a camera based on a captured image and user movement identification, according to one embodiment of the present disclosure.

[0107] The components and operations of the components described with reference to FIGS. 5a and 5b may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 4. The components and operations of the components described with reference to FIGS. 5a and 5b may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 6 to 14, which will be described later.

[0108] According to one embodiment, at least one of the operations of FIGS. 5A and 5B may be performed in exemplary cases where a notification indicating that the first camera is activated and / or an alternative notification cannot be provided. For example, at least one of the operations of FIGS. 5A and 5B may be performed in exemplary cases where a malfunction is identified in the light emitter providing the notification through operation 420 of FIG. 4 and an alternative notification method for providing the alternative notification through operation 440 of FIG. 4 is not available.

[0109] Referring to FIG. 5A, according to one embodiment, at operation 510, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A-3B) (e.g., an HMD device) may acquire a first image.

[0110] In one embodiment, the first image may include an image of the external environment surrounding the wearable electronic device. For example, the first image may include an image of the external environment in the front direction of the wearable electronic device captured by a camera (e.g., the second camera module (253) of FIG. 2 or the first camera modules (311, 312) of FIGS. 3A and 3B). In the example of FIG. 5A, the camera that captured the first image may be referred to as the first camera.

[0111] According to one embodiment, in operation 520, the wearable electronic device may identify whether a portion corresponding to a first object is present (e.g., visible) (or included) in the first image. If it is exemplary that the portion corresponding to the first object is present (or included) in the first image, operation 530 may be performed. If it is exemplary that the portion corresponding to the first object is not present (or included) in the first image, operation 540 may be performed.

[0112] In one embodiment, the first object may be an object that should not be captured by the camera without a notification or alternative notification. For example, the first object may be a person or figure or a part of a person (e.g., a face or head). The person associated with the first object may be someone other than the user of the wearable electronic device. In one embodiment, the first object may be a building or object that is a confidential object. For convenience of explanation, the first object will be described below as a person.

[0113] According to one embodiment, in operation 530, the wearable electronic device may disable the camera or prohibit capturing and / or storing an image (e.g., a video) through the camera based on identifying that a portion corresponding to a first object exists (e.g., is included) in the first image. For example, the wearable electronic device may prohibit capturing an image (e.g., a video). For example, if a portion corresponding to another person exists (e.g., is included) in the first image captured by the camera (e.g., a first image of the external environment in the front direction of the wearable electronic device), capturing and / or storing the image through the first camera without providing a notification or alternative notification may infringe upon the privacy of the other person. Therefore, there is a need for the wearable electronic device to control the camera to alleviate the concern of invading the privacy of the other person.

[0114] According to one embodiment, the wearable electronic device can deactivate the camera based on (or in response to) identifying that a portion corresponding to a first object is present (e.g., included) in the first image. In this way, the deactivated camera cannot capture images. According to an embodiment, the wearable electronic device can prevent the capture of images through the camera itself to protect the privacy of others. The wearable electronic device according to this embodiment may include, but is not limited to, a first type of wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2) that can secure a user's field of view even when the wearable electronic device does not display a display of an image captured through the first camera. The first type of wearable electronic device may include, but is not limited to, a wearable electronic device that provides an AR function, for example.

[0115] According to one embodiment, the wearable electronic device may keep the camera active based on (or in response to) identifying that a portion corresponding to a first object is present (e.g., included) in the first image, but may prevent the camera from capturing and / or storing an image (e.g., a video) through the camera. According to this embodiment, the wearable electronic device may display an image captured (e.g., captured) through the camera on a display (e.g., the display member (201) of FIG. 2 or the first display (321) of FIG. 3B) to ensure a view for a user of the wearable electronic device, but may prevent the image from being captured and stored in the form of a video to protect the privacy of others. The wearable electronic device according to this embodiment may be applied to, but is not limited to, a second type of wearable electronic device (e.g., the wearable electronic device (300) of FIGS. 3A to 3C) in which the wearable electronic device does not display an image captured through the camera and thus cannot be viewed by a user wearing the wearable electronic device. A second type of wearable electronic device may include, but is not limited to, a wearable electronic device that provides VR / VST functionality.

[0116] In one embodiment, the action of preventing the camera from capturing an image (e.g., a video) may include, for example, stopping (or disabling or turning off) the camera from capturing an image (e.g., a video) that is currently being captured. In the exemplary case where the image capture is stopped (or disabled or turned off), the image captured before the stop (e.g., an existing video) may or may not be saved, depending on the user's selection.

[0117] According to one embodiment, in operation 540, the wearable electronic device may keep the camera activated and allow an image (e.g., a video) to be captured and / or stored via the camera based on the identification that a portion corresponding to the first object is not included in the first image. For example, if a portion corresponding to another person, which is the first object, is not present (e.g., is not included) in the first image captured via the camera (e.g., a first image of the external environment in the front direction of the wearable electronic device), there is no concern about invasion of the privacy of another person even when the image is captured via the camera without providing a notification or alternative notification. Accordingly, there is no need to deactivate the first camera even if a notification or alternative notification is not provided.

[0118] According to one embodiment, the wearable electronic device may keep the camera activated based on (or in response to) identifying that a portion corresponding to a first object is not present (e.g., included) in the first image, and may allow an image (e.g., a video) to be captured and / or stored via the camera. The wearable electronic device according to this embodiment may not only allow an image captured by the camera to be displayed via a display to secure a view of a user of the wearable electronic device, but may also allow the image to be captured and stored in the form of a video.

[0119] In one embodiment, the act of allowing an image (e.g., a video) to be captured via the camera may include, but is not limited to, an act of continuing to capture an image (e.g., a video) currently being captured via the camera and / or an act of saving the captured image (e.g., a video).

[0120] In the embodiment of FIG. 5B described below, unlike the embodiment of FIG. 5A, after the wearable electronic device identifies, through operation 520, that a part corresponding to a first object does not exist (e.g., is not included) in the first image, the wearable electronic device may perform an operation of identifying a movement of a user of the wearable electronic device and permitting or prohibiting the capturing and / or storing of an image (e.g., a video) through a camera based on the result of the user movement identification.

[0121] Referring to FIG. 5B, according to one embodiment, at operation 510, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device) may acquire (e.g., capture) a first image. Operation 510 of FIG. 5B may be identical to operation 510 of FIG. 5A.

[0122] According to one embodiment, in operation 520, the wearable electronic device may identify whether a portion corresponding to the first object exists (e.g., is included) in the first image. If it is identified that the portion corresponding to the first object exists (e.g., is included) in the first image, operation 530 may be performed. If it is identified that the portion corresponding to the first object does not exist (e.g., is not included) in the first image, operation 541 may be performed. Operation 520 of FIG. 5B may be identical to operation 520 of FIG. 5A.

[0123] According to one embodiment, in operation 530, the wearable electronic device may disable the first camera or prohibit capturing and / or storing an image (e.g., a video) through the first camera based on identifying that a portion corresponding to the first object is present (e.g., included) in the first image. Operation 530 of FIG. 5B may be identical to operation 530 of FIG. 5A.

[0124] According to one embodiment, in operation 541, the wearable electronic device may determine whether a movement of the user of the wearable electronic device is identified based on the identification that a portion corresponding to the first object is not present (e.g., included) in the first image. In an exemplary case where the user movement is identified, operation 543 may be performed. In an exemplary case where the user movement is not identified, operation 545 may be performed. According to one embodiment, the wearable electronic device may determine whether a movement of the user of the wearable electronic device is identified based on sensing data acquired through at least one sensor (e.g., an acceleration sensor, a 6-axis sensor). According to one embodiment, the wearable electronic device may determine a movement of the wearable electronic device based on sensing data acquired through at least one sensor (e.g., an acceleration sensor, a 6-axis sensor). For example, in operation 441, the method may determine whether the wearable electronic device is moving based on sensing data acquired through at least one sensor.

[0125] According to one embodiment, the wearable electronic device may perform operation 541, for example, after performing operation 540 of FIG. 5A, if it is determined that a portion corresponding to the first object is not present (e.g., included) in the first image. In other words, operation 540 may be performed before operation 541.

[0126] According to one embodiment, the wearable electronic device may determine whether a movement of a user wearing the wearable electronic device is identified based on at least one image acquired through a second camera (e.g., the third camera module (255) of FIG. 2 or the second camera modules (313, 314, 315, 316) of FIGS. 3A and 3B). According to one embodiment, the wearable electronic device may determine whether a movement of a user wearing the wearable electronic device is identified based on sensing data acquired through at least one sensor (e.g., an acceleration sensor, a 6-axis sensor).

[0127] As described above, in the exemplary case where a portion corresponding to another person, i.e., a first image captured by a camera (e.g., a first image of the external environment facing the front of a wearable electronic device), is not included, there is no concern about invasion of another person's privacy even when the image is captured and / or stored by the camera without providing a notification or alternative notification. Therefore, even if a notification or alternative notification is not provided, there is no need to disable the camera. However, even in this case, if the user moves and the location and / or direction captured by the camera changes, there is a possibility that another person is present in the changed location and / or direction, which may again raise concerns about invasion of another person's privacy. Therefore, a solution is needed to address this issue.

[0128] According to one embodiment, in operation 543, the wearable electronic device may keep the camera active based on the user's movement being identified, but may prevent the camera from capturing and / or storing images (e.g., video) through the camera. For example, the wearable electronic device may keep the camera active based on the user's movement being identified, but may prevent the camera from capturing and / or storing images. The wearable electronic device according to this embodiment may display images captured by the camera on the display to secure a field of view for the user of the wearable electronic device. However, as described above, depending on the user's movement, the location and / or direction captured by the first camera may change, and another person may be present in the changed location and / or direction, which may infringe upon the privacy of another person. Therefore, the wearable electronic device may also prevent the capture and storage of images in the form of images (e.g., video) to protect the privacy of another person.

[0129] In one embodiment, an action to prevent a camera from capturing an image (e.g., a video) may include, for example, an action to stop (e.g., disable or turn off) a currently active image (e.g., video) capture (e.g., a capture function) via the camera. In an exemplary case where image capture is stopped (e.g., disabled or turned off), images captured prior to the stop (e.g., existing videos) may or may not be saved.

[0130] According to one embodiment, in operation 545, the wearable electronic device may keep the camera active and allow an image (e.g., a video) to be captured and / or stored through the camera based on the user's movement not being identified. For example, the wearable electronic device may keep the camera active and allow an image (e.g., a video) to be captured and / or stored through the camera in response to the user's movement not being identified. The wearable electronic device according to this embodiment may not only display an image captured by the camera on a display to ensure a view of the user of the wearable electronic device, but may also allow the image to be captured and stored in the form of an image (e.g., a video) without concern for invasion of privacy of others.

[0131] In one embodiment, the act of allowing an image (e.g., a video) to be captured via the camera may include, but is not limited to, an act of continuing to capture an image (e.g., a video) currently being captured via the camera and / or an act of saving the captured image (e.g., a video).

[0132] Hereinafter, with reference to FIGS. 6 to 8, various examples of operations for a wearable electronic device to identify whether a light-emitting device or a light-emitting device is abnormal are described. In the embodiments of FIGS. 6 to 8, for convenience of explanation, the light-emitting device is illustrated as an LED, but this is not limiting. For example, the descriptions of FIGS. 6 to 8 can be equally applied to other types of light-emitting devices.

[0133] FIG. 6 is a diagram illustrating an operation and camera status of a wearable electronic device for identifying an abnormality of an LED based on the current of the LED, according to one embodiment of the present disclosure.

[0134] The components and operations of the components described with reference to FIG. 6 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 5. The components and operations of the components described with reference to FIG. 6 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 7 to 14, which will be described later.

[0135] Referring to FIG. 6, according to one embodiment, an LED may be used as a light emitter (e.g., light emitter (270) of FIG. 2, or light emitter (340) of FIG. 3c) to provide notification of the operating status of the camera.

[0136] According to one embodiment, in operation 601, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device) may execute a camera (e.g., the second camera module (253) of FIG. 2 or the first camera modules (311, 312) of FIGS. 3A and 3C). The execution operation of the camera of operation 601 may include, but is not limited to, an operation of obtaining a camera execution request of operation 410 of operation 4. In the embodiment of FIG. 6, for convenience of explanation, an example in which the camera is activated through the execution operation of the camera is described.

[0137] In one embodiment, in operation 602, the wearable electronic device may activate an LED. For example, the wearable electronic device may activate the LED to provide a notification indicating that the camera is activated based on a camera activation request. The activated LED may emit light of a specified color (e.g., red) to provide a notification indicating that the camera is activated.

[0138] According to one embodiment, in operation 603, the wearable electronic device may determine whether the current of the LED is normal. If the LED current is determined to be normal, operation 607 may be performed. If the LED current is determined to be abnormal, operation 608 may be performed.

[0139] According to one embodiment, the wearable electronic device can obtain (or detect) the current flowing through the LED through a sensor (e.g., a current sensor) and compare it with a normal current to identify whether the LED current is normal.

[0140] According to one embodiment, in operation 607, the wearable electronic device can maintain the activation state of the LED based on the identification that the LED current is normal. The wearable electronic device can confirm that the LED is normally activated based on the identification that the LED current is normal. In this case, the wearable electronic device can identify that there is no abnormality in the LED (e.g., the light emitter) and maintain the activation state of the LED. Through this, a notification indicating that the camera is activated can be continuously provided through the LED.

[0141] According to one embodiment, in operation 609, the wearable electronic device may provide abnormality information and disable the camera based on the identification that the LED current is abnormal. The wearable electronic device may determine that the LED is not activating normally based on the identification that the LED current is abnormal. In this case, the wearable electronic device may identify that there is an abnormality in the LED, provide abnormality information to indicate that there is an abnormality in the LED, and disable the camera.

[0142] In one embodiment, the abnormality information may include information indicating an LED abnormality and / or information prompting the user to use the camera. An example of a screen including the abnormality information is described below with reference to FIG. 11.

[0143] In one embodiment, at operation 611, the wearable electronic device may determine whether use of the camera is selected. If use of the camera is determined to be selected, operation 615 may be performed. If use of the camera is determined to be not selected, operation 613 may be performed.

[0144] In one embodiment, the wearable electronic device can identify whether camera use has been selected based on user input. For example, the wearable electronic device can identify whether camera use has been selected based on user input (e.g., user gesture input) via anomaly information.

[0145] In one embodiment, at step 613, the wearable electronic device may maintain the camera in a disabled state based on the identification that the camera is not being used. In an example where the LED providing the notification is abnormal and the user does not wish to use the camera, the wearable electronic device may continue to maintain the camera in a disabled state.

[0146] In one embodiment, at step 615, the wearable electronic device may activate the camera based on the identification that the camera has been selected for use. The user may desire to use the camera even if the LED providing the notification is abnormal. In this case, the user may select the use of the camera through user input, and the wearable electronic device may activate the camera based on the user input selecting the use of the camera.

[0147] In one embodiment, at operation 617, the wearable electronic device may identify whether a replacement notification is available. For example, the wearable electronic device may identify whether a replacement notification is available by identifying whether an alternative notification method for providing the replacement notification is available. Operation 617 may include, for example, operation 440 of FIG. 4 . Examples of replacement notification methods and a description of providing replacement notifications using the replacement notification method may refer to the description of operation 440 of FIG. 4 . In an exemplary case where it is determined that a replacement notification is not available, operation 619 may be performed. In an exemplary case where it is determined that a replacement notification is available, operation 621 may be performed.

[0148] In one embodiment, at step 619, the wearable electronic device may disable the camera based on the identification that alternative notifications are not available. In this example, if the LED providing the notification is faulty and alternative notifications are not available, the wearable electronic device may disable the camera to protect the privacy of others, even if the user wishes to use the camera.

[0149] In one embodiment, at step 621, the wearable electronic device may keep the camera active based on the identification that a replacement notification is not available. In this example, if the LED providing the notification is faulty but a replacement notification is available, the wearable electronic device may activate the camera and provide a replacement notification indicating that the camera is active via the replacement notification method.

[0150] FIG. 7 is a diagram for explaining an operation and camera state of a wearable electronic device that identifies an abnormality of an LED based on a connector connection state of the LED according to one embodiment of the present disclosure.

[0151] The components and operations of the components described with reference to FIG. 7 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 6. The components and operations of the components described with reference to FIG. 7 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 8 to 14, which will be described later.

[0152] Referring to FIG. 7, according to one embodiment, an LED may be used as a light emitter (e.g., light emitter (270) of FIG. 2, or light emitter (340) of FIG. 3c) to provide notification of the operating status of the camera.

[0153] According to one embodiment, in operation 701, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device) may execute a camera (e.g., the second camera module (253) of FIG. 2 or the first camera modules (311, 312) of FIGS. 3A and 3C). The execution operation of the camera of operation 701 may include, but is not limited to, an operation of obtaining a camera execution request of operation 410 of operation 4. In the embodiment of FIG. 7, for convenience of explanation, an example in which the camera is not activated before operation 703 is described.

[0154] According to one embodiment, in operation 703, the wearable electronic device may identify whether the connection of a connector for connecting (e.g., electrically connecting) at least one processor of the wearable electronic device and an LED is normal. If the connector connection is identified as normal, operation 705 may be performed. If the connector connection is identified as not normal, operation 707 may be performed.

[0155] According to one embodiment, a wearable electronic device can identify whether a connector connection is normal based on a detection signal detected through at least one pin of the connector. For example, if the detection signal satisfies a specified condition (e.g., a normal signal condition), the wearable electronic device can identify that the connector connection is normal. In this case, the wearable electronic device can confirm that an LED and at least one processor are normally connected through the connector and identify that there is no problem with the LED. For example, if the detection signal does not satisfy a specified condition, the wearable electronic device can identify that the connector connection is abnormal. In this case, the wearable electronic device can confirm that an LED and at least one processor are not normally connected through the connector and identify that there is a problem with the LED.

[0156] According to one embodiment, in operation 705, the wearable electronic device may activate the camera based on the identification that the connector connection is normal. The wearable electronic device may also confirm that the LED is normal based on the identification that the connector connection is normal. In this case, the wearable electronic device may activate the camera and provide a notification indicating that the camera is activated via the activated LED.

[0157] According to one embodiment, in operation 707, the wearable electronic device may provide abnormality information based on the identification that the connector connection is abnormal. The wearable electronic device may identify an LED abnormality based on the identification that the connector connection is abnormal. Accordingly, the wearable electronic device may provide abnormality information to indicate an LED abnormality.

[0158] In one embodiment, the abnormality information may include first information indicating an LED abnormality and / or second information prompting the user to use the camera. An example of a screen including the abnormality information is described below with reference to FIG. 11.

[0159] In one embodiment, at operation 709, the wearable electronic device may determine whether use of the camera is selected. If it is determined that use of the camera is not selected, operation 711 may be performed. If it is determined that use of the camera is selected, operation 713 may be performed.

[0160] In one embodiment, the wearable electronic device can identify whether camera use has been selected based on user input. For example, the wearable electronic device can identify whether camera use has been selected based on user input (e.g., user gesture input) via anomaly information.

[0161] In one embodiment, at step 711, the wearable electronic device may maintain the camera in a disabled state based on the identification that the use of the camera is not selected. In an exemplary case where the LED providing the notification is abnormal and the user does not wish to use the camera, the wearable electronic device may continue to maintain the camera in a disabled state.

[0162] According to one embodiment, in operation 713, the wearable electronic device may activate the camera based on the identification that the camera has been selected for use. The user may desire to use the camera even if the LED providing the notification is abnormal. In this case, the user may select the use of the camera through user input, and the wearable electronic device may activate the camera based on the user input selecting the use of the camera.

[0163] According to one embodiment, in operation 715, the wearable electronic device may identify whether an alternative notification is available. For example, the wearable electronic device may identify whether an alternative notification is available by identifying whether an alternative notification method for providing the alternative notification is available. Operation 715 may include, but is not limited to, operation 440 of FIG. 4 . Examples of alternative notification methods and descriptions of providing alternative notifications using the alternative notification methods may be applied to the description of operation 440 of FIG. 4 . In an exemplary case where it is determined that an alternative notification is not available, operation 717 may be performed. In an exemplary case where it is determined that an alternative notification is available, operation 719 may be performed.

[0164] In one embodiment, at step 717, the wearable electronic device may disable the camera based on the identification that alternative notifications are not available. In this example, if the LED providing the notification is faulty and alternative notifications are not available, the wearable electronic device may disable the camera to protect the privacy of others, even if the user wishes to use the camera.

[0165] In one embodiment, at step 719, the wearable electronic device may keep the camera active based on the identification that a replacement notification is not available. In this example, if the LED providing the notification is faulty but a replacement notification is available, the wearable electronic device may activate the camera and provide a replacement notification indicating that the camera is active via the replacement notification method.

[0166] FIG. 8 is a diagram illustrating an operation and camera status of a wearable electronic device for identifying an abnormality of an LED based on light from the LED, according to one embodiment of the present disclosure.

[0167] The components and operations of the components described with reference to FIG. 8 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 7. The components and operations of the components described with reference to FIG. 8 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 9 to 14, which will be described later.

[0168] Referring to FIG. 8, according to one embodiment, an LED may be used as a light emitter (e.g., light emitter (270) of FIG. 2, or light emitter (340) of FIG. 3c) to provide notification of the operating status of the camera.

[0169] According to one embodiment, in operation 801, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device) may execute a camera (e.g., the second camera module (253) of FIG. 2 or the first camera modules (311, 312) of FIGS. 3A and 3C). The execution operation of the camera of operation 801 may include, but is not limited to, an operation of obtaining a camera execution request of operation 410 of operation 4. In the embodiment of FIG. 8, for convenience of explanation, an example in which the camera is not activated before operation 803 is described.

[0170] According to one embodiment, in operation 803, the wearable electronic device may identify light (or light) from an LED. In an exemplary case where the LED light is identified, operation 805 may be performed. In an exemplary case where the LED light is not identified, operation 807 may be performed.

[0171] According to one embodiment, the wearable electronic device can identify whether light is emitted from an LED by using a light-emitting detection device (e.g., a second camera (e.g., the third camera module (255) or the second camera module (313, 314, 315, 316) of FIG. 2) or a sensor) for detecting light emission. The wearable electronic device can identify that there is no problem with the LED when it is identified that light is emitted from the LED (when LED light is identified), and can identify that there is a problem with the LED when it is not identified that light is emitted from the LED (when LED light is not identified).

[0172] In one embodiment, the wearable electronic device can use a light detection device to determine whether the LED is emitting light at the time the LED is activated (hereinafter referred to as the activation time or on-timing) or at a time after the activation time. The activation time of the LED may be, for example, a time after a specified period of time (e.g., 0.01 seconds) from the time a camera execution request is received. A normal LED may begin emitting light at the activation time.

[0173] According to one embodiment, in operation 805, the wearable electronic device may activate the camera based on the identification of the LED light. The wearable electronic device may determine that the LED is not abnormal based on the identification of the LED light. In this case, the wearable electronic device may activate the camera and provide a notification indicating that the camera is activated via the activated LED.

[0174] According to one embodiment, in operation 807, the wearable electronic device may provide abnormality information based on the LED light not being identified. The wearable electronic device may identify that there is an abnormality in the LED (e.g., a light emitter) based on the LED light not being identified. Accordingly, the wearable electronic device may provide abnormality information to indicate that there is an abnormality in the LED.

[0175] In one embodiment, the abnormality information may include information indicating an LED abnormality and / or information prompting the user to use the camera. An example of a screen including the abnormality information is described below with reference to FIG. 11.

[0176] According to one embodiment, at operation 809, the wearable electronic device may determine whether use of the camera is selected. If it is determined that use of the camera is not selected, operation 811 may be performed. If it is determined that use of the camera is selected, operation 813 may be performed.

[0177] In one embodiment, the wearable electronic device can identify whether camera use has been selected based on user input. For example, the wearable electronic device can identify whether camera use has been selected based on user input (e.g., user gesture input) via anomaly information.

[0178] In one embodiment, at step 811, the wearable electronic device may maintain the camera in a disabled state based on the identification that the use of the camera is not selected. In an example where the LED providing the notification is abnormal and the user does not wish to use the camera, the wearable electronic device may continue to maintain the camera in a disabled state.

[0179] According to one embodiment, in operation 813, the wearable electronic device may activate the camera based on the identification that the use of the camera has been selected. The user may desire to use the camera even if the LED providing the notification is abnormal. In this case, the user may select (e.g., decide) to use the camera through user input, and the wearable electronic device may activate the camera based on the user input selecting the use of the camera.

[0180] According to one embodiment, at operation 815, the wearable electronic device may identify whether an alternative notification is available. For example, the wearable electronic device may identify whether an alternative notification is available by identifying whether an alternative notification method for providing the alternative notification is available. Operation 815 may include, but is not limited to, operation 440 of FIG. 4 . Examples of alternative notification methods and descriptions of providing alternative notifications using the alternative notification methods may be applied to the description of operation 440 of FIG. 4 . In an exemplary case where it is determined that an alternative notification is not available, operation 817 may be performed. In an exemplary case where it is determined that an alternative notification is available, operation 819 may be performed.

[0181] In one embodiment, at step 817, the wearable electronic device may disable the camera based on the identification that alternative notifications are not available. In this example, if the LED providing the notification is faulty and alternative notifications are not available, the wearable electronic device may disable the camera to protect the privacy of others, even if the user wishes to use the camera.

[0182] In one embodiment, at step 819, the wearable electronic device may keep the camera active based on the identification that a replacement notification is not available. In this example, if the LED providing the notification is faulty but a replacement notification is available, the wearable electronic device may activate the camera and provide a replacement notification indicating that the camera is active via the replacement notification method.

[0183] FIG. 9A is a diagram illustrating an operation and camera state for determining whether to deactivate a camera based on a captured image by a wearable electronic device according to one embodiment of the present disclosure.

[0184] FIG. 9B is a diagram illustrating an operation and camera state for determining whether to deactivate a camera based on a captured image and user movement of a wearable electronic device according to one embodiment of the present disclosure.

[0185] The components and operations of the components described with reference to FIGS. 9a and 9b may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 8. The components and operations of the components described with reference to FIGS. 9a and 9b may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 10 to 14, which will be described later.

[0186] Fig. 9a may be an embodiment related to the operations of Fig. 5a. Fig. 9a may be an embodiment related to the operations of Fig. 5b.

[0187] Referring to FIG. 9A, according to one embodiment, in operation 901, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device) may execute a camera (e.g., the second camera module (253) of FIG. 2 or the first camera modules (311, 312) of FIGS. 3A and 3C). The execution operation of the camera of operation 901 may include, but is not limited to, an operation of obtaining a camera execution request of operation 410 of operation 4. In the embodiment of FIG. 9A, for convenience of explanation, an example in which the camera is activated and an image (e.g., video) capturing function is activated before operation 903 is described.

[0188] According to one embodiment, at operation 903, the wearable electronic device may identify whether a notification or a replacement notification is available. The operation of identifying whether a notification is available may be, for example, the same as or include operation 420 of FIG. 4 . The operation of identifying whether a replacement notification is available may be, for example, the same as or include operation 440 of FIG. 4 . In an exemplary case where a notification or replacement notification is identified as available, operation 905 may be performed. In an exemplary case where a notification or replacement notification is identified as not available, operation 907 may be performed.

[0189] According to one embodiment, at operation 905, the wearable electronic device may keep the camera active based on identifying that a notification or alternative notification is available.

[0190] According to one embodiment, in operation 907, the wearable electronic device may identify whether a portion corresponding to a first object exists (e.g., is included) in an image acquired through a camera. If it is identified that a portion corresponding to the first object does not exist (e.g., is not included) in the image, operation 909 may be performed. If it is identified that a portion corresponding to the first object exists (e.g., is included) in the image, operation 911 may be performed. Operation 907 may be, for example, identical to operation 520 of FIGS. 5A and 5B .

[0191] In one embodiment, at operation 909, the wearable electronic device may keep the camera activated and continue capturing an image (e.g., a video) based on determining that a portion corresponding to the first object is not present (e.g., included) in the image. Operation 909 may be, for example, identical to operation 540 of FIG. 5A.

[0192] In one embodiment, at operation 911, the wearable electronic device may disable the camera or stop an image (e.g., video) recording function based on identifying that a portion corresponding to a first object is present (e.g., included) in the image. Operation 911 may be, for example, identical to operation 530 of FIGS. 5A and 5B .

[0193] In one embodiment, the wearable electronic device can repeatedly perform operations 907 to 909 while the camera remains activated. This allows operations 907 to 909 to be performed for each image continuously captured by the activated camera, thereby continuously determining whether a person is present (e.g., included) in the image while the camera is activated, even if a notification or alternative notification cannot be provided.

[0194] Referring to FIG. 9B, according to one embodiment, in operation 901, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device) may execute a camera (e.g., the second camera module (253) of FIG. 2 or the first camera modules (311, 312) of FIGS. 3A and 3C). The execution operation of the camera of operation 901 may include, but is not limited to, an operation of obtaining a camera execution request of operation 410 of operation 4. In the embodiment of FIG. 9B, for convenience of explanation, an example in which the camera is activated and an image (e.g., video) capturing function is activated before operation 903 is described.

[0195] According to one embodiment, at operation 903, the wearable electronic device may identify whether a notification or a replacement notification is available. The operation of identifying whether a notification is available may be, for example, the same as or include operation 420 of FIG. 4 . The operation of identifying whether a replacement notification is available may be, for example, the same as or include operation 440 of FIG. 4 . In an exemplary case where a notification or replacement notification is identified as available, operation 905 may be performed. In an exemplary case where a notification or replacement notification is identified as not available, operation 907 may be performed.

[0196] According to one embodiment, at operation 905, the wearable electronic device may keep the camera active based on identifying that a notification or alternative notification is available.

[0197] According to one embodiment, in operation 907, the wearable electronic device may identify whether a portion corresponding to a first object exists (e.g., is included) in an image acquired through a camera. If it is identified that a portion corresponding to the first object does not exist (e.g., is not included) in the image, operation 909 may be performed. If it is identified that a portion corresponding to the first object exists (e.g., is included) in the image, operation 911 may be performed. Operation 907 may be, for example, identical to operation 520 of FIGS. 5A and 5B .

[0198] In one embodiment, at operation 909, the wearable electronic device may keep the camera activated and maintain the image (e.g., video) capturing function based on determining that a portion corresponding to the first object is not present (e.g., included) in the image. Operation 909 may be, for example, identical to operation 540 of FIG. 5A.

[0199] In one embodiment, at operation 911, the wearable electronic device may disable the camera or stop the image (e.g., video) recording function based on identifying that a portion corresponding to a first object is included in the image. Operation 911 may be, for example, identical to operation 530 of FIGS. 5A and 5B .

[0200] In one embodiment, at operation 913, the wearable electronic device may determine whether a user movement of the wearable electronic device is identified. If the user movement is identified, operation 917 may be performed. If the user movement is not identified, operation 919 may be performed.

[0201] According to one embodiment, the wearable electronic device may determine whether movement of a user wearing the wearable electronic device is identified based on at least one image acquired through a second camera (e.g., the third camera module (255) or the second camera module (313, 314, 315, 316) of FIG. 2).

[0202] According to one embodiment, in operation 915, the wearable electronic device may keep the camera activated and continue capturing images (e.g., video) based on the user's movements not being identified. Operation 915 may be, for example, identical to operation 545 of FIG. 5B or may include at least one of the detailed operations of operation 545.

[0203] In one embodiment, at operation 917, the wearable electronic device may keep the camera active but stop capturing images (e.g., video) based on the user's movement being identified. Operation 917 may be, for example, identical to operation 543 of FIG. 5B or may include at least one of the detailed operations of operation 543.

[0204] FIG. 10A is a flowchart illustrating a method for a wearable electronic device to process a video upon detection of a video capture failure event, according to one embodiment of the present disclosure.

[0205] FIG. 10b is a flowchart illustrating a method for a wearable electronic device to process a video upon detection of a video recording resumption event, according to one embodiment of the present disclosure.

[0206] The components and operations of the components described with reference to FIGS. 10a and 10b may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 9b. The components and operations of the components described with reference to FIGS. 10a and 10b may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 11 to 14, which will be described later.

[0207] Referring to FIG. 10A, according to one embodiment, in operation 1010, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device) may execute a camera (e.g., the second camera module (253) of FIG. 2 or the first camera modules (311, 312) of FIGS. 3A and 3C) and activate a video recording function. As a result, the camera is activated, and a video may be captured through the activated camera. The camera execution operation of operation 1010 may include, but is not limited to, an operation of obtaining a camera execution request of operation 410 of operation 4, for example.

[0208] According to one embodiment, in operation 1020, the wearable electronic device may detect a video recording disable event (or a video recording stop event) while recording a video through a camera. Based on the detection of the video recording disable event (or the video recording stop event) while recording a video through the camera, the wearable electronic device may stop recording a video (or turn off the video recording function) or deactivate the camera. In an exemplary case where the video recording disable event (or the video recording stop event) is not detected while recording a video through the camera, the wearable electronic device may maintain recording a video through the camera and store the recorded video.

[0209] According to one embodiment, a video recording unavailable event (or a video recording stop event) may be detected when a situation occurs in which video recording is unavailable (or a situation in which video recording must be stopped) while video recording is being performed through an activated camera. The situation in which video recording is unavailable (video recording unavailable situation) or the situation in which video recording must be stopped (video recording stop situation) may include, but is not limited to, a first situation in which a notification is unavailable, a second situation in which a notification is unavailable and no alternative notification is available, a third situation in which neither a notification nor an alternative notification is available and a first object (e.g., a person) is identified as being present (e.g., included) in the captured image, and / or a fourth situation in which neither a notification nor an alternative notification is available and a first object (e.g., a person) is not present (e.g., included) in the captured image, but movement of a user of the wearable electronic device is identified. The first situation can be detected when, during video recording through the camera, an abnormality of a light emitting device (e.g., an LED) is identified, for example, through operation 420 of FIG. 4, operation 605 of FIG. 6, operation 703 of FIG. 7, or operation 803 of FIG. 8. The second situation can be detected when, during video recording through the camera, it is identified that an alternative notification method is not available, for example, through operation 440 of FIG. 4, operation 617 of FIG. 6, operation 715 of FIG. 7, or operation 815 of FIG. 8. The third situation can be detected when, during video recording through the camera, it is identified that a portion corresponding to the first object exists (is included) in an image captured, for example, through operation 520 of FIGS. 5A and 5B or operation 907 of FIGS. 9A and 9B. The fourth situation can be detected when the movement of the user wearing the wearable electronic device is identified during video recording via the camera, for example, through operation 541 of FIG. 5b or operation 913 of FIG. 9b.

[0210] In one embodiment, at operation 1030, the wearable electronic device may determine whether video storage is selected. If video storage is determined to be selected, operation 1040 may be performed. If video storage is determined to be not selected, operation 1050 may be performed.

[0211] According to one embodiment, a wearable electronic device may provide video storage selection information for video storage. The video storage selection information may include first information for notifying that a video recording situation has occurred and / or second information for prompting the user for video storage selection. According to one embodiment, the wearable electronic device may identify whether video storage is selected based on user input (e.g., user gesture input) via the video storage selection information. An example of a screen including the video storage selection information is described below with reference to FIG. 12 .

[0212] According to one embodiment, in operation 1040, the wearable electronic device may store the captured video before the video recording is stopped (e.g., disabled) based on identifying that video storage is selected. According to one embodiment, in operation 1050, the wearable electronic device may not store the captured video before the video recording is stopped (e.g., turned off or disabled) based on identifying that video storage is not selected. According to one embodiment of the present disclosure, the video captured before the video recording is stopped may be referred to as an existing video. As such, in the embodiment of FIG. 5A, when a video recording unavailable situation (or a video recording unavailable situation) occurs, whether to store the existing video may be determined through a user's selection.

[0213] Referring to FIG. 10B, according to one embodiment, in operation 1010, a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device) may execute a camera (e.g., the second camera module (253) of FIG. 2 or the first camera modules (311, 312) of FIGS. 3A and 3C) and activate a video recording function. As a result, the camera is activated and a video may be recorded through the camera. Operation 1010 of FIG. 10B may be identical to operation 1010 of FIG. 10A.

[0214] According to one embodiment, in operation 1020, the wearable electronic device may detect a video recording disable event (or a video recording stop event) while recording a video through the camera. The wearable electronic device may stop recording a video or deactivate the camera based on the detection of the video recording disable event (or the video recording stop event) while recording a video through the camera. In an exemplary case where the video recording disable event (or the video recording stop event) is not detected while recording a video through the camera, the wearable electronic device may maintain recording a video through the camera and store the recorded video. Operation 1020 of FIG. 10B may be identical to operation 1020 of FIG. 10A.

[0215] According to one embodiment, in operation 1021, the wearable electronic device may identify whether a video recording resume event (or a video recording start event) is detected. If the video recording resume event (or a video recording start event) is detected, in an exemplary case, operation 1022 may be performed. If the video recording resume event (or a video recording start event) is not detected, in an exemplary case, operation 1030 may be performed.

[0216] In one embodiment, a video recording resume event (or video recording start event) may be detected when a situation in which video recording can be resumed (or a video recording start situation) occurs while the camera is disabled or video recording is stopped (e.g., the video recording function is disabled or turned off). The video recording possible situation (video recording possible situation) or the video recording start situation (video recording start situation) may include, but is not limited to, a first situation in which a notification is available, a second situation in which an alternative notification is available, a third situation in which neither a notification nor an alternative notification is available but a first object (e.g., a person) is identified as not being present (e.g., included) in the captured image, and / or a fourth situation in which neither a notification nor an alternative notification is available and a first object (e.g., a person) is not identified as being present (e.g., included) in the captured image and no movement of the user of the wearable electronic device is identified. A first situation can be detected when, during camera deactivation or video recording stop, an abnormality of a light emitter (e.g., an LED) is not identified, for example, through operation 420 of FIG. 4, operation 605 of FIG. 6, operation 703 of FIG. 7, or operation 803 of FIG. 8. A second situation can be detected when, during camera deactivation or video recording stop, it is identified that an alternative notification method is available, for example, through operation 440 of FIG. 4, operation 617 of FIG. 6, operation 715 of FIG. 7, or operation 815 of FIG. 8. A third situation can be detected when, during video recording stop, it is identified that a portion corresponding to the first object is not present (e.g., included) in the captured image, for example, through operation 520 of FIGS. 5A and 5B or operation 907 of FIGS. 9A and 9B. A fourth situation can be detected when the movement of the user wearing the wearable electronic device is not identified during the video recording stop, for example, through operation 541 of FIG. 5b or operation 913 of FIG. 9b.

[0217] According to one embodiment, in operation 1022, the wearable electronic device may resume video recording based on detecting a video recording resume event, and manage to connect a video recorded before the video recording was stopped with a new video recorded after the video recording was resumed. The wearable electronic device may perform a camera activation operation and / or a video recording function activation operation to resume (or initiate) video recording. According to one embodiment, a video recorded before the video recording was stopped may be referred to as an existing video, and a new video recorded after the video recording was resumed (or initiated) may be referred to as a new video. By connecting the existing video and the new video, a single connected video may be created and stored.

[0218] According to one embodiment, at operation 1030, the wearable electronic device may determine whether video storage is selected. If it is determined that video storage is selected, operation 1040 may be performed. If it is determined that video storage is not selected, operation 1050 may be performed. Operation 1030 of FIG. 10B may be identical to operation 1030 of FIG. 10A.

[0219] According to one embodiment, in operation 1040, the wearable electronic device may store the captured video before the video recording is stopped based on identifying that video storage is selected. According to one embodiment, in operation 1050, the wearable electronic device may store the captured video before the video recording is stopped (e.g., the video recording function is disabled or turned off) based on identifying that video storage is not selected. Operations 1040 and 1050 of FIG. 10B may be identical to operations 1040 and 1050 of FIG. 10A, respectively.

[0220] FIG. 11 shows an example of a screen including abnormal information according to one embodiment of the present disclosure.

[0221] The components and operations of the components described with reference to FIG. 11 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 10. The components and operations of the components described with reference to FIG. 11 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 12 to 14, which will be described later.

[0222] Referring to FIG. 11, a wearable electronic device (e.g., a wearable electronic device (200) of FIG. 2 or a wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device) may provide abnormality information. According to one embodiment, the wearable electronic device may display a screen providing abnormality information to notify that there is an abnormality in a light-emitting device (e.g., an LED) or a notification using the light-emitting device through a display (e.g., a display member (201) of FIG. 2 or a first display (321) of FIG. 3B).

[0223] According to one embodiment, the abnormal information may include first information for notifying the user of an abnormality in the light emitter and / or second information for inquiring the user about the use of the camera. The first information may be provided, for example, through a message such as "Camera activation notification abnormality occurred!!" as exemplified in the screen displayed on the left display of FIG. 11 . The second information may be provided, for example, through a message such as "Continue using the camera?" as exemplified in the screen displayed on the right display of FIG. 11 .

[0224] In one embodiment, the wearable electronic device may provide a user interface for selecting camera use (e.g., continuing to use the camera) as the abnormal information, together with the second information. The user interface for selecting camera use may include, but is not limited to, a first selectable item (e.g., a "yes" item) for selecting camera use and / or a second selectable item (e.g., a "no" item) for not selecting camera use. The user interface for selecting camera use may be displayed on the same screen (e.g., a screen displayed on the right display of FIG. 11 ) along with the second information, but is not limited thereto.

[0225] In one embodiment, the wearable electronic device can identify whether the use of the camera (e.g., continued use) is selected based on user input (e.g., user gesture input) obtained through a user interface for selecting camera use. For example, if the wearable electronic device obtains (or receives) a user input (e.g., user gesture input) for selecting a first selectable item, the wearable electronic device can identify that the use of the camera (e.g., continued use) is selected. In this case, the wearable electronic device can keep the camera activated or activate it. This allows the camera to continue to be used according to the user's selection even when a notification via the light emitter cannot be provided. For example, if the wearable electronic device obtains (or receives) a user input (e.g., user gesture input) for selecting a second selectable item, the wearable electronic device can identify that the use of the camera (e.g., continued use) is not selected. In this case, the wearable electronic device can keep the camera in an inactive state or deactivate it. This allows the camera to be disabled at the user's option if notifications via the light source cannot be provided.

[0226] FIG. 12 shows an example of a screen including video storage selection information according to one embodiment of the present disclosure.

[0227] The components and operations of the components described with reference to FIG. 12 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 10. The components and operations of the components described with reference to FIG. 12 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 13 to 14, which will be described later.

[0228] Referring to FIG. 12, a wearable electronic device (e.g., a wearable electronic device (200) of FIG. 2 or a wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device) may provide video storage selection information. According to one embodiment, the wearable electronic device may display a screen providing video storage selection information for video storage through a display (e.g., a display member (201) of FIG. 2 or a first display (321) of FIG. 3B).

[0229] According to one embodiment, the video storage selection information may include first information for notifying the user that a video recording situation has occurred, and / or second information for prompting the user for a video storage selection. The first information may be provided, for example, via a message such as "Video recording unavailable!!" as illustrated on the screen displayed on the left side of FIG. 12 . The second information may be provided, for example, via a message such as "Video storage?" as illustrated on the screen displayed on the right side of FIG. 12 .

[0230] In one embodiment, the wearable electronic device may provide a user interface for selecting video storage along with second information as video storage selection information. The user interface for selecting video storage may include a first selectable item (e.g., a "Yes" item) for selecting to store an existing video and / or a second selectable item (e.g., a "No" item) for not selecting to store an existing video. The user interface for selecting to store the video may be displayed on the same screen (e.g., the screen shown on the right display of FIG. 12 ) along with the second information, but is not limited thereto.

[0231] According to one embodiment, the wearable electronic device can identify whether the storage of an existing video is selected based on user input (e.g., a user gesture input) obtained through a user interface for selecting video storage. For example, if the wearable electronic device obtains (or receives) a user input (e.g., a user gesture input) for selecting a first selectable item, the wearable electronic device can identify that the storage of an existing video is selected. In this case, the wearable electronic device can store the existing video based on the user's selection. For example, if the wearable electronic device obtains (or receives) a user input (e.g., a user gesture input) for selecting a second selectable item, the wearable electronic device can identify that the storage of an existing video is not selected. In this case, the wearable electronic device may not store the existing video based on the user's selection.

[0232] FIG. 13 is a diagram illustrating a camera state according to one embodiment of the present disclosure.

[0233] The components and operations of the components described with reference to FIG. 13 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 12. The components and operations of the components described with reference to FIG. 12 may be partially or entirely identical to the components and operations of the components described with reference to FIG. 14, which will be described later.

[0234] Referring to FIG. 13, according to one embodiment, the operational state of a camera (e.g., the second camera module (253) of FIG. 2 or the first camera module (311, 312) of FIGS. 3A and 3C) may include an activated state (1310) or a deactivated state (1320).

[0235] According to one embodiment, the activation state (1310) may be a state in which the camera is activated. The activation state (1310) may include, but is not limited to, a first activation state (1311) and / or a second activation state (1312).

[0236] According to one embodiment, the first activation state (1311) may be a state in which the camera is activated, but the image (e.g., video) capture and / or storage function is not activated. The camera in the first activation state (1311) may acquire (e.g., capture) an image of the external environment of a wearable electronic device (e.g., the wearable electronic device (200) of FIG. 2 or the wearable electronic device (300) of FIGS. 3A to 3B) (e.g., an HMD device). For example, the camera in the first activation state (1311) may acquire an image of the external environment in the front direction of the wearable electronic device. According to one embodiment, when the wearable electronic device is the device of FIGS. 3A to 3C, the image acquired in the first activation state (1311) may be displayed, for example, as a preview image, through a display of the wearable electronic device (e.g., the first display (321) of FIG. 3B) to provide a user wearing the wearable electronic device with an image of the external environment. According to one embodiment, when the wearable electronic device is the device of FIGS. 2A and 2B, the image acquired in the first activation state (1311) may not be displayed, for example, as a preview image, through a display of the wearable electronic device (e.g., the first display (321) of FIG. 3B). The image acquired in the first activation state (1311) may not be stored.

[0237] According to one embodiment, the second activation state (1312) may be a state in which the camera is activated and the image (e.g., video) capture and / or storage function is also activated. The camera in the second activation state (1312) may capture still or video images of the external environment. For example, the camera in the second activation state (1310) may capture and store still or video images of the external environment in the front direction of the wearable electronic device. The captured images may be displayed on the display of the wearable electronic device, thereby providing a user wearing the wearable electronic device with images of the external environment. The captured images may or may not be stored depending on the user's selection.

[0238] According to one embodiment, the wearable electronic device may, in response to detecting an image (e.g., video) capture initiation event (1303), transition an activation state (1310) of the camera from a first activation state (1311) to a second activation state (1312). The image capture initiation event (1303) may include a video capture resume event (e.g., a video capture resume event (or a video capture initiation event) of operation 1021 of FIG. 10B ). The image capture initiation event (1303) may be detected, for example, when at least one of the situations of the video capture resume event (or the video capture initiation event) of operation 1021 of FIG. 10B is identified. In an exemplary case of transitioning from the first activation state (1311) to the second activation state (1312), the wearable electronic device may activate an image capture and / or storage function.

[0239] According to one embodiment, the wearable electronic device may switch the camera activation state (1310) from the second activation state (1312) to the first activation state (1311) in response to detecting an image (e.g., video) capture stop event (1303). According to one embodiment, the image capture stop event may be detected when a situation occurs during image capture through the activated camera to stop or prohibit image capture. The image capture stop event (1304) may include an image capture disable event (e.g., the video capture disable event (or video capture stop event) of operation 1020 of FIGS. 10A and 10B ). The image capture stop event (1304) may be detected when at least one of the situations of the video capture disable event (or video capture stop event) of operation 1020 of FIGS. 10A and 10B is identified, for example. In an exemplary case where the wearable electronic device transitions from the second activation state (1312) to the first activation state (1311), the wearable electronic device may disable the image capture and / or storage function.

[0240] In one embodiment, the disabled state (1320) may be a state in which the camera is disabled. A camera in the disabled state (1320) cannot capture (e.g., capture) images of the external environment of the wearable electronic device. Therefore, captured images cannot be displayed on the display as preview images.

[0241] According to one embodiment, the wearable electronic device may transition the state of the camera from the inactive state (1320) to the active state (1310) in response to detecting an activation event (1301) in the inactive state (1320). The activation event (1301) may be detected, for example, when an abnormality of a light emitter (e.g., an LED) is not identified through operation 420 of FIG. 4 , operation 605 of FIG. 6 , operation 703 of FIG. 7 , or operation 803 of FIG. 8 . The activation event (1301) may be detected, for example, when it is identified that an alternative notification method is available through operation 440 of FIG. 4 , operation 617 of FIG. 6 , operation 715 of FIG. 7 , or operation 815 of FIG. 8 . An activation event (1301) may be detected, for example, when it is identified that a portion corresponding to a first object is not present (e.g., not included) in an image captured through operation 520 of FIGS. 5A and 5B or operation 907 of FIGS. 9A and 9B . An activation event (1301) may be identified, for example, when a movement of a user wearing a wearable electronic device is not identified through operation 541 of FIG. 5B or operation 913 of FIG. 9B .

[0242] According to one embodiment, the wearable electronic device may transition the state of the camera from the activated state (1310) to the deactivated state (1320) in response to detecting a deactivation event (1302) in the activated state (1310). The deactivation event (1302) may be detected when an abnormality in a light emitter (e.g., an LED) is identified, for example, through operation 420 of FIG. 4 , operation 605 of FIG. 6 , operation 703 of FIG. 7 , or operation 803 of FIG. 8 . The deactivation event (1302) may be detected when it is identified, for example, that an alternative notification method is not available, through operation 440 of FIG. 4 , operation 617 of FIG. 6 , operation 715 of FIG. 7 , or operation 815 of FIG. 8 . A deactivation event (1302) may be detected when it is identified that a portion corresponding to a first object exists (e.g., is included) in an image captured through, for example, operation 520 of FIGS. 5A and 5B or operation 907 of FIGS. 9A and 9B. A deactivation event (1302) may be detected when a movement of a user wearing a wearable electronic device is identified through, for example, operation 541 of FIG. 5B or operation 913 of FIG. 9B.

[0243] FIG. 14 is a diagram illustrating a configuration of an electronic device according to one embodiment of the present disclosure.

[0244] The components and operations of the components described with reference to FIG. 14 may be partially or entirely identical to the components and operations of the components described with reference to FIGS. 1 to 13.

[0245] Referring to FIG. 14, an electronic device (1400) (e.g., the electronic device (101) of FIG. 1, the wearable electronic device (200) of FIG. 2, or the wearable electronic device (300) of FIGS. 3A to 3C) may include at least one processor (1410), at least one memory (1420), at least one camera (1430), at least one display (1440), and / or a light-emitting device (1450). However, the present disclosure is not limited thereto, and for example, according to another embodiment, the electronic device (1400) may include one or more other components. For example, the electronic device (1400) may further include at least one sensor and / or other components (e.g., a communication circuit) necessary to perform at least one operation. According to one embodiment, the electronic device (1400) may be implemented identically or similarly to the electronic device (101) of FIG. 1, the wearable electronic device (200) of FIG. 2, or the wearable electronic device (300) of FIGS. 3A to 3C.

[0246] According to one embodiment, the electronic device (1400) may be worn by a user (e.g., worn on the user's head). For example, the electronic device (1400) may be implemented as either an AR type HMD device (e.g., the wearable electronic device (200) of FIG. 2) or a VR / VST type HMD device (e.g., the wearable electronic device (300) of FIGS. 3A to 3C).

[0247] According to one embodiment, at least one processor (1410) of an electronic device (1400) (e.g., processor (120) of FIG. 1 ) may be electrically and / or operatively connected to one another by a configuration such as a memory (1420), a camera (1430), a display (1440), and / or a light-emitting device (1450) via a communication bus.

[0248] According to one embodiment, at least one processor (1410) may control the overall operation of the electronic device (1400) and perform at least one operation of the electronic device (1400) (e.g., at least one of the operations described above in FIGS. 1 to 13). The processor (1410) may perform operations or data processing related to control and / or communication of at least one other component of the electronic device (1400). The processor (1410) may include at least one processing circuit that executes instructions stored in the memory (1420).

[0249] According to one embodiment, at least one processor (1410) may include various processing circuits and / or multiple processors. One or more of the at least one processor (1410) may be individually and / or collectively configured to perform various functions described herein. The processor (1410) may include various processing circuits and / or multiple processors. In this disclosure, when "a processor," "at least one processor," and "one or more processors" are described as being configured to perform various functions, these terms may cover, for example, situations where one processor performs some of the recited functions and other processor(s) perform other parts of the recited functions. For example, as used herein, the term "processor" may encompass various processing circuits including at least one processor, wherein one or more of the at least one processors may be individually and / or collectively configured to perform the various functions described herein in a distributed manner. However, the present disclosure is not limited thereto, and for example, embodiments of the present disclosure may also include, but are not limited to, situations where a single processor can perform all of the functions cited. Additionally, at least one processor (390) may comprise a combination of processors that perform various functions cited / disclosed, for example, in a distributed manner. At least one processor (1410) may execute program instructions to achieve or perform various functions.

[0250] According to one embodiment, at least one processor (1410) may include, but is not limited to, at least one of a central processing unit (CPU), an NPU, a graphics processing unit (GPU), a micro processing unit (MPU), a micro controller unit (MCU), an application processor (AP), a communication processor (CP), a system on chip (SoC), or an integrated circuit (IC), a sensor hub, a supplementary processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), and may have multiple cores.

[0251] According to one embodiment, the memory (1420) (e.g., the memory (130) of FIG. 1) may store various data that may be used to control the operation of each component of the electronic device (1400). The memory (1420) may include, for example, at least one storage medium that stores a plurality of application programs used in the electronic device (1420), data for controlling the operation of the electronic device (1400), and instructions. The instructions stored in the memory (1420), when executed by at least one processor (1410), may cause the electronic device (1400) to perform at least one operation (e.g., at least one of the operations described above in FIGS. 1 to 13).

[0252] According to one embodiment, the memory (1420) can store at least one program for processing and controlling the processor (1410), and can store input and / or output data. The memory (1420) can also store at least one artificial intelligence model. The memory (1420) can include at least one of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD (secure digital) or XD (extreme digital) memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk, but is not limited thereto. According to one example, a web storage or a cloud server that performs a storage function on the Internet may be operated by the electronic device (1400).

[0253] According to one embodiment, at least one camera (1430) can convert light input from the outside into an electrical signal. For example, at least one camera (1430) can convert light input from the outside of the housing of the electronic device (1440) or the lower housing of the camera into an electrical signal. The electronic device (1400) can capture an image (e.g., a still image or a moving image) through the camera (1430). The electronic device (1400) can include at least one camera (1430). At least one camera (1430) may include a first camera (e.g., the second camera module (253) of FIG. 2 or the first camera modules (311, 312) of FIGS. 3A and 3C), a second camera (e.g., the third camera module (255) of FIG. 2 or the second camera modules (313, 314, 315, 316) of FIGS. 3A and 3C) and / or a third camera (e.g., the first camera module (251) of FIG. 2 or the face recognition camera modules (325, 326) of FIG. 3).

[0254] According to one embodiment, at least one display (1440) can visually provide information to an external party (e.g., a user) of the electronic device (1400). The at least one display (1440) can include a first display (e.g., the display member (201) of FIG. 2 or the first display (321) of FIG. 3B) and / or a second display (e.g., the second display (350) of FIG. 3C).

[0255] In one embodiment, at least one light emitting device (1450) can emit light. For example, the light emitting device (1450) can emit light of at least one color. The at least one light emitting device (1450) can include at least one light emitter (e.g., light emitter (270) of FIG. 2 or light emitter (340) of FIG. 3C) (e.g., an LED). The light emitting device (1450) can be used to provide notification of the operating status of the camera (1430).

[0256] According to one embodiment of the present disclosure, when capturing an external environment using a camera in an HMD device, a notification indicating that the camera is activated is necessary to protect the privacy of others around the camera. This notification may be provided, for example, via a light-emitting device (e.g., an LED). The HMD device must verify whether this notification is provided normally and take appropriate action based on the confirmation.

[0257] According to one embodiment, the HMD device can determine whether there is an abnormality in a light-emitting device that provides a notification that the camera is activated, and perform actions based on the determination.

[0258] In one embodiment, in an exemplary case where there is a problem with the light-emitting device providing the notification, the HMD device can determine whether an alternative notification method is available that provides an alternative notification indicating that the camera is activated instead of the notification using the light-emitting device, and perform actions based on the determination.

[0259] In one embodiment, in an exemplary case where there is a problem with the light-emitting device providing the notification and an alternative notification method for providing an alternative notification is not available, the HMD device may determine whether the camera is available based on an image captured by the camera and / or movements of a user wearing the HMD device, and perform actions based on the determination.

[0260] According to one embodiment of the present disclosure, an HMD device (e.g., an electronic device (101) of FIG. 1, a wearable electronic device (200) of FIG. 2, a wearable electronic device (300) of FIGS. 3A to 3C, or an electronic device (1400) of FIG. 14) includes a first camera (e.g., a second camera module (253) of FIG. 2, the first camera modules (311, 312) of FIGS. 3A and 3C, or a camera (1430) of FIG. 14), a first display (e.g., a display member (201) of FIG. 2, the first display (321) of FIG. 3B, or the display (1400) of FIG. 14), a light emitter (e.g., a light emitter (270) of FIG. 2, or a light emitter (340) of FIG. 3, or a light emitter (1450) of FIG. 14), and at least one processor including a processing circuit (e.g., a processor (120) of FIG. 1 or The HMD device may include a memory (e.g., a memory (130) of FIG. 1 or a memory (1420) of FIG. 14) including at least one storage medium storing instructions (e.g., a processor (1410) of FIG. 14). The instructions, when individually or collectively executed by the at least one processor, may cause the HMD device to perform at least one operation. The at least one operation may include an operation of identifying whether there is an abnormality in the light emitter used to provide a notification notifying that the first camera is activated based on obtaining an execution request of the first camera. The at least one operation may include an operation of providing the notification using the light emitter based on identifying that there is no abnormality in the light emitter. The at least one operation may identify whether an alternative notification method used to provide an alternative notification notifying that the first camera is activated is available based on identifying that there is an abnormality in the light emitter.

[0261] According to one embodiment, the at least one action may include providing the alternative notification using the alternative notification method based on identifying that the alternative notification method is available.

[0262] In one embodiment, the at least one action may include: identifying whether a portion corresponding to the first object is present (e.g., included) within the first image based on identifying that the alternative notification method is not available; and disabling the first camera or preventing capturing via the first camera based on identifying that a portion corresponding to the first object is present (e.g., included) within the first image.

[0263] According to one embodiment, the HMD device may include a second camera (e.g., the third camera module (255) of FIG. 2 or the second camera module (313, 314, 315, 316) of FIG. 3), and the at least one operation may include an operation of determining whether movement of a user wearing the HMD device is identified based on at least one image acquired through the second camera, based on identification that a portion corresponding to a first object is not present (e.g., included) in the first image; an operation of maintaining the first camera in an activated state but prohibiting filming through the first camera if the movement of the user is identified; and an operation of maintaining the first camera in an activated state and allowing filming through the first camera if the movement of the user is not identified.

[0264] In one embodiment, the at least one action may include disabling the first camera if it is identified that the alternative notification method is not available.

[0265] According to one embodiment, the HMD device includes a second display (e.g., the second display (350) of FIG. 3c), and the alternative notification method corresponds to a method of visually providing the alternative notification through the second display, wherein the second display and the first camera are provided (or arranged) on a first surface (e.g., side) of a housing of the HMD device, and the first display may be provided (or arranged) on a second surface (e.g., side) of the housing.

[0266] According to one embodiment, the HMD device includes a speaker (e.g., speaker (216) of FIG. 2 or speaker (318) of FIGS. 3A to 3C), and the alternative notification method may correspond to a method of providing the alternative notification audibly through the speaker.

[0267] In one embodiment, the HMD device includes a second camera, and the alternative notification method may visually provide the alternative notification via an IR emitting LED included in the second camera.

[0268] In one embodiment, the HMD device may include a sensor. In one embodiment, the act or process of identifying (e.g., determining) whether there is a problem with the light emitter may include an act of identifying whether there is a problem with the light emitter based on a current of the light emitter obtained through the sensor.

[0269] According to one embodiment, the HMD device includes a connector (e.g., interface (177) or connection terminal (178) of FIG. 1) for connecting the light emitter to a PCB including the at least one processor, and the operation of identifying whether there is a problem with the light emitter may include an operation of identifying whether the light emitter and the at least one processor are electrically connected based on a detection signal detected through at least one pin of the connector, and identifying whether there is a problem with the light emitter based on the identification result.

[0270] In one embodiment, the HMD device may include a second camera. In one embodiment, the operation or process of identifying (e.g., determining) whether there is a problem with the light emitter may include identifying, via the second camera, whether light is emitted through the light emitter, and identifying, based on the identification result, whether there is a problem with the light emitter.

[0271] The embodiments of this document and the terminology used herein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0272] The term "module" used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0273] One embodiment of the present document may be implemented as software (e.g., a program (140) of FIG. 1) including one or more instructions stored in a storage medium (e.g., an internal memory (136) of FIG. 1 or an external memory (138) of FIG. 1) readable by a machine (e.g., an electronic device (101) of FIG. 1). For example, a processor (e.g., a processor (120) of FIG. 1) of the machine (e.g., an electronic device (101) of FIG. 1) may call at least one instruction from among the one or more instructions stored from the storage medium and execute at least one of the one or more instructions. This enables the machine to operate to perform at least one function according to the called at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0274] According to one embodiment, the method according to one embodiment disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the exemplary case where the computer program product is distributed online, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0275] According to one embodiment, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to one embodiment, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to one embodiment, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In the HMD (head-mounted display) device, A first camera for capturing an image in the front direction of the HMD device; First display; light emitter; At least one processor comprising a processing circuit; and A memory comprising at least one storage medium storing instructions, said instructions causing said HMD device to: Based on the acquisition of the execution request of the first camera, identifying whether there is an abnormality in the light emitter configured to provide a notification that the first camera is activated; Controlling the light emitter to output the notification based on the identification that there is no abnormality in the light emitter; Based on the identification that there is an abnormality in the above light emitter, identifying whether an alternative notification method is available to provide an alternative notification that the first camera is activated; An HMD device configured to cause the alternative notification to be output using the alternative notification method based on the identification that the alternative notification method is available.

2. In paragraph 1, The above instructions, when individually or collectively executed by the at least one processor, cause the HMD device to: Based on the identification that the above alternative notification method is not available, identifying whether a portion of a person exists in the first image acquired through the activated first camera, An HMD device further configured to deactivate the first camera or cause the first camera to stop capturing the first image based on the identification that the part of the person is present in the first image.

3. In paragraph 2, The HMD device includes a second camera, The above instructions, when individually or collectively executed by the at least one processor, cause the HMD device to: Based on the identification that the part of the person does not exist in the first image, it is determined whether movement of the HMD device is identified based on at least one image acquired through the second camera, When movement of the HDM device is identified, the first camera is kept activated, but the capturing of the first image through the first camera is stopped, An HMD device further configured to cause the first camera to remain activated and allow the capturing of the first image through the first camera when movement of the HDM device is not identified.

4. In any one of paragraphs 1 to 3, The above instructions, when individually or collectively executed by the at least one processor, cause the HMD device to: An HMD device further configured to cause the first camera to be disabled if it is identified that the above alternative notification method is not available.

5. In any one of paragraphs 1 to 4, The HMD device further comprises a second display, The above alternative notification method corresponds to a method of visually providing the alternative notification through the second display, An HMD device, wherein the second display and the first camera are provided on a first surface of a housing of the HMD device, and the first display is provided on a second surface of the housing.

6. In any one of paragraphs 1 to 5, The HMD device further comprises a speaker, The above alternative notification method corresponds to a method of audibly providing the above alternative notification through the speaker, an HMD device.

7. In any one of paragraphs 1 to 6, The HMD device further comprises a second camera, The above alternative notification method is an HMD device that visually provides the alternative notification through an infrared (IR) light-emitting diode (LED) of the second camera.

8. In any one of paragraphs 1 to 7, The HMD device further comprises a sensor, The operation of identifying whether there is an abnormality in the above light emitter is as follows: An HMD device comprising an operation for identifying whether there is an abnormality in the light emitter based on the current of the light emitter obtained through the sensor.

9. In any one of paragraphs 1 to 8, The HMD device further comprises a connector configured to connect the light emitter to a printed circuit board (PCB) including the at least one processor, The operation of identifying whether there is an abnormality in the above light emitter is as follows: An HMD device comprising an operation of identifying whether the light emitter and the at least one processor are electrically connected based on a detection signal detected through at least one pin of the connector, and identifying whether there is a problem with the light emitter based on a result of identifying whether the light emitter and the at least one processor are electrically connected.

10. In any one of paragraphs 1 to 9, The HMD device further comprises a second camera, The operation of identifying whether there is an abnormality in the above light emitter is as follows: An HMD device comprising an operation of identifying whether light is emitted through the light emitter through the second camera, and identifying whether there is an abnormality in the light emitter based on a result of identifying whether light is emitted through the light emitter through the second camera.

11. In the method of the HMD (head-mounted display) device, An operation of identifying whether there is an abnormality in a light emitter configured to provide a notification that the first camera is activated based on an execution request of the first camera for capturing an image in the front direction of the HMD device being acquired; An operation of controlling the light emitter to output the notification based on identifying that there is no abnormality in the light emitter; An operation of identifying whether an alternative notification method is available to provide an alternative notification indicating that the first camera is activated, based on the identification that there is an abnormality in the light emitter; and A method comprising an action of outputting the alternative notification using the alternative notification method based on identification that the alternative notification method is available.

12. In paragraph 11, An operation of identifying whether a part of a person exists in a first image acquired through the activated first camera based on the identification that the above alternative notification method is not available; and A method comprising: an action of deactivating the first camera or stopping capturing the first image through the first camera based on identifying that the part of the person is present in the first image.

13. In paragraph 12, The HMD device includes a second camera, The above method is: An operation of determining whether movement of the HMD device is identified based on at least one image acquired through the second camera, based on identification that the part of the person does not exist in the first image; An action of keeping the first camera activated when movement of the HMD device is identified, but stopping the shooting of the first image through the first camera; and A method further comprising an action of maintaining the first camera in an activated state and allowing capturing of the first image through the first camera when movement of the HMD device is not identified.

14. In any one of paragraphs 11 to 13, A method further comprising the action of disabling the first camera if it is identified that the alternative notification method is not available.

15. In any one of paragraphs 11 to 14, The HMD device comprises a second display, The above alternative notification method corresponds to a method of visually providing the alternative notification through the second display. A method wherein the second display and the first camera are provided on a first side of a housing of the HMD device, and the first display is provided on a second side of the housing.

Images