Electronic device including a camera and method of operating the same

Abstract

A wearable electronic device according to various embodiments includes a first sensor, a display, a camera disposed below a rear surface that is not exposed to the outside of the display, and a processor, wherein the processor identifies a first gesture toward the wearable electronic device through the first sensor, identifies the user’s face and gaze based on an image including the user’s face captured through the camera, determines whether to turn on the display based on the first gesture, the face, and the gaze, and if it is determined to turn on the display, the processor may be configured to adjust the screen displayed on the display based on the distance between the face and the wearable electronic device, the direction of the face, and the angle of the gaze.

Description

Technology Field

[0001] Various embodiments of the present invention relate to an electronic device including a camera and a method of operating the same. Background Technology

[0002] Recently, electronic devices are evolving into various forms for user convenience and providing a variety of services or functions.

[0003] Recently, electronic devices are being implemented to expand the display screen by reducing display bezels and maximizing the use of the exposed areas of the display to effectively display content and information related to the execution of various services or functions and to enhance user immersion. To achieve this, electronic devices may be implemented by including an under-display camera (UDC) that mounts the front camera beneath the display panel, instead of using a notch or hole display form.

[0004] Various services and additional functions provided through portable electronic devices, such as wearable electronic devices, are gradually expanding. In order to increase the utility value of these wearable electronic devices and satisfy the needs of various users, telecommunications service providers or electronic device manufacturers are competitively developing electronic devices to provide various functions and differentiate themselves from other companies.

[0005] In particular, a smart watch is a type of wearable electronic device that combines a watch and an electronic device. A smart watch performs various functions by being equipped with devices (e.g., cameras, sensors, etc.) that can perform not only the function of a watch but also various other functions.

[0006] A smartwatch can provide a list-up function. For example, the list-up function may refer to a 'raise wrist to see the watch' function. For example, while the user is wearing the smartwatch, the smartwatch can turn on the screen with only a gesture of raising the wrist without any separate input. The problem to be solved

[0007] When a smartwatch is equipped with a camera, its placement can reduce usability. For example, usability may decrease if the camera is not positioned on the same plane as the watch face, or even if it is positioned on the same plane, the size of the smartwatch increases. Additionally, usability may be reduced because it is inconvenient for the user to look at the camera rather than the watch face to use it.

[0008] In addition, the list-up function provided by smartwatches detects gestures through sensors, so there is a possibility of misrecognition. For example, when a user is driving or exercising, an unintended list-up function may be performed. To reduce such misrecognition, a large number of sensors can be installed in the smartwatch, but this may result in an increase in the size of the smartwatch or an increase in manufacturing costs.

[0009] Various embodiments of the present invention may provide an electronic device and a method of operation thereof that can detect a gesture for performing a list-up function through a sensor, detect a user's face and gaze through a UDC mounted on the wearable electronic device, determine whether to turn on a display included in the wearable electronic device based on the gesture, the face, and the gaze, and adjust a screen displayed on the display. means of solving the problem

[0010] A wearable electronic device according to various embodiments includes a first sensor, a display, a camera disposed below a rear surface that is not exposed to the outside of the display, and a processor, wherein the processor identifies a first gesture toward the wearable electronic device through the first sensor, identifies the user’s face and gaze based on an image including the user’s face captured through the camera, determines whether to turn on the display based on the first gesture, the face, and the gaze, and if it is determined to turn on the display, the processor may be configured to adjust the screen displayed on the display based on the distance between the face and the wearable electronic device, the direction of the face, and the angle of the gaze.

[0011] A method of operation of a wearable electronic device according to various embodiments may include: confirming a first gesture toward the wearable electronic device through a first sensor included in the wearable electronic device; confirming the face and gaze of the user based on an image including the user's face captured through a camera positioned below the back surface that is not exposed to the outside of the display of the wearable electronic device; determining whether to turn on the display of the electronic device based on the first gesture, the face, and the gaze; and, if it is determined to turn on the display, adjusting the screen displayed on the display based on the distance between the face and the wearable electronic device, the direction of the face, and the angle of the gaze.

[0012] A non-transient storage medium according to various embodiments may store an executable instruction such that, when executed by a processor, the program performs the following operations: confirming a first gesture toward the wearable electronic device through a first sensor included in the wearable electronic device; confirming the face and gaze of the user based on an image including the user's face captured through a camera positioned below the back surface that is not exposed to the outside of the display of the wearable electronic device; determining whether to turn on the display of the electronic device based on the first gesture, the face, and the gaze; and, if it is determined to turn on the display, adjusting the screen displayed on the display based on the distance between the face and the wearable electronic device, the direction of the face, and the angle of the gaze. Effects of the invention

[0013] A wearable electronic device according to various embodiments of the present invention can reduce misrecognition of the list-up function and increase usability by using a UDC. Brief explanation of the drawing

[0014] FIG. 1 is a diagram showing a network environment according to various embodiments. FIG. 2 is a drawing showing examples of configurations of wearable electronic devices according to various embodiments. FIG. 3 is a drawing showing examples of configurations of electronic devices according to various embodiments. FIG. 4 is a block diagram showing the schematic configuration of a wearable electronic device according to various embodiments. FIG. 5 is a flowchart for explaining the operation method of a wearable electronic device according to various embodiments. FIG. 6 is a diagram illustrating the operation of a wearable electronic device according to various embodiments confirming the user's face and gaze. FIG. 7 is a flowchart illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on the distance from the user's face. FIG. 8 is a diagram illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on the distance from the user's face. FIG. 9a is a flowchart for explaining the operation of adjusting the screen when a user other than the user is identified in a wearable electronic device according to various embodiments. FIG. 9b is a diagram illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen when a user other than the user is identified. FIG. 10 is a diagram illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on the user's gaze. FIG. 11 is a flowchart for explaining the operation of a wearable electronic device according to various embodiments adjusting a screen based on the user's biometric information. FIG. 12 is a diagram illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on the user's biometric information. FIG. 13 is a flowchart illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on a user's gesture. FIG. 14 is a diagram illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on a user's gesture. FIG. 15 is a flowchart illustrating the operation of a wearable electronic device according to various embodiments transmitting an unlock signal to another electronic device. Specific details for implementing the invention

[0015] Hereinafter, electronic devices according to various embodiments will be examined with reference to the attached drawings. In the various embodiments, the term "user" may refer to a person using the electronic device or a device using the electronic device (e.g., an artificial intelligence electronic device).

[0016] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with an electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0017] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0018] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence is performed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0019] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0020] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0021] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0022] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0023] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0024] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0025] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0026] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0027] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0028] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0029] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0030] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0031] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0032] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0033] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna.

[0034] The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., electronic device (104)), or a network system (e.g., a second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0035] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0036] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0037] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0038] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services, for example, using distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0039] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0040] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0041] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0042] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0043] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0044] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0045] FIG. 2 is a drawing showing an example of the configuration of an electronic device according to various embodiments, and FIG. 3 is a drawing showing an example of the configuration of an electronic device according to various embodiments.

[0046] Referring to FIG. 2, an electronic device (101) according to one embodiment (e.g., the electronic device (101) of FIG. 1) may include a display module (160) (e.g., the display module (160) of FIG. 1) and a camera (210) (e.g., the camera module (180) of FIG. 1) disposed on the front (201) of a housing (200). An electronic device (101) according to one embodiment may be configured to include a memory (130) and a display module (160), a camera (210), and at least one processor (120) electrically connected to the memory (130). In addition, an electronic device (101) according to one embodiment may be configured to include other components described in FIG. 1. According to one embodiment, the electronic device (101) may be implemented as a wearable electronic device. For example, the electronic device (101) may be implemented as a watch-type wearable electronic device (e.g., a smart watch).

[0047] Referring to FIG. 3, according to one embodiment, a display module (160) is positioned to be exposed to the front (201) of a housing (200), a window (310) is positioned so that a first surface is exposed to the outside, and a display panel (320) is positioned below a second surface (back) of the window (310). The display panel (320) may be configured to include a substrate (e.g., a flexible circuit board (FPCB)) (340) and a display element layer (311) disposed on the substrate (340). The display panel (320) may be configured in the form of a touch-sensitive panel (TSP). The display element layer (311) may include a circuit layer comprising a thin-film transistor (TFT) (not shown), an organic light-emitting diode (OLED) (not shown) as a display element, and an insulating layer (IL) (not shown) between them. The display panel (320) may be configured to include a display driver integrated circuit (not shown). According to one embodiment, the transparent glass layer (310) and the display panel (320) may be configured in a shape in which at least a portion is curved. The display panel (320) may be formed from a flexible polymer film and may include, for example, polyimide, polyethylene terephthalate, or other polymer materials. The display panel (320) may be configured to include a first polymer layer (313) (e.g., polyimide) and a second polymer layer (315) (e.g., polyethylene terephthalate) disposed below the display element layer (311).

[0048] According to one embodiment, the camera (210) may be positioned below a back surface (e.g., a second surface) that is not exposed to the outside of the display module (160). As shown in FIG. 3, the camera (210) may be an under-display camera (e.g., an Under Display Camera (UDC) camera) (210) in which at least a portion of the camera module (180) is positioned below the display panel (320). The camera (210) may be positioned between the display panel (320) and the substrate (340). For example, the camera (210) may be at least a portion of the camera module (180) of FIG. 1 and may be contained within the housing (200) so as not to be exposed. The camera (210) may be configured to include at least one of a camera sensor (e.g., an image sensor) (not shown) that detects light incident through the window (310) through a lens and converts the light into a digital signal to acquire an image, an image processing module (not shown) that processes the image, or a memory (not shown) that stores the image.

[0049] FIG. 4 is a block diagram showing the schematic configuration of a wearable electronic device according to various embodiments.

[0050] Referring to FIG. 4, according to various embodiments, a wearable electronic device (401) may include a camera (410), a processor (420), a memory (430), a first sensor (440), a second sensor (450), a display (460), and a communication module (480). For example, the wearable electronic device (401) may be implemented identically or similarly to the electronic device (101) of FIG. 1. For example, the wearable electronic device (401) may be implemented as a watch-type wearable electronic device (e.g., a smart watch).

[0051] According to various embodiments, the camera (410) may be implemented as an under-display camera (UDC). For example, the camera (410) may be implemented identically or similarly to the camera (210) of FIG. 2. For example, the camera (410) may be located in the center area of ​​the display (460) (e.g., the display (160) of FIG. 2).

[0052] According to various embodiments, the processor (420) can control the overall operation of the wearable electronic device (420). For example, the processor (420) may be implemented identically or similarly to the processor (120) of FIG. 1.

[0053] According to various embodiments, the processor (420) can detect a first gesture toward the wearable electronic device (401) through the first sensor (440). For example, the first sensor (440) can sense a gesture toward the wearable electronic device (401). For example, the first sensor (440) may include at least one of an accelerometer, a speed sensor, or a motion sensor. For example, the first gesture may include a gesture of raising the wrist while the user is wearing the wearable electronic device (401) (e.g., a smart watch).

[0054] According to various embodiments, the processor (420) may capture and acquire an image including the user's face through the camera (410). The processor (420) may identify the user's face and gaze based on the image including the user's face captured through the camera (410). For example, the processor (420) may identify the size of the user's face, the direction of the face, and the direction of the gaze based on the image. For example, the processor (420) may determine the direction of the face based on the nose and / or two eyes included in the face. For example, the processor (420) may determine the direction of the gaze by identifying the two eyes included in the face. The processor (420) may identify whether the direction of the face and the direction of the gaze match the camera (410).

[0055] According to various embodiments, the processor (420) may determine whether to turn on the display (460) based on the first gesture, the user's face, and the user's gaze. For example, the processor (420) may check whether the first gesture matches a gesture stored in advance in memory (430) (e.g., memory (130) of FIG. 1) (e.g., a gesture of the user raising their wrist while wearing the wearable electronic device (401)). Additionally, the processor (420) may check whether the direction of the user's face and / or the direction of the gaze matches the camera (410). The processor (420) may turn on the display if the first gesture matches the stored gesture and the direction of the user's face and the direction of the gaze match the camera (410). Alternatively, the processor (420) may turn on the display if the first gesture matches a pre-stored gesture and the direction of the user's gaze matches the camera (410). For example, the processor (420) may turn on the display (460) if the first gesture matches a pre-stored gesture and the direction of the gaze matches the camera (410), even if the direction of the face does not match the camera (410).

[0056] According to various embodiments, when the processor (420) determines to turn on the display (460), it may display a screen on the display (460). For example, the screen may include at least one of a watch face, a message, a user interface, an image, an application execution screen, or a home screen. The processor (420) may adjust the screen displayed on the display based on the distance between the user's face and the wearable electronic device (401) (or camera (410)), the direction of the user's face, and the direction (or angle) of the user's gaze. For example, the adjustment of the screen may include changing at least one object included in the screen (e.g., changing color, changing position, and / or changing size) or changing from an existing screen to another screen.

[0057] According to various embodiments, the processor (420) may enlarge or reduce the size of the screen displayed on the display (460) based on the distance between the user's face and the wearable electronic device (401) when the user's gaze remains the same for a specified period of time. For example, the processor (420) may enlarge the size of the screen as the distance between the user's face and the wearable electronic device (401) increases. Alternatively, the processor (420) may reduce the size of the screen as the distance between the user's face and the wearable electronic device (401) decreases. For example, the processor (420) may determine the size of the user's face and the distance between the user's eyes based on an image containing the user's face, and determine the distance between the user's face and the wearable electronic device based on the size of the user's face and the distance between the user's eyes.

[0058] According to various embodiments, the processor (420) may display only some information on the screen displayed on the display (460) when a user other than a pre-registered user is identified in an image captured through the camera (410). For example, the processor (420) may check the privacy level of the content displayed on the screen and determine the information displayed on the screen according to the privacy level. Additionally, the processor (420) may change the display location of the information displayed on the display (460) when a user other than a pre-registered user is identified in an image captured through the camera (410). For example, the processor (420) may display information in an area where a user other than a pre-registered user cannot see in detail (e.g., an area at a location far from other users).

[0059] According to various embodiments, the processor (420) can adjust the position of a notification window displayed on the display (460) based on the direction of the face and the direction (or angle) of the gaze. For example, the processor (420) can display a notification window at the position where the user's face and gaze are directed. According to another embodiment, the processor (420) can adjust the position of a watch face displayed on the display (460) based on the direction of the face and the direction (or angle) of the gaze. For example, the processor (420) can display a watch face at the position where the user's face and gaze are directed.

[0060] According to various embodiments, the processor (420) can detect a second gesture (e.g., a hand-opening gesture) of the user's hand movement through the camera (410) and adjust the screen displayed on the display (460) based on the second gesture (e.g., enlarge or reduce the screen size). For example, the processor (420) can detect the second gesture of the user's movement based on images continuously captured through the camera (410) while the screen is displayed on the display (460), and check whether the second gesture matches a designated gesture. If the second gesture matches a designated gesture, the processor (420) can adjust the screen of the display (460) according to a function corresponding to the second gesture.

[0061] According to various embodiments, the processor (420) may obtain the user's biometric information through the second sensor (450). For example, the second sensor (450) may include a PPG sensor, an EKG sensor, and / or an ECG sensor capable of measuring the user's biometric signal. The processor (420) may update the screen displayed on the display (460) based on the biometric information obtained through the second sensor (450). For example, the processor (420) may check the user's mood state and / or health state through the biometric information and update the screen according to the checked mood state and / or health state.

[0062] According to various embodiments, the processor (420) may transmit and receive data (or signals) to and from an external electronic device (402) through a communication module (480). For example, the external electronic device (402) may include a terminal, a smartphone, and an AI speaker. The processor (420) may determine whether a user is a pre-registered user based on the user's face and gaze included in an image captured through the camera (410). If the processor (420) confirms that the user is a pre-registered user, it may transmit a signal requesting unlocking to the external electronic device (402).

[0063] According to various embodiments, the processor (420) may display data received from an external electronic device (402) on the display (460) when the user is identified as a pre-registered user. Additionally, after the user is identified as a pre-registered user, the processor (420) may display privacy-related data received from an external electronic device (402) on the display (460) when the user's face and gaze are directed toward the wearable electronic device (401).

[0064] Meanwhile, at least some of the operation of the electronic device below may be performed by the processor (420) of FIG. 4. However, for convenience of explanation, the operation below will be described as being performed by the processor (420).

[0065] FIG. 5 is a flowchart for explaining the operation method of a wearable electronic device according to various embodiments.

[0066] Referring to FIG. 5, according to various embodiments, in operation 501, a wearable electronic device (e.g., the wearable electronic device (401) of FIG. 4) can detect a first gesture for the wearable electronic device (401) through a first sensor (e.g., the first sensor (440) of FIG. 4). For example, a processor (420) can check whether the first gesture matches a pre-registered gesture.

[0067] According to various embodiments, in operation 503, the wearable electronic device (401) can identify the user's face and gaze based on an image including the user's face captured through a camera (UDC) (e.g., camera (410) of FIG. 4). For example, the processor (420) can determine whether the user is using the wearable electronic device (401) or looking at it based on the direction of the user's face and the angle of the gaze.

[0068] According to various embodiments, in operation 505, the wearable electronic device (401) may determine whether to turn on a display (e.g., the display (460) of FIG. 4) based on a first gesture, a face, and a gaze. For example, the processor (420) may turn on the display (460) if it is confirmed that the first gesture matches a pre-registered gesture and that the user is looking at the wearable electronic device (401) based on the face direction and the angle of the gaze.

[0069] According to various embodiments, in operation 507, when the wearable electronic device (401) decides to turn on the display, it can adjust the screen displayed on the display (460) based on the distance between the user's face and the wearable electronic device, the direction of the face, and the angle of gaze. For example, the processor (420) can change the size of the screen or change the display position based on the distance between the user's face and the wearable electronic device, the direction of the face, and the angle of gaze.

[0070] FIG. 6 is a diagram illustrating the operation of a wearable electronic device according to various embodiments confirming the user's face and gaze.

[0071] Referring to FIG. 6, according to various embodiments, a wearable electronic device (e.g., the wearable electronic device (401) of FIG. 4) can determine whether a user is using or not using the wearable electronic device (401) through a camera (UDC) (e.g., the camera (410) of FIG. 4). For example, the wearable electronic device (401) can determine whether the user is using the wearable electronic device (401) by checking the direction of the user's face and the angle of gaze (or direction of gaze) in an image captured through the camera (UDC) (410), and based on the direction of the face and the angle of gaze.

[0072] According to various embodiments, the wearable electronic device (401) can determine whether the user is using the wearable electronic device (401) based on the direction of gaze while the front of the user's face is visible in an image captured through the camera (410).

[0073] According to various embodiments, assuming the camera (410) is positioned in the front, the wearable electronic device (401) can acquire an image (610, 620, 630, or 640) containing the user's face. For example, as shown in FIG. 6 (a), the wearable electronic device (401) can determine that the user is using the wearable electronic device (401) or intends to use it if it confirms that the direction of the face and the direction of the gaze in the image (610) containing the user's face match the camera (410). Additionally, as shown in FIG. 6 (b), the wearable electronic device (401) can determine that the user is using the wearable electronic device (401) or intends to use it if it confirms that the direction of the face in the image (620) containing the user's face does not match the camera (410) but the direction of the gaze matches the camera (410). On the other hand, as shown in FIG. 6 (c), if the wearable electronic device (401) confirms that in an image (630) containing the user's face, the direction of the face aligns with the camera (410) but the direction of the gaze does not align with the camera (410), it can determine that the user is not using the wearable electronic device (401) or has no intention of using it. Additionally, as shown in FIG. 6 (d), if the wearable electronic device (401) confirms that in an image (640) containing the user's face, the direction of the face and the direction of the gaze do not align with the camera (410), it can determine that the user is not using the wearable electronic device (401) or has no intention of using it.

[0074] According to various embodiments, if the user's face is not visible or the front of the face is not visible in an image captured through the camera (410), the wearable electronic device (401) may determine that the user is not using the wearable electronic device (401) or has no intention of using it.

[0075] According to various embodiments, the wearable electronic device (401) may determine whether to turn on a display (e.g., the display (460) of FIG. 4) based on a first gesture, the direction of the face, and the angle of gaze (or the direction of gaze).

[0076] FIG. 7 is a flowchart illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on the distance from the user's face.

[0077] According to various embodiments, the wearable electronic device (401) may determine whether to turn on a display (e.g., the display (460) of FIG. 4) based on a first gesture, the direction of the face, and the angle of gaze (or the direction of gaze).

[0078] According to various embodiments, the wearable electronic device (401) can adjust the screen displayed on the display (460) based on the user's face and gaze in an image captured through the camera (410) after turning on the display (460).

[0079] Referring to FIG. 7, according to various embodiments, in operation 701, the wearable electronic device (401) can check whether the user's gaze remains the same for a specified period of time by taking a plurality of images (continuously) through the camera (410). For example, the wearable electronic device (401) can check whether the user's gaze is looking at the display (460) of the camera (410) for a specified period of time.

[0080] According to various embodiments, in operation 703, the wearable electronic device (401) can check the size of the user's face and the spacing of the eyes included in the face.

[0081] According to various embodiments, in operation 705, the wearable electronic device (401) can determine the distance between the user's face and the wearable electronic device (401) based on the size of the user's face and the distance between the eyes.

[0082] According to various embodiments, in operation 707, the wearable electronic device (401) may enlarge or reduce the size of the screen displayed on the display (460) based on the distance between the user's face and the wearable electronic device (401). For example, the wearable electronic device (401) may enlarge the size of the screen as the distance between the user's face and the wearable electronic device (401) increases. Additionally, the wearable electronic device (401) may reduce the size of the screen as the distance between the user's face and the wearable electronic device (401) decreases.

[0083] FIG. 8 is a diagram illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on the distance from the user's face.

[0084] Referring to FIG. 8, a wearable electronic device (e.g., the wearable electronic device (401) of FIG. 4) can identify the user's face included in an image captured through a camera (410) and determine the distance between the user's face and the wearable electronic device (401) based on the size of the user's face and the spacing of the eyes.

[0085] According to various embodiments, the wearable electronic device (401) can identify a user face (810) of a first size. Based on the user face (810) of the first size, the wearable electronic device (401) can determine that the distance between the user's face and the wearable electronic device (401) is a first distance (d1). Based on the first distance (d1), the wearable electronic device (401) can display a first screen (820) on a display (e.g., the display (460) of FIG. 4).

[0086] According to various embodiments, the wearable electronic device (401) can detect a user face (830) of a second size. For example, the second size may be smaller than the first size. Based on the user face (830) of the second size, the wearable electronic device (401) may determine that the distance between the user's face and the wearable electronic device (401) is a second distance (d2). Based on the second distance (d2), the wearable electronic device (401) may display a second screen (840) on the display (460). For example, the size of the second screen (840) may be larger than the size of the first screen (820). The second distance (d2) may be longer than the first distance.

[0087] Although FIG. 8 only describes a method for determining the screen size based on the size of the face, the technical concept of the present invention may not be limited thereto. For example, the wearable electronic device (401) can determine the distance between the user's face and the wearable electronic device (401) based on the change in the distance between the user's eyes in the same or similar way as the method described above.

[0088] FIG. 9a is a flowchart for explaining the operation of adjusting the screen when a user other than the user is identified in a wearable electronic device according to various embodiments.

[0089] Referring to FIG. 9a, according to various embodiments, in operation 901, a wearable electronic device (wearable electronic device (401) of FIG. 4) can determine whether a user other than the user is included in an image containing the face of a user captured through a camera (UDC) (e.g., camera (410) of FIG. 4). For example, the wearable electronic device (401) can determine whether a user other than a user registered in advance (e.g., user of the wearable electronic device (401)) is included in the image.

[0090] According to various embodiments, if it is confirmed that another user is included (e.g., in operation 903), in operation 905, the wearable electronic device (401) may display only some information of a message on a screen displayed on a display (e.g., the display (460) of FIG. 4). For example, the information of the message that is displayed may be determined by the privacy level of the message. For example, for a message with the highest privacy level (e.g., a security authentication message), only the message title may be displayed. Or, for a message with an intermediate privacy level (e.g., a private message), the message title and part of the message content may be displayed.

[0091] According to various embodiments, if it is not confirmed that another user is included (No in operation 903), in operation 907, the wearable electronic device (401) may display all information of the message on the screen displayed on the display (460). However, the wearable electronic device (401) may be configured to display only some information of the message according to the privacy level set by the user.

[0092] FIG. 9b is a diagram illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen when a user other than the user is identified.

[0093] Referring to (a) of FIG. 9b, according to various embodiments, a wearable electronic device (e.g., the wearable electronic device (401) of FIG. 4) can display a message (950) on a display (e.g., the display (460) of FIG. 4).

[0094] According to various embodiments, the wearable electronic device (401) can determine whether another user other than the user is included in an image containing the user's face captured through a camera (UDC) (e.g., camera (410) of FIG. 4). If the wearable electronic device (401) determines that another user is included, it may display only some information of the message displayed on the display (460). For example, referring to (b) of FIG. 9b, the wearable electronic device (401) may display only the title (960) of the message on the display (460). The wearable electronic device (401) may display the title (960) of the message in the lower area of ​​the display (460). Additionally, the size of the title (960) of the message may be reduced compared to the size of the existing message (950).

[0095] FIG. 10 is a diagram illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on the user's gaze.

[0096] Referring to FIG. 10, according to various embodiments, a wearable electronic device (e.g., the wearable electronic device (401) of FIG. 4) can identify a user's face included in an image captured through a camera (UDC) (e.g., the camera (410) of FIG. 4). The wearable electronic device (401) can adjust the position of a notification window (1010, 1020, 1030, 1040, or 1050) displayed on a display (460) based on the direction of the face and the direction (or angle) of the gaze.

[0097] According to various embodiments, the wearable electronic device (401) may display a notification window at a location where the user's face and gaze are directed. For example, when the wearable electronic device (401) confirms that the location where the user's face and gaze are directed is a first location (e.g., the center area of ​​the display (460)), it may display a notification window (1010) in an area corresponding to the first location (e.g., the center area of ​​the display (460)). For example, when the wearable electronic device (401) confirms that the location where the user's face and gaze are directed is a second location (e.g., the left area of ​​the display (460)), it may display a notification window (1020) in an area corresponding to the second location (e.g., the left area of ​​the display (460)). For example, if the wearable electronic device (401) confirms that the position where the user's face and gaze are directed is a third position (e.g., the upper area of ​​the display (460)), it may display a notification window (1030) in an area corresponding to the third position (e.g., the upper area of ​​the display (460)). For example, if the wearable electronic device (401) confirms that the position where the user's face and gaze are directed is a fourth position (e.g., the right area of ​​the display (460)), it may display a notification window (1040) in an area corresponding to the fourth position (e.g., the right area of ​​the display (460)). For example, if the wearable electronic device (401) confirms that the position where the user's face and gaze are directed is a fifth position (e.g., the lower area of ​​the display (460)), it may display a notification window (1050) in an area corresponding to the fifth position (e.g., the lower area of ​​the display (460)). At this time, the wearable electronic device (401) can adjust the shape of the notification window displayed on the display (460) based on the location where the notification window is displayed.

[0098] FIG. 11 is a flowchart for explaining the operation of a wearable electronic device according to various embodiments adjusting a screen based on the user's biometric information.

[0099] Referring to FIG. 11, according to various embodiments, in operation 1101, a wearable electronic device (e.g., the wearable electronic device (401) of FIG. 4) can acquire the user's biometric information through a second sensor.

[0100] According to various embodiments, in operation 1103, the wearable electronic device (401) can update the screen displayed on the display (e.g., the display (460) of FIG. 4) in consideration of the user's biometric information. For example, the wearable electronic device (401) can check the user's mood or health condition based on the user's biometric information. The wearable electronic device (401) can update the watch face in consideration of the user's mood or health condition.

[0101] FIG. 12 is a diagram illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on the user's biometric information.

[0102] According to various embodiments, a wearable electronic device (e.g., the wearable electronic device (401) of FIG. 4) may display a first watch face (1210) on a display (e.g., the display (460) of FIG. 4). For example, the first watch face (1210) may be a basic watch face.

[0103] According to various embodiments, the wearable electronic device (401) can obtain the user's biometric information through a second sensor (e.g., the second sensor (450) of FIG. 4). For example, the wearable electronic device (401) can determine the user's mood (or health condition) based on the user's biometric information.

[0104] According to various embodiments, the wearable electronic device (401) may update the watch face (1210) in consideration of the user's mood (or health) state. For example, the wearable electronic device (401) may update the first watch face (1210) to the second watch face (1220) if, as a result of checking biometric information, the user is in a "fatigued" state. For example, the second watch face (1220) may include a color indicating the degree of "fatigued". For example, the wearable electronic device (401) may update the first watch face (1210) to the third watch face (1230) if, as a result of checking biometric information, the user is in a "depressed" state. For example, the third watch face (1230) may include a color indicating the degree of "depressed" state. For example, the wearable electronic device (401) can update the first watch face (1210) to the fourth watch face (1240) if, as a result of checking biometric information, the user is in a "tensed" state. For example, the fourth watch face (1240) may include a color indicating the degree of "tension."

[0105] According to various embodiments, the wearable electronic device (401) can provide notifications or feedback indicating the user's mood (or health condition) using sound or vibration while updating the watch face.

[0106] FIG. 13 is a flowchart illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on a user's gesture.

[0107] Referring to FIG. 13, according to various embodiments, in operation 1301, a wearable electronic device (e.g., the wearable electronic device (401) of FIG. 4) can detect a second gesture of the user through a camera (UDC) (e.g., the camera (410) of FIG. 4). For example, the second gesture may include the movement of a specific user's hand (or finger) and / or the shape of the hand. For example, the wearable electronic device (401) can set a face detection area and a hand detection area through the camera (UDC) (410) and thereby detect the user's face and hand.

[0108] According to various embodiments, in operation 1303, the wearable electronic device (401) may adjust the screen displayed on the display (e.g., the display (460) of FIG. 4) by taking into account a second gesture in addition to the user's face and gaze identified in the captured image by the camera (UDC) (410). According to another embodiment, the wearable electronic device (401) may adjust the screen displayed on the display (460) by taking into account only the identified second gesture by taking into account the camera (UDC) (410).

[0109] FIG. 14 is a diagram illustrating the operation of a wearable electronic device according to various embodiments adjusting a screen based on a user's gesture.

[0110] Referring to FIG. 14, according to various embodiments, a wearable electronic device (e.g., the wearable electronic device (401) of FIG. 4) can check the movement (or shape of the hand) (1405) of a user's hand through a camera (UDC) (e.g., the camera (410) of FIG. 4). The wearable electronic device (401) can check whether the movement of the user's hand matches a designated gesture.

[0111] According to various embodiments, the wearable electronic device (401) can perform a function corresponding to a specified gesture when the movement of the user's hand matches a specified gesture. For example, the wearable electronic device (401) can change a first screen (1410) having a first size displayed on a display (e.g., the display (460) of FIG. 4) into a second screen (1420) having a second size. That is, the wearable electronic device (401) can enlarge the size of the screen previously displayed on the display (1460) based on the movement of the user's hand. According to another embodiment, the wearable electronic device (401) can change a second screen (1420) having a second size displayed on the display (460) into a first screen (1410) having a first size. That is, the wearable electronic device (401) may reduce the size of the screen that was previously displayed on the display (1460) based on the movement of the user's hand.

[0112] Although FIG. 14 only describes the operation of enlarging or reducing the size of the screen, the technical concept of the present invention may not be limited thereto. For example, a wearable electronic device (401) can adjust the screen in various ways based on a gesture including the movement (or shape of the hand) of the user confirmed through a camera (UDC) (410).

[0113] FIG. 15 is a flowchart illustrating the operation of a wearable electronic device according to various embodiments transmitting an unlock signal to another electronic device.

[0114] Referring to FIG. 15, according to various embodiments, in operation 1501, a wearable electronic device (e.g., the wearable electronic device (401) of FIG. 4) can check the user's face and gaze through a camera (UDC) (e.g., the camera (410) of FIG. 4).

[0115] According to various embodiments, in operation 1503, the wearable electronic device (401) can determine whether the user matches a registered user based on the user's face and gaze. For example, the wearable electronic device (401) can determine whether the user's information (e.g., information about the face and gaze) stored in memory (e.g., memory (430) of FIG. 4) matches the user's face and gaze confirmed through the camera (410). If the user's information (e.g., information about the face and gaze) stored in memory (430) matches the user's face and gaze confirmed through the camera (410), the wearable electronic device (401) can determine that the user confirmed through the camera (410) is a registered user.

[0116] According to various embodiments, if it is determined that the user does not match the registered user (No of action 1503), the wearable electronic device (401) may not perform any additional action. At this time, the wearable electronic device (401) may monitor whether a new user is identified through the camera (410).

[0117] According to various embodiments, when it is confirmed that the user matches the registered user (e.g., operation 1503), in operation 1505, the wearable electronic device (401) may transmit a signal requesting unlocking to an external electronic device (e.g., the external electronic device (402) of FIG. 4). For example, the external electronic device (402) may unlock or maintain the unlocked state of the external electronic device (402) based on the reception of the signal requesting unlocking.

[0119] A wearable electronic device according to various embodiments includes a first sensor, a display, a camera disposed below a rear surface that is not exposed to the outside of the display, and a processor, wherein the processor identifies a first gesture toward the wearable electronic device through the first sensor, identifies the user’s face and gaze based on an image including the user’s face captured through the camera, determines whether to turn on the display based on the first gesture, the face, and the gaze, and if it is determined to turn on the display, the processor may be configured to adjust the screen displayed on the display based on the distance between the face and the wearable electronic device, the direction of the face, and the angle of the gaze.

[0120] The processor may be configured to enlarge or reduce the size of the screen based on the distance between the face and the wearable electronic device if the gaze remains the same for a specified period of time.

[0121] The processor may be configured to determine the distance between the face and the wearable electronic device based on the size of the face and the distance between the eyes of the face.

[0122] The above processor may be configured to display only some information on the screen when a user other than the user is identified through the camera.

[0123] The processor may be configured to change the display position of some information included on the screen when another user other than the user is identified through the camera.

[0124] The processor may be configured to adjust the position of the notification window displayed on the display based on the direction of the face and the angle of the gaze.

[0125] The processor may be configured to detect a second gesture regarding the movement of the user's hand through the camera and to adjust the configuration of the screen based on the second gesture.

[0126] The electronic device further includes a second sensor for acquiring biometric information, and the processor may be configured to acquire the user's biometric information through the second sensor and update the screen based on the biometric information.

[0127] The processor may be configured to determine whether the user is a pre-registered user based on the face and the gaze, and if the user is confirmed to be a pre-registered user, to transmit a signal requesting unlocking to an external electronic device.

[0128] The above-mentioned wearable electronic device can be implemented as a smart watch.

[0129] A method of operation of a wearable electronic device according to various embodiments may include: confirming a first gesture toward the wearable electronic device through a first sensor included in the wearable electronic device; confirming the face and gaze of the user based on an image including the user's face captured through a camera positioned below the back surface that is not exposed to the outside of the display of the wearable electronic device; determining whether to turn on the display of the electronic device based on the first gesture, the face, and the gaze; and, if it is determined to turn on the display, adjusting the screen displayed on the display based on the distance between the face and the wearable electronic device, the direction of the face, and the angle of the gaze.

[0130] The operation of adjusting the screen may include, if the gaze is maintained the same for a specified period of time, an operation of enlarging or reducing the size of the screen based on the distance between the face and the wearable electronic device.

[0131] The operation of adjusting the screen may include the operation of determining the distance between the face and the wearable electronic device based on the size of the face and the distance between the eyes of the face.

[0132] The operation of adjusting the above screen may include an operation of displaying only some information on the screen when a user other than the user is identified through the camera.

[0133] The operation of adjusting the screen may include changing the display position of some information included in the screen when another user other than the user is identified through the camera.

[0134] The operation of adjusting the screen may include the operation of adjusting the position of the notification window displayed on the display based on the direction of the face and the angle of the gaze.

[0135] The method of operation of the electronic device may further include the operation of confirming a second gesture regarding the movement of the user's hand through the camera and the operation of adjusting the configuration of the screen based on the second gesture.

[0136] The method of operation of the electronic device may further include the operation of acquiring a user's biometric information through a second sensor included in the wearable electronic device and the operation of updating the screen based on the biometric information.

[0137] The method of operation of the electronic device may further include an operation of verifying whether the user is a pre-registered user based on the face and the gaze, and an operation of transmitting a signal requesting unlocking to an external electronic device when the user is verified as the pre-registered user.

[0138] A non-transient storage medium according to various embodiments may store an executable instruction such that, when executed by a processor, the program performs the following operations: confirming a first gesture toward the wearable electronic device through a first sensor included in the wearable electronic device; confirming the face and gaze of the user based on an image including the user's face captured through a camera positioned below the back surface that is not exposed to the outside of the display of the wearable electronic device; determining whether to turn on the display of the electronic device based on the first gesture, the face, and the gaze; and, if it is determined to turn on the display, adjusting the screen displayed on the display based on the distance between the face and the wearable electronic device, the direction of the face, and the angle of the gaze. Explanation of the symbols

[0140] 101: Electronic devices 160: Display 200: Housing 201: Front of the housing 210: Under-display camera (UDC)

Claims

Claim 1 In a wearable electronic device, a first sensor; a display; a camera disposed below a rear surface that is not exposed to the outside of the display; at least one processor; The system includes a memory for storing instructions, wherein when the instructions are executed individually or collectively by at least one processor, the wearable electronic device is caused to confirm a first gesture toward the wearable electronic device through the first sensor, confirm the user's face and gaze based on an image including the user's face captured through the camera, determine whether to turn on the display based on the first gesture, the face, and the gaze, and if it is determined to turn on the display, display a screen on the display, check whether the gaze remains the same for a specified time while displaying the screen on the display, and if it is confirmed that the gaze remains the same for the specified time, check the size of the face, and if the size of the face is confirmed as a first size, check the distance between the face and the wearable electronic device as a first distance, and display the screen as a first screen, and if the size of the face is confirmed as a second size smaller than the first size, check the distance between the face and the wearable electronic device as a second distance greater than the first distance, and display the screen as a second screen larger than the first screen A wearable electronic device configured to display on a screen. Claim 2 delete Claim 3 delete Claim 4 A wearable electronic device according to claim 1, wherein, when the instructions are executed individually or collectively by the at least one processor, the wearable electronic device is configured to display only some information on the screen when a user other than the user is identified through the camera. Claim 5 In paragraph 4, the wearable electronic device is configured such that, when the above commands are executed individually or collectively by the at least one processor, the wearable electronic device changes the display position of the said part of the information included on the screen when the other user other than the user is identified through the camera. Claim 6 A wearable electronic device according to claim 1, wherein the instructions, when executed individually or collectively by the at least one processor, are configured to cause the wearable electronic device to adjust the position of a notification window displayed on the display based on the direction of the face and the angle of the gaze. Claim 7 A wearable electronic device configured such that, in the first paragraph, the instructions are executed individually or collectively by the at least one processor to detect a second gesture regarding the movement of the user's hand through the camera and to adjust the configuration of the screen based on the second gesture. Claim 8 A wearable electronic device according to claim 1, further comprising a second sensor for acquiring biometric information, wherein the instructions, when executed individually or collectively by the at least one processor, are configured to cause the wearable electronic device to acquire a user's biometric information through the second sensor and update the screen based on the biometric information. Claim 9 A wearable electronic device configured such that, when the instructions are executed individually or collectively by at least one processor, the wearable electronic device determines whether the user is a pre-registered user based on the face and the gaze, and if the user is confirmed to be the pre-registered user, transmits a signal requesting unlocking to an external electronic device. Claim 10 In claim 1, the wearable electronic device is a wearable electronic device implemented as a smart watch. Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete

Images