Method, apparatus and storage medium for providing user interface

Abstract

According to various embodiments, the electronic device comprises: a memory; a display; an audio output unit; at least one camera disposed below the display; and at least one processor operatively connected to the memory, the display, and the at least one camera, wherein the at least one processor may be configured to detect the distance between the at least one camera and at least one first subject, compare a preview image of the at least one first subject collected by the at least one camera with a reference image stored in the memory, determine whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or the result of comparing the preview image and the reference image, and provide a user interface (UI) through either the display or the audio output unit to instruct shooting through the at least one camera when the shooting condition is satisfied, and other embodiments may also be possible.

Description

Technology Field

[0001] Various embodiments relate to a method, device, and storage medium for providing a user interface. Background Technology

[0002] In electronic devices such as smartphones or tablet terminals, a camera may generally be placed on the front of the electronic device with a display to provide selfie shooting or video calling functions.

[0003] Cameras, which were traditionally positioned at the top front of electronic devices, are evolving to be placed at the bottom of the display to optimize screen size and enhance visual immersion. For example, under-display cameras (UDCs) are being proposed to maximize the immersive experience by placing the camera beneath the display so that it is not visible on the device.

[0004] UDC can be displayed on the screen when running and not displayed on the screen when not running, thereby enabling the implementation of a full display. The problem to be solved

[0005] Since the location of the UDC placed at the bottom of the display is not easily visible to the user, a method may be required to provide the user with the location of the UDC in situations where its use is necessary.

[0006] Due to the nature of UDC requiring the camera sensor to be hidden as much as possible from the display, there may be limitations on the size of the camera sensor exposed on the display. Consequently, it may be difficult to implement autofocus functions in UDC due to the restricted size of the camera sensor.

[0007] It can be difficult to implement autofocus functions even in conventional hole-punching type cameras. Therefore, general cameras placed on the front of electronic devices typically provide a fixed focus function. When using a UDC with a fixed focus function, the user must adjust the distance between the UDC and the subject until it corresponds to the focal length. However, since the user cannot clearly know the focal length, they may experience inconvenience in adjusting the distance between the UDC and the subject.

[0008] According to various embodiments, a method, device, and storage medium for providing a user interface (UI) in an electronic device can be provided.

[0009] According to various embodiments, the electronic device may provide a UI that enables the user to recognize the location of the UDC.

[0010] According to various embodiments, the electronic device may provide a UI that enables a user to recognize the location of at least one UDC to be used among a plurality of UDCs in an environment where a plurality of UDCs are used.

[0011] According to various embodiments, the electronic device can provide ideal shooting results without using an autofocus function by providing a UI that induces distance adjustment so that the distance between the UDC and the subject corresponds to the focal length of the UDC.

[0012] According to various embodiments, the electronic device can provide a UI that enables the composition of a subject to be set stably and clearly based on a learned reference model.

[0013] The technical problems to be solved in the various embodiments are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art to which the various embodiments of the present disclosure belong from the description below. means of solving the problem

[0014] According to various embodiments, the electronic device comprises: a memory; a display; an audio output unit; at least one camera disposed below the display; and at least one processor operatively connected to the memory, the display, and the at least one camera, wherein the at least one processor may be configured to detect the distance between the at least one camera and at least one first subject, compare a preview image of the at least one first subject collected by the at least one camera with a reference image stored in the memory, determine whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or the result of comparing the preview image and the reference image, and provide a user interface (UI) through either the display or the audio output unit to instruct shooting through the at least one camera when the shooting condition is satisfied.

[0015] According to various embodiments, a method for providing a user interface (UI) in an electronic device may include: detecting a distance between at least one camera positioned at the bottom of a display and at least one first subject; comparing a preview image of the at least one first subject collected by the at least one camera with a reference image stored in advance; determining whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or the result of comparing the preview image and the reference image; and performing at least one of displaying a first UI on the display to instruct shooting through the at least one camera or outputting the first UI as a voice message when the shooting condition is satisfied.

[0016] According to various embodiments, in a non-transient storage medium storing commands, the commands are configured to cause at least one processor to perform at least one operation when executed by at least one processor, wherein the at least one operation may include: detecting the distance between at least one camera positioned at the bottom of a display and at least one first subject; comparing a preview image of the at least one first subject collected by the at least one camera with a reference image stored in advance; determining whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or the result of comparing the preview image and the reference image; and, if the shooting condition is satisfied, displaying a first user interface (UI) on the display to instruct shooting through the at least one camera or outputting the first UI as a voice message. Effects of the invention

[0017] According to various embodiments, the electronic device can clearly indicate the location of the UDC to the user through the UI.

[0018] According to various embodiments, the electronic device can instruct the user via a UI the location of at least one UDC to be used among a plurality of UDCs.

[0019] According to various embodiments, the electronic device can provide the user with an image having clear focus by allowing the distance between the UDC and the subject to correspond to the focal length of the UDC through the UI.

[0020] According to various embodiments, the electronic device can provide the user with an image of a stable composition through the UI.

[0021] According to various embodiments, appropriate interaction between an electronic device and a user and a corresponding user experience (UX) can be provided based on the use of UDC and dynamic UI, thereby increasing user satisfaction and convenience regarding the use of the electronic device. Brief explanation of the drawing

[0022] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments. FIG. 2 is a block diagram illustrating a camera module according to various embodiments. FIG. 3 is a block diagram of an electronic device according to various embodiments. FIG. 4 is a flowchart illustrating a method of providing a UI in an electronic device according to various embodiments. FIG. 5 is a flowchart illustrating an example of a method for providing a UI in an electronic device including a UDC according to various embodiments. FIG. 6 is a flowchart illustrating how an electronic device according to various embodiments generates a category-specific reference model. FIG. 7 is a flowchart illustrating another example of a method in which an electronic device including a UDC according to various embodiments provides a UI. FIG. 8 is a diagram illustrating a UI for indicating a UDC in an electronic device according to various embodiments. FIGS. 9a and 9b are flowcharts illustrating an example of a method for providing a UI in an electronic device comprising a plurality of UDCs according to various embodiments. FIG. 10 is a diagram illustrating a UI for indicating an activated UDC among a plurality of UDCs in an electronic device according to various embodiments. Specific details for implementing the invention

[0023] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0024] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0025] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0026] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0027] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0028] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0029] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0030] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0031] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0032] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0033] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0034] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0035] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0036] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0037] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0038] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0039] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0040] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) may support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0041] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0042] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0043] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0044] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0045] FIG. 2 is a block diagram (200) illustrating a camera module (201) (e.g., camera module (180) of FIG. 1) according to various embodiments. Referring to FIG. 2, the camera module (201) may include a lens assembly (210), a flash (220), an image sensor (230), an image stabilizer (240), a memory (250) (e.g., a buffer memory), or an image signal processor (ISP, 260). The lens assembly (210) may collect light emitted from a subject that is the target of image capture. The lens assembly (210) may include one or more lenses. According to various embodiments, the camera module (201) may include a plurality of lens assemblies (210). In this case, the camera module (201) may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies (210) may have the same lens properties (e.g., angle of view, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties different from the lens properties of other lens assemblies. The lens assemblies (210) may include, for example, a wide-angle lens or a telephoto lens.

[0046] A flash (220) may emit light used to enhance light emitted or reflected from a subject. According to various embodiments, the flash (220) may include one or more light-emitting diodes (e.g., RGB (red-green-blue) LED, white LED, infrared LED, or ultraviolet LED), or a xenon lamp. An image sensor (230) may acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through a lens assembly (210) into an electrical signal. According to various embodiments, the image sensor (230) may include, for example, one image sensor selected from image sensors with different properties such as an RGB sensor, a BW (black and white) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same properties, or a plurality of image sensors having different properties. Each image sensor included in the image sensor (230) can be implemented using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

[0047] The image stabilizer (240) may move at least one lens or image sensor (230) included in the lens assembly (210) in a specific direction or control the operational characteristics of the image sensor (230) (e.g., adjusting read-out timing) in response to the movement of the camera module (201) or the electronic device (201) containing it. This allows for compensating for at least some of the negative effects caused by the movement on the image being captured. According to various embodiments, the image stabilizer (240) may detect such movement of the camera module (201) or the electronic device (201) using a gyroscope sensor (not shown) or an accelerometer sensor (not shown) placed inside or outside the camera module (201). According to various embodiments, the image stabilizer (240) may be implemented, for example, as an optical image stabilizer. The memory (250) may temporarily store at least a portion of the image acquired through the image sensor (230) for the next image processing operation. For example, if image acquisition by the shutter is delayed or multiple images are acquired at high speed, the acquired original image (e.g., a Bayer-patterned image or a high-resolution image) is stored in the memory (250), and the corresponding copy image (e.g., a low-resolution image) can be previewed through the display module (160). Subsequently, when a specified condition is satisfied (e.g., user input or system command), at least a portion of the original image stored in the memory (250) may be acquired and processed, for example, by an image signal processor (260). According to various embodiments, the memory (250) may be configured as at least a portion of the memory (130) or as a separate memory that operates independently thereof.

[0048] The image signal processor (260) can perform one or more image processing operations on an image obtained through the image sensor (230) or an image stored in memory (250). The above one or more image processing methods may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softing). Additionally or generally, the image signal processor (260) may perform control (e.g., exposure time control, or readout timing control, etc.) over at least one of the components included in the camera module (201) (e.g., image sensor (230)). The image processed by the image signal processor (260) may be stored back in memory (250) for further processing or provided to an external component of the camera module (201) (e.g., memory (130), display module (160), electronic device (102), electronic device (104), or server (108)). According to various embodiments, the image signal processor (260) may be configured as at least part of the processor (120), or It may be configured as a separate processor that operates independently of the processor (120). If the image signal processor (260) is configured as a separate processor from the processor (120), at least one image processed by the image signal processor (260) may be displayed through the display device (260) as is or after additional image processing by the processor (120).

[0049] According to various embodiments, the electronic device (101) may include a plurality of camera modules (201), each having different attributes or functions. In this case, for example, at least one of the plurality of camera modules (201) may be a wide-angle camera and at least another may be a telephoto camera. Similarly, at least one of the plurality of camera modules (201) may be a front camera and at least another may be a rear camera.

[0050] FIG. 3 is a block diagram (300) of an electronic device according to various embodiments.

[0051] Referring to FIG. 3, the electronic device (301) may include a display (310), a first camera (312), a second camera (314), a memory (320), an audio output unit (330), and a processor (340).

[0052] According to various embodiments, the electronic device (301) is not limited to the configurations shown in FIG. 3 and may be implemented to include more or fewer configurations. For example, the electronic device (301) may be implemented to include additional configurations described in FIG. 1 or FIG. 2. The electronic device (301) according to various embodiments may be the electronic device (101) of FIG. 1 or the electronic device to be described later.

[0053] In one embodiment, the display (310) may display at least one of information generated by the processor (340), an image collected by at least one of the first camera (312) or the second camera (314), or a message to be provided to the user. The information displayed by the display (310) is not limited to the examples described above.

[0054] In one embodiment, the display (310) can perform a display operation by controlling pixels on the display (310) based on at least one of a passive matrix driving method or an active matrix driving method. In one embodiment, the display (310) may include a first pixel area positioned at a location corresponding to a first camera (312) or a second pixel area positioned at a location corresponding to a second camera (314). At least one pixel included in the first pixel area may be disabled when the first camera (312) is running and may be enabled for display when the first camera (312) is not running. At least one pixel included in the second pixel area may be disabled when the second camera (314) is running and may be enabled for display when the second camera (312) is not running. The display (310) may be included in the display module (160) of FIG. 1.

[0055] In one embodiment, the first camera (312) and the second camera (314) may be UDCs positioned at the bottom of the display (310). In one embodiment, the first camera (312) and the second camera (314) may be positioned at different locations.

[0056] For example, the first camera (312) may be positioned at the top of the display (310) when viewing the display (310) from the front, and the second camera (314) may be positioned at the bottom of the display (310). The top of the display (310) may refer to the end relatively close to the sound output unit (330) (or microphone), and the bottom of the display (310) may refer to the end relatively far from the sound output unit (330) (or microphone).

[0057] As another example, the first camera (312) may be positioned to the right of the display (310) when viewing the display (310) from the front, and the second camera (314) may be positioned to the left of the display (310). The positions of the first camera (312) and the second camera (314) are not limited to the examples described above and can be changed in various ways.

[0058] According to one embodiment, FIG. 3 illustrates that two cameras (e.g., a first camera (312) and a second camera (314)) are included in the electronic device (301), but the electronic device (301) may include one camera or more than two cameras.

[0059] In one embodiment, at least one of the first camera (312) or the second camera (314) may be included in the camera module (180) of FIG. 1 or the camera module (201) of FIG. 2.

[0060] In one embodiment, the memory (320) may store information generated by the processor (340), information for face authentication operations, commands, and / or indicators. The memory (320) may store images collected by at least one of the first camera (312) or the second camera (314).

[0061] In one embodiment, the memory (320) may store images collected by at least one of the first camera (312) or the second camera (314) by category based on an image classification model. In one embodiment, the category may be related to at least one of the location where the image was taken (e.g., indoor, outdoor, landmark, or place name), the time of time the image was taken (e.g., morning, afternoon, or season), the type of subject included in the image (e.g., person, food, landscape, animal, or object), or the number of subjects included in the image.

[0062] In one embodiment, the memory (320) may store as a reference image one of the images having the most frequently used composition by category or one of the user-preferred images selected by the user by category.

[0063] In one embodiment, the memory (320) may include volatile memory or non-volatile memory and may be included in the memory (130) of FIG. 1.

[0064] In one embodiment, the sound output unit (330) may output a message related to a user interface (UI) as voice. For example, the sound output unit (330) may output to the outside of the electronic device (301) at least one of a voice message for indicating at least one of the first camera (312) or the second camera (314), a voice message for adjusting the distance between the first camera (312) or the second camera (314) and a subject, or a voice message for indicating shooting through at least one of the first camera (312) or the second camera (314).

[0065] In one embodiment, the acoustic output unit (330) may be included in the acoustic output module (155) of FIG. 1.

[0066] In one embodiment, the processor (340) controls the display (310), the first camera (312), the second camera (314), and the memory (320) to provide a UI associated with at least one of the first camera (312) or the second camera (314) through at least one of the display (310) or the sound output unit (330). In one embodiment, the processor (340) may perform operations performed in an electronic device according to various embodiments described below.

[0067] In one embodiment, the processor (340) may include one or more processors and may be included in the processor (120) of FIG. 1.

[0068] According to various embodiments, the electronic device comprises: a memory; a display; an audio output unit; at least one camera disposed below the display; and at least one processor operatively connected to the memory, the display, and the at least one camera, wherein the at least one processor may be configured to detect the distance between the at least one camera and at least one first subject, compare a preview image of the at least one first subject collected by the at least one camera with a reference image stored in the memory, determine whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or the result of comparing the preview image and the reference image, and provide a user interface (UI) through either the display or the audio output unit to instruct shooting through the at least one camera when the shooting condition is satisfied.

[0069] In one embodiment, the at least one processor may be configured to detect the distance between the at least one camera and the at least one first subject based on at least one of distance information by size of the subject in the image stored in the memory, the time taken until a signal output from a sensor placed at the bottom of the display is reflected by the at least one subject and input to the sensor, or, in the case of the at least one camera, parallax information of stereoscopic images of the at least one subject acquired by the plurality of cameras.

[0070] In one embodiment, the at least one processor may be configured to determine whether the detected distance corresponds to a focal length set for the at least one camera, and if the detected distance does not correspond to the focal length, to provide a UI through at least one of the display or the sound output unit to instruct to adjust the distance between the at least one camera and the at least one first subject.

[0071] In one embodiment, the at least one processor may be configured to provide a UI through at least one of the display or the sound output unit that instructs to change the shooting angle of the at least one camera so that the composition of the at least one first subject in the preview image corresponds to the composition of the at least one second subject when, as a result of comparing the preview image and the reference image, the composition of the at least one first subject in the preview image does not correspond to the composition of the at least one second subject in the reference image.

[0072] In one embodiment, the at least one processor may be configured to determine that the shooting condition for the at least one camera is satisfied when the detected distance corresponds to a focal length set for the at least one camera, and as a result of comparing the preview image and the reference image, the composition of the at least one first subject in the preview image corresponds to the second composition of the at least one subject in the reference image.

[0073] In one embodiment, the at least one processor may be configured to detect at least one of the position of the at least one first subject or the gaze direction of the at least one subject within the preview image when a shooting condition for the at least one camera is satisfied and the number of the at least one cameras is multiple, and based on at least one of the detected position or gaze direction, select at least one camera among the multiple cameras to collect an image of the at least one first subject, and provide a UI for indicating the selected at least one camera through at least one of the display or the sound output unit.

[0074] In one embodiment, the at least one processor may be configured to store a user-preferred image selected by a user among at least one images collected by the at least one camera as the reference image in the memory, or to analyze at least one image collected by the at least one camera to identify an image having the most frequently used composition information and store the identified image as the reference image in the memory.

[0075] The operation of an electronic device according to various embodiments is described below. The operations of the electronic device according to various embodiments described in FIGS. 4 to 10 may be performed by an electronic device (e.g., the electronic device (101) of FIG. 1 or the electronic device (301) of FIG. 3), at least one processor (e.g., the processor (120) of FIG. 1 or the processor (340) of FIG. 3), or a combination thereof. Hereinafter, the operations shown in FIGS. 4 to 10 are described as being performed by an electronic device as an example.

[0076] At least one camera, a plurality of cameras, a UDC, or a plurality of UDCs described in FIGS. 4 to 10 may each be included in the camera module (180) of FIG. 1 and the camera module (201) of FIG. 2, and may be the first camera (312) or the second camera (314) of FIG. 3. Additionally, the display described in FIGS. 4 to 10 may be included in the display module (160) of FIG. 1 and may be the display (310) of FIG. 3.

[0077] FIG. 4 is a flowchart (400) illustrating a method of providing a UI in an electronic device according to various embodiments.

[0078] Referring to FIG. 4, in operation 402, the electronic device can detect the distance between at least one camera and at least one first subject.

[0079] In operation 404, the electronic device can compare a preview image of at least one first subject collected by at least one camera with a stored reference image.

[0080] In operation 406, the electronic device can determine whether shooting conditions for at least one camera are satisfied based on at least one of the detected distance or the result of comparing a preview image with a reference image.

[0081] In operation 408, the electronic device may display a UI on the display to indicate at least one camera when shooting conditions are satisfied, or output a message related to the UI as voice.

[0082] According to various embodiments, one camera or multiple cameras may be used in the electronic device. A method for providing a UI according to various embodiments may include a method for providing a UI when one camera is used in the electronic device and a method for providing a UI when multiple cameras are used in the electronic device.

[0083] A method for providing a UI when one camera is used in an electronic device according to various embodiments will be described below with reference to FIGS. 5 to 8, and a method for providing a UI when multiple cameras are used in an electronic device according to various embodiments will be described with reference to FIGS. 9a to 10. In the embodiments to be described later, the case where one camera or multiple cameras are each UDC will be described as an example.

[0084] FIG. 5 is a flowchart (500) illustrating an example of a method for providing a UI in an electronic device including a UDC according to various embodiments.

[0085] Referring to FIG. 5, the electronic device can determine the occurrence of a UDC execution event in operation 502. In one embodiment, the UDC execution event is an event for the execution of the UDC and may be generated based on at least one of a user action on the electronic device (e.g., a user holding, lifting, or tilting the electronic device), a touch input on the screen of the electronic device, a user's voice input, or a physical key input.

[0086] In the 504 operation, the electronic device executes the UDC based on a generated UDC execution event and can display a UI on the display to indicate the executed UDC. In one embodiment, the UI to indicate the executed UDC may be provided using at least one pixel among the surrounding pixels of a display pixel area placed at a location corresponding to the executed UDC. For example, the UI may be provided by the at least one pixel being activated for a set period of time or until a specific event occurs (e.g., until the user completes shooting through the UDC). As another example, the UI may be provided by the at least one pixel periodically repeating an activation operation and a deactivation operation. The activation operation of the at least one pixel may indicate an operation in which an electrical signal is input to the at least one pixel and the at least one pixel emits light (pixel-on operation), and the deactivation operation of the at least one pixel may indicate an operation in which an electrical signal is not input to the at least one pixel and the at least one pixel does not emit light (pixel-off operation).

[0087] In operation 506, the electronic device can measure the distance to the subject. For example, the electronic device can measure the distance to the subject based on the time taken for a signal (e.g., an infrared signal or a laser signal) output from a sensor placed at the bottom of the display to be reflected by the subject and input to the sensor. The sensor may be, for example, an infrared sensor or a time of flight (TOF) sensor placed at a location different from the UDC, and the signal output from the sensor may be output to the outside of the electronic device, for example, through the space between pixels of the display. According to one embodiment, instead of measuring the distance to the subject, the electronic device may determine the size of the subject within a preview image collected by the UCD and identify the distance corresponding to the determined size of the subject based on pre-stored distance information by size of the subject.

[0088] In operation 508, the electronic device can identify the focal length of the UDC. In one embodiment, the UDC may support a fixed focus function. The fixed focus function may indicate a function in which the subject being photographed by the UDC is focused when the distance between the UDC and the subject corresponds to a set distance, i.e., the focal length.

[0089] In operation 510, the electronic device can determine whether the measured distance corresponds to the focal length of the UDC. If the measured distance does not correspond to the focal length of the UDC, in operation 512, the electronic device may provide a UI to instruct distance adjustment. In one embodiment, if the distance between the UDC and the subject is greater than or equal to the focal length, the electronic device may provide at least one of a voice message saying "Please reduce the distance from the camera a little more" or a UI on the display to instruct the UDC. The UI on the display to instruct the UDC may, for example, be a UI in which the surrounding pixels of a display pixel area placed at a position corresponding to the UDC are activated in the order from the outermost pixel to the innermost pixel.

[0090] In one embodiment, when the distance between the UDC and the subject is less than the focal length, the electronic device may provide at least one of a voice message saying "Increase the distance from the camera a little more" or a UI on a display for directing the UDC. The UI on a display for directing the UDC may, for example, be a UI in which the surrounding pixels of a display pixel area placed at a position corresponding to the UDC are activated in the order from the innermost pixel to the outermost pixel.

[0091] In operation 510, the electronic device determines that the subject is in focus when the measured distance corresponds to the focal length of the UDC, and in operation 514, may provide a UI to instruct shooting. In one embodiment, the UI to instruct shooting may provide at least one of a voice message saying "Please start shooting" or a UI on a display to instruct the UDC. In one embodiment, the UI on the display to instruct shooting may be the same as or different from the UI provided in operation 504.

[0092] According to the various embodiments illustrated in FIG. 5, the user can easily recognize the location of the UDC and focus based on the UI in the UDC where autofocus support is difficult, thereby obtaining a better quality image.

[0093] FIG. 6 is a flowchart (600) illustrating how an electronic device according to various embodiments generates a category-specific reference model.

[0094] Referring to FIG. 6, the electronic device can collect and store images in operation 602. In one embodiment, image collection and storage may represent an operation of collecting images through the executed UDC and storing the collected images based on a user's selection when the UDC is executed based on the occurrence of a UDC execution event.

[0095] In operation 604, the electronic device can classify images by category based on an image classification model. In one embodiment, the image classification model may be generated by training based on a dataset containing images stored in the electronic device (e.g., images collected by a UDC) and data labeled for each image.

[0096] In one embodiment, the data labeled on each image may be related to at least one of the image shooting location (e.g., indoor, outdoor, landmark, or place name), the image shooting time (e.g., morning, afternoon, or season), the type of subject included in the image (e.g., person, food, landscape, animal, or object), or the number of subjects included in the image. The data labeled on each image may be input by a user, or input based on at least one of location information obtained from GPS, time information set during UCD shooting, or image analysis operations based on big data obtained from the Internet.

[0097] In one embodiment, the category of each image may be determined based on data labeled on each image. The category of each image may be related to at least one of the location where the image was taken, the time the image was taken, the type of subject included in the image, or the number of subjects included in the image.

[0098] In operation 606, the electronic device can generate a reference image by learning composition information by category. In one embodiment, the electronic device learns composition information by category related to at least one of the location where the image was taken, the time the image was taken, the type of subject included in the image, or the number of subjects included in the image, and can determine as a reference image one of an image having the most frequently used composition by category or a user-preferred image selected by the user by category. According to one embodiment, the determined reference image can be used to provide a UI as shown in FIG. 7.

[0099] FIG. 7 is a flowchart (700) showing another example of a method in which an electronic device including a UDC according to various embodiments provides a UI.

[0100] Referring to FIG. 7, in operation 702, the electronic device can determine the occurrence of a UDC execution event. In one embodiment, the UDC execution event may be generated based on at least one of a user action on the electronic device (e.g., a user holding, lifting, or tilting the electronic device), a touch input on the screen of the electronic device, a user's voice input, or a physical key input.

[0101] In operation 704, the electronic device can execute the UDC based on the generated UDC execution event and display a preview image.

[0102] In operation 706, the electronic device can identify a reference image associated with the preview image. In one embodiment, the electronic device can identify a category for the preview image based on an image classification model and identify a reference image corresponding to the identified category. The category for the preview image may be identified based on at least one of, for example, location information obtained from GPS for the preview image, time information when the preview image was obtained, or information about a subject included in the preview image. Information about a subject included in the preview image may be obtained based on, for example, at least one of an image analysis operation based on big data obtained from the Internet, a face recognition technology, or an eye tracking technology.

[0103] In operation 708, the electronic device can determine the difference in composition of the subject by comparing the preview image and the reference image. In one embodiment, the difference in composition of the subject may represent the difference between the composition of the subject included in the preview image and the composition of the subject included in the reference image. In one embodiment, if the subject is a person, the face angle may be detected based on face recognition technology, and the direction of the person's gaze may be detected based on eye tracking technology.

[0104] In operation 710, if the difference in the judged composition is not below a threshold, i.e., above the threshold, the electronic device may provide a UI to instruct to change the shooting angle in operation 712. In one embodiment, if the subject is a human face, and the composition of the face included in the preview image is tilted to the right compared to the composition of the face included in the reference image and the gaze direction is upward, a voice message such as “Please tilt your head to the left and look straight ahead” may be output. According to one embodiment, while the voice message is being output, the surrounding pixels of the display pixel area placed at a position corresponding to the UDC may periodically repeat activation and deactivation actions to provide an additional UI in which the area around the UDC blinks.

[0105] In operation 710, if the difference in the determined composition is less than a threshold value, the electronic device can measure the distance to the subject and identify the focal length of the UDC in operation 714. According to one embodiment, if the difference in the determined composition is less than a threshold value, the electronic device may provide a UI to instruct the user to take a shot instead of performing operation 714.

[0106] In operation 716, the electronic device can determine whether the measured distance corresponds to the focal length of the UDC. If the distance to the subject does not correspond to the focal length of the UDC, in operation 718, the electronic device can provide a UI to instruct distance adjustment.

[0107] In one embodiment, when the distance between the UDC and the subject is greater than the focal length, the electronic device may provide at least one of a voice message saying "Please reduce the distance from the camera a little more" or a UI on a display for directing the UDC. The UI on a display for directing the UDC may, for example, be a UI in which the surrounding pixels of a display pixel area placed at a position corresponding to the UDC are activated in the order from the outermost pixel to the innermost pixel.

[0108] In one embodiment, when the distance between the UDC and the subject is less than the focal length, the electronic device may provide at least one of a voice message saying "Increase the distance from the camera a little more" or a UI on a display for directing the UDC. The UI on a display for directing the UDC may, for example, be a UI in which the surrounding pixels of a display pixel area placed at a position corresponding to the UDC are activated in the order from the innermost pixel to the outermost pixel.

[0109] In operation 716, if the measured distance corresponds to the focal length of the UDC, the electronic device may provide a UI to instruct the user to take a shot in operation 720. In one embodiment, the UI to instruct to take a shot may provide at least one of a voice message saying "Please start taking a shot" or a UI on a display to instruct the UDC. In one embodiment, the UI on the display to instruct to take a shot may be the same as or different from the UI provided in operation 704.

[0110] According to the various embodiments illustrated in FIG. 7, by providing a UI related to the shooting of the UDC based on context conditions such as the distance or angle between the UDC and the subject, it is possible to provide the user with a better quality image and shooting experience.

[0111] FIG. 8 is a drawing (800) illustrating a UI for indicating a UDC in an electronic device according to various embodiments.

[0112] Referring to FIG. 8, in one embodiment, a display (807) may be placed on the front of an electronic device (801), and a UDC (805) may be placed on the lower part of the display (807).

[0113] According to one embodiment, when the UDC (805) is executed, a preview image of the subject (809) and a UDC peripheral UI (803) for directing the execution of the UDC (805) may be displayed on the display (807). For example, the UDC peripheral UI (803) may be provided in a form where the peripheral pixels of a display pixel area placed at a position corresponding to the UDC (805) are activated. For example, the peripheral pixels may be varied in real time as pixel-on and pixel-off operations are repeatedly performed in sequence. For another example, the peripheral pixels may be activated by being pixel-on for a set time or until a specific event occurs (e.g., until the user completes shooting through the UDC). The UDC peripheral UI (803) may be changed in various forms and is not limited to the examples described above.

[0114] A method for providing a UI when a plurality of UDCs are used in an electronic device according to various embodiments below will be explained with reference to FIGS. 9a to 10.

[0115] FIGS. 9a and 9b are flowcharts (900) illustrating an example of a method for providing a UI in an electronic device including a plurality of UDCs according to various embodiments.

[0116] Referring to FIG. 9a, in operation 902, the electronic device can determine the occurrence of a UDC execution event. In one embodiment, the UDC execution event may be generated to execute at least one of a plurality of UDCs based on at least one of a user action on the electronic device (e.g., a user holding, lifting, or tilting the electronic device), a touch input on the screen of the electronic device, a user's voice input, or a physical key input.

[0117] In operation 904, the electronic device may execute at least one of a plurality of UDCs based on a UDC execution event and display a first preview image. In one embodiment, the first preview image may be an image collected by at least one executed UDC.

[0118] In operation 906, the electronic device can identify a first reference image associated with a first preview image. In one embodiment, the electronic device can identify a category for the first preview image based on an image classification model and identify a first reference image corresponding to the identified category.

[0119] In operation 908, the electronic device may select at least one of a plurality of UDCs based on a first reference image. In one embodiment, at least one UDC selected in operation 908 may be the same as or different from at least one UDC executed in operation 904.

[0120] In one embodiment, the electronic device may select at least one of a plurality of UDCs based on at least one of the position or angle of a subject within a first reference image. For example, when the first reference image is displayed on a display, the electronic device may select a UDC placed below pixels closest to the subject within the first reference image among the plurality of UDCs.

[0121] As another example, when the subject is a person, the electronic device can select a UDC located in the direction the person is looking from among multiple UDCs based on the angle of the person's face. The angle of the person's face can be determined by detecting facial landmarks (e.g., eyes, nose, mouth, or ears) based on face recognition technology, and identifying at least one of whether the user has turned, lowered, or tilted their face based on the relative size or position of the detected landmarks. For example, if the distance between two eyes detected on the face is shorter than a set value and the right eye is larger than the left eye, it can be identified that the face is turned to the left.

[0122] In operation 910, the electronic device can acquire a second preview image of at least one subject using at least one selected UDC.

[0123] In operation 912, the electronic device can identify a second reference image associated with the second preview image. In operation 914, the electronic device can determine whether to continue using at least one selected UDC based on the identified second reference image.

[0124] In one embodiment, the electronic device may determine whether to continue using at least one selected UDC based on at least one of the position or angle of the subject within the second reference image.

[0125] For example, when a second reference image is displayed on a display, the electronic device may decide to continue using the selected at least one UDC if at least one UDC placed under the pixels closest to the subject in the second reference image among the plurality of UDCs corresponds to the selected at least one UDC. The electronic device may decide not to continue using the selected at least one UDC if at least one UDC placed under the pixels closest to the subject in the second reference image does not correspond to the selected at least one UDC.

[0126] As another example, the electronic device may decide to continue using the selected at least one UDC if, among the plurality of UDCs, at least one UDC located in the direction in which a person's gaze is directed corresponds to the selected at least one UDC. The electronic device may decide not to continue using the selected at least one UDC if, among the plurality of UDCs, at least one UDC located in the direction in which a person's gaze is directed does not correspond to the selected at least one UDC.

[0127] In operation 914, if the electronic device decides not to continue using at least one selected UDC, it may repeat operation 908, and if it decides to continue using at least one selected UDC, it may perform operation 916 of FIG. 9b indicated by “A”.

[0128] Referring to FIG. 9b, in operation 916, the electronic device can compare a second preview image and a second reference image to determine the difference between the composition of the subject included in the second preview image and the composition of the subject included in the second reference image.

[0129] In operation 918, if the difference in the judged composition is greater than or equal to a threshold value, the electronic device may provide a UI to instruct to change the shooting angle in operation 920. The electronic device may perform operation 920 and repeat the operations of operation 910 and below of FIG. 9a as instructed by “B”.

[0130] In operation 918, if the difference in the judged composition is less than a threshold value, the electronic device can measure the distance to at least one subject and identify the focal length of at least one selected UDC in operation 922.

[0131] According to one embodiment, if the difference in the judged composition is less than a threshold value, the electronic device may provide a UI to instruct the user to take a picture instead of performing a 922 operation.

[0132] In operation 924, if the distance to at least one subject does not correspond to the focal length of at least one selected UDC, in operation 926, the electronic device may provide a UI for instructing distance adjustment.

[0133] In operation 924, if the distance to at least one subject corresponds to the focal length of at least one selected UDC, in operation 928, the electronic device may provide a UI to instruct the user to take a shot.

[0134] According to the various embodiments illustrated in FIGS. 9a and 9b, in an environment where multiple UDCs exist, the user may be encouraged to use a specific UDC to obtain a better quality image based on context conditions such as the distance or angle between each UDC and the subject.

[0135] FIG. 10 is a drawing (1000) illustrating a UI for indicating an activated UDC among a plurality of UDCs in an electronic device according to various embodiments.

[0136] Referring to FIG. 10, in one embodiment, a display (1007) may be disposed on the front of an electronic device (1001), and a plurality of UDCs (1003, 1007) may be disposed on the lower part of the display (1007).

[0137] According to one embodiment, when the first UDC (1007) among the plurality of UDCs (1003, 1005) is executed, a preview image of the subject (1011) collected by the first UDC (1007) and a UDC peripheral UI (1005) for directing the execution of the first UDC (1007) may be displayed on the display (1009).

[0138] For example, the UDC peripheral UI (1005) may be provided with the peripheral pixels of the display pixel area placed at a position corresponding to the first UDC (1007) in an activated state. For example, the peripheral pixels may be varied in real time as pixel-on and pixel-off operations are performed repeatedly in sequence. For another example, the peripheral pixels may be activated by being pixel-on for a set period of time or until a specific event occurs (e.g., until the user completes shooting through the UDC). The UDC peripheral UI (1005) may be changed in various forms and is not limited to the examples described above.

[0139] According to one embodiment, among the plurality of UDCs (1003, 1005), the remaining UDCs excluding the first UDC (1007), for example, the second UDC (1003), may not be executed while the first UDC (1007) is executed, and the pixels of the display pixel area placed at a position corresponding to the second UDC (1003) may be activated to display a preview image of the subject (1011) collected by the first UDC (1007).

[0140] According to various embodiments, a method for providing a user interface (UI) in an electronic device may include: detecting a distance between at least one camera positioned at the bottom of a display and at least one first subject; comparing a preview image of the at least one first subject collected by the at least one camera with a reference image stored in advance; determining whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or the result of comparing the preview image and the reference image; and performing at least one of displaying a first UI on the display to instruct shooting through the at least one camera or outputting the first UI as a voice message when the shooting condition is satisfied.

[0141] In one embodiment, the operation of detecting the distance may include an operation of detecting the distance between the at least one camera and the at least one first subject based on at least one of distance information by size of the subject in the image stored in the memory, the time taken until a signal output from a sensor placed at the bottom of the display is reflected by the at least one subject and input to the sensor, or, in the case where there are multiple cameras, parallax information of stereoscopic images of the at least one subject obtained by multiple cameras.

[0142] In one embodiment, the method may further include: determining whether the detected distance corresponds to a focal length set for the at least one camera; and, if the detected distance does not correspond to the focal length, displaying a second UI on the display to instruct to adjust the distance between the at least one camera and the at least one first subject, or outputting the second UI as a voice message.

[0143] In one embodiment, if, as a result of comparing the preview image and the reference image, the composition of at least one first subject in the preview image does not correspond to the composition of at least one second subject in the reference image, the method may further include an operation of displaying a third UI on the display to instruct changing the shooting angle of the at least one camera so that the composition of the at least one first subject corresponds to the composition of the at least one second subject, or an operation of outputting the third UI as a voice message.

[0144] In one embodiment, the operation of determining whether the shooting condition is satisfied may include determining that the shooting condition for the at least one camera is satisfied when the detected distance corresponds to a focal length set for the at least one camera, and as a result of comparing the preview image and the reference image, the composition of the at least one first subject in the preview image corresponds to the second composition of the at least one subject in the reference image.

[0145] In one embodiment, when a shooting condition for the at least one camera is satisfied and the number of the at least one cameras is multiple, the method may further include an operation of detecting at least one of the position or gaze direction of the at least one first subject in the preview image, an operation of selecting at least one camera among the multiple cameras to collect an image of the at least one first subject based on at least one of the detected position or gaze direction, and an operation of displaying a fourth UI for indicating the selected at least one camera on the display or an operation of outputting the fourth UI as a voice message.

[0146] In one embodiment, the method may further include the operation of storing a user-preferred image selected by a user among at least one image collected by at least one camera as the reference image, or the operation of analyzing at least one image collected by at least one camera to identify an image having the most frequently used composition information and storing the identified image as the reference image.

[0147] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0148] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0149] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0150] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0151] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0152] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

Claim 1 In an electronic device, a memory for storing instructions; a display; an audio output unit; and at least one camera disposed below the display; The electronic device comprises at least one processor operatively connected to the memory, the display, and the at least one camera, wherein, when the instructions are executed individually or collectively by the at least one processor, the electronic device detects the distance between the at least one camera and at least one first subject, compares a preview image of the at least one first subject collected by the at least one camera with a reference image stored in the memory, determines whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or the result of comparing the preview image and the reference image, and provides a user interface (UI) through either the display or the sound output unit to instruct shooting through the at least one camera if the shooting condition is satisfied, and if the shooting condition for the at least one camera is satisfied and the number of the at least one cameras is multiple, detects at least one of the position of the at least one first subject within the preview image or the gaze direction of the at least one first subject, and based on at least one of the detected position or the gaze direction, among the multiple cameras, the An electronic device that selects at least one camera to collect an image of at least one first subject, and provides a UI through the display via surrounding pixels of a pixel area placed at a position corresponding to the selected at least one camera to indicate the selected at least one camera. Claim 2 An electronic device according to claim 1, wherein, when the instructions are executed individually or collectively by the at least one processor, the electronic device detects the distance between the at least one camera and the at least one first subject based on at least one of distance information by size of a subject in an image stored in the memory, the time taken until a signal output from a sensor placed at the bottom of the display is reflected by the at least one first subject and input to the sensor, or, in the case where there are multiple cameras, parallax information of stereoscopic images of the at least one first subject acquired by multiple cameras. Claim 3 An electronic device according to claim 1, wherein, when the instructions are executed individually or collectively by the at least one processor, the electronic device provides a UI through at least one of the display or the sound output unit to instruct the electronic device to determine whether the detected distance corresponds to a focal length set for the at least one camera, and if the detected distance does not correspond to the focal length, to adjust the distance between the at least one camera and the at least one first subject. Claim 4 An electronic device according to claim 1, wherein the instructions, when executed individually or collectively by the at least one processor, cause the electronic device to provide, through at least one of the display or the sound output unit, a UI that instructs the electronic device to change the shooting angle of the at least one camera so that the composition of the at least one first subject in the preview image corresponds to the composition of the at least one second subject when, as a result of comparing the preview image and the reference image, the composition of the at least one first subject in the preview image does not correspond to the composition of the at least one second subject in the reference image. Claim 5 An electronic device according to claim 1, wherein, when the instructions are executed individually or collectively by the at least one processor, the electronic device determines that the shooting condition for the at least one camera is satisfied if the detected distance corresponds to a focal length set for the at least one camera and, as a result of comparing the preview image and the reference image, the composition of the at least one first subject in the preview image corresponds to the second composition of the at least one subject in the reference image. Claim 6 delete Claim 7 An electronic device according to claim 1, wherein, when the instructions are executed individually or collectively by the at least one processor, the electronic device stores a user-preferred image selected by a user among at least one images collected by the at least one camera as a reference image in the memory, or analyzes at least one image collected by the at least one camera to identify an image having the most frequently used composition information and stores the identified image as a reference image in the memory. Claim 8 A method for providing a user interface (UI) in an electronic device comprises: detecting a distance between at least one camera disposed at the bottom of a display and at least one first subject; comparing a preview image of the at least one first subject collected by the at least one camera with a previously stored reference image; determining whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or the result of comparing the preview image and the reference image; performing at least one of displaying a first UI on the display to instruct shooting through the at least one camera or outputting the first UI as a voice message when the shooting condition is satisfied; detecting at least one of the position of the at least one first subject within the preview image or the gaze direction of the at least one first subject when the shooting condition for the at least one camera is satisfied and the number of the at least one cameras is multiple; and selecting at least one camera among the multiple cameras to collect an image of the at least one first subject based on at least one of the detected position or the gaze direction. A method comprising the operation of providing a UI through the display via surrounding pixels of a pixel area positioned at a location corresponding to the selected at least one camera for indicating the selected at least one camera. Claim 9 In claim 8, the operation of detecting the distance comprises an operation of detecting the distance between the at least one camera and the at least one first subject based on at least one of distance information by size of a subject in an image stored in the memory of the electronic device, the time taken until a signal output from a sensor placed at the bottom of the display is reflected by the at least one first subject and input to the sensor, or parallax information of stereoscopic images of the at least one first subject acquired by the plurality of cameras. Claim 10 In a non-transient storage medium storing commands, the commands are configured to cause at least one processor to perform at least one operation when executed by at least one processor, wherein the at least one operation comprises: detecting the distance between at least one camera positioned below a display and at least one first subject; comparing a preview image of the at least one first subject collected by the at least one camera with a previously stored reference image; determining whether a shooting condition for the at least one camera is satisfied based on at least one of the detected distance or the result of comparing the preview image and the reference image; performing at least one of displaying a first user interface (UI) on the display to instruct shooting through the at least one camera or outputting the first UI as a voice message when the shooting condition is satisfied; and, when the shooting condition for the at least one camera is satisfied and the number of the at least one cameras is multiple, detecting at least one of the position of the at least one first subject within the preview image or the gaze direction of the at least one first subject. A storage medium comprising: an operation of selecting at least one camera to collect an image of at least one first subject among a plurality of cameras based on at least one of the detected position or the line of sight direction; and an operation of providing a UI through the display via surrounding pixels of a pixel area placed at a position corresponding to the selected at least one camera to indicate the selected at least one camera. Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete

Images