Electronic device and photographing method of same

Abstract

Disclosed are an electronic device and a photographing method thereof. A photographing method of an electronic device, according to one embodiment, may comprise the operations of: classifying a category of a subject included in a first preview image acquired by using a camera; identifying a reference composition corresponding to the subject on the basis of the category of the subject; adjusting zoom of the camera on the basis of the reference composition to acquire a second preview image; and outputting a result obtained by performing processing to adjust the composition of the subject in the second preview image to the reference composition. Various other embodiments may also be possible.

Description

Electronic devices and photographing methods thereof

[0001] Embodiments of the present invention relate to an electronic device and a photographing method thereof.

[0002] When a user wishes to capture an object using an electronic device, he or she may need to modify the field of view (FoV) centered on the subject to achieve the best results. To adjust the FoV, the user may need to manually zoom in or out repeatedly by touching the screen of the electronic device. Manual zooming can result in a loss of precision and an aesthetically unsatisfactory composition. A method is needed that automatically performs zooming while capturing an image with a composition optimized for the subject.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] An electronic device according to one embodiment may include a camera. The electronic device may include at least one processor. The electronic device may include a memory that stores instructions. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to classify a category of an object included in a first preview image acquired using the camera. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to identify a reference frame corresponding to the object based on the category of the object. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to adjust a zoom of the camera based on the reference frame to acquire a second preview image.

[0005] The above instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to output a result of performing processing for adjusting the composition of the subject in the second preview image to the reference composition.

[0006] A method for photographing an electronic device according to one embodiment may include an operation for classifying a category of a subject included in a first preview image acquired using a camera. The method may include an operation for identifying a reference composition corresponding to the subject based on the category of the subject. The method may include an operation for adjusting a zoom of the camera based on the reference composition to acquire a second preview image. The method may include an operation for outputting a result of performing processing for adjusting the composition of the subject to the reference composition in the second preview image.

[0007] According to one embodiment, a computer-readable recording medium storing one or more computer programs may include instructions for performing the method in a processor.

[0008] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0009] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0010] FIG. 2 is a diagram for explaining a generative artificial intelligence system according to one embodiment.

[0011] FIG. 3 is a flowchart illustrating a photographing method of an electronic device according to one embodiment.

[0012] FIG. 4 is a drawing for explaining the operation of an electronic device according to one embodiment.

[0013] FIG. 5 is a drawing for explaining the operation of an electronic device according to one embodiment.

[0014] Figure 6 is a drawing for explaining a shooting composition according to one embodiment.

[0015] Figure 7 is a drawing for explaining a shooting environment according to one embodiment.

[0016] FIGS. 8A and 8B are diagrams for explaining an operation of an electronic device according to one embodiment of the present invention to photograph a group subject.

[0017] FIGS. 9A and 9B are diagrams for explaining cropping processing performed by an electronic device according to one embodiment.

[0018] FIGS. 10A and 10B are drawings for explaining outpainting processing performed by an electronic device according to one embodiment.

[0019] FIG. 11A and FIG. 11B are diagrams for explaining an operation of an electronic device according to one embodiment to photograph a plurality of subjects of different categories.

[0020] FIG. 12 is a drawing for explaining an operation of an electronic device according to one embodiment of the present invention to photograph a subject.

[0021] FIGS. 13A to 13E are drawings for explaining an automatic zoom control operation performed by an electronic device according to one embodiment.

[0022] Fig. 14 shows a flowchart of a photographing method of an electronic device according to one embodiment.

[0023] FIG. 15A is a front perspective view of an electronic device according to one embodiment.

[0024] FIG. 15b is a plan view of the rear side of an electronic device according to one embodiment.

[0025] FIG. 16 is an exploded perspective view of a portion of the electronic device of FIGS. 15A and 15B including a hinge device according to one embodiment.

[0026] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0027]

[0028] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100), according to one embodiment.

[0029] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0030] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134).

[0031] According to one embodiment, the processor (120) may be implemented as a circuit (e.g., a processing circuit) such as a system on chip (SoC) or an integrated circuit (IC). The processor (120) may include one or more processors. For example, the processor (120) may include a combination of one or more processors such as a CPU, a GPU, an MPU, an AP, and a CP.

[0032] According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0033] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0034] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0035] According to one embodiment, the memory (130) may include one or more memories. Instructions stored in the memory (130) may be stored in a single memory. Instructions stored in the memory (130) may be divided and stored in multiple memories. The instructions stored in the memory (130) may be executed by the processor (120) to cause the electronic device (101) to perform and / or control the operations of the electronic device (101) described with reference to FIGS. 2 to 16.

[0036] According to one embodiment, instructions stored in memory (130) may cause electronic device (101) to perform one or more operations when individually or collectively executed by at least one processor (e.g., main processor (121) and / or auxiliary processor (123)). For example, instructions stored in memory (130) may be executed by one processor (e.g., main processor (121) or auxiliary processor (123) such as a communication processor) or by multiple processors operating cooperatively (e.g., main processor (121) and auxiliary processor (123)).

[0037] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0038] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0039] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0040] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0041] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0042] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0043] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0044] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0045] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0046] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0047] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0048] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0049] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0050] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0051] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0052] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0053] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0054] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0055]

[0056] FIG. 2 is a diagram for explaining a generative artificial intelligence system according to one embodiment.

[0057] Referring to FIG. 2, according to one embodiment, the generative artificial intelligence system (200) may be a program (e.g., a software module) implemented on an electronic device (e.g., an electronic device (101) of FIG. 1) and / or a server (e.g., a server (108) of FIG. 1).

[0058] According to one embodiment, a user query / response interface (210) may receive user input. The user input may be an input of a type (or modality) such as natural language, image, audio, and / or video. Additionally, context information may also be transmitted when the user input is transmitted. The context information may include various side information related to the time when the user input is input to the artificial intelligence system (200). For example, there is side information such as application information currently being used by the user or location information of the user. Additionally, the user input may be an input of a mixed type of natural language, image, audio, video, and / or context information as described above. Additionally, the user input may include non-natural language input, such as selecting a menu.

[0059] According to one embodiment, a user query / response interface (210) may provide output from a generative artificial intelligence system to a user. The output may include a natural language-based response and / or specific content. The output may also include an action requested by the user.

[0060] In one embodiment, an AI framework (220) may receive user input. Based on the user input (e.g., a user's query), the AI ​​framework (220) may coordinate and / or control one or more components necessary to perform an action corresponding to the user's intent.

[0061] According to one embodiment, user input received from the user query / response interface (210) may be transmitted to a prompt design component (221). The prompt design component (221) may be used to generate a prompt suitable as input to a generative model (e.g., a large language model (LLM) and / or a large multimodal model (LMM)) based on the user input.

[0062] In one embodiment, the prompt design component (221) may be an AI component that utilizes a machine learning algorithm or a neural network. The prompt design component (221) may generate improved prompts over time through learning. The prompt design component (221) may access a knowledge repository (230) to generate prompts based on user input. The knowledge repository (230) may include user preference data, a prompt library, and / or prompt examples. The prompt design component (221) may provide the generated prompts to a generative model (e.g., an LLM and / or an LMM).

[0063] According to one embodiment, the APIs / Plugins management component (223) can communicate with an external information source based on a request for additional information when user input is transmitted to the generative model.

[0064] In one embodiment, the APIs / Plugins management component (223) can establish a communication channel for communication with the outside of the system (200) via the API. The APIs / Plugins management component (223) can enable access to various data sources via the communication channel. The acquired information can be used to generate prompts by the prompt design component (221) along with user input, or can be used as input to the generative model (250).

[0065] According to one embodiment, the APIs / Plugins management component (223) may request the application / service component (240) via the API for a final action in response to user input, rather than an intermediate action, if the final action must be performed by the application or service.

[0066] In one embodiment, the refiner component (225) can fine-tune the output of the generative model (250). For example, the refiner component (225) can determine the relevance (e.g., a score) between the output (e.g., content) of the generative model and the user input. For example, the refiner component (225) can determine whether the output contains biased information (e.g., selective information). For example, the refiner component (225) can determine whether the output contains harmful information (e.g., violent content or profanity).

[0067] In one embodiment, the refinement component (225) may determine the degree of matching (e.g., a score) between the output of the generative model (250) and the user input (e.g., the intent of the user input). If the refinement component (225) determines that the output of the generative model (250) does not correspond to the user input, the refinement component (225) may modify the output to correspond to the user input.

[0068] In one embodiment, the refinement component (225) may provide hints to the user (e.g., hints for prompt generation) to enable the user to obtain information that matches the user's intent from the generative model (250).

[0069] According to one embodiment, a generative model (250) may refer to an artificial intelligence neural network that generates new data (e.g., text, images, audio, or video) based on user input (e.g., user utterance). The generative model (250) may include an image generation model and / or a language generation model.

[0070] In one embodiment, the image generation model may include a generative adversarial network (GAN) and / or a variational autoencoder (VAE). An example of an image generation model is a diffusion-based generative model having the structure of a VAE and a transformer.

[0071] In one embodiment, a language generation model (e.g., ChatGPT) may be a model trained to generate statistically most appropriate output based on input. The language generation model may include a large multimodal model (LMM). An LMM can identify various types of input, such as text, images, audio (e.g., speech), and / or video, and generate new data corresponding to the input.

[0072] It is apparent that the AI ​​neural network of the present disclosure may include various foundation models such as a code model, an image model, and / or other AI neural network models.

[0073] 'LMM' may include an inference engine using an LMM neural network model. For example, 'inputting a multimodal input prompt to an LMM' may mean 'inputting the multimodal input prompt to an inference engine based on an LMM.' For example, 'the output of an LMM for a multimodal input prompt' may mean the output information of the last neural network layer of the LMM obtained when inputting the input prompt to an LMM-based inference engine and / or the output information modified through additional processing.

[0074] The attention mechanism is a technology that enables an AI model to focus (attention) on important parts of input data. The attention mechanism can be used to predict output data by predicting the extent to which some parts of multimodal input data (e.g., multimodal input data such as voice, video, and images) contribute to the output of the intermediate layers of a neural network and / or the final output. While a recurrent neural network (RNN) structure, which sequentially processes each element of a sequence, may degrade prediction performance when there is information dependency between long time series distances, the attention mechanism can account for information dependency between long time series distances by controlling the level of weight concentration (attention) within the entire and / or partial context of the input data. A transformer can be configured with an encoder-decoder structure. The encoder can process the input data and output compressed information (e.g., a contextual representation). The decoder can process the compressed information and output data. Each of the encoder and decoder may include an independent attention network, and may further include a cross-attention network connecting the encoder and decoder.

[0075]

[0076] FIG. 3 is a flowchart illustrating a photographing method of an electronic device according to one embodiment.

[0077] Referring to FIG. 3, according to one embodiment, operations 310 to 380 may be understood to be performed in a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0078] In operation 310, the electronic device (101) may obtain a first preview image using a camera (e.g., the camera module (180) of FIG. 1). The preview image is an image received in real time from the camera and may be displayed on a display (e.g., the display module (160)) of the electronic device (101). When a user (e.g., a user of the electronic device (101)) takes a picture of a subject using the electronic device (101), the user may adjust the composition and / or settings (e.g., the zoom level) of the subject while checking the preview image. The preview image may include a first preview image and a second preview image. The electronic device (101) may display an image (e.g., the preview image) of the subject received in real time from the camera on a display (e.g., the display module (160)). Throughout the specification, subject may be used as a term to refer to the object that the user wishes to photograph.

[0079] In operation 320, the electronic device (101) can determine whether the type of a scene can be detected from the first preview image. The electronic device (101) can analyze the scene corresponding to the first preview image to detect the type of the scene. The type of the scene can be determined based on information about objects included in the scene and / or a background located around the objects. For example, the type of the scene can be determined based on information such as the number of objects included in the scene, the category of the objects, and the category of the background. The electronic device (101) can detect the type of the scene using an artificial intelligence (AI) model. The AI ​​model can include a model such as a deep-learning model or a machine learning (ML) model. The electronic device (101) can recognize an object included in the first preview image. The electronic device (101) can recognize a subject included in a first preview image using an artificial intelligence model (e.g., a deep learning model). For example, the electronic device (101) can detect information about the subject from a scene corresponding to the first preview image. The electronic device (101) can classify the category of the subject included in the first preview image using an artificial intelligence model (e.g., a deep learning model). Based on the result of recognizing the subject (e.g., the category of the subject), the electronic device (101) can detect the type of the scene corresponding to the first preview image. The type of the scene may include a single scene and a complex scene. A single scene may include a scene in which the main subject has one type of category.For example, a single scene may include a scene composed of subjects of one category, such as scenery, person, things, and animals other than people. A composite scene may include a scene with two or more categories of main subjects. For example, a composite scene may include a scene composed of subjects of two or more categories, such as person+scenery or animal+scenery. Scenery refers to the overall appearance of a wide area in a natural or artificial environment, and the electronic device (101) may classify the type of the scene as scenery in response to detecting information about a space (e.g., a space such as a mountain, an ocean, a beach, a city, or an indoor) from the scene. The electronic device (101) may classify the subject as a person and classify the type of the scene as being about people in response to detecting information about a person (e.g., information about a face, a baby, or people) from the scene. The electronic device (101) may, in response to detecting information about an object (e.g., information about an object such as food, clothing, or a vehicle) from a scene, classify the subject as an object and classify the type of the scene as being about an object. The electronic device (101) may, in response to detecting information about an animal (e.g., information about an animal such as a bird, a dog, or a cat) from a scene, classify the subject as an animal and classify the type of the scene as being about an animal. It should be noted that the types of scenes described in the specification are merely exemplary and the types of scenes that the electronic device (101) can detect are not limited thereto. The electronic device (101) may determine whether the type of scene corresponding to the first preview image is detectable.For example, the electronic device (101) may not be able to detect the type of the scene if information about the subject and / or background included in the first preview image is not detected. The electronic device (101) may perform operation 350 in response to detecting the type of the scene corresponding to the first preview image. The electronic device (101) may perform operation 330 in response to not detecting the type of the scene corresponding to the first preview image.

[0080] In operation 330, the electronic device (101) can determine whether a subject is located within a boundary line of a first preview image. When multiple subjects exist within the first preview image, the electronic device (101) can determine whether a main subject is located within the boundary line, or can view the multiple subjects as a group (e.g., a mass) and determine whether the group (e.g., a mass) is located within the boundary line. For example, the electronic device (101) can determine a subject occupying the largest area within the first preview image as a main subject and determine whether the main subject is located within the boundary line of the first preview image. For example, the electronic device (101) can view multiple subjects existing within the first preview image as a mass and determine whether they are located within the boundary line. The boundary line is a virtual line set with a predetermined margin from the edge of the image, and may include a virtual line for indicating a work area within the image. Even if the electronic device (101) cannot classify the category of the subject included in the first preview image (e.g., if information about the subject is not detected), the electronic device (101) can detect the presence of the subject and the position of the subject within the image. The electronic device (101) can recognize the subject as a mass and determine whether the subject is located within the boundary line of the first preview image (e.g., inside the boundary line). For example, if the electronic device (101) cannot detect information about the subject included in the first preview image, the electronic device (101) can determine whether a virtual mass including all of the subjects existing in the first preview image is located within the boundary line of the first preview image.The electronic device (101) may perform operation 380 in response to determining that the subject is located within the boundary of the first preview image. The electronic device (101) may perform operation 340 in response to determining that the subject is not located within the boundary of the first preview image.

[0081] In operation 340, the electronic device (101) can adjust the zoom. The electronic device (101) can automatically adjust the zoom of a camera (e.g., the camera module (180) of FIG. 1). For example, the electronic device (101) can automatically adjust the zoom of the camera without any operation by the user. The camera (e.g., the camera module (180) of FIG. 1) of the electronic device (101) can include one or more lenses. For example, the camera can include an ultra-wide angle lens, a wide angle lens, and a telephoto lens. The electronic device (101) can adjust the zoom of the camera by adjusting the focal length of the currently selected lens or by changing the lens. For example, the electronic device (101) can adjust the zoom of the camera by changing the focal length of the currently selected lens. For example, the electronic device (101) can change the focal length and adjust the zoom of the camera by changing the lens to be used with a lens having a different focal length range. The electronic device (101) can adjust the zoom of the camera based on the boundary line of the first preview image. The electronic device (101) can adjust the zoom of the camera so that the subject can be positioned within the boundary line in the first preview image. The electronic device (101) can adjust the zoom of the camera based on the center line of the first preview image. The center line may include a virtual line for indicating the central point of the image, such as a crossline where the horizontal midpoint and the vertical midpoint of the first preview image intersect. For example, the electronic device (101) can adjust the zoom of the camera by considering whether the subject is positioned above or below the center line of the first preview image.The electronic device (101) can perform optical zoom, which physically enlarges and reduces an image by moving the optical lens back and forth. The operation of the electronic device (101) adjusting the zoom to position a subject within the boundary will be described in detail with reference to FIGS. 12 to 13e below.

[0082] In operation 350, the electronic device (101) can identify a reference composition corresponding to the subject. The electronic device (101) can identify the reference composition corresponding to the subject based on the category of the subject. The electronic device (101) can identify the reference composition corresponding to the subject based on the category of the subject included in the first preview image classified using an artificial intelligence model (e.g., a deep learning model). The reference composition can include a structure in which the subject is aesthetically appropriately arranged. For example, the reference composition can include a composition in which a part of the subject is aesthetically appropriately arranged within the image without being cropped or going beyond the boundary. The types of compositions in which the subject can be arranged (e.g., shooting compositions) will be described in detail later with reference to FIG. 6.

[0083] In operation 360, the electronic device (101) can adjust the zoom of the camera to obtain a second preview image. The electronic device (101) can adjust the zoom of the camera based on the category of the subject. The electronic device (101) can detect the type of scene corresponding to the first preview image based on the category of the subject, and can adjust the zoom of the camera based on the type of scene. For example, the electronic device (101) can determine the zoom level of the camera based on the type of scene corresponding to the first preview image obtained in operation 320 (e.g., the type of scene such as a person, a landscape, an object, an animal, or a composite scene), and can adjust the zoom of the camera based on the zoom level. The electronic device (101) can identify a reference frame corresponding to the subject based on the category of the subject, and can adjust the zoom of the camera based on the reference frame. For example, the electronic device (101) can adjust the zoom of the camera to change the composition of the subject into the most appropriate and aesthetically pleasing composition according to the subject category. The camera can include one or more lenses. For example, the camera can include an ultra-wide angle lens, a wide angle lens, and a telephoto lens. The electronic device (101) can adjust the zoom of the camera by adjusting the focal length of the currently selected lens or changing the lens. The electronic device (101) can determine the type of lens and / or zoom level to use based on the subject category. For example, the electronic device (101) can determine an estimated zoom level that is estimated to be appropriate for the type of scene according to the subject category. The estimated zoom level that is estimated to be appropriate for the type of scene can be determined based on information about the lens and / or zoom level that is mainly used according to the type of scene. The zoom ratio can be determined within a range of magnifications that can be implemented in the electronic device (101).For example, the electronic device (101) may determine the zoom level within a range of optical zoom magnification that can be implemented using a lens of a camera (e.g., a lens such as an ultra-wide-angle lens, a wide-angle lens, a telephoto lens) to a digital zoom magnification (e.g., a range such as 0.6x to 100x). The electronic device (101) may adjust the zoom of the camera based on the category of the subject to obtain the second preview image. For example, the electronic device (101) may obtain the second preview image by enlarging or reducing the subject or maintaining the zoom magnification of the first preview image. The second preview image may include an image with a different zoom magnification from the first preview image. The second preview image may include an image with the same zoom magnification as the first preview image. For example, the second preview image may include an image with the zoom magnification of the first preview image maintained. The electronic device (101) can adjust the zoom of the camera to obtain a second preview image in which the subject's composition is improved compared to the first preview image. The subject's composition can be improved as it approaches a reference composition corresponding to the subject. The reference composition can include a structure in which the subject is aesthetically appropriately arranged.

[0084] In operation 370, the electronic device (101) may perform processing to modify the composition of the subject to a reference composition on the second preview image. The electronic device (101) may determine whether the composition of the subject in the second preview image is suitable for the reference composition corresponding to the subject, and may perform processing to modify the composition of the subject to the reference composition. For example, if the electronic device (101) cannot place the subject to the reference composition only with the zoom adjustment performed in operation 360, the electronic device (101) may perform additional processing on the second preview image. The processing for the electronic device (101) to modify the composition of the subject to the reference composition may include at least one of fixing, cropping, and outpainting. For example, the electronic device (101) may perform digital zoom processing (e.g., digital zoom processing through cropping) on ​​the second preview image. Digital zoom may refer to a method of enlarging or reducing a digital image displayed on an image sensor (e.g., an image sensor of a camera module (180)) through digital signaling.

[0085] In operation 380, the electronic device (101) may output a result. The result may include an image automatically captured by the electronic device (101) using a camera. For example, if the electronic device (101) determines that the subject is appropriately positioned in the reference composition within a preview image (e.g., a first preview image, a second preview image), the electronic device (101) may automatically capture the image and output the result without a user input. The result may include an image captured by the electronic device (101) using a camera in response to a user input. For example, the electronic device (101) may output the result in response to a user input (e.g., a user input such as a button input, a touch input, or a voice input) indicating an intention to capture the image using the camera. The electronic device (101) may determine whether the result satisfies the quality of an optical zoom level. The electronic device (101) may perform upscaling on the result in response to determining that the result does not meet the image quality of the optical zoom level. For example, in the case of an image that has been digitally zoomed, the image quality may not meet the image quality of the optical zoom level, and the electronic device (101) may perform upscaling on the result. Upscaling is a process for improving the image quality of the result, and may include processes such as noise removal, brightness adjustment, sharpening, and detail enhancement. The electronic device (101) may output the result and display it on a display (e.g., the display module (160) of FIG. 1). The electronic device (101) may store the result. The electronic device (101) may store the result in a memory (e.g., the memory (130) of FIG. 1). The result may include a plurality of images.The electronic device (101) can separate the subject included in the first preview image and / or the second preview image from the background and store the result. For example, the electronic device (101) can separate the subject recognized in the first preview image and / or the second preview image from the background and store the result as a separate result. The user can check the result and perform editing.

[0086] According to one embodiment, operations 310 to 380 may be performed sequentially, but are not limited thereto. For example, two or more operations may be performed in parallel. The electronic device (101) may not perform all of operations 310 to 380. For example, one or more of operations 310 to 380 may be omitted.

[0087]

[0088] FIG. 4 is a drawing for explaining the operation of an electronic device according to one embodiment.

[0089] Referring to FIG. 4, according to one embodiment, a display of an electronic device (101) (e.g., a display module (160) of FIG. 1) may include a first display (e.g., a main display, a foldable display) and / or a second display (e.g., a sub display, a cover display). The first display may include a display disposed on the front side of the electronic device (101). The second display may include a display disposed on the rear side of the electronic device (101). The electronic device (101) may include a camera (408) (e.g., a camera module (180) of FIG. 1). The camera (408) may include one or more lenses. A user may capture an image using the camera (408) of the electronic device (101). For example, a user may capture an image while checking his or her appearance in real time through the second display (e.g., the cover display). The image may include a still photograph and / or individual frames of a video.

[0090] According to one embodiment, the electronic device (101) may provide an auto-zoom function. The auto-zoom function may be a function in which the electronic device (101) automatically adjusts the zoom of the camera (408). For example, the auto-zoom function may be a function in which the electronic device (101) automatically controls the zoom ratio of the camera (408) without user operation. The electronic device (101) may use an icon (410) to visually indicate that the auto-zoom function is activated. The user may select the icon (410) to change the activation status of the auto-zoom function. For example, the user may select the icon (410) to turn the auto-zoom function on or off. The electronic device (101) may display an interface (e.g., interface (411), interface (413)) indicating the activation status of the auto-zoom function. The electronic device (101) may, in response to a change in the activation status of the auto-zoom function, display an interface (e.g., interface (411), interface (413)) indicating the activation status of the auto-zoom function as a pop-up. For example, the electronic device (101) may temporarily display an interface (411) including text (e.g., “auto-zoom off”) indicating that the auto-zoom function is disabled through the screen (401). For example, the electronic device (101) may, in response to the auto-zoom function being turned on (e.g., in response to a user input to the icon (410)), temporarily display an interface (413) including text (e.g., “auto-zoom on”) indicating that the auto-zoom function is enabled. The icon (410) may visually change depending on whether the auto-zoom function is enabled. For example, when a user selects the icon (410) to disable the auto-zoom function, the electronic device (101) may change the color and / or shape of the icon (410) and display it on the display.For example, the electronic device (101) may display the icon (410) in an enlarged form when the auto-zoom function is activated, and in a reduced form when the auto-zoom function is deactivated. It should be noted that the icon (410) illustrated in FIG. 4 is merely an example for indicating that the auto-zoom function of the electronic device (101) is activated, and that the operation indicating that the auto-zoom function of the electronic device (101) is activated is not limited thereto.

[0091] According to one embodiment, the electronic device (101) may display a first preview image (401) acquired using the camera (408) on a display (e.g., a cover display). The electronic device (101) may display an interface (e.g., interface (420), interface (440)) indicating a current zoom ratio of the camera (408) on the display. For example, the interface (420) may include text (e.g., 2x) indicating that the current zoom ratio of the camera (408) is set to 2x. The zoom ratio may be a ratio indicating the degree to which a subject is enlarged or reduced. For example, when the zoom ratio of the camera (408) is set to 2x, the subject may be captured at a size approximately twice larger than its original size. For example, the first preview image (401) may be an image acquired using the camera (408) with the zoom ratio set to 2x. The higher the zoom ratio, the larger the subject becomes, and the lower the zoom ratio, the smaller the subject becomes, allowing for a wider overall field of view.

[0092] According to one embodiment, the electronic device (101) can recognize a subject (e.g., a person (430), a bridge) included in the first preview image (401). The electronic device (101) can classify a category of the subject included in the first preview image (401). For example, the electronic device (101) can recognize the subject, the person (430), the bridge, and the background location included in the first preview image (401), and classify the category of the subject (e.g., a person, a landscape, a landmark). For example, the electronic device (101) can recognize the bridge included in the first preview image (401) as the Manhattan Bridge, and classify the category as a landmark. The electronic device (101) can detect the type of the scene corresponding to the first preview image (401) based on the category of the subject, and adjust the zoom of the camera (408). The electronic device (101) can generate a second preview image (402) by adjusting the zoom of the camera (408). For example, the electronic device (101) can automatically adjust the zoom of the camera (408) to 1x to generate the second preview image (402). The electronic device (101) can automatically adjust the zoom of the camera (408) based on the composition of the subject in the first preview image (401). The electronic device (101) can adjust the zoom of the camera (408) by determining whether the composition of the subject (e.g., a person (430), legs) in the first preview image (401) is suitable for a reference composition corresponding to the subject. The reference composition can include a structure in which the subject is aesthetically suitable. The electronic device (101) may not adjust the zoom of the camera (408) if the composition of the subject (e.g., person (430), leg) in the first preview image (401) is already suitable for the reference composition. In this case, the second preview image (402) may be the same image as the first preview image (401).The electronic device (101) can display the second preview image (402) on the display. The electronic device (101) can visually indicate that the zoom ratio of the camera (408) has changed. For example, the electronic device (101) can visually indicate that the zoom ratio has changed by displaying an interface (440) indicating the current zoom ratio of the camera (408) instead of the interface (420). For example, when the zoom ratio of the camera (408) has changed, the electronic device (101) can dynamically change the interface (420) to the interface (440) by applying an animation effect. The interface (440) can indicate the current zoom ratio of the camera (408) (e.g., the changed zoom ratio). For example, the interface (440) can include text (e.g., 1x) indicating that the zoom ratio of the camera (408) is currently set to 1x.

[0093] According to one embodiment, the electronic device (101) may determine whether the composition of the subject (e.g., person (430), legs) in the second preview image (402) is suitable for a reference composition corresponding to the subject (e.g., person (430), legs). For example, in the case of the second preview image (402), the reference composition may be such that the upper body of the person (430) is positioned at a predetermined ratio (e.g., about 60% to about 80%) within the screen, the Manhattan Bridge behind the person (430) is not cut off, and the person (430) is positioned in a vanishing point composition centered on the person. It should be noted that the reference composition corresponding to the subject may vary depending on the image, and the above description is merely an example. The electronic device (101) may output a result of performing processing to adjust the composition of the subject (e.g., person (430)) in the second preview image (402) to the reference composition. The processing may include at least one of fixing, cropping, and outpainting. Fixing may be to keep the image (e.g., the second preview image (402)) as is. For example, the composition and zoom ratio of the second preview image may be kept as is. Cropping may be to crop a desired area from an image (e.g., the second preview image) to perform digital zoom. For example, cropping may include a digital zoom process of selecting a desired area from the second preview image, cropping it, and enlarging it. Outpainting may be to create a new portion or add a new object outside of an image (e.g., the second preview image (402)). The electronic device (101) may perform outpainting using a generative model (e.g., the generative model (250) of FIG. 2). The electronic device (101) can analyze the composition of the subject included in the second preview image and selectively perform processing to appropriately adjust the reference composition corresponding to the subject.For example, the electronic device (101) can fix the composition and / or zoom ratio of the second preview image (402) as it is because the upper body of the person (430) is positioned at about 60% to 80% of the screen in the second preview image (402), the Manhattan Bridge behind the person (430) is not cut off, and the composition is positioned with the person (430) as the center in a vanishing point composition. The electronic device (101) can output a result of performing processing on the second preview image (402) (e.g., fixing the composition and / or zoom ratio). The result may include an image automatically captured by the electronic device (101) using the camera (408). For example, if the electronic device (101) determines that a subject (e.g., a person (430), a leg) is appropriately positioned in the reference composition within a preview image (e.g., a second preview image (402)), the electronic device (101) can automatically capture the image and output the result without a user input. The electronic device (101) can output the result in response to a user input. For example, the electronic device (101) can output the result in response to a user input of touching an arbitrary area within the screen. For example, the electronic device (101) can output the result in response to a user input for a button included in the electronic device (e.g., a volume control button). For example, the electronic device (101) can output the result in response to a user input for an external electronic device (e.g., a smart watch, a smart pen) connected to the electronic device (101) via a communication module (e.g., the communication module (190) of FIG. 1). For example, the electronic device (101) may output a result in response to a user's voice input. For example, the electronic device (101) may output a result in response to a user's gesture input (e.g., extending a palm). Note that the user input refers to an input including a photographing intention, and is not limited to the examples listed above. The electronic device (101) may display a screen (403) in response to the output of a result (e.g., photographing).The screen (403) may include an interface (450). The interface (450) may indicate that a result has been output (e.g., that an image has been captured). The electronic device (101) may display the result on the display in response to a user's input to the interface (450). For example, the electronic device (101) may display the result by executing a gallery application in response to a user's selection input to the interface (450). The interface (450) may include an indicator (e.g., a thumbnail) corresponding to the result.

[0094]

[0095] FIG. 5 is a drawing for explaining the operation of an electronic device according to one embodiment.

[0096] Referring to FIG. 5, according to one embodiment, FIG. 5 is a diagram for explaining an interface displayed on a display (e.g., a first display) disposed on the front of an electronic device (101). The electronic device (101) can display a preview image (e.g., a first preview image (501)) acquired using a camera (e.g., a camera 408 of FIG. 4, a camera module (180) of FIG. 1) on the first display (e.g., a main display, a foldable display). A user can capture an image using the camera (e.g., a camera 408 of FIG. 4) of the electronic device (101). For example, the user can capture an image while checking a subject in real time through the first display (e.g., a main display, a foldable display). The image can include a still photograph and / or an individual frame of a video.

[0097] According to one embodiment, the electronic device (101) may provide an auto-zoom function. The auto-zoom function may be a function in which the electronic device (101) automatically adjusts the zoom of the camera. For example, the auto-zoom function may be a function in which the electronic device (101) automatically controls the zoom ratio of the camera without user operation. The electronic device (101) may use an icon (510) to visually indicate that the auto-zoom function is activated. The user may select the icon (510) to change the activation status of the auto-zoom function. For example, the user may select the icon (510) to turn the auto-zoom function on or off. The icon (510) may visually change as the activation status of the auto-zoom function changes. For example, if the user selects the icon (510) to deactivate the auto-zoom function, the electronic device (101) may change the color and / or shape of the icon (510) and display it on the display. For example, the electronic device (101) may display the icon (510) in an enlarged form when the auto-zoom function is activated, and in a reduced form when the auto-zoom function is deactivated. It should be noted that the icon (510) illustrated in FIG. 5 is merely an example for indicating that the auto-zoom function of the electronic device (101) is activated, and that the operation of the electronic device (101) indicating that the auto-zoom function is activated is not limited thereto.

[0098] According to one embodiment, the electronic device (101) may display a first preview image (501) acquired using a camera (e.g., camera (408)) on a first display (e.g., main display, foldable display). The electronic device (101) may display an interface (e.g., interface (520), interface (540)) indicating a current zoom ratio of the camera on the display. For example, the interface (520) may include text (e.g., 2x) indicating that the current zoom ratio of the camera (408) is set to 2x. The zoom ratio may be a ratio indicating the degree to which a subject is enlarged or reduced. For example, when the zoom ratio of the camera (408) is set to 2x, the subject may be captured at a size approximately twice larger than its original size. For example, the first preview image (501) may be an image acquired using the camera (408) with the zoom ratio set to 2x. The higher the zoom ratio, the larger the subject becomes, and the lower the zoom ratio, the smaller the subject becomes, allowing for a wider overall field of view.

[0099] According to one embodiment, the electronic device (101) can recognize a subject (e.g., a doll (551) and a doll (552)) included in the first preview image (501). The electronic device (101) can classify a category of the subject included in the first preview image (501). For example, the electronic device (101) can recognize two dolls (e.g., a doll (551) and a doll (552)) included in the first preview image (501) and classify a category of the subject (e.g., things, doll). The electronic device (101) can detect a type of a scene corresponding to the first preview image (501) based on the category of the subject and adjust the zoom of the camera. The electronic device (101) can adjust the zoom of the camera to generate a second preview image (502). For example, the electronic device (101) can automatically adjust the zoom of the camera to 1x to generate the second preview image (502). The electronic device (101) can automatically adjust the zoom of the camera based on the composition of the subject in the first preview image (501). The electronic device (101) can adjust the zoom of the camera by determining whether the composition of the subject (e.g., doll (551), doll (552)) in the first preview image (501) is suitable for the reference composition corresponding to the subject. The electronic device (101) may not adjust the zoom of the camera if the composition of the subject (e.g., doll (551), doll (552)) in the first preview image (501) is already suitable for the reference composition. In this case, the second preview image (502) may be the same image as the first preview image (501). The electronic device (101) can display a second preview image (502) on the display. The electronic device (101) can visually indicate that the zoom ratio of the camera (408) has changed.For example, the electronic device (101) can visually indicate that the zoom ratio has changed by displaying an interface (540) indicating the current zoom ratio of the camera instead of the interface (520). The interface (540) can indicate the current zoom ratio of the camera (e.g., the changed zoom ratio). For example, the electronic device (101) can display an interface (540) indicating that the zoom ratio of the camera has changed to 1x on the display. For example, when the zoom ratio of the camera (408) is changed, the electronic device (101) can dynamically change the interface (520) to the interface (540) by applying an animation effect.

[0100] According to one embodiment, the electronic device (101) can determine whether the composition of the subject (e.g., doll (551), doll (552)) in the second preview image (502) is suitable for a reference composition corresponding to the subject (e.g., doll (551), doll (552)). For example, in the case of the doll (551) and the doll (552), the arrangement in which the subject fits within the screen without any cut-off portions may be the reference composition. The reference composition may include a structure in which the subject is aesthetically suitable. It should be noted that the reference composition corresponding to the subject may vary depending on the image, and the above description is merely an example. The electronic device (101) can output a result of performing processing to adjust the composition of the subject (e.g., doll (551) and doll (552)) in the second preview image (502) to the reference composition. The processing that the electronic device (101) can perform to adjust the composition of the subjects (e.g., dolls (551) and dolls (552)) in the second preview image (502) to a reference composition is the same as the processing described with reference to FIG. 4, and thus, a duplicate description thereof will be omitted. The electronic device (101) can analyze the composition of the subjects (e.g., dolls (551) and dolls (552)) included in the second preview image and selectively perform processing to appropriately adjust the composition to a reference composition corresponding to the subjects. For example, the electronic device (101) can fix the composition and / or zoom ratio of the second preview image (502) as it is because the dolls (551) and dolls (552) are both positioned without being cropped in the second preview image (502). The electronic device (101) can output a result of performing the processing on the second preview image (e.g., fixing the composition and / or zoom ratio). The result may include an image automatically captured by the electronic device (101) using a camera.For example, if the electronic device (101) determines that a subject (e.g., a doll (551), a doll (552)) is appropriately positioned in the reference composition within a preview image (e.g., a second preview image (502)), the electronic device (101) can automatically capture the subject and output the result without a user input. The electronic device (101) can output the result in response to a user input. For example, the electronic device (101) can output the result in response to a user input to a capture interface (560). The electronic device (101) can display the screen (503) in response to the result being output (e.g., captured). The screen (503) can include an interface (550). The interface (550) can indicate that the result has been output (e.g., the image has been captured). The electronic device (101) can display the result on the display in response to a user input to the interface (550). For example, the electronic device (101) may, in response to a user's selection input on the interface (550), execute a gallery application to display a result. The interface (550) may include an indicator (e.g., a thumbnail) corresponding to the result.

[0101]

[0102] Figure 6 is a drawing for explaining a shooting composition according to one embodiment.

[0103] Referring to FIG. 6, according to one embodiment, a photographic composition may refer to the arrangement of an object and / or background within a frame. The frame may include a frame of an image that an electronic device (e.g., the electronic device (101) of FIG. 1) acquires in real time from a camera (e.g., the camera module (180) of FIG. 1, the camera (408) of FIG. 4). For example, the frame may include a first preview image and a second preview image. The shooting composition may include compositions such as triangle composition (601), vertical composition (602), horizontal composition (603), vertical-horizontal composition (604), cross composition (605), fan composition (606), radial composition (607), diagonal composition (608), bi-diagonal composition (609), diamond composition (610), bilateral symmetrical composition (611), circular composition (612), spiral composition (613), diagonal composition (614), inverted triangle composition (615), full-light composition (616), arch composition (617), C-shaped composition (618), S-shaped composition (619), and U-shaped composition (620). The shooting composition can play an important role in emphasizing the subject in a picture or creating balance between subjects. The shooting composition can be a factor in determining the overall mood of the picture. For example, a photo taken with a triangular composition (601) can have a stable and heavy atmosphere. It should be noted that the shooting compositions (601 to 620) illustrated in FIG. 6 are merely examples for explaining the shooting compositions, and the shooting compositions that can be considered when the electronic device (101) performs shooting using a camera (e.g., the camera module (180) of FIG. 1, the camera (408) of FIG. 4) are not limited thereto.

[0104]

[0105] Figure 7 is a drawing for explaining a shooting environment according to one embodiment.

[0106] Referring to FIG. 7, according to one embodiment, a user (750) can take a photo and / or a video using an electronic device (101). The electronic device (101) may include a foldable device. The electronic device (101) may be operated while maintaining one of a first state that is fully unfolded (e.g., an unfolded state or an unfolded state), a second state that is fully folded (e.g., a folded state or a folded state), and a third state between the first state and the second state (e.g., an intermediate state). The user (750) can take a photo and / or a video by holding the electronic device (101) in a folded state (e.g., the third state). For example, a user (750) can take a selfie by placing the electronic device (101) on a desk in a folded state (e.g., the third state) and making the camera (408) (e.g., the camera module (180) of FIG. 1) look at the user. The selfie may be a photo and / or video taken of the user using a camera (e.g., the camera (408)). The user (750) can check his / her appearance in real time through a display (e.g., the display module (160) of FIG. 1, the second display (1531) of FIGS. 15A and 15B) arranged on the rear side of the electronic device (101). If the angle at which the electronic device (101) is folded changes while it is placed, the viewpoint of the camera (408) may change. For example, depending on the angle at which the electronic device (101) is folded, the position of the camera (408) may change, and thus the composition may change even for the same subject and background. For example, if the electronic device (101) is folded more than it is in a vertically folded state, the subject can be photographed in a composition looking up from below.

[0107]

[0108] FIG. 8A and FIG. 8B are drawings for explaining an operation of an electronic device according to one embodiment of the present invention to photograph a group subject.

[0109] FIG. 8a is a drawing for explaining an operation of a user taking a group selfie using an electronic device according to one embodiment, and FIG. 8b is a drawing for explaining an operation of an electronic device according to one embodiment outputting a result and providing it to a user.

[0110] Referring to FIGS. 8A and 8B , according to one embodiment, the electronic device (101) may include a camera (408) (e.g., the camera module (180) of FIG. 1 ). The camera (408) may include one or more lenses. The display (e.g., the display module (160) of FIG. 1 ) of the electronic device (101) may include a first display (e.g., a main display, a foldable display) and / or a second display (e.g., a sub display, a cover display). The first display may include a display disposed on the front side of the electronic device (101). The second display may include a display disposed on the rear side of the electronic device (101). A user may capture an image using the camera (408) of the electronic device (101). The image may include a still photograph and / or individual frames of a video. A user may capture a group selfie using the electronic device (101). A group selfie can be a selfie taken by multiple people together. For example, a user can take a group selfie while checking their appearance in real time on a secondary display (e.g., a cover display).

[0111] According to one embodiment, the electronic device (101) may provide an auto-zoom function. The auto-zoom function may be a function in which the electronic device (101) automatically adjusts the zoom of the camera (408). For example, the auto-zoom function may be a function in which the electronic device (101) automatically controls the zoom ratio of the camera (408) without user operation. The electronic device (101) may use an icon (810) to visually indicate that the auto-zoom function is activated. The icon (810) is substantially the same as the icon (410) described with reference to FIG. 4 and the icon (510) described with reference to FIG. 5, and thus, any overlapping description thereof will be omitted. The electronic device (101) may display a first preview image (801) acquired using the camera (408) on a display (e.g., a cover display). The electronic device (101) may display an interface (e.g., interface (820), interface (840)) indicating the current zoom ratio of the camera (408) on the display. The interface (820) may indicate the current zoom factor of the camera (408). For example, the interface (820) may include text (e.g., 2x) indicating that the current zoom factor of the camera (408) is set to 2x.

[0112] According to one embodiment, the electronic device (101) can recognize subjects (e.g., person (851), person (852), person (853)) included in the first preview image (801). The first preview image (801) can include a plurality of subjects. Based on the result of recognizing the subjects (e.g., person (851), person (852), and person (853)), the electronic device (101) can classify a category (e.g., people) of the subjects included in the first preview image (801). For example, in response to recognizing the subjects (e.g., person (851), person (852), and person (853)) included in the first preview image (801), the electronic device (101) can classify the category of the subjects, which are two or more people, as people. The electronic device (101) can detect the type of the scene based on the category of the subjects. For example, the electronic device (101) can detect the type of scene corresponding to the first preview image (801) including people as a group (e.g., group selfie). The electronic device (101) can adjust the zoom of the camera (408) based on the category of the subject to generate the second preview image (802). The electronic device (101) can automatically adjust the zoom of the camera based on the composition of the subject in the first preview image (801). The electronic device (101) can adjust the zoom of the camera (408) by determining whether the composition of the subject (e.g., person (851), person (852), and person (853)) in the first preview image (801) is suitable for the reference composition corresponding to the subject. For example, if a part of a person (853) is cut off in the first preview image (801) and the arrangement of the person (851), the person (852), and the person (853) is determined to be not suitable for the reference composition, the zoom of the camera (408) can be adjusted. The electronic device (101) can adjust the zoom of the camera (408) to generate a second preview image (802).For example, the electronic device (101) can automatically adjust the zoom of the camera to 1x to generate the second preview image (802). The electronic device (101) can display the second preview image (802) on the display. The person (853) in the second preview image (802) can be displayed completely without being cut off. The electronic device (101) can visually indicate that the zoom ratio of the camera (408) has changed. For example, the electronic device (101) can visually indicate that the zoom ratio has changed by displaying an interface (840) indicating the current zoom ratio of the camera (408) instead of the interface (820). For example, when the zoom ratio of the camera (408) is changed, the electronic device (101) can dynamically change the interface (820) to the interface (840) by applying an animation effect. The interface (840) can indicate the current zoom ratio of the camera (408). For example, the electronic device (101) may display an interface (840) on the display indicating that the zoom factor of the camera (408) has been changed to 1x. The interface (840) may indicate the current zoom factor of the camera (408) (e.g., the changed zoom factor). For example, the interface (840) may include text (e.g., 1x) indicating that the zoom factor of the camera (408) is currently set to 1x.

[0113] According to one embodiment, the electronic device (101) can determine whether the composition of the subjects (e.g., people (851), people (852), and people (853)) within the second preview image (802) is suitable for a reference composition corresponding to the subjects (e.g., people (851), people (852), and people (853)). For example, in the case of people (851), people (852), and people (853), the reference composition may be an arrangement in which all subjects are properly placed within the screen without any part being cut off. The reference composition may include a structure in which the subjects are aesthetically suitable. It should be noted that the reference composition corresponding to the subjects may vary depending on the image, and the above description is merely an example. The electronic device (101) can output a result of performing processing to adjust the composition of subjects (e.g., people (851), people (852), and people (853)) in the second preview image (802) to a reference composition. The electronic device (101) can analyze the composition of subjects (e.g., people (851), people (852), and people (853)) included in the second preview image (802) and selectively perform processing to appropriately adjust the composition to a reference composition corresponding to the subjects. For example, since the people (851) to the people (853) are all arranged without being cropped in the second preview image (802), the electronic device (101) can fix the composition and zoom ratio of the second preview image (802) as they are. The electronic device (101) can output a result of fixing the second preview image (802). The result may include an image automatically captured by the electronic device (101) using the camera (408). The electronic device (101) may output the result in response to a user input. For example, the electronic device (101) may output the result in response to a user input of touching an arbitrary area within the screen. The electronic device (101) may display the screen (803) in response to the output (e.g., capture) of the result.The screen (803) may include an interface (850). The interface (850) may indicate that a result has been output (e.g., that an image has been captured). The electronic device (101) may display the result on the display in response to a user input to the interface (850). For example, the electronic device (101) may display the result by executing a gallery application in response to a user selection input to the interface (850). The interface (850) may include an indicator (e.g., a thumbnail) corresponding to the result. The electronic device (101) may display the screen (804) on the display in response to a user input to the interface (850).

[0114] According to one embodiment, the screen (804) may include an interface for a gallery application (hereinafter, referred to as a gallery interface). The electronic device (101) may display the gallery interface on the screen (804). The electronic device (101) may display the gallery interface on the screen (804) through a display (e.g., a first display, a main display, a foldable display) to provide a result to a user. The result may include an image acquired by the electronic device (101) using the camera (408). The result may include an image automatically captured by the electronic device (101) using the camera (408). The result may include a plurality of images (e.g., images (881 to 884)). The plurality of images may include a representative image and image candidates to be recommended to the user. The representative image may be an image in which the electronic device (101) determines that the composition of the subject is most suitable for the reference composition. For example, the electronic device (101) may determine an image (881) in which the composition of the subject is adjusted to a reference composition, as a representative image. The electronic device (101) may display an interface (870) to indicate that the image is a representative image in a portion of the image (881). The electronic device (101) may display the representative image on the screen (804). For example, the electronic device (101) may display the image (881) determined as the representative image in an area (805) within the screen (804). The electronic device (101) may display the image (881) in a large size in the area (805) and display it in a small size together with candidate images (e.g., images (882 to 884)) at the bottom. In addition to the representative image, the candidate images may include images (e.g., images (882 to 884)) that the electronic device (101) recommends to the user.The composition of the subjects (e.g., person (851), person (852), and person (853)) in the candidate images may be different from the composition of the subjects (e.g., person (851), person (852), and person (853)) in the representative image. For example, the image (882) may be an image that is zoomed in more than the image (881) and has a different composition of the persons (851) and (853). When the electronic device (101) captures multiple subjects (e.g., captures a group selfie), the electronic device (101) may provide an image for each subject (e.g., a selfie for an individual) as a candidate image. For example, the electronic device (101) may provide an image (883) for the person (852) and an image (884) for the person (853) as candidate images.

[0115] According to one embodiment, the electronic device (101) can receive a user's selection input for a plurality of images (e.g., images (881 to 884)). The electronic device (101) can determine an image selected by the user as a representative image. For example, in response to the user's selection input for image (882), the electronic device (101) can change the representative image from image (881) to image (882). The electronic device (101) can modify a reference composition corresponding to a subject based on the user's selection input. For example, the electronic device (101) can generate a user's preference score for the composition based on the user's selection input for image (882) captured with a zoom greater than that of image (881). The electronic device (101) can adjust the zoom of the camera (408) differently based on the preference score reflecting the user's selection input. For example, when taking a group selfie later, the electronic device (101) can adjust the zoom to position the subject in a composition with a high user preference score. The electronic device (101) can generate personalized results for the user by differently adjusting the zoom of the camera (408) based on the preference score reflected in the user's selection input.

[0116]

[0117] FIGS. 9A and 9B are diagrams for explaining cropping processing performed by an electronic device according to one embodiment.

[0118] FIG. 9a is a drawing for explaining an operation of an electronic device according to one embodiment performing cropping processing to capture a subject, and FIG. 9b is a drawing for explaining an operation of an electronic device according to one embodiment outputting a result and providing it to a user.

[0119] Referring to FIGS. 9A and 9B , according to one embodiment, the electronic device (101) may include a camera (408) (e.g., the camera module (180) of FIG. 1 ). The camera (408) may include one or more lenses. The display (e.g., the display module (160) of FIG. 1 ) of the electronic device (101) may include a first display (e.g., a main display, a foldable display) and / or a second display (e.g., a sub display, a cover display). The first display may include a display disposed on the front side of the electronic device (101). The second display may include a display disposed on the rear side of the electronic device (101). A user may capture an image using the camera (408) of the electronic device (101). The image may include a still photograph and / or individual frames of a video. A user may capture a subject using the electronic device (101). For example, a user can take a picture of a cat (960) climbing a tree using an electronic device (101). The electronic device (101) can provide a zoom function. The auto-zoom function can be a function in which the electronic device (101) automatically adjusts the zoom of the camera (408). For example, the auto-zoom function can be a function in which the electronic device (101) automatically controls the zoom ratio of the camera (408) without any operation by the user. The electronic device (101) can use an icon (910) to visually indicate that the auto-zoom function is activated. The icon (910) is substantially the same as the icon (410) described with reference to FIG. 4 and the icon (510) described with reference to FIG. 5, and any overlapping descriptions thereof will be omitted. The electronic device (101) can display a first preview image (901) acquired using the camera (408) on a display (e.g., a cover display).The electronic device (101) may display an interface (e.g., interface (920), interface (940)) indicating the current zoom ratio of the camera (408) on the display. The interface (920) may indicate the current zoom ratio of the camera (408). For example, the interface (820) may include text (e.g., 2x) indicating that the current zoom ratio of the camera (408) is set to 2x.

[0120] According to one embodiment, the electronic device (101) can recognize a subject (e.g., a cat (960), a tree) included in the first preview image (901). The electronic device (101) can generate a second preview image (902) by adjusting the zoom of the camera (408) based on the category of the subject. For example, the electronic device (101) can control the zoom of the camera (408) based on the subject, the cat (960), to generate a second preview image (902) in which the cat (960) is enlarged. The electronic device (101) can visually notify that the zoom ratio of the camera (408) has changed. For example, the electronic device (101) can visually notify that the zoom ratio has changed by displaying an interface (940) indicating the current zoom ratio of the camera (408) instead of the interface (920). For example, when the zoom ratio of the camera (408) is changed, the electronic device (101) can dynamically change the interface (920) to the interface (940) by applying an animation effect. The interface (940) can indicate the current zoom ratio of the camera (408) (e.g., the changed zoom ratio). For example, the interface (940) can include text (e.g., 5x) indicating that the zoom ratio of the camera (408) is currently set to 5x. The electronic device (101) can determine whether the composition of the subject (e.g., the cat (960), the tree) in the second preview image (902) is suitable for a reference composition corresponding to the subject (e.g., the cat (960), the tree). The electronic device (101) can perform processing to adjust the composition of the subject (e.g., the cat (960), the tree) in the second preview image (902) to the reference composition. The processing may include at least one of fixing, cropping, and outpainting. Cropping may be performing digital zoom by cropping a specific area from an image (e.g., the second preview image (902)).For example, cropping may include digital zoom processing that selects and crops a desired area in the second preview image and enlarges it. The electronic device (101) may perform cropping processing on the second preview image (902) to adjust the composition of a subject (e.g., a cat (960), a tree) included in the second preview image (902) to fit the reference composition. For example, if the electronic device (101) determines that the subject, a cat (960), cannot be captured in a desired composition by performing optical zoom alone, the electronic device (101) may perform cropping processing on the second preview image (902). The electronic device (101) may display a cropping guide (930) on the second preview image (902). For example, the electronic device (101) may display a cropping guide (930) on the second preview image (902) to guide the user to an area to be cropped within the second preview image (902). The electronic device (101) may display the cropping guide (930) on the display for a set period of time (e.g., n seconds), and then output a result of performing processing (e.g., performing cropping processing) on ​​the second preview image (902). The image (903) may be an image obtained by performing cropping processing on the second preview image (902) by the electronic device (101). The electronic device (101) may display the image (903) on the display (e.g., cover display), and then automatically capture the image and output the result. For example, if the electronic device (101) determines that a subject (e.g., a cat (960)) is appropriately positioned in the reference composition within the image (903), the electronic device (101) can automatically capture the image and output the result without user input. The electronic device (101) can output the result in response to the user input. For example, the electronic device (101) can output the result image (903) in response to a user input of touching an arbitrary area within the screen.When the electronic device (101) outputs an image on which cropping processing has been performed as a result (e.g., when capturing an image), the deleted area can be stored together with the result. The deleted area may be an area not included in the final result image (903). The interface (950) may indicate that the result has been output (e.g., that the image has been captured). The electronic device (101) may display the result on the display in response to a user's input to the interface (950). For example, the electronic device (101) may display the result by executing a gallery application in response to a user's selection input to the interface (950). The interface (450) may include an indicator (e.g., a thumbnail) corresponding to the result.

[0121] According to one embodiment, the electronic device (101) may display a screen (904) on the display in response to a user input to the interface (950). The screen (904) may include an interface for a gallery application (hereinafter, referred to as a gallery interface). The electronic device (101) may display the gallery interface on the screen (904). The electronic device (101) may display the gallery interface on the screen (904) through a display (e.g., a first display, a main display, a foldable display) to provide a result to the user. The result may include an image acquired by the electronic device (101) using the camera (408). The result may include an image automatically captured by the electronic device (101) using the camera (408). The result may include a plurality of images (e.g., images (981 to 983)). The plurality of images may include a representative image and candidate images to be recommended to the user. The representative image may be an image in which the electronic device (101) determines that the composition of the subject (e.g., a cat (960)) is most suitable for the reference composition. For example, the electronic device (101) may determine an image (981) in which the composition of the subject, a cat (960), is adjusted to the reference composition by performing cropping processing on the second preview image (902), as the representative image. The image (981) may be generated by storing the image (903) displayed on the second display (e.g., a cover display) of the electronic device (101) in the electronic device (101). The electronic device (101) may display an interface (970) to indicate that the image is the representative image in a portion of the image (981). The electronic device (101) may display the representative image on the screen (904). For example, the electronic device (101) can display an image (981) determined as a representative image in an area (905) within the screen (904).The electronic device (101) can display the image (981) in a large size in the area (905) and display it in a small size together with candidate images (e.g., images (982, 983)) at the bottom. The candidate images can include images (e.g., images (982, 983)) that the electronic device (101) recommends to the user in addition to the representative image. The composition of the subject (e.g., cat (960)) in the candidate image can be different from the composition of the subject (e.g., cat (960)) in the representative image. For example, the image (982) can be an image that is zoomed in more than the image (981) and has a different composition of the cat (960).

[0122] According to one embodiment, the electronic device (101) may receive a user's selection input for a plurality of images (e.g., images (981 to 983)). The electronic device (101) may determine an image selected by the user as a representative image. For example, in response to the user's selection input for image (982), the electronic device (101) may change the representative image from image (981) to image (982). The electronic device (101) may modify a reference composition corresponding to a subject based on the user's selection input. For example, the electronic device (101) may generate a user's preference score for the composition based on the user's selection input for image (982) captured with a zoom level greater than that of image (981). The electronic device (101) may adjust the zoom of the camera differently based on the preference score reflecting the user's selection input.

[0123] According to one embodiment, the electronic device (101) may provide a function for editing an image. The electronic device (101) may display an interface (990) on a display. In response to a user input to the interface (990), the electronic device (101) may display an interface for editing an image on the display (e.g., a first display). The screen (906) may include an interface for editing an image. The electronic device (101) may also directly display the screen (906) on the display in response to a user input to the interface (950). For example, in response to a user input to the interface (950), the electronic device (101) may display the screen (906) on the display so that the user can directly edit the result of the cropping process. The electronic device (101) may display an interface for editing an image on which the cropping process has been performed on the screen (906). The screen (906) may include a crop frame (908). The crop frame (908) is a frame whose size and / or shape can be adjusted and displayed on the image, and may be a guide for indicating an area to be cropped and saved. The electronic device (101) may display a deleted area together with the outer area of ​​the crop frame (908). The deleted area may include an area deleted while the electronic device (101) performs cropping processing on the second preview image (902). The deleted area may include an image that the electronic device (101) simultaneously saves while performing a photographing process. The electronic device (101) may display the inside of the crop frame (908) and the deleted area separately. For example, the electronic device (101) may display the inside of the crop frame (908) brightly and display the deleted area relatively dark and translucently. The user may adjust the area by dragging the corners and sides of the crop frame (908).The user can change the size and / or shape of the crop frame (908) by dragging the corners and sides of the crop frame (908) to specify the area to be saved. The electronic device (101) can save the image inside the crop frame (908) in response to the user's input to the interface (992). The electronic device (101) can restore the image in response to the user's input to the interface (991). For example, the electronic device (101) can restore the original image including the deleted area in response to the user's input to the interface (991).

[0124]

[0125] FIGS. 10A and 10B are drawings for explaining outpainting processing performed by an electronic device according to one embodiment.

[0126] FIG. 10A is a drawing for explaining an operation of an electronic device according to one embodiment performing outpainting processing to capture a subject, and FIG. 10B is a drawing for explaining an operation of an electronic device according to one embodiment outputting a result and providing it to a user.

[0127] Referring to FIGS. 10A and 10B , according to one embodiment, the electronic device (101) may include a camera (408) (e.g., the camera module (180) of FIG. 1 ). The camera (408) may include one or more lenses. The display (e.g., the display module (160) of FIG. 1 ) of the electronic device (101) may include a first display (e.g., a main display, a foldable display) and / or a second display (e.g., a sub display, a cover display). The first display may include a display disposed on the front of the electronic device (101). The second display may include a display disposed on the back of the electronic device (101). A user (e.g., a person (1030)) may capture an image using the camera (408) of the electronic device (101). The image may include a still photograph and / or individual frames of a video. The user may capture a subject using the electronic device (101). For example, a user can take a picture of himself (e.g., a person (1030)) in front of the Statue of Liberty (1035) using the electronic device (101). The electronic device (101) can provide a zoom function. The auto-zoom function can be a function in which the electronic device (101) automatically adjusts the zoom of the camera (408). For example, the auto-zoom function can be a function in which the electronic device (101) automatically controls the zoom ratio of the camera (408) without any operation by the user. The electronic device (101) can use an icon (1010) to visually indicate that the auto-zoom function is activated. The icon (1010) is substantially the same as the icon (410) described with reference to FIG. 4 and the icon (510) described with reference to FIG. 5, and any overlapping descriptions thereof will be omitted. The electronic device (101) can display a first preview image (1001) acquired using the camera (408) on a display (e.g., a cover display).The electronic device (101) may display an interface (e.g., interface (1020), interface (1040)) indicating the current zoom ratio of the camera (408) on the display. The interface (1020) may indicate the current zoom ratio of the camera (408). For example, the interface (1020) may include text (e.g., 3x) indicating that the current zoom ratio of the camera (408) is set to 3x.

[0128] According to one embodiment, the electronic device (101) can recognize a subject (e.g., a person (1030), a Statue of Liberty (1035)) included in a first preview image (1001). The electronic device (101) can classify a category of the subject (e.g., a person (1030), a Statue of Liberty (1035)) included in the first preview image (1001). For example, the electronic device (101) can classify the category of the person (1030) included in the first preview image (1001) as a person and the category of the Statue of Liberty (1035) as a landmark. The landmark can be stored in a database in the electronic device (101). For example, the landmark can be stored in a memory (e.g., the memory (130) of FIG. 1) of the electronic device (101) as an image and data converted from the image to text can be stored in a database in the memory. The electronic device (101) can detect the type of a scene based on the category of the subject. For example, the electronic device (101) can detect the type of the scene corresponding to the first preview image (1001) including the Statue of Liberty (1035) and a person (1030) as a composite scene (e.g., person + landmark). The electronic device (101) can generate a second preview image (1002) by adjusting the zoom of the camera (408) based on the category of the subject. For example, the electronic device (101) can generate a second preview image (1002) with a wider overall field of view of the subject, the person (1030) and the Statue of Liberty (1035), by controlling the zoom of the camera (408). The electronic device (101) can visually notify that the zoom magnification of the camera (408) has been changed. For example, the electronic device (101) can visually indicate that the zoom ratio has changed by displaying an interface (1040) indicating the current zoom ratio of the camera (408) instead of the interface (1020). For example, when the zoom ratio of the camera (408) is changed, the electronic device (101) can dynamically change the interface (1020) to the interface (1040) by applying an animation effect.The interface (1040) may indicate the current zoom ratio (e.g., a changed zoom ratio) of the camera (408). For example, the interface (1040) may include text (e.g., 1x) indicating that the current zoom ratio of the camera (408) is set to 1x. The electronic device (101) may determine whether the composition of the subject (e.g., the person (1030), the Statue of Liberty (1035)) in the second preview image (1002) is suitable for a reference composition corresponding to the subject (e.g., the person (1030), the Statue of Liberty (1035)). The electronic device (101) may perform processing to adjust the composition of the subject (e.g., the person (1030), the Statue of Liberty (1035)) in the second preview image (1002) to the reference composition. The processing may include at least one of fixing, cropping, and outpainting. Outpainting may involve creating new portions or adding new objects outside of an image (e.g., the second preview image (1002)). The electronic device (101) may perform outpainting using a generative model (e.g., the generative model (250) of FIG. 2). The second preview image (1002) may not be suitable for the reference composition because a part of the subject (e.g., a person (1030), the Statue of Liberty (1035)) is cut off. The electronic device (101) may perform outpainting to adjust the composition of the subject (e.g., a person (1030), the Statue of Liberty (1035)) included in the second preview image (1002) to be suitable for the reference composition. The electronic device (101) may display an interface (1050) on the display to notify that outpainting will be performed. For example, if the electronic device (101) determines that outpainting is necessary for the second preview image (1002), the electronic device (101) may display the interface (1050) on the display to notify the user that outpainting will be performed. After displaying the interface (1050) on the display, the electronic device (101) may use the generative model (250) to, Outpainting can be performed.The electronic device (101) may display an interface (1060) on the display while performing outpainting. For example, the electronic device (101) may display an interface (1060) on the display indicating that outpainting is being performed on the second preview image (1002). The electronic device (101) may perform outpainting to generate a cropped portion of a subject (e.g., a person (1030), the Statue of Liberty (1035)). The electronic device (101) may output a result of performing processing (e.g., performing outpainting) on ​​the second preview image (1002). The result may include an image automatically captured by the electronic device (101) using the camera (408). The electronic device (101) may output a result (e.g., image (1004)) in response to a user's input. For example, the electronic device (101) may output a result image (1004) in response to a user input of touching an arbitrary area within the screen. The interface (1070) may indicate that the result has been output (e.g., that an image has been captured). The electronic device (101) may display the result on the display in response to a user input to the interface (1070). For example, the electronic device (101) may display the result by executing a gallery application in response to a user selection input to the interface (1070). The interface (1070) may include an indicator (e.g., a thumbnail) corresponding to the result.

[0129] According to one embodiment, the electronic device (101) may display a screen (1004) on a display in response to a user input to the interface (1070). The screen (1004) may include an interface for a gallery application (hereinafter, referred to as a gallery interface). The electronic device (101) may display the gallery interface on the screen (1004). The electronic device (101) may display the gallery interface on the screen (1004) through a display (e.g., a first display, a main display, a foldable display) to provide a result to the user. The result may include an image acquired by the electronic device (101) using the camera (408). The result may include an image automatically captured by the electronic device (101) using the camera (408). The result may include a plurality of images (e.g., images (1081 to 1084)). The plurality of images may include a representative image and candidate images to be recommended to the user. The representative image may be an image in which the electronic device (101) determines that the composition of the subject (e.g., person (1030), Statue of Liberty (1035)) is most suitable for the reference composition. For example, the electronic device (101) may determine an image (1081) that is an image adjusted to the reference composition by performing outpainting on the second preview image (1002) so that the person (1030) and Statue of Liberty (1035) are not cut off, as the representative image. The image (1081) may be generated by storing the image (1003) displayed on the second display (e.g., cover display) of the electronic device (101) in the electronic device (101). The electronic device (101) may display an interface (1080) to indicate that it is the representative image in a portion of the image (1081). The electronic device (101) may display the representative image on the screen (1004). For example, the electronic device (101) can display an image (1081) determined as a representative image in an area (1007) within the screen (1004).The electronic device (101) can display the image (1081) in a large area (1007) and display candidate images (e.g., images (1082 to 1084)) in a small size at the bottom. The candidate images can include images (e.g., images (1082 to 1084)) that the electronic device (101) recommends to the user in addition to the representative image. The composition of the subject (e.g., person (1030), Statue of Liberty (1035)) in the candidate image can be different from the composition of the subject (e.g., person (1030), Statue of Liberty (1035)) in the representative image. For example, the image (1082) can be an image that is zoomed in more than the image (1081) and has a different composition of the person (1030).

[0130] According to one embodiment, the electronic device (101) can receive a user's selection input for a plurality of images (e.g., images (1081 to 1084)). The electronic device (101) can determine an image selected by the user as a representative image. For example, in response to the user's selection input for image (1082), the electronic device (101) can change the representative image from image (1081) to image (1082). The electronic device (101) can modify a reference composition corresponding to a subject based on the user's selection input. For example, the electronic device (101) can generate a user's preference score for the composition based on the user's selection input for image (1082) captured with a zoom level greater than that of image (1081). The electronic device (101) can adjust the zoom of the camera differently based on the preference score reflecting the user's selection input.

[0131] According to one embodiment, the electronic device (101) may provide a function for editing an image. The electronic device (101) may display an interface (1090) on the display. In response to a user input to the interface (1090), the electronic device (101) may display an interface for editing an image on the display (e.g., a first display). The screen (1006) may include an interface for editing an image. The electronic device (101) may also directly display the screen (1006) on the display in response to a user input to the interface (1070). For example, in response to a user input to the interface (1050), the electronic device (101) may display the screen (1006) on the display so that the user can directly edit the result of outpainting. The electronic device (101) may display an interface for editing an image on which outpainting has been performed on the screen (1006). The screen (1006) may include an interface (1091) for restoring an image (1081) on which outpainting has been performed to an original image. The electronic device (101) may display the original image on the display in response to a user's input to the interface (1091). For example, the electronic device (101) may display an image (e.g., a second preview image (1002), a first preview image (1001)) before outpainting is performed on the display in response to a user's input to the interface (1091). The electronic device (101) may terminate image editing in response to a user's input to the interface (1092).

[0132]

[0133] FIG. 11A and FIG. 11B are diagrams for explaining an operation of an electronic device according to one embodiment to photograph a plurality of subjects of different categories.

[0134] FIG. 11a is a drawing for explaining an operation of an electronic device according to one embodiment performing automatic zoom adjustment to photograph a subject, and FIG. 11b is a drawing for explaining an operation of an electronic device according to one embodiment outputting a result and providing it to a user.

[0135] Referring to FIGS. 11A and 11B , according to one embodiment, the electronic device (101) may include a camera (408) (e.g., the camera module (180) of FIG. 1 ). The camera (408) may include one or more lenses. The display (e.g., the display module (160) of FIG. 1 ) of the electronic device (101) may include a first display (e.g., a main display, a foldable display) and / or a second display (e.g., a sub display, a cover display). The first display may include a display disposed on the front of the electronic device (101). The second display may include a display disposed on the back of the electronic device (101). A user may capture an image using the camera (408) of the electronic device (101). The image may include a still photograph and / or individual frames of a video. A user (e.g., a person (1130)) may capture a subject using the electronic device (101). For example, a user can take a picture of himself (e.g., a person (1130)) in front of the Statue of Liberty using an electronic device (101). The electronic device (101) can provide a zoom function. The auto-zoom function can be a function in which the electronic device (101) automatically adjusts the zoom of the camera (408). For example, the auto-zoom function can be a function in which the electronic device (101) automatically controls the zoom ratio of the camera (408) without any operation by the user. The electronic device (101) can use an icon (1110) to visually indicate that the auto-zoom function is activated. The icon (1110) is substantially the same as the icon (410) described with reference to FIG. 4 and the icon (510) described with reference to FIG. 5, and any overlapping descriptions thereof will be omitted. The electronic device (101) can display a first preview image (1101) acquired using the camera (408) on a display (e.g., a cover display).The electronic device (101) may display an interface (e.g., interface (1120), interface (1140)) indicating the current zoom ratio of the camera (408) on the display. The interface (1120) may indicate the current zoom ratio of the camera (408). For example, the interface (1120) may include text (e.g., 2x) indicating that the current zoom ratio of the camera (408) is set to 2x.

[0136] According to one embodiment, the electronic device (101) can recognize a subject (e.g., a person (1130) or the Statue of Liberty) included in a first preview image (1101). The electronic device (101) can detect a type of a scene based on a category of the subject. For example, the electronic device (101) can detect a type of a scene corresponding to the first preview image (1101) including a person (1130) and the Statue of Liberty as a composite scene. The electronic device (101) can generate a second preview image (1102) by adjusting the zoom of the camera (408) based on the category of the subject. For example, the electronic device (101) can generate a second preview image (1102) with a wider overall field of view of the subject, that is, the person (1130) and the Statue of Liberty, by controlling the zoom of the camera (408). The electronic device (101) can visually indicate that the zoom ratio of the camera (408) has changed. For example, the electronic device (101) can visually indicate that the zoom ratio has changed by displaying an interface (1140) indicating the current zoom ratio of the camera (408) instead of the interface (1120). The interface (1140) can indicate the current zoom ratio of the camera (408) (e.g., a changed zoom ratio). For example, the interface (1140) can include text (e.g., 1x) indicating that the zoom ratio of the camera (408) is currently set to 1x. The electronic device (101) can determine whether the composition of the subject (e.g., the person (1130), the Statue of Liberty) in the second preview image (1102) is suitable for a reference composition corresponding to the subject (e.g., the person (1130), the Statue of Liberty). For example, the electronic device (101) can determine that the second preview image (402) is suitable for the reference composition because the person (1130) and the Statue of Liberty are arranged in a triangular composition without being cut off.The electronic device (101) may perform processing to adjust the composition of the subject (e.g., person (1130), Statue of Liberty) in the second preview image (1102) to a reference composition. For example, since the person (1130) and Statue of Liberty are arranged in a triangular composition and are suitable for the reference composition in the second preview image (1102), the electronic device (101) may fix the composition and / or zoom ratio of the second preview image (1102) as it is. The electronic device (101) may output a result of performing processing (e.g., fixing the composition and / or zoom ratio) on the second preview image (1102). The result may include an image automatically captured by the electronic device (101) using the camera (408). The electronic device (101) may output a result (e.g., image (1103)) in response to a user's input. For example, the electronic device (101) may output a result image (1103) in response to a user input of touching an arbitrary area within the screen. The interface (1150) may indicate that the result has been output (e.g., that an image has been captured). The electronic device (101) may display the result on the display in response to a user input to the interface (1150). For example, the electronic device (101) may display the result by executing a gallery application in response to a user selection input to the interface (1150). The interface (1150) may include an indicator (e.g., a thumbnail) corresponding to the result.

[0137] According to one embodiment, the electronic device (101) may display a screen (1104) on a display in response to a user input to the interface (1150). The screen (1104) may include an interface for a gallery application (hereinafter, referred to as a gallery interface). The electronic device (101) may display the gallery interface on the screen (1104). The electronic device (101) may display the gallery interface on the screen (1104) through a display (e.g., a first display, a main display, a foldable display) to provide a result to the user. The result may include an image acquired by the electronic device (101) using the camera (408). The result may include an image automatically captured by the electronic device (101) using the camera (408). The result may include a plurality of images (e.g., images (1181 to 1184)). The plurality of images may include a representative image and candidate images to be recommended to the user. The representative image may be an image that the electronic device (101) determines to be the most suitable for the reference composition in terms of the composition of the subject (e.g., a person (1130), the Statue of Liberty). For example, the electronic device (101) may determine an image (1181) in which the person (1130) and the Statue of Liberty are not cropped and form a triangular composition, thereby being suitable for the reference composition, as the representative image. The image (1181) may be generated by storing an image (1103) displayed on a second display (e.g., a cover display) of the electronic device (101) in the electronic device (101). The electronic device (101) may display an interface (1170) to indicate that the image is the representative image in a portion of the image (1181). The electronic device (101) may display the representative image on the screen (1104). For example, the electronic device (101) can display an image (1181) determined as a representative image in an area (1105) within the screen (1104).The electronic device (101) can display the image (1181) in a large size in the area (1105) and display candidate images (e.g., images (1182 to 1184)) in a small size at the bottom. The candidate images can include images (e.g., images (1182 to 1184)) that the electronic device (101) recommends to the user in addition to the representative image. The composition of the subject (e.g., person (1130), Statue of Liberty) in the candidate image can be different from the composition of the subject (e.g., person (1130), Statue of Liberty) in the representative image. For example, the image (1182) can be an image with a different composition due to zooming in on the Statue of Liberty more than the image (1181). When there are two or more subjects, the electronic device (101) can generate an animated image file (e.g., a file in gif format) by synthesizing images focused on each subject. For example, the electronic device (101) can generate an animation image file by synthesizing an image focused on a person (1130) and an image focused on the Statue of Liberty, and provide the file to the user as a candidate image. The image (1184) may be an animation image file. When there are two or more subjects, the electronic device (101) can provide the user with enlarged images centered on each subject as candidate images. For example, the electronic device (101) can provide the user with an image (1182) centered on the Statue of Liberty and an image (1183) centered on the person (1130) as candidate images.

[0138] According to one embodiment, the electronic device (101) may receive a user's selection input for a plurality of images (e.g., images (1181 to 1184)). The electronic device (101) may determine an image selected by the user as a representative image. For example, in response to the user's selection input for image (1182), the electronic device (101) may change the representative image from image (1181) to image (1182). The electronic device (101) may modify a reference composition corresponding to a subject based on the user's selection input. For example, the electronic device (101) may generate a user's preference score for the composition based on the user's selection input for image (1182), which is captured with a zoomed-in view centered on the Statue of Liberty, rather than image (1181). The electronic device (101) may adjust the zoom of the camera differently based on the preference score reflecting the user's selection input.

[0139] According to one embodiment, the electronic device (101) may provide a screen for image editing (e.g., a screen for image editing such as screen (906) of FIG. 9B and screen (1006) of FIG. 10B) in response to a user input to the interface (1190). The electronic device (101) may separate a subject recognized in the first preview image and / or the second preview image from the background and store the result. For example, the electronic device (101) may separate a subject, a person (1130), recognized in the first preview image (1101) and / or the second preview image (1102) from the background and / or another subject, the Statue of Liberty, and store the result. The user may check the result and perform editing. For example, the user may edit an image using an image of the subject that is separately stored on the screen for image editing.

[0140]

[0141] FIG. 12 is a drawing for explaining an operation of an electronic device according to one embodiment of the present invention to photograph a subject.

[0142] Referring to FIG. 12, according to one embodiment, FIG. 12 may be for explaining an operation of adjusting zoom when the electronic device (101) does not detect a type of scene in a first preview image (1201) acquired using a camera (408) (e.g., the camera module (180) of FIG. 1). The electronic device (101) may provide an auto-zoom function. The electronic device (101) may use an icon (1210) to visually indicate that the auto-zoom function is activated. The icon (1210) is substantially the same as the icon (410) described with reference to FIG. 4 and the icon (510) described with reference to FIG. 5, and any overlapping descriptions thereof will be omitted. The electronic device (101) may display the first preview image (1201) acquired using the camera (408) on a display (e.g., a cover display). The electronic device (101) may display an interface (e.g., interface (1220), interface (1240)) indicating the current zoom ratio of the camera (408) on the display. The interface (1220) may indicate the current zoom ratio of the camera (408). For example, the interface (1220) may include text (e.g., 3x) indicating that the current zoom ratio of the camera (408) is set to 3x.

[0143] According to one embodiment, the electronic device (101) may not be able to classify a specific category of a subject (e.g., objects (1231 to 1233)) included in the first preview image (1201). For example, the electronic device (101) may not be able to classify a specific category of objects (1231 to 1233) that are cupcakes, and may only recognize the presence and location of the subject. The electronic device (101) may recognize the subject (e.g., objects (1231 to 1233)) as a single entity and determine whether the subject is located within a boundary line (1270) of the first preview image (1201). The boundary line (e.g., boundary line (1270)) is a virtual line set with a predetermined margin from the edge of the image, and may include a virtual line for indicating a work area within the image. The electronic device (101) can adjust the zoom of the camera (408) so that the subject (e.g., objects (1231 to 1233)) can be positioned within the boundary line (1270) within the first preview image (1201). For example, the electronic device (101) can adjust the zoom of the camera (408) in response to determining that the objects (1231 to 1233) within the first preview image (1201) are not positioned within the boundary line. The electronic device (101) can adjust the zoom of the camera (408) so that the subject can be positioned within the boundary line (1270). For example, the electronic device (101) can lower the zoom magnification of the camera (408) to widen the overall field of view and adjust the object (1231) to the object (1233) so that the object (1231) to the object (1233) is positioned within the boundary line (1270). The centerline (e.g., centerline (1280)) is a crossline where the horizontal midpoint and the vertical midpoint of the first preview image intersect, and may include an imaginary line to indicate the central point of the image.For example, the electronic device (101) can adjust the zoom of the camera by considering whether the subject is located above or below the center line (1280) of the first preview image. The electronic device (101) can adjust the zoom of the camera (408) to generate the second preview image (1202). The electronic device (101) can visually indicate that the zoom ratio of the camera (408) has changed. For example, the electronic device (101) can visually indicate that the zoom ratio has changed by displaying an interface (1240) indicating the current zoom ratio of the camera (408) instead of the interface (1220). The interface (1240) can indicate the current zoom ratio (e.g., the changed zoom ratio) of the camera (408). For example, the interface (1240) may include text (e.g., 1x) indicating that the current zoom ratio of the camera (408) is set to 1. If the electronic device (101) determines that a subject (e.g., objects (1231-1233)) included in the second preview image (1202) is located within the boundary (1270) of the second preview image (1202), the electronic device (101) may output a result. The result may include an image automatically captured by the electronic device (101) using the camera (408). The electronic device (101) may output the result (e.g., image (1203)) in response to a user input. For example, the electronic device (101) may output the result image (1203) in response to a user input of touching an arbitrary area within the screen. The interface (1250) may indicate that the result has been output (e.g., that the image has been captured). The electronic device (101) may display a result on the display in response to a user's input to the interface (1250). For example, the electronic device (101) may display a result by executing a gallery application in response to a user's selection input to the interface (1250).The interface (1250) may include an indicator (e.g., a thumbnail) corresponding to the result. The interface (1250) is substantially the same as the interface (1250) described with reference to FIG. 12, and any duplicate description thereof will be omitted.

[0144]

[0145] FIGS. 13A to 13E are drawings for explaining an automatic zoom control operation performed by an electronic device according to one embodiment.

[0146] Referring to FIGS. 13A to 13E, according to one embodiment, FIGS. 13A to 13E may be for explaining an operation of adjusting zoom to adjust the composition in a state where an electronic device (e.g., the electronic device (101) of FIG. 1) does not recognize a person as a subject when taking a picture of the subject. The electronic device (101) can adjust the zoom. The electronic device (101) can automatically adjust the zoom of a camera. For example, the electronic device (101) can automatically adjust the zoom of a camera (e.g., the camera (408) of FIG. 4, the camera module (180) of FIG. 1) without a user's operation. The electronic device (101) can take a picture of a subject using the camera. The electronic device (101) can divide the subject into two parts and adjust the composition. For example, the electronic device (101) can adjust the composition by simplifying the subject into a face (1301) and a torso (1303). The electronic device (101) can adjust the composition of the subject based on the boundary line and the center line of the preview image. The electronic device (101) can adjust the composition by adjusting the zoom of the camera (408) so that the position of the subject changes based on the boundary line and the center line. The reference composition (1310) can be an ideal composition corresponding to the upper body of a human subject. For example, the reference composition (1310) can be a composition in which the face (1301) is positioned at the upper end of the center line (1350) and exactly inside the boundary line (1330), and the torso (1303) is positioned inside the boundary line (1330). It should be noted that the reference composition (1310) is merely an example. The composition of the subject included in the preview image (1311) may be such that the face (1301) is located at the bottom of the center line (1350) and is away from the inside of the boundary line (1330), but the torso (1303) is located inside the boundary line (1330).The electronic device (101) can adjust the composition of the subject included in the preview image (1311) by adjusting the zoom of the camera (408) so that the composition is suitable for the reference composition (1310). For example, the electronic device (101) can adjust the composition of the subject included in the preview image (1311) so that the composition is suitable for the reference composition (1310) by performing a zoom-out using the camera (408). The composition of the subject included in the preview image (1312) may be such that the face (1301) is located at the top of the center line (1350), but is separated from the inside of the boundary line (1330), and the torso (1303) is located outside the boundary line (1330). The electronic device (101) can adjust the composition so that the composition of the subject included in the preview image (1312) is suitable for the reference composition (1310) by adjusting the zoom of the camera (408). For example, the electronic device (101) can adjust the composition so that the composition of the subject included in the preview image (1312) is suitable for the reference composition (1310) by performing a zoom-out using the camera (408). The preview image (1313) may be a composition in which the face (1301) straddles the boundary line (1330). The electronic device (101) may not adjust the zoom of the camera (408) when the face (1301) straddles the boundary line (1330). The electronic device (101) can adjust the zoom of the camera (408) even if the face (1301) is within the boundary line (1330) when there are two or more subjects (e.g., two or more people).

[0147] In one embodiment, the reference composition (1320) may be an ideal composition corresponding to the entire body of the subject. For example, the reference composition (1320) may be a composition in which the face (1301) is positioned at the upper end of the center line (1350) and is positioned exactly inside the boundary line (1330), and the torso (1303) is positioned inside the boundary line (1330) and the lower end of the torso (1303) (e.g., feet) is positioned exactly inside the boundary line (1370) of the entire body. Note that the reference composition (1320) is merely an example. The composition of the subject included in the preview image (1321) may be such that the face (1301) is located at the top of the center line (1350), but is separated from the inside of the boundary line (1330), and the torso (1303) is located within the boundary line (1330), but is separated from the inside of the full body boundary line (1370). The electronic device (101) may adjust the composition so that the composition of the subject included in the preview image (1321) is suitable for the reference composition (1320) by adjusting the zoom of the camera (408). For example, the electronic device (101) may adjust the composition so that the composition of the subject included in the preview image (1321) is suitable for the reference composition (1320) by performing a zoom-in using the camera (408). The composition of the subject included in the preview image (1322) may be such that the face (1301) is located at the bottom of the center line (1350) and is separated from the inside of the boundary line (1330), and the torso (1303) is located within the boundary line (1330) but is separated from the inside of the full body boundary line (1370). The electronic device (101) may adjust the composition so that the composition of the subject included in the preview image (1322) is suitable for the reference composition (1320) by adjusting the zoom of the camera (408). For example, the electronic device (101) may adjust the composition so that the composition of the subject included in the preview image (1322) is suitable for the reference composition (1320) by performing a zoom-in using the camera (408).

[0148] In one embodiment, the reference composition (1340) may be an ideal composition corresponding to the entire body of a subject. The reference composition (1340) may be the same as the reference composition (1320). For example, the reference composition (1340) may be a composition in which the face (1301) is positioned at the upper end of the center line (1350) and is positioned exactly inside the boundary line (1330), and the torso (1303) is positioned inside the boundary line (1330) and the lower end (e.g., feet) of the torso (1303) is positioned exactly inside the boundary line (1370) of the entire body. Note that the reference composition (1340) is merely an example. The composition of the subject included in the preview image (1341) may be such that the face (1301) is located at the top of the center line (1350), but is separated from the inside of the boundary line (1330), and the torso (1303) is located inside the boundary line (1330), but a part of the torso (1303) is located outside the full body boundary line (1370). The electronic device (101) may adjust the composition so that the composition of the subject included in the preview image (1341) is suitable for the reference composition (1340) by adjusting the zoom of the camera (408). For example, the electronic device (101) may adjust the composition so that the composition of the subject included in the preview image (1341) is suitable for the reference composition (1340) by performing a zoom-out using the camera (408). The composition of the subject included in the preview image (1342) may be such that the face (1301) is located at the top of the center line (1350), but is outside the boundary line (1330), and the torso (1303) is also outside the boundary line (1330). The electronic device (101) may adjust the composition so that the composition of the subject included in the preview image (1342) is suitable for the reference composition (1340) by adjusting the zoom of the camera (408). For example, the electronic device (101) may adjust the composition so that the composition of the subject included in the preview image (1342) is suitable for the reference composition (1340) by performing a zoom-out using the camera (408).The composition of the subject included in the preview image (1343) may be such that the face (1301) is positioned at the top of the center line (1350), but both the face (1301) and the torso (1303) are positioned outside the boundary line (1330). The electronic device (101) may adjust the composition so that the composition of the subject included in the preview image (1343) fits the reference composition (1340) by adjusting the zoom of the camera (408). For example, the electronic device (101) may adjust the composition so that the composition of the subject included in the preview image (1343) fits the reference composition (1340) by performing a zoom-out using the camera (408).

[0149] According to one embodiment, the preview image (1361) may be acquired using the camera (408) of the electronic device (101). The electronic device (101) may adjust the composition of subjects (e.g., people (1353) and people (1354)) included in the preview image (1361) by adjusting the zoom of the camera (408). For example, the electronic device (101) may perform a zoom-out using the camera (408) to adjust the composition so that the subjects, people (1353) and people (1354), are both located inside the boundary line (1330). The image (1360) may be an image in which the composition of the subject is adjusted.

[0150] According to one embodiment, the preview image (1391) may be acquired using the camera (408) of the electronic device (101). The electronic device (101) may adjust the composition of a subject (e.g., a person) included in the preview image (1351) by adjusting the zoom of the camera (408). For example, the electronic device (101) may perform a zoom-in using the camera (408) to adjust the composition so that the head of the subject, that is, the person, is positioned exactly within the boundary line (1330). The image (1390) may be an image in which the composition of the subject is adjusted by eliminating the margin (1392) between the subject and the boundary line (1330).

[0151]

[0152] Fig. 14 shows a flowchart of a photographing method of an electronic device according to one embodiment.

[0153] Referring to FIG. 14, according to one embodiment, operations 1410 to 1470 may be substantially identical to operations performed by the electronic device (e.g., the electronic device (101) of FIG. 1) described with reference to FIGS. 1 to 13. Therefore, any redundant description will be omitted.

[0154] According to one embodiment, operations 1410 to 1470 may be understood to be performed in a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1).

[0155] In operation 1410, the electronic device (101) can classify a category of a subject included in a first preview image acquired using a camera (e.g., camera module (180) of FIG. 1, camera (408) of FIG. 4).

[0156] In operation 1430, the electronic device (101) can identify a reference composition corresponding to the subject based on the subject's category. The reference composition can include a structure in which the subject is aesthetically appropriately arranged.

[0157] In operation 1450, the electronic device (101) may acquire a second preview image by adjusting the zoom of the camera based on the reference frame. The electronic device (101) may adjust the zoom of the camera by adjusting the zoom magnification of the lens or changing the lens to a different type of lens. The second preview image may be an image with an adjusted zoom magnification of the first preview image.

[0158] In operation 1470, the electronic device (101) may output a result of performing processing to adjust the composition of the subject in the second preview image to a reference composition. The processing that the electronic device (101) may perform to adjust the composition of the subject in the second preview image to a reference composition may include at least one of fixing, cropping, and outpainting.

[0159] In one embodiment, operations 1410 through 1470 may be performed sequentially, but are not limited thereto. For example, two or more operations may be performed in parallel.

[0160]

[0161] FIG. 15A is a perspective view of the front of an electronic device according to one embodiment. FIG. 15B is a plan view of the rear of an electronic device according to one embodiment.

[0162] Referring to FIGS. 15A and 15B , an electronic device (1500) (e.g., the electronic device (101) of FIG. 1 ) may include a first housing (1510) (e.g., a first housing structure) including a first side member (1513) (e.g., a side bezel) and a second housing (1520) (e.g., a second housing structure) including a second side member (1523) (e.g., a side bezel) that are foldably coupled to each other with respect to a folding axis (F) through at least one hinge device (1540, 1540-1) (e.g., a hinge module or a hinge structure). For example, the first housing (1510) and the second housing (1520) may be configured as a foldable housing (e.g., a housing structure). For example, the electronic device (1500) may include a first display (1530) (e.g., a flexible display, a foldable display, or a main display) arranged to be supported by a first housing (1510) and a second housing (1520). For example, the first housing (1510) may include a first side (1511) and a second side (1512) facing in an opposite direction (e.g., in the -z-axis direction) of the first side (1511). For example, the second housing (1520) may include a third side (1521) and a fourth side (1522) facing in an opposite direction (e.g., in the -z-axis direction) of the third side (1521). For example, the first housing (1510) may include a first rear cover (1514) coupled with a first side member (1513). For example, the second housing (1520) may include a second rear cover (1524) coupled with a second side member (1523). For example, when the electronic device (1500) is in a fully unfolded first state (e.g., an unfolded state or a spread state), the first side (1511) and the third side (1521) may be operated such that they face substantially the same direction (e.g., the z-axis direction).For example, when the electronic device (1500) is in a fully folded second state (e.g., a folded state or a collapsed state), the first side (1511) and the third side (1521) may be operated so that they face each other or face in opposite directions. For example, the electronic device (1500) may also be operated so as to maintain a third state (e.g., an intermediate state) between the first state and the second state.

[0163] According to one embodiment, the electronic device (1500) may include a receiver (1501), at least one sensor module (1504) (e.g., an ambient light sensor) and / or at least one first camera module (1505) (e.g., a UDC, under display camera) disposed on a first side (1511) of the first housing (1510). For example, the electronic device (1500) may include at least one key (1506) disposed on a first side member (1513). For example, the electronic device (1500) may include a second display (1531) (e.g., a sub-display), at least one second camera module (1508) and / or a flash (1509) disposed on a second side (1512) of the first housing (1510) (e.g., a first rear cover (1514)). For example, the second display (1531) may be arranged to be visible from the outside through at least a portion of the first rear cover (1514). For example, the electronic device (1500) may include a speaker (1502), a microphone (1503), or a connector port (1507) arranged on the second side member (1523). At least some of the aforementioned components may be arranged in the first housing (1510) and / or the second housing (1520).

[0164] According to one embodiment, the first display (1530) (e.g., a flexible display) may include a first region (1530a) (e.g., a first planar portion) corresponding to at least a portion of the first surface (1511), a second region (1530b) (e.g., a second planar portion) corresponding to at least a portion of the third surface (1521), and a third region (1530c) (e.g., a flexible portion) connecting the first region (1530a) and the second region (1530b) and allowing the electronic device (1500) to be deformed in a second state (e.g., a folded state) and / or a third state (e.g., an intermediate state). For example, the third region (1530c) may be positioned to at least partially overlap at least one hinge device (1540, 1540-1) when the first display (1530) is viewed from above (e.g., in the z-axis direction). For example, the first display (1530) may be arranged so that it is not visible from the outside in the second state by having the first side (1511) and the third side (1521) face each other (e.g., inward-fold type). For example, the first display (1530) may be arranged so that it is visible from the outside in the second state by having the first side (1511) and the third side (1521) face each other in opposite directions (e.g., outward-fold type).

[0165]

[0166] FIG. 16 is an exploded perspective view of a portion of the electronic device of FIGS. 15A and 15B including a hinge device according to one embodiment.

[0167] Referring to FIG. 16, the electronic device (1500) may include at least one hinge device (1540, 1540-1) (e.g., a hinge module or a hinge structure) connecting the first housing (1510) and the second housing (1520) under the first display (1530) (e.g., in the -z-axis direction). For example, the at least one hinge device (1540, 1540-1) may include the first hinge device (1540) and a second hinge device (1540-1) spaced apart from the first hinge device (1540) along a direction parallel to the folding axis (F) (e.g., in the ±y-axis direction). For example, at least one hinge device (1540, 1540-1) may be supported by a first support member (15131) extending from the first side member (1513) into the first space (15101) of the first housing (1510) and a second support member (15231) extending from the second side member (1523) into the second space (15201) of the second housing (1520). For example, at least one hinge device (1540, 1540-1) may be arranged between the first housing (1510) and the second housing (1520) so as to be invisible from the outside through a hinge housing (1550) (e.g., a hinge cover).

[0168] In one embodiment, the first hinge device (1540) may include a first rotation member (1541) (e.g., a first arm or a first rotator) disposed on a first support member (15131) of a first housing (1510), a second rotation member (1542) (e.g., a second arm or a second rotator) disposed on a second support member (15231) of a second housing (1520), and a gear assembly (1543) connected to the first rotation member (1541) and the second rotation member (1542) such that the first housing (1510) and the second housing (1520) rotate symmetrically with respect to each other. For example, the gear assembly (1543) can include a plurality of gears (e.g., spur gears and / or worm gears) gear-coupled with respect to one another. For example, the gear assembly (1543) can include a cam coupling structure to urge the first housing (1510) and the second housing (1520) in a direction in which they are to transition from a first state (e.g., an unfolded state or an unfolded state) to a second state (e.g., a folded state or a collapsed state) or from the second state to the first state, based on a predetermined angle, and to provide a free stop at various folding angles. For example, the second hinge device (1540-1) can be substantially symmetrical with respect to the first hinge device (1540), or can have a configuration that is substantially the same.

[0169] According to one embodiment, the electronic device (1500) may be coupled with at least one hinge device (1540, 1540-1) and may include at least one detent module (1544) for providing a stopping sensation at various angles. For example, the at least one hinge device (1540, 1540-1) and / or the detent module (1544) may form substantially the same plane as the first support member (15131) and the second support member (15231) when the electronic device (1500) is in the first state. For example, the second hinge device (1540-1) may be substantially symmetrical with the first hinge device (1540) or may have a substantially identical configuration.

[0170]

[0171] An electronic device (101; 1500) according to one embodiment may include a camera (180; 408). The electronic device (101; 1500) may include at least one processor (120). The electronic device (101; 1500) may include a memory (130) that stores instructions. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101; 1500) to classify a category of an object included in a first preview image acquired using the camera (180; 408). The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101; 1500) to identify a reference composition corresponding to the subject based on a category of the subject. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101; 1500) to adjust a zoom of the camera (180; 408) based on the reference composition to obtain a second preview image. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101; 1500) to output a result of performing processing for adjusting the composition of the subject to the reference composition in the second preview image.

[0172] In one embodiment, the processing may include at least one of fixing, cropping, and outpainting.

[0173] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101; 1500) to determine whether the result satisfies the quality of an optical zoom level. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101; 1500) to perform up-scaling on the result in response to determining that the result does not meet the quality of an optical zoom level.

[0174] According to one embodiment, the result may include an image automatically captured by the electronic device (101; 1500) using the camera (180; 408).

[0175] According to one embodiment, the reference composition may include a structure in which the subject is aesthetically appropriately positioned.

[0176] In one embodiment, the result may include a plurality of images. The plurality of images may include a representative image and an image candidate to be recommended to the user.

[0177] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101; 1500) to receive a user's selection input for the plurality of images. The instructions, when individually or collectively executed by the at least one processor (120), may cause the electronic device (101; 1500) to modify a reference composition corresponding to the subject based on the selection input.

[0178] According to one embodiment, the composition of the subject in the candidate image may be different from the composition of the subject in the representative image.

[0179] A method for photographing an electronic device (101) according to one embodiment may include an operation for classifying a category of a subject included in a first preview image acquired using a camera (180; 408). The method may include an operation for identifying a reference composition corresponding to the subject based on the category of the subject. The method may include an operation for adjusting a zoom of the camera based on the reference composition to acquire a second preview image. The method may include an operation for outputting a result of performing processing for adjusting the composition of the subject to the reference composition in the second preview image.

[0180] In one embodiment, the processing may include at least one of fixing, cropping, and outpainting.

[0181] In one embodiment, the photographing method may further include an operation of determining whether the result satisfies the quality level of optical zoom. The photographing method may further include an operation of performing upscaling on the result in response to determining that the result does not satisfy the quality level of optical zoom.

[0182] According to one embodiment, the result may include an image automatically captured by the electronic device (101; 1500) using the camera (180; 408).

[0183] According to one embodiment, the reference composition may include a structure in which the subject is aesthetically appropriately positioned.

[0184] In one embodiment, the result may include a plurality of images. The plurality of images may include a representative image and an image candidate to be recommended to the user.

[0185] According to one embodiment, the photographing method may further include an operation of receiving a user's selection input for the plurality of images. The photographing method may further include an operation of modifying a reference composition corresponding to the subject based on the selection input.

[0186] In one embodiment, the composition of the subject in the candidate image may be different from the composition of the subject in the representative image.

[0187] According to one embodiment, a computer-readable recording medium storing one or more computer programs may include instructions for performing the method in a processor.

[0188] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from this document.

[0189]

[0190] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0191] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0192] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0193] Various embodiments of the present document may be implemented as software (e.g., a program (1740)) including one or more instructions stored in a storage medium (e.g., an internal memory (1736) or an external memory (1738)) readable by a machine (e.g., an electronic device (1701)). For example, a processor (e.g., a processor (1720)) of the machine (e.g., an electronic device (1701)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0194] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0195] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separately arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101; 1500), Camera (180; 408); At least one processor (120) comprising processing circuitry; and Memory for storing instructions (130) Including, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101; 1500) to: Classify the category of the object included in a first preview image obtained using the above camera (180; 408), Identify a reference composition corresponding to the subject based on the category of the subject; Based on the above reference composition, the zoom of the camera (180; 408) is adjusted to obtain a second preview image, Outputting the result of performing processing to adjust the composition of the subject to the reference composition in the second preview image. Electronic devices (101; 1500).

2. In paragraph 1, The above processing is, At least one of fixing, cropping, and outpainting including, Electronic devices (101; 1500).

3. In any one of paragraphs 1 and 2, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101; 1500) to: Determine whether the above result meets the quality level of optical zoom, In response to determining that the above result does not meet the image quality of the optical zoom level, performing up-scaling on the above result, Electronic devices (101; 1500).

4. In any one of paragraphs 1 to 3, The above results are, The electronic device (101; 1500) includes an image automatically captured using the camera (180; 408). Electronic devices (101; 1500).

5. In any one of paragraphs 1 to 4, The above results are, Contains multiple images, The above multiple images are, representative image; and Image candidate to recommend to users including, Electronic devices (101; 1500).

6. In any one of paragraphs 1 to 5, The above instructions, when individually or collectively executed by the at least one processor (120), cause the electronic device (101; 1500) to: Receiving a user's selection input for the above multiple images, Based on the above selection input, modify the reference composition corresponding to the subject. Electronic devices (101; 1500).

7. In any one of paragraphs 1 to 6, The composition of the subject in the above candidate image is, Different from the composition of the subject in the representative image above, Electronic devices (101; 1500).

8. An operation of classifying a category of a subject included in a first preview image acquired using a camera (180; 408); An operation of identifying a reference composition corresponding to the subject based on the category of the subject; An operation of adjusting the zoom of the camera based on the above reference composition to obtain a second preview image; and An operation of outputting a result of performing processing to adjust the composition of the subject to the reference composition in the second preview image. including, method.

9. In paragraph 8, The above processing is, At least one of fixing, cropping, and outpainting including, method.

10. In any one of paragraphs 8 to 9, An operation for determining whether the above result satisfies the quality of optical zoom; and An operation of performing up-scaling on the result in response to determining that the result does not meet the image quality of the optical zoom level. including more, method.

11. In any one of paragraphs 8 to 10, The above results are, Contains images automatically captured using the above camera (180; 408), method.

12. In any one of paragraphs 8 to 11, The above results are, Contains multiple images, The above multiple images are, representative image; and Image candidate to recommend to users including, method.

13. In any one of paragraphs 8 to 12, An operation of receiving a user's selection input for the above plurality of images; and An operation of modifying a reference composition corresponding to the subject based on the above selection input. including more, method.

14. In any one of paragraphs 8 to 13, The composition of the subject in the above candidate image is, Different from the composition of the subject in the representative image above, method.

15. A computer program stored on a computer-readable recording medium for executing the method of any one of claims 8 to 14 in combination with hardware.

Images