Method for acquiring image, electronic device supporting same, and storage medium

Abstract

An electronic device comprises a camera and at least one processor, wherein the at least one processor is configured to: acquire motion information corresponding to a plurality of objects included in image frames acquired through the camera; remove at least one first object from the image frames; acquire a background image; acquire weighted sums corresponding to the respective image frames comprising the at least one first object; and acquire a panning image by synthesizing, from among the image frames comprising the at least one first object, an image frame having the greatest weighted sum from among the acquired weighted sums, and the background image.

Description

A method for acquiring an image, an electronic device supporting the same, and a storage medium

[0001] The present disclosure relates to a method for acquiring an image, an electronic device supporting the same, and a storage medium.

[0002] Thanks to the remarkable advancements in information and communication technology and semiconductor technology, the distribution and use of various electronic devices are increasing rapidly. Electronic devices are being developed to allow users to carry them around and communicate. The term "electronic device" may refer to a device that performs specific functions according to an installed program, such as mobile communication terminals, tablet PCs, video / audio devices, desktop / laptop computers, or in-vehicle navigation systems.

[0003] Recently, users have shown great interest in acquiring high-quality images provided by advanced camera equipment, going beyond simply capturing images using electronic devices. To enhance the utility value of mobile terminals and satisfy the diverse needs of users, various User Interfaces (UIs) and various functions utilizing them are being provided. Electronic devices provide image editing functions. Electronic devices can provide an environment in which users can edit images stored on the device using an image editing application.

[0004] An electronic device according to one embodiment of the present disclosure may include a camera and at least one processor. The at least one processor may be configured to confirm a first user input for acquiring a panning image. The at least one processor may be configured to acquire motion information corresponding to a plurality of objects included in image frames acquired through the camera. The at least one processor may be configured to identify at least one first object among the plurality of objects based on the acquired motion information. The at least one processor may be configured to acquire a background image including at least one blurred background object based on removing the at least one first object from the image frames and synthesizing the image frames from which the at least one first object has been removed. The at least one processor may be configured to acquire weighted sums corresponding to each of the image frames including the at least one first object by acquiring a weighted sum of at least one pose score associated with the aesthetics of the movement corresponding to the at least one first object for each of the image frames including the at least one first object among the acquired image frames. The above at least one processor may be configured to acquire a panning image by synthesizing the image frame having the largest weighted sum among the acquired weighted sums and the background image among the image frames including the at least one first object.

[0005] According to one embodiment, the method may include an operation of acquiring motion information corresponding to a plurality of objects included in image frames acquired through a camera of an electronic device. The method may include an operation of identifying at least one first object among the plurality of objects based on the acquired motion information. The method may include an operation of acquiring a background image including at least one blurred background object based on removing the at least one first object from the image frames and synthesizing the image frames from which the at least one first object has been removed. The method may include an operation of acquiring weighted sums corresponding to each of the image frames including the at least one first object by acquiring a weighted sum of at least one pose score associated with the aesthetics of the movement corresponding to the at least one first object for each of the image frames including the at least one first object among the acquired image frames. The method may include an operation of acquiring a panning image by synthesizing the image frame having the largest weighted sum among the acquired weighted sums and the background image among the image frames including the at least one first object.

[0006] According to one embodiment, in a storage medium storing computer-executable instructions, the instructions may cause the electronic device to perform at least one operation when executed by at least one processor of the electronic device. The at least one operation may include an operation of acquiring motion information corresponding to a plurality of objects included in image frames acquired through a camera of the electronic device. The at least one operation may include an operation of identifying at least one first object among the plurality of objects based on the acquired motion information. The at least one operation may include an operation of acquiring a background image including at least one blurred background object based on removing the at least one first object from the image frames and synthesizing the image frames from which the at least one first object has been removed. The at least one operation may include an operation of acquiring weighted sums corresponding to each of the image frames including the at least one first object by acquiring a weighted sum of at least one pose score associated with the aesthetics of the movement corresponding to the at least one first object for each of the image frames including the at least one first object among the acquired image frames. The above at least one operation may include the operation of acquiring a panning image by synthesizing the image frame having the largest weighted sum among the acquired weighted sums among the image frames including the at least one first object and the background image.

[0007] The means for solving the problem according to one embodiment of the present disclosure are not limited to the means for solving the problem described above, and means for solving the problem not mentioned will be clearly understood by those skilled in the art from the present specification and the accompanying drawings.

[0008] FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment.

[0009] FIG. 2 is a block diagram of electronic device configurations according to one embodiment.

[0010] FIG. 3 is a flowchart illustrating a method for acquiring a panning image according to one embodiment.

[0011] FIGS. 4a, FIGS. 4b, FIGS. 4c, FIGS. 4d, FIGS. 4e, and FIGS. 4f are drawings for explaining a method of acquiring a panning image according to one embodiment.

[0012] Figure 5 is a flowchart illustrating a method for verifying a main object based on motion vectors.

[0013] Figure 6 is a diagram illustrating a method for identifying a main object based on motion vectors.

[0014] FIG. 7 is a flowchart illustrating a method for identifying multiple main objects based on relative motion vectors.

[0015] FIG. 8 is a diagram illustrating a method for identifying multiple main objects based on relative motion vectors.

[0016] FIG. 9 is a flowchart illustrating a method for obtaining weighted sums of pose scores corresponding to image frames according to one embodiment.

[0017] FIG. 10 is a diagram illustrating a method for obtaining weighted sums of pose scores corresponding to image frames according to one embodiment.

[0018] FIG. 11 is a diagram illustrating a method for obtaining a weighted sum of pose scores corresponding to an image frame based on a layer corresponding to an object according to one embodiment.

[0019] FIG. 12 is a diagram illustrating a method for obtaining a weighted sum of pose score values ​​corresponding to an image frame according to one embodiment.

[0020] FIG. 13 is a flowchart illustrating a method for acquiring a panning image based on a pose correction according to one embodiment.

[0021] FIGS. 14a and FIGS. 14b are drawings for explaining a method of acquiring a panning image based on a pose correction according to one embodiment.

[0022] FIG. 15 is a flowchart illustrating a method for acquiring a panning image based on an image enhancement function according to one embodiment.

[0023] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to one embodiment. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through the server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0024] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0025] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0026] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0027] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0028] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0029] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0030] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0031] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0032] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0033] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0034] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0035] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0036] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0037] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0038] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0039] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0040] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0041] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0042] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0043] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0044] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0045] In the following detailed description, reference numbers in the drawings may be assigned identically or omitted for configurations that can be easily understood through prior embodiments, and detailed descriptions thereof may also be omitted. An electronic device according to one embodiment disclosed in this document may be implemented by selectively combining configurations of different embodiments, and a configuration of one embodiment may be replaced by a configuration of another embodiment. For example, it should be noted that the present invention is not limited to specific drawings or embodiments.

[0046] FIG. 2 is a block diagram of electronic device configurations according to one embodiment.

[0047] In one embodiment, the electronic device (200) may be the electronic device (101) of FIG. 1.

[0048] Referring to FIG. 2, in one embodiment, the electronic device (200) may include a camera (210), a display (220), a memory (230), and / or a processor (240).

[0049] In one embodiment, the camera (210) may be included in the camera module (180) of FIG. 1.

[0050] In one embodiment, the camera (210) can acquire images of objects outside the electronic device (200).

[0051] In one embodiment, the display (220) may be included in the display module (160) of FIG. 1.

[0052] In one embodiment, when a panning image is acquired, the display (220) may display the panning image on at least a portion of the screen of the display (220). The display (220) may acquire user input for correcting the pose of a main object included in the panning image. The main object may be an object that is required to be represented more clearly than other objects within the panning image and may be referred to as a "target object."

[0053] In one embodiment, panning is a technique that makes a moving object appear relatively fast by processing the image of the object relative to the background sharply when acquiring an image of a moving object. In one embodiment, panning may be performed based on moving the camera (210) in a direction corresponding to the direction in which the object is moving. By camera panning, the image of the moving object may be acquired relatively sharply. By camera panning, the background and static objects may appear blurred. The operation of panning the camera (210) may include, for example, moving the camera (210) vertically or horizontally, but is not limited thereto. The operation of panning the camera (210) may also include moving the camera (210) diagonally. The operation of panning the camera (210) may be referred to as "panning" or "camera panning." In one embodiment, an image acquired based on camera panning may be referred to as a "panning image."

[0054] In one embodiment, the memory (230) may be included in the memory (130) of FIG. 1.

[0055] In one embodiment, the memory (230) may store an artificial intelligence (AI) model for modifying the pose of an object identified within an image frame. The AI ​​model may include an artificial intelligence model trained to output information associated with a changed pose corresponding to an object included in the image frame based on receiving an image frame as input. The AI ​​model may learn, for example, images acquired by a camera (210). In one embodiment, the AI ​​model may be stored in an external electronic device (e.g., the electronic device (104) or server (108) of FIG. 1).

[0056] In one embodiment, the processor (240) may be included in the processor (120) of FIG. 1.

[0057] In one embodiment, the processor (240) can control the overall operation for acquiring a panning image. The processor (240) may include one or more processors for performing the operation of acquiring a panning image. For example, the processor (240) may correspond to a plurality of processors that divide a plurality of operations among the processors and perform them collectively. The operation performed by the processor (240) will be described later.

[0058] In one embodiment, the processor (240) may include a neural processing unit (NPU) for performing a panning image acquisition operation. For example, the processor (240) may include a graphic processing unit (GPU) capable of performing a panning image acquisition operation using a specified algorithm when the panning image acquisition operation is performed using a specified algorithm.

[0059] In one embodiment, the processor (240) can detect a first user input for acquiring a panning image. For example, the processor (240) can detect a user input for entering a panning mode (e.g., a touch of the display (220)) through the display (220). The user input for entering a panning mode may include a touch input to an object (e.g., a button) for capturing a panning image displayed through the display (200), but is not limited thereto.

[0060] In one embodiment, the processor (240) may control the camera (210) to acquire a plurality of image frames based on a first user input. The image frames acquired through the camera (210) may be referred to as “first image frames.” The first image frames may be images captured based on the movement of the camera. The first image frames may be acquired based on the camera’s exposure time being set short. The first image frames may also be referred to as “original image frames.” The processor (240) may acquire a plurality of image frames sequentially, for example, based on the camera (210)’s exposure time being set short. Each of the acquired first image frames may include a plurality of objects. The processor (240) may acquire a plurality of first image frames while the user moves the camera (210) in correspondence with the direction of movement of an object (e.g., main object) that the user wishes to clearly represent within the panning image.

[0061] In one embodiment, the processor (240) can identify at least one main object among a plurality of objects included in each of the image frames. Each of the image frames may include a plurality of objects, for example, a main object and a background object. The background object may include at least one object located in the background area of ​​the image frame. The background object may be an object that is relatively less important than the main object. The background object may be blurred in the panning image. The processor (240) can identify at least one main object among a plurality of image frames (e.g., first image frames) based on at least two reference images. A specific method by which the processor (240) identifies the main object will be described later in FIGS. 5 and FIGS. 6.

[0062] In one embodiment, the processor (240) may acquire a background image based on removing at least one main object from the first image frames. The processor (240) may acquire a background image based on synthesizing the images from which the main object has been removed. The background image may include, for example, at least one blurred background object. The processor (240) may acquire a background image including a background object that appears relatively blurry compared to the main object, even without applying a blur effect.

[0063] In one embodiment, the processor (240) may calculate a score associated with the aesthetics of the movement of the main object for each of the image frames containing the main object. The image frames containing the main object may be referred to as “second image frames,” but there is no limitation on the naming of the image frames. If multiple main objects are detected within an image frame, the processor (240) may obtain posture scores corresponding to the main objects. The processor may obtain a weighted sum corresponding to the image frame containing the main object by calculating a weighted sum among the obtained posture scores. The weighted sum may be referred to as a “weighted posture score (“WPS”).” The processor (240) may obtain weighted sums corresponding to the image frames for each of the image frames containing the main object. The processor (240) may identify the image frame having the largest weighted sum among the obtained weighted sums. The image frame having the largest weighted sum may be, for example, the image frame having the largest score associated with the aesthetics of the moving pose (e.g., running, walking, or cycling) corresponding to the main object. By identifying the image with the largest score, the processor can identify the image frame containing the main object having the best pose among multiple image frames. The processor (240) can acquire a panning image based on compositing the image frame having the largest weighted sum with the background image. In the acquired panning image, the main object may be represented as relatively sharp in contrast to the background object. The processor (240) can store the acquired panning image in memory (230). The processor (240) can display the acquired panning image through the display (220).

[0064] In one embodiment, the processor (240) may obtain image frames from which at least one main object has been removed by performing image inpainting on each of the image frames containing at least one main object. The processor (240) may obtain an image frame containing only a background object by performing image inpainting on an area corresponding to at least one main object within the image frame.

[0065] In one embodiment, the processor (240) may obtain the background image based on synthesizing image frames from which at least one main object has been removed. The operation of synthesizing image frames from which at least one main object has been removed may be referred to as the operation of overlapping image frames.

[0066] In one embodiment, the processor (240) may set a value corresponding to a fast shutter speed as the shutter speed of the camera (210). The processor (240) may change the shutter speed value of the camera (210) to a value corresponding to a fast shutter speed based on confirming the occurrence of a switching event for operating in panning mode. The switching event may include user input for switching to panning mode (e.g., touch input to a panning mode button). The switching event may include the satisfaction of a condition for switching to panning mode. The condition for switching to panning mode may include, for example, the detection of a moving subject located within the field of view of the camera (210). In one embodiment, a fast shutter speed of the camera (210) may be required to acquire an image frame containing a main object having little motion blur. As the shutter speed of the camera (210) is set faster, the exposure time corresponding to the camera (210) may be shortened.

[0067] In one embodiment, the processor (240) can acquire original image frames (or first image frames) through the camera (210) based on the shutter speed corresponding to the panning mode. The processor (240) can detect the occurrence of an event for switching from the panning mode to another mode (e.g., default mode). The event for switching from the panning mode to another mode may include user input for switching the operating mode of the camera (210) (e.g., touch input for the default mode). The processor (240) can terminate the panning mode by switching from the panning mode to another mode. The processor (240) can change the setting of the shutter speed of the camera (210), for example, based on the shutter speed value corresponding to the default mode. The exposure time corresponding to the camera (210) in the default mode may be relatively longer than the exposure time corresponding to the camera (210) in the panning mode.

[0068] In one embodiment, the processor (240) can identify at least two reference images, including a first reference image and a second reference image, from image frames (e.g., first image frames). The processor (240) can, for example, select the first original image frame as the first reference image from among N image frames (N is a natural number greater than or equal to 2). The processor (240) can, for example, select the last original image frame as the second reference image from among N image frames.

[0069] In one embodiment, the processor (240) may obtain motion information corresponding to the plurality of objects among the plurality of objects included in each of the reference images. The motion information may include, for example, motion vector values ​​corresponding to the plurality of objects. The processor (240) may identify at least one main object among the plurality of objects based on the obtained motion vector values. The processor (240) may identify the motion vector values ​​by checking the difference between the position corresponding to the main object in the first reference image and the position corresponding to the main object in the second reference. A method by which the processor (240) identifies the first object (e.g., the main object) based on the motion vector values ​​among the plurality of moving objects will be described in detail in FIGS. 5 and 6.

[0070] In one embodiment, the processor (240) can identify the object having the smallest motion vector value among the motion vector values ​​among the plurality of objects included in the reference image as the primary object. The method by which the processor (240) identifies the primary object will be described in detail in FIGS. 7 and FIGS. 8.

[0071] In one embodiment, the processor (240) can obtain a relative motion vector value corresponding to at least one object excluding the primary object among a plurality of objects included in a reference image. The relative motion vector may represent the relative movement of an object with respect to the primary object. The processor (240) can check whether at least one object is identified where the relative motion vector is smaller than a set threshold value. The threshold value may be set to an appropriate value to identify a secondary object moving at a speed similar to that of the primary object when a plurality of main objects with movement are detected.

[0072] In one embodiment, the processor (240) may identify at least one object as a secondary object based on identifying at least one object having a relative motion vector value smaller than a threshold value. The processor (240) may set the weight of the secondary object lower than the weight of the primary object. The weight corresponding to the object may be used to select an image frame with excellent aesthetics of the pose corresponding to the moving object.

[0073] In one embodiment, the processor (240) may obtain weighted sums of pose scores corresponding to at least one main object for each of the image frames (e.g., second image frames) containing at least one main object. The processor (240) may select the image frame having the highest weighted sum among the image frames containing at least one main object. The image frame having the highest weighted sum may be referred to as the “third image frame,” and there is no limitation on the naming of the image frames.

[0074] In one embodiment, the processor (240) may acquire a panning image based on synthesizing an image frame having the highest weighted sum to the background image. In the panning image, the main object (or subject of interest) may be rendered sharper than the background object. The processor (240) may provide the acquired panning image through a display (220).

[0075] In one embodiment, the processor (240) may obtain weighted sums between a first pose score corresponding to a primary object and at least one second pose score corresponding to at least one secondary object for each of the image frames containing at least one main object. The processor (240) may identify the image frame having the largest weighted sum among the plurality of weighted sums. The processor (240) may set the weight for the first pose score higher than the weight for at least one second pose score. The total sum of the weight corresponding to the primary object and the at least one weight corresponding to the at least one secondary object may be 1.

[0076] In one embodiment, the processor (240) can perform pose correction on an image frame. The processor (240) can perform pose correction based on information output from a generative AI model by inputting the image frame to a generative AI model trained to output a pose-corrected image based on receiving the image frame as input. The image frame input to the generative AI model may be the image frame having the largest weighted sum. If the image frame having the largest weighted sum is not identified, the image frame input to the generative AI model may be any one of the image frames acquired through the camera (210).

[0077] In one embodiment, the processor (240) may display at least one candidate image with a modified pose through the display (220). The processor (240) may provide at least one candidate image through the display (220) to acquire a panning image based on one of the at least one candidate image with a modified pose.

[0078] In one embodiment, the processor (240) may obtain a panning image by compositing the selected candidate image onto the background image based on confirming user input for selecting one of the at least one candidate image. The panning image obtained by compositing the selected candidate image and the background image may differ from the panning image obtained by compositing the image frame and the background image having the largest weighted sum. The processor (240) may obtain a panning image including a main object with a modified pose by performing compositing with the background image based on the candidate image corresponding to the user input.

[0079] In one embodiment, the processor (240) may display a panning image obtained using a generative AI model through a display (220). The processor (240) may also perform additional posture correction based on user input regarding the displayed panning image (e.g., touch input to an object for posture correction).

[0080] In one embodiment, the processor (240) can determine whether image frames were acquired in a low-light environment. The processor (240) can determine whether the brightness of an image frame (e.g., a first image frame) is lower than a set threshold brightness value based on checking the pixel values ​​included in the image frame.

[0081] In one embodiment, the processor (240) may apply an image enhancement function to the image frames based on confirming that the image frames were acquired in a low-light environment. The processor (240) may increase the brightness of the image frames based on applying the image enhancement function to the image frames, taking into account that an image with low brightness may be acquired when the image is acquired with a fast shutter speed.

[0082] In FIG. 2, the electronic device (200) is illustrated as including a camera (210), a display (220), a memory (230), and / or a processor (240), but is not limited thereto. In one embodiment, the electronic device (200) may further include at least one of the components included in the electronic device (101) of FIG. 1.

[0083] FIG. 3 is a flowchart illustrating a method for acquiring a panning image according to one embodiment. The embodiment of FIG. 3 will be described with reference to FIG. 4a, 4b, 4c, 4d, 4e, and 4f. FIG. 4a, 4b, 4c, 4d, 4e, and 4f are drawings illustrating a method for acquiring a panning image according to one embodiment.

[0084] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Depending on the implementation, certain operations may be omitted.

[0085] Referring to FIG. 3, in operation 301, in one embodiment, an electronic device (200) (e.g., processor (240) of FIG. 2) may acquire motion information corresponding to a plurality of objects included in first image frames acquired through a camera (e.g., camera (210) of FIG. 2). In one embodiment, the motion information may include vector values. The motion vector values ​​may represent the amount of change in the position of an object between two or more image frames. The smaller the motion vector value, the greater the probability that an object located within an image frame is identified as a subject of interest. The larger the motion vector value, the less the probability that an object located within an image frame is identified as a subject of interest. A method for the electronic device (200) to acquire motion information corresponding to an object will be described later in FIG. 5 and FIG. 6.

[0086] Referring to FIG. 4a, in one embodiment, an electronic device (200) may acquire a plurality of image frames including background objects (403, 405, 407) and the first object (401) based on confirming user input for acquiring a panning image of a moving first object (401). The plurality of image frames may be used, for example, to acquire an aesthetic panning image of a moving subject. The panning image may be a vivid image including a sharp specific subject and a blurred background subject. The first object may be an object that is required to be represented more clearly than other objects within the panning image. The first object may be replaced with terms such as "main object" or "subject of interest." The user of the electronic device (200) can maintain the direction in which the lens (not shown) of the camera (e.g., camera (210) of FIG. 2) is facing from the electronic device (200) toward the main object (411) while image frames are being acquired. In one embodiment, the camera included in the electronic device (200) can move in a direction (413) corresponding to the direction (409) in which the main object (401) is moving while a plurality of image frames are being acquired. The action of the camera moving at a speed similar to the speed of movement of the main object (401) while shooting is being performed may be referred to as "camera panning."

[0087] In one embodiment, the electronic device (200) can set a first shutter speed as the shutter speed of the camera. The first shutter speed can be set short, for example, to acquire an image including a main object having small motion blur. The electronic device (200) can acquire image frames based on setting the shutter speed fast.

[0088] Referring to FIG. 4b, in one embodiment, an electronic device (200) may acquire N original image frames (420a, 420b, to 420n). The first original image frame (420a) may include, for example, a plurality of objects (421a, 423a, 425a, 427a). The second original image frame (420b) may include a plurality of objects (421b, 423b, 425b, 427b). The Nth original image frame (420n) may include a plurality of objects (421n, 423n, 425n, 427n). The electronic device (200) can obtain motion vector values ​​corresponding to a plurality of objects based on at least two reference images (420a, 420n) among N original image frames (420a, 420b, to 420n). The electronic device (200) can calculate motion vector values ​​using, for example, a block matching algorithm or a Lucas-Kanade method, and there are no limitations on the specific method of calculating motion vector values. The electronic device (200) can obtain motion vector values ​​corresponding to an object (e.g., object (401)) by checking the difference between the position of an object (421a) identified from the first original image frame (420a) and the position of an object (421n) identified from the last original image frame (420n). The electronic device (200) can obtain a motion vector value corresponding to an object (e.g., object (403)) by determining the difference between the position of an object (423a) identified from the first original image frame (420a) and the position of an object (423n) identified from the last original image frame (420n).The electronic device (200) can obtain a motion vector value corresponding to an object (e.g., object (405)) by checking the difference between the position of an object (425a) identified from the first original image frame (420a) and the position of an object (425n) identified from the last original image frame (420n). The electronic device (200) can obtain a motion vector value corresponding to an object (e.g., object (407)) by checking the difference between the position of an object (427a) identified from the first original image frame (420a) and the position of an object (427n) identified from the last original image frame (420n).

[0089] In operation 303, in one embodiment, the electronic device (200) can identify at least one first object among a plurality of objects based on acquired motion information. The electronic device (200) can identify at least one first object, for example, using a motion vector value. The electronic device (200) can identify at least one first object having a motion vector value smaller than a first threshold value among vector values ​​corresponding to a plurality of objects identified within an image frame. The first threshold value may be set to an appropriate value to distinguish an object required to be clearly represented within a panning image from an object to be processed as background. The plurality of objects may be referred to as "candidate objects" or "candidate subjects."

[0090] Referring again to FIG. 4b, in one embodiment, the electronic device (200) can identify a first object (421a) having a motion vector value smaller than a first threshold. Based on the movement of the camera in a direction corresponding to the movement direction (429) of the object (421a) required to be clearly represented in the panning image, the amount of position change of the objects (421a, 421b, to 421n) identified from each of the original image frames (420a, 420b, to 420n) may be small. The electronic device (200) can identify objects (423a, 425a, 427a) having a motion vector value larger than the first threshold. Objects (423a, 425a, 427a) having a motion vector value larger than the first threshold may be referred to as "background objects". A specific method for identifying a main object based on the difference between the positions corresponding to the object identified by the electronic device (200) based on reference images will be described later in FIGS. 5 and 6.

[0091] In operation 305, in one embodiment, the electronic device (200) may obtain a background image based on removing at least one first object from first image frames and compositing the image frames from which at least one first object has been removed. The background image may include at least one blurred background object.

[0092] Referring to FIG. 4c, in one embodiment, the electronic device (200) may obtain image frames (430a, 430b, to 430n) from which the first object has been removed, based on removing the first object (e.g., the first object (421a)) from image frames (e.g., first image frames (420a, 420b, to 420n)). The electronic device (200) may obtain image frames (430a, 430b, to 430n) from which the at least one first object has been removed, based on performing image inpainting on each of the image frames containing the first object. For example, the electronic device (200) may select an object of interest (e.g., the first object (421a, 421b, to 421n)) included in the first image frames. The electronic device (200) can extract a selected subject of interest from each of the image frames. The electronic device (200) can fill in the missing area based on applying a background filling approach to each of the image frames. Image inpainting may include a region fill based on content-aware fill. For example, the electronic device (200) can reconstruct the missing area by analyzing the pixels surrounding the missing area after removing the subject of interest. The electronic device (200) can obtain inpainted image frames output from a CNN-based model by, for example, inputting the image frames from which the subject of interest has been removed into a CNN-based model. The electronic device (200) can fill in the background area while preserving background content based on performing image inpainting.An image frame (430a) from which a first object has been removed from a first original image frame (e.g., first original image frame (420a)) may include background objects (433a, 435a, 437a). An image frame (430b) from which a first object has been removed from a second original image frame (e.g., second original image frame (420b)) may include background objects (433b, 435b, 437b). An image frame (430n) from which a first object has been removed from a last original image frame (e.g., last original image frame (420n)) may include background objects (433n, 435n, 437n). The electronic device (200) may obtain a background image (440) based on synthesizing (431) the image frames (430a, 430b, to 430n) from which a first object has been removed. The electronic device (200) can obtain a background image (440) including blurred background objects (441) without applying a blur effect.

[0093] In operation 307, in one embodiment, the electronic device (200) can obtain weighted sums corresponding to each of the image frames by obtaining a weighted sum of at least one pose score associated with the aesthetics of the movement corresponding to at least one first object for each of the image frames including at least one first object.

[0094] Referring to FIG. 4d, in one embodiment, an electronic device (200) can obtain a pose score (451a, 451b, to 451n) corresponding to the first object (421a, 421b, to 421n) for each of the image frames (450a, 450b, to 450n) including the first object (421a, 421b, to 421n). The image frames (450a, 450b, to 450n) may include the first object (421a, 421b, to 421n) extracted from each of the image frames obtained through a camera (e.g., the first image frames (420a, 420b, to 420n)), and may be referred to as "extracted subject frames" or "second image frames." The electronic device (200) can, for example, obtain a pose score (451a) corresponding to a first object (421a) identified from an image frame (450a). The electronic device (200) can obtain a pose score (451b) corresponding to a first object (421b) identified from an image frame (450b). The electronic device (200) can obtain a pose score (451n) corresponding to a first object (421n) identified from an image frame (450n).

[0095] Referring to FIG. 4e, the electronic device (200) can obtain a pose score corresponding to the first object (401) for an image frame (450) containing the first object (401). The pose score may be an indicator of the aesthetics of the movement of the first object (401). Elements associated with the aesthetics of the movement corresponding to the first object (401) may include, for example, at least one of an image structure (453), a head posture (455), or body coordination, and specific elements for evaluating aesthetics are not limited to the examples described above. The electronic device (200) can obtain a pose score corresponding to the first object (401) by using a scoring method based on pose estimation.

[0096] In one embodiment, the electronic device (200) may obtain scores corresponding to body parts of the first object (401) in order to obtain a posture score corresponding to the first object (401). The electronic device (200) may detect key points corresponding to the first object (401). The key points corresponding to the first object (401) may define the image structure (453) of the first object (401). Based on the detected key points, the electronic device (200) may evaluate the structure and coordination of each of the body parts of the first object (401). To evaluate body coordination, the electronic device (200) may obtain the angles formed by the body parts corresponding to the first object (401). For example, the electronic device (200) can analyze the head posture (455) of the first object (401) by measuring an angle (e.g., angle B) corresponding to the head position based on detected key points. The electronic device (200) can obtain angles corresponding to at least some of the body parts of the first object (401) (e.g., at least one of angle A, angle B, angle C, angle D, angle E, angle F, angle G, or angle H). The electronic device (200) can also control the body posture of the first object (401) based on changing the angles. The electronic device (200) can calculate the angles by using a cosine formula based on the coordinate values ​​corresponding to each of the key points, and the specific method by which the electronic device (200) calculates the angles associated with the body parts corresponding to the first object (401) is not limited thereto. The electronic device (200) can obtain scores (or reference scores) corresponding to each of the angles by comparing the angles with a reference range (or reference value) for different postures. The reference range may be calculated in advance, and there are no limitations thereon.Each of the scores may have a value between 0 and 1, for example. The electronic device (200) may obtain a posture score corresponding to the first object (401) based on information output from the regression model by inputting scores corresponding to body parts of the first object (401) into a regression model (or classification model). The higher the posture score, the higher the aesthetic quality of the first object (401) may be evaluated.

[0097] Referring again to FIG. 4d, the electronic device (200) can obtain a pose score corresponding to the first object as a weighted sum of at least one pose score based on identifying only one first object from each of the image frames containing the first object. The electronic device (200) can obtain weighted sums corresponding to each of the image frames containing the first object. The electronic device (200) can select the image frame having the largest weighted sum among the image frames. The electronic device (200) may also determine the object having the best pose based on analyzing different aspects such as image structure, head pose, or body coordination among the original image frames (e.g., original image frames (420a, 420b, to 420n)). The electronic device (200) can identify an image frame (460) containing the first object (461) having the best pose among the weighted sums corresponding to each of the image frames containing the first object, for example, by confirming that the image frame (450n) has the largest pose score (451n). The image frame having the largest weighted sum may be referred to as the "third image frame." The electronic device (200) can identify a single frame containing the first object (461) that is most suitable for acquiring a panning image. The electronic device (200) can automatically identify an image frame with excellent aesthetics (e.g., image frame (460)) by acquiring the weighted sums corresponding to each of the image frames, in contrast to the case where the image frame having the best pose is manually selected by the user (or photographer).

[0098] In one embodiment, the electronic device (200) may identify an image frame (460) having the largest weighted sum of pose scores based on information output by the artificial intelligence model by inputting image frames (450a, 450b, to 450n) containing a first object (421a, 421b, to 421n) into an artificial intelligence model based on supervised learning. The information output from the artificial intelligence model may include, for example, weighted sums of pose scores corresponding to each of the image frames (450a, 450b, to 450n). The electronic device (200) may identify an image frame (460) having the largest weighted sum among the weighted sums among the image frames (450a, 450b, to 450n). The image frame (460) having the largest weighted sum may include a clear image of a fast-moving subject.

[0099] In operation 309, in one embodiment, the electronic device (200) can obtain a panning image by synthesizing an image frame and a background image having the largest weighted sum among the obtained weighted sums.

[0100] Referring to FIG. 4f, the electronic device (200) can obtain a panning image (470) by synthesizing an image frame (460) having the largest weighted sum and a background image (440). The electronic device (101) can obtain a panning image (470) by synthesizing (453) an image frame (460) containing a relatively clear first object (461) to a background image (440) containing relatively blurry background objects (441). The electronic device (200) can obtain a panning image with excellent aesthetics of a moving subject by synthesizing an image frame containing the first object (461) with the highest pose score to the background image (440).

[0101] FIG. 5 is a flowchart illustrating a method for identifying a main object based on motion vectors. An embodiment of FIG. 5 will be described with reference to FIG. 6. FIG. 6 is a diagram illustrating a method for identifying a main object based on motion vectors.

[0102] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Depending on the implementation, certain operations may be omitted. The operations described in FIG. 5 may be at least partially identical or similar to operation 303 of FIG. 3, and redundant descriptions may not be repeated here.

[0103] Referring to FIG. 5, in operation 501, in one embodiment, an electronic device (200) (e.g., processor (240) of FIG. 2) can identify at least two reference images, including a first reference image and a second reference image, among the image frames.

[0104] Referring to FIG. 6, in one embodiment, an electronic device (200) may select the first original image frame as the first reference image (610) among a plurality of image frames (e.g., original image frames (420a, 420b, to 420n) or first image frames) acquired using a camera (e.g., camera (210) of FIG. 2). The plurality of image frames may be acquired, for example, while the camera is moving. The electronic device (200) may select the last original image frame as the second reference image (620) among the plurality of image frames. The electronic device (200) may select other image frames among the plurality of image frames as reference image frames, without being limited to the examples described above.

[0105] In operation 503, in one embodiment, the electronic device (200) can obtain motion information corresponding to a plurality of objects by checking the difference between a first position and a second position. The motion information may include, for example, motion vector values, but is not limited thereto. The electronic device (200) can obtain motion vector values ​​corresponding to a plurality of objects by checking the difference between a first position corresponding to each of a plurality of objects in a first reference image and a second position corresponding to each of a plurality of objects in a second reference image.

[0106] Referring again to FIG. 6, the electronic device (200) can identify a plurality of objects (601, 603, 605, 607) included in the first reference image (610). The electronic device (200) can identify a first location (611, 613, 615, 617) corresponding to each of the plurality of objects within the first reference image (610). The electronic device (200) can identify a second location (621, 623, 625, 627) corresponding to each of the plurality of objects within the second reference image (620). The electronic device (200) can obtain motion vector values ​​corresponding to a plurality of objects by checking the difference (631, 633, 635, 637) between a first position (611, 613, 615, 617) and a second position (621, 623, 625, 627) for each of the plurality of objects. For example, the electronic device (200) can obtain a motion vector value corresponding to an object (601) by checking the difference (631) between a first position (611) and a second position (621) corresponding to an object (601). The electronic device (200) can obtain a motion vector value corresponding to an object (603) by checking the difference (633) between a first position (613) and a second position (623) corresponding to an object (603). The electronic device (200) can obtain a motion vector value corresponding to the object (605) by checking the difference (635) between the first position (615) and the second position (625) corresponding to the object (605). The electronic device (200) can obtain a motion vector value corresponding to the object (607) by checking the difference (637) between the first position (617) and the second position (627) corresponding to the object (607).

[0107] In operation 505, in one embodiment, the electronic device (200) can identify at least one first object based on comparing motion vector values ​​with a first threshold value. The first threshold value may be set to an appropriate value to determine an object that is required to be clearly represented within a panning image. Referring to FIG. 6, the electronic device (200) can identify the first object (601) based on checking whether each of the acquired motion vector values ​​is smaller than the first threshold value. For example, a user of the electronic device (200) can photograph the subject of interest in panning mode by moving the electronic device (200) along the direction (602) in which the subject of interest (e.g., the first object (601)) is moving. In the image frames captured in panning mode, the position change of the subject of interest may be smaller than the position change of other objects. The electronic device (200) can identify the first object (601) by comparing each of the acquired motion vector values ​​with the set first threshold value. The electronic device (200) can identify the first object (601) based on the fact that among a plurality of objects (601, 603, 605, 607), there is one object having a motion vector value smaller than the first threshold value. The electronic device (200) can also identify the first object (601) having the smallest motion vector value among the acquired motion vector values ​​among the plurality of objects (601, 603, 605, 607). The electronic device (200) can identify the objects (603, 605, 605) having motion vector values ​​larger than the first threshold value as background objects.

[0108] FIG. 7 is a flowchart illustrating a method for identifying multiple main objects based on relative motion vectors. An embodiment of FIG. 7 will be described with reference to FIG. 8. FIG. 8 is a diagram illustrating a method for identifying multiple main objects based on relative motion vectors.

[0109] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Depending on the implementation, certain operations may be omitted. The operations described in FIG. 7 may be at least partially identical or similar to operation 303 of FIG. 3, and redundant descriptions may not be repeated here.

[0110] Referring to FIG. 7, in operation 701, in one embodiment, an electronic device (200) (e.g., processor (240) of FIG. 2) may determine the object having the smallest motion vector value among the motion vector values ​​as the primary object. The electronic device (200) may determine the object having the smallest motion vector value among the acquired motion vector values ​​among at least one first object as the primary object.

[0111] Referring to FIG. 8, in one embodiment, an electronic device (200) can obtain motion vector values ​​corresponding to a plurality of objects (801, 803, 805, 807, 809) based on two reference images (810, 820). The electronic device (200) can obtain a motion vector value corresponding to an object (801) by checking the difference (831) between a first position (811) and a second position (821) corresponding to an object (801). The difference between the first position and the second position may be referred to as a motion vector. The electronic device (200) can obtain a motion vector value corresponding to an object (803) by checking the difference (833) between a first position (813) and a second position (823) corresponding to an object (803). The electronic device (200) can obtain a motion vector value corresponding to the object (805) by checking the difference (835) between the first position (815) and the second position (825) corresponding to the object (805). The electronic device (200) can obtain a motion vector value corresponding to the object (807) by checking the difference (837) between the first position (817) and the second position (827) corresponding to the object (807). The electronic device (200) can obtain a motion vector value corresponding to the object (809) by checking the difference (839) between the first position (819) and the second position (829) corresponding to the object (809). The electronic device (200) can determine, among a plurality of objects (801, 803, 805, 807, 809), the objects (801, 803) having a motion vector value smaller than a first threshold value among the acquired motion vector values ​​as the first objects. The electronic device (200) can determine, among the first objects, the object (801) having the smallest motion vector value among the motion vector values ​​as the primary object.The electronic device (200) may determine the object (801) having the smallest motion vector value among the motion vector values ​​among a plurality of objects (801, 803, 805, 807, 809) as the primary object. In one embodiment, reference image frames (810, 820) may be acquired while a camera (e.g., camera (210)) moves in a direction corresponding to the direction (802, 804) in which the subjects of interest (801, 803) are moving. While shooting is performed in panning mode, the motion vector values ​​of the subjects of interest (801, 803) may be calculated to be relatively smaller than the motion vector values ​​of other subjects due to the camera moving along the subjects of interest (801, 803). For example, the motion vector values ​​of stationary subjects (805, 807, 809) may be calculated to be relatively larger than the motion vector values ​​of subjects of interest (801, 803). The electronic device (200) may determine the stationary subjects (805, 807, 809) as background objects based on confirming that the motion vector values ​​of the stationary subjects (805, 807, 809) are smaller than a first threshold value.

[0112] In operation 703, in one embodiment, the electronic device (200) can determine at least one relative motion vector value corresponding to at least one remaining object excluding the primary object. The electronic device (200) can obtain at least one relative motion vector value corresponding to at least one remaining object excluding the primary object among at least one first object.

[0113] In one embodiment, the relative motion vector value may be a measurement of the relative movement with respect to the primary object. The electronic device (200) may obtain a relative motion vector corresponding to each of the remaining objects (803, 805, 807, 809) excluding the primary object (801) among a plurality of objects (801, 803, 805, 807, 809). Referring again to FIG. 8, for example, the electronic device (200) may obtain a relative motion vector (841) corresponding to the object (803) based on the difference between the motion vector (831) corresponding to the primary object (801) and the motion vector (833) corresponding to the remaining object (803).

[0114] In operation 705, in one embodiment, the electronic device (200) can determine whether at least one object having a relative motion vector value smaller than a second threshold value is identified. The specific numerical value of the second threshold value for determining the secondary object may be changed according to the embodiment. The secondary object may be an object of interest that moves at a speed similar to that of the primary object among a plurality of objects of interest. Referring to FIG. 8, in one embodiment, the electronic device (200) can determine that the relative motion vector value corresponding to the remaining object (803) excluding the primary object (801) among the objects of interest (801, 803) is smaller than the second threshold value. The electronic device (200) can determine that each of the motion vector values ​​corresponding to the background objects (805, 807, 809) is larger than the second threshold value.

[0115] In operation 707, in one embodiment, the electronic device (200) may determine at least one object having a relative motion vector value smaller than the second threshold as a secondary object based on identifying at least one object having a relative motion vector value smaller than the second threshold (operation 705: yes). Referring to FIG. 8, the electronic device (200) may determine an object (803) having a relative motion vector (841) value smaller than the second threshold as a secondary object.

[0116] In one embodiment, the subjects of interest may move in the same direction. The speeds at which the subjects of interest move may differ from each other. In one embodiment, the user of the electronic device (200) may want to obtain panning images of multiple subjects moving at different speeds. The user of the electronic device (200) may photograph the subjects of interest by moving the camera in a direction corresponding to the direction in which the subjects of interest are moving in the panning mode of the electronic device (200). The electronic device (200) may determine a primary object (801) and a secondary object (803) among a plurality of objects (801, 803, 805, 807, 809) based on reference images (810, 820) among image frames obtained through the camera. The electronic device (200) may determine the object having the smallest motion vector value as the primary object (801). The electronic device (200) can determine, among a plurality of objects (803, 805, 807, 809) excluding the primary object (801), an object having a relative motion vector value smaller than the second threshold value as the secondary object (803).

[0117] FIG. 9 is a flowchart illustrating a method for obtaining weighted sums of pose scores corresponding to image frames according to one embodiment. The embodiment of FIG. 9 will be described with reference to FIG. 10, FIG. 11, and FIG. 12. FIG. 10 is a diagram illustrating a method for obtaining weighted sums of pose scores corresponding to image frames according to one embodiment. FIG. 11 is a diagram illustrating a method for obtaining a weighted sum of pose scores corresponding to an image frame based on a layer corresponding to an object according to one embodiment. FIG. 12 is a diagram illustrating a method for obtaining a weighted sum of pose scores corresponding to an image frame according to one embodiment.

[0118] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Depending on the implementation, certain operations may be omitted.

[0119] Referring to FIG. 9, in operation 901, in one embodiment, an electronic device (200) (e.g., processor (240) of FIG. 2) may acquire a first pose score corresponding to a primary object (e.g., primary object (801)) and a second pose score corresponding to at least one secondary object (e.g., secondary object (803)) for each of the image frames (e.g., second image frames) containing at least one first object (e.g., main object). The primary object may be an object having the smallest motion vector value among the acquired motion vector values ​​among a plurality of objects included in the reference image. The secondary object may be an object having a relative motion vector value smaller than a threshold value (e.g., second threshold value) among the acquired relative motion vector values ​​among a plurality of objects.

[0120] Referring to FIG. 10, the electronic device (200) can obtain a first pose score (1021a, 1021b, to 1021n) corresponding to a primary object (1011a, 1011b, to 1011n) and a pose score corresponding to a secondary object (1013a, 1013b, to 1013n) for each of the second image frames (1010a, 1010b, to 1010n). The electronic device (200) can obtain a first pose score and a second pose score for each of the image frames (1010a, 1010b, to 1010n) containing the first objects in order to select an image frame with excellent aesthetics (e.g., a third image frame (460)). The specific method by which the electronic device (200) obtains a pose score corresponding to an object is described in detail in FIG. 4e, so redundant descriptions may not be repeated here.

[0121] In operation 903, in one embodiment, the electronic device (200) can obtain a plurality of weighted sums corresponding to each of image frames containing at least one first object by obtaining a weighted sum of a first pose score and at least one second pose score. Referring to FIG. 10, the electronic device (200) can obtain a weighted sum (1031a) of a first pose score (1021a) and a second pose score (1023a) corresponding to an image frame (1010a). The electronic device (200) can obtain a weighted sum (1031b) of a first pose score (1021b) and a second pose score (1023b) corresponding to an image frame (1010b). In the same way, the electronic device (200) can obtain weighted sums of a first pose score and a second score corresponding to image frames containing at least one first object. For example, the electronic device (200) can obtain a weighted sum (1031n) of a first pose score (1021n) and a second pose score (1023n) corresponding to an image frame (1010n).

[0122] Referring to reference numeral 1110 of FIG. 11, in one embodiment, an electronic device (200) can determine a weight corresponding to each of a plurality of objects (1111, 1113, 1115, 1117, 1119) identified within an image frame. The electronic device (200) can determine a primary object (1111) and secondary objects (1113, 1115, 1117, 1119) among the plurality of objects.

[0123] When the electronic device (200) simply sums the pose scores corresponding to each of the plurality of (1111, 1113, 1115, 1117, 1119) without weighting, an image frame containing a primary object having a low pose score may be selected as the image frame having the best pose. For example, if the average of the pose scores corresponding to the secondary objects is greater than the pose score corresponding to the primary object, an image frame containing a primary object having a low pose score may be selected as the image frame having the best pose. Referring to reference numeral 1120, the electronic device (200) may apply a higher weight to the primary object (1121) than to the secondary objects (1123, 1125, 1127, 1129) in order to give priority to an image frame containing a primary object (1121) having a high pose score.

[0124] When an electronic device (200) selects an image frame to be composited onto a background image without considering whether each of the secondary objects (1113, 1115, 1117, 1119) within the image frame is located in the front layer or the rear layer, a subject of less interest (e.g., a secondary object in the rear layer) in the panning image may be represented more clearly than other subjects (e.g., a secondary object in the front layer). For example, if the pose score corresponding to the secondary object in the rear layer is greater than the pose score corresponding to the secondary object in the front layer, the image frame containing the secondary object in the front layer having the lower pose score may be selected as the image frame having the best pose. Referring to reference numeral 1130, the electronic device (200) can identify the layer corresponding to each of the secondary objects (1133, 1135, 1137, 1139) in order to give priority to the secondary objects of the front layer. For example, the electronic device (200) can identify that among a plurality of layers (1130a, 1130b, 1130c, 1130d, 1130e, 1130f, 1130g), the secondary object (1139) is located in the frontmost layer (e.g., the third layer (1130c)). The electronic device (200) can identify that the secondary objects (1133, 1135) are located in the backmost layer (e.g., the seventh layer (1130g)). The 7th layer (1130g) may also be referred to as the "last layer." The electronic device (200) can confirm that the secondary object (1137) is located on the 6th layer (1130f). If the secondary object is located on the front layer rather than the back layer, it may be visible closer to the user of the electronic device (200) (or the photographer of the panning image).The secondary object (1139) of the front layer may be a more important secondary object than the secondary objects (1133, 1135, 1137) of the back layer.

[0125] In one embodiment, the electronic device (200) can calculate a weighted pose score (WPS) corresponding to an image frame based on assigning the largest weight to the primary object (1131) and assigning a larger weight to the secondary object (1139) of the front layer than to other secondary objects (1133, 1135, 1137). The electronic device (200) can assign the smallest weight to the secondary objects (1133, 1135) of the last layer. The same weight may be assigned to the secondary objects (1133, 1135) on the same layer. For example, the pose score (PSp) corresponding to the primary object (1131) may be 0.7. The pose score (PS1) corresponding to the secondary object (1139) on the third layer (1130c) may be 0.5. The attitude score (PS2) corresponding to the secondary object (1137) on the 6th layer (1130f) may be 0.5. The attitude score (PS3) corresponding to the secondary object (1133) on the 7th layer (1130g) may be 0.7. The attitude score (PS4) corresponding to the secondary object (1135) on the 7th layer (1130g) may be 0.8. The specific numerical values ​​of the attitude scores corresponding to the objects in FIG. 11 are for convenience of explanation and may be calculated differently depending on the embodiment.

[0126] In one embodiment, the electronic device (200) may initialize the weight (h) to 1 to assign a weight corresponding to each of the secondary objects (1133, 1135, 1137, 1139). The electronic device (200) may sequentially increase the weights from the furthest back layer to the front layer. For example, the weight (W3) corresponding to the secondary object (1133) on the seventh layer (1130g) may be 1. The weight (W4) corresponding to the secondary object (1135) on the seventh layer (1130g) may be 1. The electronic device (200) may update the weights based on Equation 1.

[0127] [Mathematical Formula 1]

[0128] h = h * n + 1

[0129] n may be the number of secondary objects on the current layer. Based on the fact that the weight assigned to the 7th layer (1130g) is 1 and the number of secondary objects on the 7th layer (1130g) is 2, the electronic device (200) may update the weight (W2) corresponding to the secondary object (1137) on the 6th layer (1130f) to 3. Based on the fact that the weight assigned to the 6th layer (1130f) is 3 and the number of secondary objects on the 6th layer (1130f) is 1, the electronic device (200) may update the weight (W1) corresponding to the secondary object (1139) on the 3rd layer (1130c) to 4. After determining the weight (W1) corresponding to the secondary object (1139) on the front layer (e.g., the third layer (1130c)), the electronic device (200) can determine the weight (Wp) corresponding to the primary object (1131) based on Equation 1. Based on the weight assigned to the third layer (1130c) being 4 and the number of secondary objects on the third layer (1130c) being 1, the electronic device (200) can update the weight (Wp) corresponding to the primary object (1131) to 5. The electronic device (200) can determine the weights so that the primary object (1131) has the largest weight. The electronic device (200) can determine weights such that the secondary object (1129) on the front layer has a greater weight than the secondary objects (1123, 1125, 1127, 1129) on the back layer. In one embodiment, the electronic device (200) may determine each of the weights such that the sum of the weights is 1. The electronic device (200) can obtain a weighted sum (e.g., WPS) corresponding to an image frame based on Equation 2.

[0130] [Mathematical Formula 2]

[0131] WPS = (Wp * PSp) + (W1 * PS1) + (W2 * PS2) + (W3 * PS3) + (W4 * PS4)

[0132] The electronic device (200) can obtain a weighted sum of 8.5 of the pose scores corresponding to the image frame based on the obtained weights and Equation 2. Referring again to FIG. 10, the electronic device (200) can identify a third image frame (1040) having the largest weighted sum (e.g., weighted sum (1031b)) among the weighted sums (1031a, 1031b, to 1031n) obtained based on the scoring method described in FIG. 11. The third image frame (1040) may include objects (1041, 1043) that cause the largest weighted sum to be calculated. Referring to FIG. 12, in one embodiment, the electronic device (200) can obtain a weighted sum corresponding to an image frame based on selecting a plurality of subjects of interest (1211, 1213, 1215) among a plurality of objects (1211, 1213, 1215, 1217) included in the image frame.

[0133] Referring to reference numeral 1210, the electronic device (200) can identify the primary object (1211) having the smallest motion vector value among a plurality of objects (1211, 1213, 1215, 1217) included in an image frame. The electronic device (200) can identify the objects (1213, 1215) having relative motion vector values ​​less than a set threshold among the remaining objects (1213, 1215, 1217) as secondary objects.

[0134] Referring to reference numeral 1220, the electronic device (200) can obtain a weighted sum of pose scores corresponding to an image frame based on assigning weights corresponding to each of the main objects (1221, 1223, 1225). To obtain the weighted sum of pose scores, the electronic device (200) can obtain pose scores corresponding to a plurality of main objects (1221, 1223, 1225). For example, the electronic device (200) can obtain a pose score (PS2) of 0.8 corresponding to the primary object (1221). The electronic device (200) can obtain a pose score (PS1) of 0.4 corresponding to the secondary object (1225). The electronic device (200) can obtain a pose score (PS0) of 0.7 corresponding to the secondary object (1223). The electronic device (200) can confirm that a secondary object (1225) is located on a front layer (e.g., layer zero) among a plurality of layers. The electronic device (200) can confirm that a secondary object (1223) is located on a back layer (e.g., layer 2). The electronic device (200) can confirm that a primary object (1221) is located on layer 1. Based on Equation 1, the electronic device (200) can determine a weight (W0) corresponding to the secondary object (1223) on the back layer to be 1. The electronic device (200) can determine a weight (W1) corresponding to the secondary object (1225) on the front layer to be 2. The electronic device (200) can determine a weight (W2) corresponding to the primary object (1221) to be 3. The electronic device (200) can determine the weights so that the primary object (1221) has the largest weight. The electronic device (200) can determine the weights so that the secondary object (1225) on the front layer has a larger weight than the secondary object (1223) on the back layer.The electronic device (200) can obtain a weighted sum of attitude scores based on mathematical formula 3 using the obtained attitude scores and determined weights.

[0135] [Mathematical Formula 3]

[0136]

[0137] n may be the number of main objects. i may be an index corresponding to a main object. The electronic device (200) may obtain a weighted sum of pose scores corresponding to an image frame, 3.9, according to Equation 3. The electronic device (200) may identify an image frame containing main objects that have the best overall pose based on obtaining weighted sums for each of the image frames. The electronic device (200) may obtain a panning image containing sharply rendered main objects based on compositing the image frame with the largest weighted sum onto a background image (e.g., background image (440)).

[0138] FIG. 13 is a flowchart illustrating a method for acquiring a panning image based on attitude correction according to one embodiment. The embodiment of FIG. 13 will be described with reference to FIG. 14a and FIG. 14b. FIG. 14a and FIG. 14b are drawings illustrating a method for acquiring a panning image based on attitude correction according to one embodiment.

[0139] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Depending on the implementation, certain operations may be omitted.

[0140] Referring to FIG. 13, in operation 1301, in one embodiment, an electronic device (200) (e.g., processor (240) of FIG. 2) can acquire at least one candidate image with a modified pose. Referring to FIG. 14a, in one embodiment, the electronic device (200) can acquire at least one candidate image by inputting an image frame (e.g., a third image frame) having the largest weighted sum to an artificial intelligence model (1410) trained to output an image frame with a modified pose based on receiving an image frame as input. The candidate image may be, for example, an image containing a main object having at least one reference pose stored in the electronic device (200). The artificial intelligence model (1410) may be a generative AI model trained to convert a given reference image into a target pose. The generative AI model may be implemented, for example, as a generative adversarial network (GAN) including a generator and a discriminator, and there are no restrictions on the specific implementation method. The generative AI model may adjust a set reference pose or generate a main object having a new pose different from the poses corresponding to the image frames (e.g., source poses). The generative AI model may change the pose of the main object for aesthetic purposes. The electronic device (200) may, for example, select an image frame having the largest weighted sum of pose scores among the image frames containing the main object. If an image frame with excellent aesthetic quality is not identified, the electronic device (200) may input at least some of the image frames containing the main object into the artificial intelligence model (1410) to obtain at least one candidate image with a modified pose based on information output from the artificial intelligence model (1410).

[0141] Referring to FIG. 14b, in one embodiment, the electronic device (200) can acquire at least one candidate image (1413) based on information output from the artificial intelligence model by inputting an image frame (1411) into the artificial intelligence model. The electronic device (200) can identify key points corresponding to the primary object included in the third image frame. Since the method of acquiring key frames from the image frame has been described in FIG. 4e, redundant descriptions may not be repeated here. Key points corresponding to the best pose can be stored in the electronic device (200) as a reference pose. The electronic device (200) can measure the degree of deformation of the primary object based on comparing the key points corresponding to the primary object with the key points of the reference pose. The artificial intelligence model can output an image with a modified pose corresponding to the image containing the main object having the reference pose, based on receiving the image frame, using the degree of deformation.

[0142] In operation 1303, in one embodiment, the electronic device (200) may display at least one candidate image with a modified pose through a display (e.g., display (220)). The electronic device (200) may obtain at least one candidate image based on output information output from a generative AI model by inputting an image frame having the largest weighted sum into a generative AI model. The modified poses may be provided in the form of a list suggested to the user.

[0143] In operation 1305, in one embodiment, the electronic device (200) can obtain a panning image by compositing a selected candidate image onto a background image based on checking user input. Referring to FIG. 14b, the electronic device (200) can check user input (1415). The user input can be a user input for selecting any one of at least one provided candidate image. The user input may include an input for dragging at least some of the key points through the user input. The electronic device (200) can modify the pose of the main object by adjusting the position (or coordinates) of the key points based on the user input. The electronic device (200) can obtain a panning image by compositing a selected candidate image onto a background image. The electronic device (200) can obtain a panning image including the main object with the modified pose by compositing a candidate image corresponding to the user input with the background image.

[0144] In one embodiment, the electronic device (200) may provide an acquired panning image (1417). For example, the electronic device (200) may display the panning image through a display. The electronic device (200) may also perform additional posture correction based on user input regarding the displayed panning image.

[0145] FIG. 15 is a flowchart illustrating a method for providing a panning image based on an image enhancement function according to one embodiment.

[0146] In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. Depending on the implementation, certain operations may be omitted.

[0147] Referring to FIG. 15, in operation 1501, in one embodiment, an electronic device (200) (e.g., processor (240) of FIG. 2) can determine whether image frames (e.g., first image frames acquired through a camera) are acquired in a low-light environment.

[0148] In one embodiment, the electronic device (200) can determine whether image frames were acquired in a low-light environment. The electronic device (200) can determine whether the brightness of an image frame is lower than a set threshold brightness value based on checking the pixel values ​​included in the image frame.

[0149] In operation 1503, in one embodiment, the electronic device (200) may apply an image enhancement function to the image frames based on confirming that the image frames are acquired in a low-light environment (operation 1501: yes).

[0150] In one embodiment, the electronic device (200) may apply an image enhancement function to image frames based on confirming that the image frames were acquired in a low-light environment. The electronic device (200) may increase the brightness of the image frames based on applying an image enhancement function to the image frames, taking into account that an image with low brightness may be acquired when the image is acquired with a fast shutter speed.

[0151] In operation 1505, in one embodiment, the electronic device (200) can identify main objects included in image frames based on motion information. The specific method by which the electronic device (200) identifies main objects is described in detail in FIGS. 5 to 8, so redundant descriptions may not be repeated.

[0152] In operation 1507, in one embodiment, the electronic device (200) may acquire a background image based on removing main objects from image frames. The specific method by which the electronic device (200) acquires the background image is described in detail in FIG. 4c, so redundant descriptions may not be repeated.

[0153] In operation 1509, in one embodiment, the electronic device (200) can identify the image frame having the largest weighted sum among the image frames based on obtaining a weighted sum of pose scores for each of the image frames. The specific method by which the electronic device (200) identifies the image frame having the largest weighted sum is described in detail in FIGS. 9 and FIGS. 10, so redundant descriptions may not be repeated.

[0154] In operation 1511, in one embodiment, the electronic device (200) can obtain a panning image by synthesizing an image frame having the largest weighted sum to a background image. The specific method by which the electronic device (200) obtains the panning image is described in detail in FIGS. 4a to 4f, so redundant descriptions may not be repeated.

[0155] In one embodiment, the electronic device (200) can apply an image enhancement function to image frames captured in a low-light environment. The electronic device (200) can identify image frames with excellent aesthetics of the main object based on a pose score. The electronic device (200) can acquire a background image including a blurred background object without applying a blur effect. By synthesizing the image frame and the background image having the largest weighted sum, the electronic device (200) can acquire a panning image without interaction with the photographer (e.g., selecting an image frame with excellent aesthetics of the main object).

[0156] An electronic device according to one embodiment of the present disclosure may include a camera and at least one processor. The at least one processor may be configured to acquire motion information corresponding to a plurality of objects included in image frames acquired through the camera. The at least one processor may be configured to identify at least one first object among the plurality of objects based on the acquired motion information. The at least one processor may be configured to acquire a background image including at least one blurred background object based on removing the at least one first object from the image frames and synthesizing the image frames from which the at least one first object has been removed. The at least one processor may be configured to acquire weighted sums corresponding to each of the image frames including the at least one first object by acquiring a weighted sum of at least one pose score associated with the aesthetics of the movement corresponding to the at least one first object for each of the image frames including the at least one first object among the acquired image frames. The above at least one processor may be configured to acquire a panning image by synthesizing the image frame having the largest weighted sum among the acquired weighted sums and the background image among the image frames including the at least one first object.

[0157] In one embodiment, the at least one processor may be further configured to identify at least two reference images, including a first reference image and a second reference image, among the image frames. The at least one processor may be further configured to obtain motion vector values ​​corresponding to the plurality of objects by identifying the difference between a first position corresponding to each of the plurality of objects in the first reference image and a second position corresponding to each of the plurality of objects in the second reference image. The at least one processor may be further configured to identify the at least one first object based on comparing the motion vector values ​​with a first threshold value.

[0158] In one embodiment, the at least one processor may be further configured to determine, among the at least one first object, the object having the smallest motion vector value among the acquired motion vector values ​​as the primary object. The at least one processor may be further configured to check at least one relative motion vector value corresponding to at least one object excluding the primary object among the at least one first object. The at least one processor may be further configured to determine, among the at least one relative motion vector value, at least one object having a relative motion vector value smaller than a second threshold value as at least one secondary object. The relative motion vector value may be a measurement of movement relative to the primary object.

[0159] In one embodiment, the at least one processor may be further configured to check a first user input for acquiring a panning image. The at least one processor may be further configured to control the camera to acquire the image frames based on the first user input. The image frames may be images captured based on the movement of the camera.

[0160] In one embodiment, the at least one first object may include a primary object and at least one secondary object. The at least one processor may be further configured to obtain a first pose score corresponding to the primary object and at least one second pose score corresponding to the at least one secondary object for each of the image frames including the at least one first object. The at least one processor may be further configured to obtain a plurality of weighted sums corresponding to each of the image frames including the at least one first object by obtaining a weighted sum of the first pose score and the at least one second pose score.

[0161] In one embodiment, the element associated with the aesthetics of the movement corresponding to the at least one first object may include at least one of an image structure, head posture, or body coordination corresponding to the at least one first object.

[0162] In one embodiment, the electronic device may further include a display. The at least one processor may be further configured to acquire at least one candidate image with a modified pose based on information output from the artificial intelligence model by inputting the image frame having the largest weighted sum into the artificial intelligence model. The at least one processor may be further configured to display the at least one candidate image with a modified pose through the display. The at least one processor may be further configured to acquire a panning image by compositing the selected candidate image onto the background image based on confirming user input for selecting one of the at least one candidate images.

[0163] In one embodiment, the at least one processor may be further configured to acquire image frames from which the at least one first object has been removed, based on performing image inpainting on each of the image frames containing the at least one first object. The at least one processor may be further configured to acquire the background image by synthesizing the image frames from which the at least one first object has been removed.

[0164] In one embodiment, the at least one processor may be further configured to set the first shutter speed as the shutter speed of the camera. The at least one processor may be further configured to acquire the image frames based on the first shutter speed.

[0165] In one embodiment, the at least one processor may be configured to determine whether the image frames were acquired in a low-light environment. The at least one processor may be configured to apply an image enhancement function to the image frames based on determining that the image frames were acquired in a low-light environment.

[0166] According to one embodiment, the method may include the operation of obtaining motion vector values ​​corresponding to a plurality of objects included in image frames obtained through a camera of an electronic device. The method may include the operation of identifying at least one first object among the plurality of objects based on the obtained motion information. The method may include the operation of obtaining a background image including at least one blurred background object by removing the at least one first object from the image frames and synthesizing the image frames from which the at least one first object has been removed. The method may include the operation of obtaining weighted sums corresponding to each of the image frames including the at least one first object by obtaining a weighted sum of at least one pose score associated with the aesthetics of the movement corresponding to the at least one first object for each of the image frames including the at least one first object among the obtained image frames. The method may include the operation of obtaining a panning image by synthesizing the image frame having the largest weighted sum among the obtained weighted sums and the background image among the image frames including the at least one first object.

[0167] In one embodiment, the method may further include an operation of identifying at least two reference images, including a first reference image and a second reference image, among the image frames. The method may further include an operation of obtaining motion vector values ​​corresponding to a plurality of objects by identifying a difference between a first position corresponding to each of the plurality of objects in the first reference image and a second position corresponding to each of the plurality of objects in the second reference image. The method may further include an operation of identifying at least one first object based on comparing the motion vector values ​​with a first threshold value.

[0168] In one embodiment, the method may further include an operation of determining, among the at least one first object, the object having the smallest motion vector value among the acquired motion vector values ​​as the primary object. The method may further include an operation of obtaining at least one relative motion vector value corresponding to at least one object excluding the primary object among the at least one first object. The method may further include an operation of determining, among the at least one relative motion vector value, at least one object having a relative motion vector value smaller than a second threshold value as at least one secondary object. The relative motion vector value may be a measurement of movement relative to the primary object.

[0169] In one embodiment, the method may further include an operation of confirming a first user input for acquiring a panning image. The method may further include an operation of controlling the camera to acquire the image frames based on the first user input. The image frames may be images captured based on the movement of the camera.

[0170] In one embodiment, the at least one first object may include a primary object and at least one secondary object. The method may further include the operation of obtaining a first pose score corresponding to the primary object and at least one second pose score corresponding to the at least one secondary object for each of the image frames including the at least one first object. The method may further include the operation of obtaining a plurality of weighted sums corresponding to each of the image frames including the at least one first object by obtaining a weighted sum of the first pose score and the at least one second pose score.

[0171] In one embodiment, the element associated with the aesthetics of the movement corresponding to the at least one first object may include at least one of an image structure, head posture, or body coordination corresponding to the at least one first object.

[0172] In one embodiment, the method may further include the operation of acquiring at least one candidate image with a modified pose based on information output from the artificial intelligence model by inputting the image frame having the largest weighted sum into the artificial intelligence model. The method may further include the operation of displaying the at least one candidate image with a modified pose through the display of the electronic device. The method may further include the operation of acquiring a panning image by compositing the selected candidate image onto the background image based on confirming user input for selecting one of the at least one candidate image.

[0173] In one embodiment, the method may further include the operation of obtaining image frames from which the at least one first object has been removed, based on performing image inpainting on each of the image frames containing the at least one first object. The method may further include the operation of obtaining the background image from which the at least one main object has been removed, based on compositing the image frames from which the at least one main object has been removed.

[0174] In one embodiment, the method may further include an operation of setting a first shutter speed as the shutter speed of the camera. The method may further include an operation of acquiring the image frames based on the first shutter speed.

[0175] According to one embodiment, in a storage medium storing computer-executable instructions, the instructions may cause the electronic device to perform at least one operation when executed by at least one processor of the electronic device. The at least one operation may include an operation of acquiring motion information corresponding to a plurality of objects included in image frames acquired through a camera of the electronic device. The at least one operation may include an operation of identifying at least one first object among the plurality of objects based on the acquired motion information. The at least one operation may include an operation of acquiring a background image including at least one blurred background object by removing the at least one first object from the image frames and synthesizing the image frames from which the at least one first object has been removed. The at least one operation may include an operation of acquiring weighted sums corresponding to each of the image frames including the at least one first object by acquiring a weighted sum of at least one pose score associated with the aesthetics of the movement corresponding to the at least one first object for each of the image frames including the at least one first object among the acquired image frames. The above at least one operation may include the operation of acquiring a panning image by synthesizing the image frame having the largest weighted sum among the acquired weighted sums among the image frames including the at least one first object and the background image.

[0176] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs.

[0177] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0178] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0179] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0180] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0181] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0182] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device, camera; Memory containing computer-executable instructions; and It includes at least one processor, When the above instructions are executed individually or integrally by the at least one processor, the electronic device: Motion information corresponding to a plurality of objects included in the first image frames acquired through the above camera is obtained, and Among the plurality of objects above, at least one first object is identified based on the acquired motion information, and Based on identifying second image frames containing at least one first object among the first image frames, removing at least one first object from the second image frames, and synthesizing third image frames from which at least one first object has been removed, a background image including at least one blurred background object is obtained. For each of the above second image frames, identify a fourth image frame that satisfies a specified condition associated with the aesthetics of the movement corresponding to at least one first object, and An electronic device characterized by including instructions that cause a panning image to be obtained by synthesizing the identified fourth image frame and the background image.

2. In Paragraph 1, An electronic device characterized by the above-mentioned specified condition including a condition for identifying a fourth image frame having the largest weighted sum among the second image frames based on weighted sums corresponding to each of the second image frames obtained by obtaining a weighted sum of at least one pose score associated with the aesthetics of the movement corresponding to the at least one first object.

3. In Paragraph 1, The above instructions cause the electronic device, Among the first image frames above, at least two reference images including a first reference image and a second reference image are identified, and By determining the difference between a first position corresponding to each of the plurality of objects in the first reference image and a second position corresponding to each of the plurality of objects in the second reference image, motion vector values ​​corresponding to the plurality of objects are obtained, and An electronic device characterized by including instructions that cause at least one first object to be identified based on comparing the motion vector values ​​with a first threshold value.

4. In Paragraph 3, The above instructions cause the electronic device to determine, among the at least one first object, the object having the smallest motion vector value among the acquired motion vector values ​​as the primary object, and Obtaining at least one relative motion vector value corresponding to at least one object excluding the primary object among the at least one first object, and Among the above at least one relative motion vector value, the method includes instructions that cause at least one object having a relative motion vector value smaller than a second threshold value to be determined as at least one secondary object. An electronic device characterized in that the relative motion vector value is a measurement of the relative motion with respect to the primary object.

5. In Paragraph 1, The above instructions cause the electronic device to check a first user input for acquiring a panning image, and Based on the first user input, the method includes instructions that cause the camera to be controlled to acquire the first image frames, and An electronic device characterized in that the first image frames are images captured based on the movement of the camera.

6. In Paragraph 1, The above at least one first object includes a primary object and at least one secondary object, and The above instructions cause the electronic device to obtain, for each of the second image frames including the at least one first object, a first pose score corresponding to the primary object and at least one second pose score corresponding to the at least one secondary object, and An electronic device characterized by including instructions that cause a plurality of weighted sums corresponding to each of the second image frames including the at least one first object to be obtained by obtaining a weighted sum of the first pose score and the at least one second pose score.

7. In Paragraph 1, The elements associated with the aesthetics of the movement corresponding to the above-mentioned at least one first object are, An electronic device characterized by including at least one of an image structure, head posture, or body coordination corresponding to at least one first object.

8. In Paragraph 1, Includes more displays, The above instructions cause the electronic device to acquire at least one candidate image with a modified pose based on information output from the artificial intelligence model by inputting the fourth image frame having the largest weighted sum into the artificial intelligence model, and At least one candidate image with the above posture modified is displayed through the above display, and An electronic device characterized by including instructions that cause a panning image to be obtained by synthesizing a selected candidate image to a background image based on confirming user input for selecting one of the at least one candidate image.

9. In Paragraph 1, The above instructions enable the electronic device to obtain third image frames from which the at least one first object has been removed, based on performing image inpainting on each of the second image frames containing the at least one first object. An electronic device characterized by including instructions that cause the background image to be obtained based on synthesizing third image frames from which at least one first object has been removed.

10. In Paragraph 1, The above instructions cause the electronic device to set a first shutter speed as the shutter speed of the camera, and An electronic device characterized by including instructions that cause the first image frames to be acquired based on the first shutter speed.

11. Regarding the method, An operation of acquiring motion information corresponding to a plurality of objects included in first image frames acquired through a camera of an electronic device; Among the plurality of objects above, an operation of identifying at least one first object based on the acquired motion information; An operation to obtain a background image including at least one blurred background object based on identifying second image frames containing at least one first object among the first image frames, removing at least one first object from the second image frames, and synthesizing third image frames from which at least one first object has been removed; An operation of identifying a fourth image frame that satisfies a specified condition associated with the aesthetics of the movement corresponding to at least one first object for each of the second image frames; and A method characterized by including the operation of obtaining a panning image by synthesizing the identified fourth image frame and the background image.

12. In Paragraph 11, A method characterized by the above-mentioned specified condition including a condition for identifying a fourth image frame having the largest weighted sum among the second image frames based on weighted sums corresponding to each of the second image frames obtained by obtaining a weighted sum of at least one pose score associated with the aesthetics of the movement corresponding to at least one first object.

13. In Paragraph 11, Among the first image frames, an operation of identifying at least two reference images including a first reference image and a second reference image; An operation of obtaining motion vector values ​​corresponding to the plurality of objects by confirming the difference between a first position corresponding to each of the plurality of objects in the first reference image and a second position corresponding to each of the plurality of objects in the second reference image; and A method characterized by further including an operation of identifying at least one first object based on comparing the above motion vector values ​​with a first threshold value.

14. In Paragraph 13, Among the above at least one first object, the operation of determining the object having the smallest motion vector value among the acquired motion vector values ​​as the primary object; Among the above at least one first object, the operation of obtaining at least one relative motion vector value corresponding to at least one remaining object excluding the primary object; and Among the above at least one relative motion vector value, the method further includes the operation of determining at least one object having a relative motion vector value smaller than a second threshold value as at least one secondary object. A method characterized in that the relative motion vector value is a measurement of the relative movement with respect to the primary object.

15. In a non-transient storage medium for storing computer-executable instructions, said instructions cause said electronic device to perform at least one operation when executed individually or integrally by at least one processor of the electronic device, and The above at least one operation is, The operation of acquiring motion information corresponding to a plurality of objects included in first image frames acquired through the camera of the electronic device; Among the plurality of objects above, an operation of identifying at least one first object based on the acquired motion information; An operation of obtaining a background image including at least one blurred background object based on identifying second image frames containing at least one first object from the first image frames, removing at least one first object from the second image frames, and synthesizing third image frames from which at least one first object has been removed; An operation of identifying a fourth image frame that satisfies a specified condition associated with the aesthetics of the movement corresponding to at least one first object for each of the second image frames; and A non-transient storage medium characterized by including an operation to acquire a panning image by synthesizing the identified fourth image frame and the background image.

Images