Electronic device and method for generating video on basis of still image, and non-transitory computer-readable storage medium

Abstract

This electronic device may: acquire two-dimensional information about at least one object from a still image; identify a type of the at least one object related to the three-dimensional motion effect of the at least one object by using the two-dimensional information; and generate, on the basis of a motion model corresponding to the type of the at least one object, a video including the three-dimensional motion effect of the at least one object.

Description

Electronic device, method, and non-temporal computer-readable storage medium for generating video based on still images

[0001] Certain exemplary embodiments may apply to electronic devices, methods, and / or non-transient computer-readable storage media for generating video based on still images.

[0002] An electronic device can provide multiple functions. For example, the electronic device can generate images through an artificial intelligence (AI) model. For example, the electronic device can generate images (e.g., still images or videos) based on inputting text prompts into a generative AI model.

[0003] In certain exemplary embodiments, the electronic device may include at least one processor comprising a processing circuit; and a memory comprising one or more storage media for storing instructions. When the instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause to acquire two-dimensional information about at least one object from a still image. When the instructions are executed individually and / or collectively by the at least one processor, the electronic device may use the two-dimensional information to identify the type of the at least one object associated with a three-dimensional motion effect of the at least one object. When the instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause to generate a video containing the three-dimensional motion effect of the at least one object based on a motion model corresponding to the type of the at least one object.

[0004] According to certain exemplary embodiments, the electronic device may include at least one processor comprising a display and a processing circuit; and a memory comprising one or more storage media for storing instructions. When the instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause the electronic device to identify an object in a still image comprising one or more objects. When the instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause the electronic device to identify a motion model corresponding to motion attributes to be applied to a three-dimensional object corresponding to the identified object. The motion attributes may include one or more types of motion among a plurality of motion types. When the instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause the electronic device to display a first frame containing the three-dimensional object corresponding to the object at a first position through the display. When the above instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause the position of the 3D object to be updated according to the motion information of the 3D object from the motion model. When the above instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause the second frame, which includes the 3D object at a second position instead of a first position according to the motion information, to be displayed after the first frame through the display.

[0005] An exemplary method may be performed by an electronic device and may include the operation of obtaining two-dimensional information about at least one object from a still image by said electronic device. The method may include the operation of identifying the type of said at least one object related to the three-dimensional motion effect of said at least one object using said two-dimensional information. The method may include the operation of generating a video containing the three-dimensional motion effect of said at least one object based on a motion model corresponding to the type of said at least one object.

[0006] An exemplary method may be performed by an electronic device and may include an operation of identifying an object in a still image containing one or more objects. The method may include an operation of identifying a motion model corresponding to a motion attribute to be applied to a three-dimensional object corresponding to the identified object. The motion attribute may include one or more types of motion among a plurality of types of motion. The method may include an operation of displaying a first frame containing the three-dimensional object corresponding to the object at a first position through the display. The method may include an operation of updating the position of the three-dimensional object according to motion information of the three-dimensional object from the motion model. The method may include an operation of displaying a second frame containing the three-dimensional object at a second position instead of the first position according to the motion information through the display after the first frame.

[0007] An exemplary non-transitory computer-readable storage medium may store a program comprising instructions. When the instructions are executed individually and / or collectively by at least one processor, the electronic device may cause the electronic device to acquire two-dimensional information about at least one object from a still image. When the instructions are executed individually and / or collectively by the at least one processor, the electronic device may use the two-dimensional information to identify the type of the at least one object associated with a three-dimensional motion effect of the at least one object. When the instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause the electronic device to generate a video containing the three-dimensional motion effect of the at least one object based on a motion model corresponding to the type of the at least one object.

[0008] An exemplary non-transient computer-readable storage medium may store a program comprising instructions. When the instructions are executed individually and / or collectively by at least one processor, the electronic device may cause the electronic device to identify an object in a still image comprising one or more objects. When the instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause the electronic device to identify a motion model corresponding to motion attributes to be applied to a three-dimensional object corresponding to the identified object. The motion attributes may include one or more types of motion among a plurality of motion types. When the instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause the electronic device to display a first frame containing the three-dimensional object corresponding to the object at a first position through the display. When the above instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause the position of the 3D object to be updated according to the motion information of the 3D object from the motion model. When the above instructions are executed individually and / or collectively by the at least one processor, the electronic device may cause the second frame, which includes the 3D object at a second position instead of a first position according to the motion information, to be displayed after the first frame through the display.

[0009] FIG. 1 is a block diagram of an electronic device in a network environment according to various exemplary embodiments.

[0010] FIG. 2 is a block diagram of an electronic device according to an exemplary embodiment.

[0011] FIG. 3 is a diagram showing the operation of an electronic device generating a video according to an exemplary embodiment.

[0012] FIG. 4 is a drawing showing a still image according to an exemplary embodiment.

[0013] FIG. 5 is a diagram illustrating the operation of generating data related to a 3D object according to an exemplary embodiment.

[0014] FIG. 6a is a drawing showing an example of a frame included in a video according to an exemplary embodiment.

[0015] FIG. 6b is a drawing showing an example of a frame included in a video according to an exemplary embodiment.

[0016] FIG. 7a is a drawing showing an example of a frame included in a video generated by real-time input according to an exemplary embodiment.

[0017] FIG. 7b is a drawing showing an example of a frame included in a video generated by real-time input according to an exemplary embodiment.

[0018] FIG. 8a is a drawing showing an example of a frame included in a video generated by real-time input according to an exemplary embodiment.

[0019] FIG. 8b is a drawing showing an example of a frame included in a video generated by real-time input according to an exemplary embodiment.

[0020] FIG. 9a is a drawing showing an example of a frame included in a video displayed in a gallery application according to an exemplary embodiment.

[0021] FIG. 9b is a drawing showing an example of a frame within specified content of a gallery application according to an exemplary embodiment.

[0022] FIG. 10a is a drawing showing an example of a lock screen displayed in a locked state of an electronic device according to an exemplary embodiment.

[0023] FIG. 10b is a drawing illustrating an example of user input for unlocking an electronic device according to an exemplary embodiment.

[0024] FIG. 11a is a drawing showing an example of a frame displayed according to a notification of an electronic device, according to an exemplary embodiment.

[0025] FIG. 11b is a drawing showing an example of a frame displayed according to a notification of an electronic device, according to an exemplary embodiment.

[0026] FIG. 12 is a flowchart illustrating the operation of an electronic device determining acceleration based on real-time input according to an exemplary embodiment.

[0027] FIG. 13 is a flowchart illustrating the operation of an electronic device according to an exemplary embodiment.

[0028] FIG. 14 is a flowchart illustrating the operation of an electronic device according to an exemplary embodiment.

[0029] FIG. 15 is a drawing showing an example of a frame displayed on a wearable device according to an exemplary embodiment.

[0030] FIG. 16 is a drawing illustrating various types of electronic devices according to exemplary embodiments.

[0031] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments.

[0032] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0033] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof. Each "unit" herein may include a circuit.

[0034] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0035] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0036] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0037] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0038] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0039] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0040] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0041] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0042] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0043] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0044] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0045] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0046] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0047] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0048] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0049] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 664 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 6 ms or less) for realizing URLLC.

[0050] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0051] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0052] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0053] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0054] The electronic device (101) can utilize a technology that estimates 3D (three-dimensional) information from a 2D (two-dimensional) image in relation to a computer vision algorithm. In particular, the electronic device (101) can utilize an algorithm that estimates depth information from a single 2D image and uses it to generate a 3D model. In particular, the electronic device (101) can estimate more precise depth information from a single 2D image by using a large amount of training data through a deep learning-based depth estimation model. In this way, by utilizing a deep learning-based depth estimation model, the electronic device (101) can generate a 3D model by analyzing the shape and spatial arrangement of an object in a single 2D image. Additionally, the electronic device (101) can generate a video by generating and displaying frames rendered from the 3D model generated through graphics technology using the generated 3D model.

[0055] FIG. 2 is a block diagram of an electronic device according to one embodiment. FIG. 3 is a diagram showing an operation in which an electronic device generates a video according to one embodiment. FIG. 4 is a diagram showing a still image according to one embodiment. FIG. 5 is a diagram showing an operation in which data related to a 3D object is generated according to one embodiment. FIG. 6a is a diagram showing an example of a frame included in a video according to one embodiment. FIG. 6b is a diagram showing an example of a frame included in a video according to one embodiment.

[0056] FIGS. 2 to 6b can be explained with reference to FIG. 1.

[0057] Referring to FIG. 2, the electronic device (101) may include a processor (120), memory (130), an audio output module (155), a sensor module (176), and a display (260). In one embodiment, the display (260) may correspond to the display module (160) of FIG. 1.

[0058] In one embodiment, the memory (130) may include, as a program (140), an object recognition module (201), an object separation module (210), an object selection module (215), a three-dimensional modeling module (220), an object classification module (230), a motion information generation module (240), and a rendering module (250). In one embodiment, the object recognition module (201), the object separation module (210), the object selection module (215), the three-dimensional modeling module (220), the object classification module (230), the motion information generation module (240), and / or the rendering module (250) may include instructions. In one embodiment, instructions for an object recognition module (201), an object separation module (210), an object selection module (215), a 3D modeling module (220), an object classification module (230), a motion information generation module (240), and / or a rendering module (250) may be executed by a processor (120). According to an embodiment, at least one additional module may be added to the electronic device (101). According to an embodiment, one or more of the modules included in the electronic device (101) may be integrated into a single module. For example, the object recognition module (201) and the object separation module (210) may be integrated into a single module.

[0059] In one embodiment, the functions performed by at least one module among the object recognition module (201), object separation module (210), 3D modeling module (220), object classification module (230), motion information generation module (240), and / or rendering module (250) may be performed by an artificial intelligence model. For example, the functions of at least one module among the object recognition module (201), object separation module (210), 3D modeling module (220), object classification module (230), motion information generation module (240), and / or rendering module (250) described below may be performed by an artificial intelligence model. For example, the object recognition module (201) may recognize objects in an image through an artificial intelligence model trained to recognize objects in an image. For example, the object separation module (210) may separate objects in an image through an artificial intelligence model trained to separate objects in an image.

[0060] In one embodiment, the electronic device (101) can generate an output image (or video) capable of interacting with a user from an input image (or still image) through an object recognition module (201), an object separation module (210), an object selection module (215), a 3D modeling module (220), an object classification module (230), a motion information generation module (240), and / or a rendering module (250). In one embodiment, the electronic device (101) can generate an output image (or video) with motion effects applied to one or more objects included in the input image (or still image). In one embodiment, regarding the movement of one or more objects, the electronic device (101) can generate an output image (or video) with motion effects applied to one or more objects by using information within the input image (or still image) and / or information obtainable by the electronic device (101). In one embodiment, the input image may be referred to as a 2D (two-dimensional) image, an original image, or at least one frame among one or more frames in a video. In one embodiment, one or more objects may be referred to as an object selected by user input or an object satisfying specified conditions. In one embodiment, the output image may be referred to as a video, a motion image, or an image capable of interacting with the user (e.g., an image capable of playing at least some area (movement)).

[0061] Hereinafter, with reference to FIGS. 3 to 6b, an operation in which an electronic device (101) generates an output image (or video) from an input image (or still image) through an object recognition module (201), an object separation module (210), an object selection module (215), a 3D modeling module (220), an object classification module (230), a motion information generation module (240), and a rendering module (250) may be described. Hereinafter, the object recognition module (201) and the object separation module (210) are exemplified as separate modules, but according to an embodiment, the object recognition module (201) and the object separation module (210) may be integrated into a single module.

[0062] In one embodiment, the object recognition module (201) may receive an input image. In one embodiment, referring to FIG. 3, the object recognition module (201) may receive a still image (310) as an input image. However, the embodiment is not limited thereto. For example, the object recognition module (201) may receive two or more still images as input images. For example, the object recognition module (201) may receive a video (or at least one frame among the frames in the video) as an input image.

[0063] In one embodiment, the object recognition module (201) may receive user input for the input image along with the input image (e.g., still image (310)). In one embodiment, the user input for the input image may be an input for adding at least one object to the input image. For example, the user input for the input image may represent a drawing object for adding an object to the input image. For example, the user input for the input image may include a prompt for adding an object to the input image. In one embodiment, the prompt may include text and / or a drawing describing the object to be added to the input image. For example, an artificial intelligence model may add an object to the input image based on the prompt.

[0064] For example, the object recognition module (201) may receive user input (e.g., drawing, prompt (e.g., text, or drawing)) for adding a butterfly on top of a flower in an input image containing a flower as an object. For example, the object recognition module (201) may add a butterfly around the flower in the input image according to the user input through an artificial intelligence model. In one embodiment, the object added to the input image by user input may be identified as an object capable of interacting with the user. For example, the butterfly object added to the input image by user input may move within the output image in association with the movement of the flower (e.g., shaking due to wind, shaking due to user input, shaking due to the state of the electronic device (101).

[0065] In one embodiment, the object recognition module (201) can recognize one or more objects within a still image (310). For example, referring to FIG. 4, the object recognition module (201) can recognize (or identify) one or more objects (401, 402) and a background (403) within a still image (310). In one embodiment, one or more objects (401, 402) and a background (403) may have different depth values. In one embodiment, the depth value may represent the distance between the camera and the subject when the still image (310) is acquired. One or more objects (401, 402) (or regions, or parts) may be multiple foreground objects (e.g., people, animals, objects). In one embodiment, the background (402) may be regions other than multiple foreground objects (e.g., people, animals, objects) and may be referred to as background objects.

[0066] For example, the object recognition module (201) can recognize (or identify) the location (or center location) (or boundary surface) of each of one or more objects (401, 402) and / or the region of each of one or more objects (401, 402) within the still image (310). For example, the object recognition module (201) can generate bounding boxes (410, 420) representing the location and / or region of each of one or more objects (401, 402) within the still image (310). In one embodiment, the bounding box (410, 420) may be a substantially rectangular virtual box containing the identified object. In one embodiment, the bounding box (410, 420) may be the smallest size box capable of containing each of the identified one or more objects. In one embodiment, the bounding box (410, 420) is not limited to a rectangular shape and can have various shapes.

[0067] In one embodiment, the object selection module (215) can select at least one object of interest among one or more identified objects (401, 402). For example, the object selection module (215) can display a user interface (UI) through a display (260) that queries the object to which a motion effect is to be applied among one or more identified objects (401, 402). In one embodiment, the object selection module (215) can select an object of interest corresponding to the user input according to the user input selecting at least one of one or more identified objects (401, 402). In FIG. 3, processing of a still image (310) is exemplified as being performed in the order of the object recognition module (201), the object selection module (215), and the object separation module (210), but this is merely an example. According to an embodiment, after object recognition by the object recognition module (201) and object separation by the object separation module (210), an object can be selected through the object selection module (215).

[0068] In one embodiment, the object separation module (210) can separate (or segment) each of one or more objects (401, 402) identified within the still image (310) from the still image (310). For example, the object separation module (210) can separate (or segment) pixels representing the regions of each of the one or more identified objects (401, 402) among the pixels of the still image (310) from the image.

[0069] In one embodiment, the object separation module (210) can identify object information for each of one or more objects (401, 402) separated from a still image (310). In one embodiment, the object separation module (210) can identify object information for an object within a bounding box (410, 420). In one embodiment, the object information may indicate the type of object.

[0070] In one embodiment, the object separation module (210) can generate object separation information (320). In one embodiment, the object separation information (320) may include data and object information representing pixels representing regions of each of one or more identified objects (401, 402).

[0071] In one embodiment, the object separation module (210) may generate object separation information (320) for an object of interest among one or more identified objects (401, 402). In one embodiment, the object of interest may be an object selected by user input or an object satisfying specified conditions. Hereinafter, at least one object selected by user input (or satisfying specified conditions) may be referred to as an object of interest.

[0072] In one embodiment, the object separation module (210) can generate object separation information (320) for at least one object of interest selected by user input among one or more identified objects (401, 402).

[0073] For example, the object separation module (210) can generate object separation information (320) for an object of interest that satisfies specified conditions among one or more identified objects (401, 402). In one embodiment, the object of interest (or, main object) may be a main subject that determines the subject of the still image (310). In one embodiment, the object of interest (or, main object) may be a main subject located at a point of interest (e.g., line of sight) according to each of the shooting angles (e.g., C-shape, V-shape, S-curve, L-shape, golden spiral, diagonal) in the field of view (FoV) of the still image (310).

[0074] In one embodiment, the object separation module (210) can determine the number of objects of interest to which motion effects are to be applied in a still image (310). In one embodiment, the object separation module (210) can determine only one object among a plurality of objects (401, 402) included in the still image (310) as the object of interest to which motion effects are to be applied. In one embodiment, the object separation module (210) can determine two or more objects among a plurality of objects (401, 402) included in the still image (310) as the objects of interest to which motion effects are to be applied.

[0075] In one embodiment, the object separation module (210) may determine at least one of two or more objects as an object of interest. For example, the object classification module (230) may determine at least one object as an object of interest based on the relationship between each of the two or more objects (relationship between subjects represented by the objects (e.g., couple, parent-child, friend, colleague)), attributes (e.g., type of subject represented by the object (e.g., person, animal, object)), a subject of interest specified by the user, type of movement of the object, range of movement of the object), and / or priority.

[0076] For example, the object separation module (210) can identify an object as an object of interest that satisfies a specified condition when the object has specified depth information. For example, the object separation module (210) can identify an object that satisfies a specified condition among one or more identified objects (401, 402) based on depth information (e.g., depth map) of a still image (310). For example, referring to FIG. 5, based on the depth map (510) of the still image (310), the object (511) located furthest forward among the objects (511, 513) can be identified as an object of interest that satisfies a specified condition.

[0077] For example, the object separation module (210) can determine that an object is an object of interest that satisfies specified conditions when the object is a salient object. For example, the object separation module (210) can identify an object whose degree of salientity exceeds a specified degree as an object of interest that satisfies specified conditions based on a salient object detection algorithm (SOD). For example, the object separation module (210) can detect a salient object as an object of interest based on a still image (310) (or a depth map (510)) using an artificial intelligence model trained to detect salient objects through a training dataset. In one embodiment, the protruding object detection algorithm (SOD) may be an algorithm that detects a protruding object (or attention-grabbing object) within a still image (310) (or depth map (510)) and segments an area containing the detected object from the still image (310) (or depth map (510)). For example, the protruding object detection algorithm (SOD) may be an algorithm that detects and separates the salient object (or objects having the same attribute) among the objects constituting the foreground other than the background within the still image (310) (or depth map (510)). For example, the protruding object detection algorithm (SOD) can select candidate protruding object clusters among objects in a still image (310) (or depth map (510)) that are classified according to a specified criterion (e.g., similarity) (e.g., classified through spectral clustering). For example, the protruding object detection algorithm (SOD) can select at least one candidate protruding object cluster among the selected candidate protruding object clusters.For example, a protruding object detection algorithm (SOD) can select one or more candidate protruding objects classified into at least one candidate protruding object cluster as protruding objects. For example, among objects (e.g., people) within an image, objects (e.g., people) performing the same action (e.g., a soccer match) can be evaluated as having the same attribute.

[0078] In one embodiment, the object separation module (210) may transmit object separation information (320) to the 3D modeling module (220) and / or the object classification module (230). For example, the object separation module (210) may transmit object separation information (320) for an object of interest (401) to the 3D modeling module (220) and / or the object classification module (230). For the sake of explanation, the object (401) among one or more objects (401, 402) identified in the still image (310) may be exemplified as being selected as the object of interest.

[0079] In one embodiment, referring to FIG. 3, the 3D modeling module (220) can receive a still image (310). In one embodiment, the 3D modeling module (220) can receive object separation information (320) from the object separation module (210). In one embodiment, the 3D modeling module (220) can receive object separation information (320) for a selected object (401) from the object separation module (210).

[0080] In one embodiment, the 3D modeling module (220) may generate 3D model information (340) (or stereoscopic information) for a selected object (401) within a still image (310). In one embodiment, the 3D model information (340) (or stereoscopic information) may represent the selected object of interest (401) as a 3D object based on a plurality of meshes. In one embodiment, each of the plurality of meshes may include vertices, edges connecting the vertices, and faces formed by the edges. In one embodiment, the 3D model information (340) (or stereoscopic information) may include information regarding the 3D coordinates and / or colors of each of the vertices. In one embodiment, the 3D model information (340) (or stereoscopic information) may include information regarding the texture of each of the meshes. In one embodiment, the 3D model information (340) (or, stereoscopic information) may indicate whether the 3D object is deformed.

[0081] In one embodiment, the vertices, edges, and faces included in the 3D model information (340) (or, stereoscopic information) may include some vertices, some edges, and some faces represented in the still image (310) and other vertices, other edges, and other faces not represented in the still image (310). For example, other vertices, other edges, and other faces not represented in the still image (310) may be estimated (or identified) (or computed) by the 3D modeling module (220) by a specified 3D graphics algorithm (e.g., 3D reconstruction algorithm, depth estimation algorithm). However, the embodiment is not limited thereto. For example, the 3D modeling module (220) can estimate (or identify) (or calculate) other vertices, other edges, and other faces that are not represented in the still image (310) by using an artificial intelligence model for constructing objects in the still image (310) into 3D objects. According to an embodiment, if the still image (310) includes two or more still images, the 3D modeling module (220) can estimate (or identify) (or calculate) vertices, edges, and faces that represent viewpoints not represented in one still image (310) by using an artificial intelligence model (e.g., SfM (structure from motion)) for constructing objects in the multiple still images into 3D objects. In one embodiment, the two or more still images may include other photos stored in memory (130) in addition to the still image (310). In one embodiment, two or more still images may include other photos obtained from a server (108) (or web search) in addition to the still image (310).

[0082] In one embodiment, the 3D modeling module (220) can identify the coordinate values ​​in the depth direction of each of the objects (511, 513) based on the depth map (510) of the still image (310). In one embodiment, the 3D modeling module (220) can identify the coordinate values ​​in the depth direction of each of the pixels included in the objects (511, 513) based on the depth map (510) of the still image (310). In one embodiment, the 3D modeling module (220) can identify the 3D coordinates of each of the vertices based on the coordinate values ​​in the depth direction and the coordinate values ​​in the horizontal and vertical directions within the still image (310).

[0083] In one embodiment, the 3D modeling module (220) can generate 3D model information (340) (or, solid information) including edges connected to the vertices and surfaces formed by the edges, based on the 3D coordinates of each of the vertices. For example, referring to FIG. 5, the 3D modeling module (220) can generate 3D model information (530) (or, solid information) representing 3D models (531, 533) for objects (511, 513), based on the 3D coordinates of each of the vertices for the objects (511, 513).

[0084] In one embodiment, the 3D modeling module (220) may generate 3D model information (340) (or, stereoscopic information) including vertices, edges, and faces for a part of an object (401) not displayed in a still image (310) when generating 3D model information (340) (or, stereoscopic information). For example, the 3D modeling module (220) may estimate a part of an object (401) not displayed in a still image (310) through out-painting and generate 3D model information (340) (or, stereoscopic information) including vertices, edges, and faces for the estimated part of an object (401). For example, out-painting may be performed using a specified artificial intelligence model (e.g., a diffusion model, a transformer model, a GAN (generative adversarial network)).

[0085] In one embodiment, when generating 3D model information (340) (or stereoscopic information), the 3D modeling module (220) may generate 3D model information (340) (or stereoscopic information) of an object (401) by referring to another image containing an object (401) (or another object corresponding to the object (401)). In one embodiment, the other image containing the object (401) (or another object corresponding to the object (401)) may be an image containing the object (401) (or another object corresponding to the object (401)) in a different composition and / or at a different angle from the input image. For example, the 3D modeling module (220) may identify another image containing the object (401) (or another object corresponding to the object (401)) among images captured within a specified time interval from the time of capture of the input image. However, the embodiment is not limited thereto. For example, the 3D modeling module (220) can identify other images retrieved from the server (108) (or the Internet) by using information about an object (401) included in the input image. In one embodiment, the other images may be still images and / or videos.

[0086] In one embodiment, referring to another image may include the 3D modeling module (220) identifying the depth-direction coordinate values ​​of each pixel included in the objects based on the depth map of the other image, and identifying the 3D coordinates of each vertex based on the depth-direction coordinate values ​​and the horizontal and vertical coordinate values ​​within the other image. In one embodiment, referring to another image may include the 3D modeling module (220) generating 3D model information (340) (or stereoscopic information) of an object (401) by summing the 3D coordinates of each vertex identified through a still image (310) and the 3D coordinates of each vertex identified through the other image.

[0087] In one embodiment, when the 3D modeling module (220) generates 3D model information (340) (or, 3D information) for an object (401), it may generate 3D model information (340) (or, 3D information) for another object associated with the object (401). In one embodiment, the other object associated with the object (401) may be an object linked to the movement of the object (401). For example, the other object associated with the object (401) may be an object that guides, restricts, or induces the movement of the object (401). For example, if the object (401) is a balloon, the other object may be a string that ties the balloon. For example, if the object (401) is a person, the other object may be a cane held by the person. For example, if the object (401) is a person, the other object may be a hat worn by the person.

[0088] In one embodiment, the 3D modeling module (220) may transmit 3D model information (340) to the motion information generation module (240) and / or the rendering module (250). For example, the 3D modeling module (220) may transmit 3D model information (340) for at least one object (401) selected by user input to the 3D motion information generation module (240) and / or the rendering module (250). For example, the 3D modeling module (220) may transmit 3D model information (340) for an object (401) that satisfies specified conditions to the motion information generation module (240) and / or the rendering module (250).

[0089] According to an embodiment, the 3D modeling module (220) may store 3D model information (340) in memory (130) separately from the video (360). In one embodiment, the 3D modeling module (220) may reuse the 3D model information (340) stored in memory (130) separately from the video (360). For example, reusing the 3D model information (340) may include creating a 3D model of an object included in another still image through the 3D model information (340) when creating a video based on a still image (310) and another still image. In one embodiment, the other still image may be an image containing an object (401) represented by the 3D model information (340).

[0090] In one embodiment, the 3D modeling module (220) can update 3D model information (340) stored in memory (130) separately from the video (360). For example, updating the 3D model information (340) may include updating the 3D model information (340) through other 3D model information generated based on other still images when creating a video based on a still image different from the still image (310). In one embodiment, the other still image may be an image containing an object (401) represented by the 3D model information (340).

[0091] In one embodiment, the object classification module (230) may receive a still image (310). In one embodiment, the object classification module (230) may receive object separation information (320) from the object separation module (210). In one embodiment, the object classification module (230) may receive object separation information (320) for a selected object (401) from the object separation module (210). Each embodiment described herein may be used in combination with any other embodiment(s) described herein.

[0092] In one embodiment, the object classification module (230) may define (or identify) motion attributes associated with the selected object (401). For example, the motion attributes of the selected object (401) may represent one or more types of motion (or one or more motion classifications) among a plurality of types of motion (or a plurality of motion classifications). In one embodiment, the object classification module (230) may define (or identify) one or more types of motion (or one or more motion classifications) to define the motion effects of the selected object (401) (or a 3D object corresponding to the object (401)) among a plurality of types of motion (or a plurality of motion classifications). For example, the types of motion may be defined by one or more motions (e.g., circular motion, and reciprocating motion), or motions based on physical laws (e.g., elastic motion, sliding motion, uniform motion, free fall, accelerated motion).

[0093] For example, the object classification module (230) can define (or identify) the motion attributes of an object (401) based on predefined data representing motion attributes corresponding to types of objects. For example, the predefined data can map motion attributes representing types of motion applicable to each type of object. For example, if the object is a 'ball', the predefined data can map motion attributes including circular motion and falling motion as types of motion applicable to the 'ball' to the 'ball'. For example, if the object is a 'wheel', the predefined data can map motion attributes including circular motion as types of motion applicable to the 'wheel' to the 'wheel'. For example, if the object is a 'flower' or 'grass', the predefined data can map motion attributes including reciprocating motion and elastic motion as types of motion applicable to the 'flower' or 'grass' to the 'flower' or 'grass'. For example, if the object is a 'skate', the predefined data can map motion attributes including sliding motion as a type of motion applicable to the 'skate' to the 'skate'. For example, if the object is a 'cloud' or a 'river', the predefined data can map motion attributes including constant velocity motion as a type of motion applicable to the 'cloud' or the 'river' to the 'cloud' or the 'river'. For example, if the object is a 'waterfall', the predefined data can map motion attributes including falling motion as a type of motion applicable to the 'waterfall' to the 'waterfall'.

[0094] For example, the object classification module (230) can define (or identify) motion attributes, including reciprocating motion and elastic motion, as types of motion applicable to the ‘flower’, based on the fact that the object (401) is a ‘flower’.

[0095] According to an embodiment, the object classification module (230) can update predefined data representing motion attributes corresponding to types of objects. For example, the object classification module (230) can identify common objects included in at least some of the one or more images stored in the memory (130) of the electronic device (101) (or a cloud database linked to a user account of the electronic device (101)). For example, the object classification module (230) can obtain motion attributes corresponding to the type of the identified common object. For example, the object classification module (230) can obtain motion attributes corresponding to the type of the identified common object from the server (108). For example, the object classification module (230) can obtain motion attributes corresponding to the type of the identified common object based on user input obtained from a user interface (UI) that queries motion attributes for the identified common object. In one embodiment, the object classification module (230) can add movement attributes corresponding to the type of common object obtained to the predefined data.

[0096] For example, the object classification module (230) can transmit object classification information (330) including motion attributes to the motion information generation module (240).

[0097] In one embodiment, the motion information generation module (240) may receive object classification information (330) from the object classification module (230). In one embodiment, the motion information generation module (240) may receive 3D model information (340) from the 3D modeling module (220).

[0098] In one embodiment, the motion information generation module (240) can identify changed coordinate values ​​of the vertices of the 3D object corresponding to the object (401) based on object classification information (330) and 3D model information (340).

[0099] In one embodiment, the motion information generation module (240) can identify a motion model corresponding to the motion attributes of an object (401) according to object classification information (330). In one embodiment, the motion information generation module (240) can generate motion information (350) for an object (401) using the motion model for the object (401). In one embodiment, the motion information (350) may include information for transforming the position of each of the vertices included in the 3D object corresponding to the object (401). For example, the information for transforming the position may include a matrix representing rotation with respect to the 3D coordinates of each of the vertices (e.g., a rotation matrix), and / or a matrix representing translation (e.g., a translation matrix). In one embodiment, the motion model may include at least one of a motion model based on a position table, a motion model based on a position formula, and a motion model based on a physics formula.

[0100] In one embodiment, the location table may be a table in which location and time information is mapped. In one embodiment, the motion information generation module (240) may generate motion information (350) for an object (401) using the location table. For example, the motion information generation module (240) may generate motion information (350) indicating the location of the object (401) at each of one or more time points using the location table. In one embodiment, if any one of the one or more time points is not defined in the location table, the motion information generation module (240) may identify the location at any one time point using an interpolation (or extrapolation) algorithm. For example, if any one of the one or more time points is not defined in the location table, the motion information generation module (240) may identify the location at any one time point by using the locations of two time points adjacent to that one time point.

[0101] According to the movement information (350) generated based on the position table described above, the movement freedom of the object (401) may be high. Accordingly, the movement information generation module (240) can generate movement information (350) that represents various movements suitable for the object (401) based on the position table.

[0102] In one embodiment, the motion information generation module (240) can generate motion information (350) using a position formula representing the position of an object (401) according to circular motion or reciprocating motion.

[0103] For example, the motion information generation module (240) can generate motion information (350) of an object (401) using a position formula for circular motion such as Equation 1 below. For example, the motion information generation module (240) can generate motion information (350) of an object (401) that is in circular motion (e.g., a wheel, or a ball) using a position formula for circular motion such as Equation 1 below.

[0104]

[0105] Mathematical formula 1 is the positions (x) of the object (401) at time t. t , y t It can represent ). The locations (x) of the object (401). t , y t ) can be changed along the radius (r) according to the initial position (x0, y0), initial angle (w0), and angular velocity w.

[0106] For example, when an object (401) (e.g., a wheel or a ball) moves at a constant speed along a predetermined circular trajectory, the motion information generation module (240) can generate motion information (350) of the object (401) based on Equation 1.

[0107] For example, the motion information generation module (240) can generate motion information (350) of an object (401) using a position formula for reciprocating motion such as Equation 2 below. For example, the motion information generation module (240) can generate motion information (350) of an object (401) using a position formula for reciprocating motion such as Equation 2 below for an object that reciprocates (e.g., a flower, grass, or tree that reciprocates due to an external force (e.g., wind) in a specific direction).

[0108]

[0109] Mathematical formula 2 is the positions (x) of the object (401) at time t. rec t , y rec t It can represent ). The locations (x) of the object (401). rec t , y rec t ) is, circular motion coordinates(x cir t , y cir t It can be the position projected onto the line ax+by=c.

[0110] For example, when an object (401) (e.g., a flower, grass, or tree that moves back and forth due to an external force (e.g., wind) in a specific direction) moves back and forth along a predetermined path, the motion information generation module (240) can generate motion information (350) of the object (401) based on Equation 2.

[0111] The motion information (350) generated based on the position formula as described above does not require memory space for the position table and can represent various movements by changing parameters (e.g., initial position (x0, y0), initial angle (w0) and angular velocity w, straight line ax+by=c).

[0112] In one embodiment, the motion information generation module (240) can generate motion information (350) using a physical formula representing the position of an object (401) according to elastic motion, sliding motion, constant velocity motion, or falling motion.

[0113] For example, the motion information generation module (240) can generate motion information (350) of an object (401) using a physical formula representing velocity and displacement such as Equation 3 below. In one embodiment, the motion information generation module (240) can express the motion of the object (401) in terms of velocity and acceleration, and calculate the displacement of the motion by adding a velocity that changes in proportion to the acceleration for every frame of time (or unit time).

[0114]

[0115] In mathematical equation 3, a t is the acceleration at time t, and v t is the velocity at time t, and x t may be the displacement at time point t. In one embodiment, the acceleration a at time point t is t can be calculated differently depending on elastic motion, sliding motion, uniform motion, or falling motion. Equation 3 describes displacement, velocity, and acceleration in the x-direction, but Equation 3 is also applicable to displacement, velocity, and acceleration in the y-direction perpendicular to the x-direction.

[0116] For example, the motion information generation module (240) can generate motion information (350) of an object (401) using a physical formula for elastic motion such as Equation 4 below. In one embodiment, the motion information generation module (240) can determine that when an object (401) (or a part of the object (401)) receives a force while being fixed by another object (e.g., an object distinct from the object (401), or an object extending from the object (401)), movement according to elastic motion occurs in the object (401). For example, if a part of the object (401) is a fixed object (e.g., a flower, or grass), the motion information generation module (240) can determine that the object (401) performs reciprocating motion (e.g., up and down, or left and right) due to a force in a specific direction applied to the object (401). For example, the variables determining the acceleration in the reciprocating motion of the object (401) are elastic force and damping force, and the motion information generation module (240) can change the acceleration of the object (401) in proportion to the displacement and velocity of the object (401), respectively. In one embodiment, the motion information (350) of the object (401), which is an elastic body, can be generated by considering λ (elastic modulus) and c (damping modulus) in Equation 4.

[0117]

[0118] Mathematical Equation 4 is the acceleration (a) due to the elastic motion of the object (401) at time t. t It can represent ). In mathematical equation 4, λ is the elastic modulus and c can be the damping coefficient.

[0119] For example, the motion information generation module (240) can generate motion information (350) of an object (401) using a physical formula for sliding motion such as Equation 5 below. In one embodiment, the motion information generation module (240) can determine that if it is determined that the object (401) is moving by sliding motion, an equal acceleration motion occurs due to a frictional force acting in the opposite direction of the sliding motion. For example, the motion information generation module (240) can generate motion information (350) for an object (401) that is sliding motion by using Equation 5, using the kinetic friction coefficient (or μ) and the weight (or N) of the object (401).

[0120]

[0121] Equation 5 is the acceleration (a) due to the sliding motion of the object (401) at time t. t It can represent ). In mathematical formula 5, μ is the (kinetic) friction coefficient, and N may be the (estimated) weight of the object (401).

[0122] For example, the motion information generation module (240) can generate motion information (350) of an object (401) using a physical formula for uniform motion such as Equation 6 below.

[0123]

[0124] Equation 6 is the acceleration (a) due to the uniform motion of the object (401) at time t. t It can represent ).

[0125] For example, the motion information generation module (240) can generate motion information (350) of an object (401) using a physical formula for falling motion such as Equation 7 below. In one embodiment, when the motion information generation module (240) determines that the object (401) is falling due to a force (e.g., gravity) in the downward direction (or the ground direction of the electronic device (101)), it can obtain motion information (350) of the object (401) through Equation 7 by applying a constant gravitational acceleration (g) in the downward direction of the object (401).

[0126]

[0127] Equation 7 is the acceleration (a) due to the uniform motion of the object (401) at time t. t g can represent gravitational acceleration.

[0128] According to an embodiment, the velocity of an object (401) when it collides with an object such as the ground can be calculated using Equation 8 or Equation 9 below. For example, when an object has elasticity like a ball, when it collides with another object (e.g., the ground), the object moves in the opposite direction. In this case, if the object is a perfectly elastic object, the object may start moving in the direction of reflection with a velocity having a reflection angle vector calculated from the angle of incidence to the other object (e.g., the ground). In this case, when the normal vector of the other object (e.g., the ground) is denoted as n, the velocity vector of the object in the direction of the reflection angle can be calculated as Equation 8 below. Additionally, if the object is a perfectly elastic object, the object may start moving in the direction of reflection with a velocity having a reflection angle vector calculated from the angle of incidence relative to the other object (e.g., the ground). In this case, when the normal vector is denoted as n, the velocity vector of the object in the direction of the reflection angle can be calculated as Equation 9 below.

[0129]

[0130]

[0131] Equation 8 can represent the velocity of the object (401) at time t+1 when the object (401) is a complete elastic body. In Equation 8, n can represent the normal vector of the object that the object (401) collides with. Equation 9 can represent the velocity of the object (401) at time t+1 when the object (401) is not a complete elastic body. k can be the elastic modulus of the object (401).

[0132] In one embodiment, the motion information generation module (240) is the velocity (v) of Equation 3. t+1 ) and displacement(x t+1 Acceleration (a) for calculating ) t ), speed(v t ), or displacement(x t At least one of ) can be changed (or adjusted). For example, the motion information generation module (240) can change (or adjust) acceleration (a t ), speed(v t ), or displacement(x t At least one of ) can be changed based on real-time input or a specified input. In one embodiment, the motion information generation module (240) calculates acceleration (a) based on pre-prepared input information (331) as a specified input. t ), speed(v t ), or displacement(x t At least one of ) can be changed. In one embodiment, the pre-prepared input information (331) is an acceleration (a) defined or identified before playing the video (360) obtained from the still image (310). t ), speed(v t ), or displacement(x t Data for changing at least one of ) can be represented. In one embodiment, the motion information generation module (240) based on real-time input information (335) as real-time input, acceleration (a t ), speed(vt ), or displacement(x t At least one of ) can be changed. In one embodiment, the real-time input information (335) is an acceleration (a) defined or identified according to the input (or the state of the electronic device (101)) obtained simultaneously (or substantially simultaneously) while playing the video (360) obtained from the still image (310). t ), speed(v t ), or displacement(x t Data can be represented to change at least one of ).

[0133] In one embodiment, the motion information generation module (240) can identify an operation toward the user’s electronic device (101) (e.g., touch input, voice input, operation causing movement of the electronic device (101)) and / or a change in the state of the electronic device (101) as a real-time input (or real-time input information (335)).

[0134] In one embodiment, the motion information generation module (240) is based on the intensity, speed, time, distance traveled, pattern, and / or direction of the touch input (or drag input) to the display (260), the acceleration (a) of the object (401) toward which the touch input is directed or toward which the touch input is initiated. t ), speed(v t ), or displacement(x t At least one of ) can be changed. For example, the motion information generation module (240) can change the acceleration (a) to be applied to the object (401) to which the drag input is initiated (or to which the drag input is directed), based on the direction, speed, pattern, and distance of the drag input to the display (260). t ) can be changed (or adjusted). The acceleration (a) of the object (401) that changes according to touch input (or drag input). t ), speed(v t ), or displacement(x tAt least one of ) may be based on physical properties specified for the object (401) (e.g., mass, elasticity, shape, hardness, pattern, roughness, texture). For example, the higher the mass, the smaller the change in acceleration may be.

[0135] In one embodiment, the motion information generation module (240) can identify the movement (or change in inertia) of the electronic device (101) (or the housing of the electronic device (101)) (or as real-time input information (335)) through the sensor module (176). In one embodiment, the motion information generation module (240) can identify that the electronic device (101) (or the housing of the electronic device (101)) is rotated (e.g., clockwise or counterclockwise) through the sensor module (176). In one embodiment, the motion information generation module (240) can identify the acceleration (a) to be applied to the object (401) such that the direction of the gravitational acceleration is directed toward the ground as the electronic device (101) is rotated. t ) can be changed (or adjusted). In one embodiment, the acceleration (a) applied to the object (401) according to the rotation of the electronic device (101) t ) may be based on the rotation direction or rotation angle of the electronic device (101). For example, acceleration (a) to be applied to the object (401) so that the direction of gravitational acceleration is adjusted in the opposite direction of the rotation direction. t ) can be changed (or adjusted). For example, the acceleration (a) applied to the object (401) so that the direction of the gravitational acceleration is adjusted in proportion to the angle. t ) can be changed (or adjusted).

[0136] In one embodiment, the motion information generation module (240) can identify that the electronic device (101) (or the housing of the electronic device (101)) is moved through the sensor module (176). In one embodiment, the motion information generation module (240) can apply acceleration (a) to the object (401) such that acceleration in the opposite direction of the direction of movement is applied as the electronic device (101) moves. t ) can be changed (or adjusted). In one embodiment, acceleration (a) applied to the object (401) according to the movement of the electronic device (101) t ) may be based on the direction of movement, speed of movement, time of movement, or distance of movement of the electronic device (101). For example, acceleration (a) applied to the object (401) in the opposite direction of the direction of movement t ) can be changed (or adjusted). For example, acceleration (a) applied to the object (401) proportional to the speed of movement. t ) can be changed (or adjusted). For example, acceleration (a) applied to the object (401) in proportion to the travel time or travel distance. t The time for ) to be changed (or adjusted) may be extended.

[0137] In one embodiment, the motion information generating module (240) may receive a sound signal (or pressure signal) through a microphone (e.g., the input module (150) of FIG. 1). In one embodiment, the motion information generating module (240) may, according to the sound signal, apply an acceleration (a) to floating objects among a plurality of objects (e.g., objects capable of reciprocating motion, constant velocity motion, or falling motion). t ) can be changed (or adjusted). In one embodiment, the motion information generation module (240) can change (or adjust) the acceleration (a) to be applied to floating objects (e.g., objects capable of reciprocating motion, constant velocity motion, or falling motion) based on the intensity of the sound signal or time. t) can be changed (or adjusted). For example, acceleration (a) applied to floating objects in proportion to the intensity of the sound signal. t ) can be changed (or adjusted). For example, acceleration (a) applied to floating objects so as to be proportional to the time of the sound signal. t The time for ) to be changed (or adjusted) may be extended.

[0138] In one embodiment, the motion information generation module (240) can identify a notification (e.g., a phone call notification from a phone application, a message reception notification from a message application). In one embodiment, the motion information generation module (240) can determine the acceleration (a) to be applied to the object (401) according to the notification. t ) can be changed (or adjusted). For example, the motion information generation module (240) can change (or adjust) the acceleration (a) to be applied to the object (401) depending on the type of notification (or the application that generated the notification), the content of the notification, and / or the sender associated with the notification. t ) can be changed (or adjusted). For example, acceleration (a) to be applied to the object (401) according to the intensity set according to the type of notification. t ) can be changed (or adjusted). For example, the intensity set for a phone call notification may be stronger than the intensity set for a message notification. For example, the acceleration (a) applied to the object (401) according to the intensity set according to the caller associated with the notification t ) may be changed (or adjusted). For example, the intensity set for notifications for callers stored in contacts may be stronger than the intensity set for notifications for callers not stored in contacts. For example, the intensity set for notifications for designated callers (e.g., family, friends) among callers stored in contacts may be stronger than the intensity set for notifications for undesignated callers (e.g., colleagues) among callers stored in contacts.

[0139] In one embodiment, the motion information generation module (240) can identify weather information. In one embodiment, the weather information may include the weather within the still image (310) and / or the weather within the physical environment where the electronic device (101) is located. In one embodiment, the weather within the still image (310) may include the weather depicted by the still image (310) and / or the actual weather at the time and / or location where the still image (310) was taken. Hereinafter, the weather depicted by the still image (310) and / or the actual weather at the time and / or location where the still image (310) was taken may be referred to as the weather of the still image (310).

[0140] In one embodiment, the motion information generation module (240) determines the acceleration (a) to be applied to the object (401) according to the weather information. t ) can be changed (or adjusted). For example, the motion information generation module (240) can change (or adjust) the acceleration (a) to be applied to the object (401) according to the type of weather (e.g., clear, cloudy, rain, snow, overcast, foggy), temperature, sunrise time, or sunset time. t You can change (or adjust) ). For example, the acceleration (a) applied to the object to be proportional to the temperature. t ) can be changed (or adjusted). For example, acceleration (a) applied to the object (401) in proportion to the temperature t ) can be changed (or adjusted). For example, acceleration (a) applied to the object (401) at image temperature t ) is the acceleration (a) to be applied to the object (401) at a sub-zero temperature t It can be higher than ). For example, acceleration (a) applied to the object (401) according to the intensity set according to the type of weather. t) may be changed (or adjusted). However, embodiments are not limited thereto. For example, the motion information generation module (240) may apply acceleration (a) to the object (401) according to the time, season, weather, etc. represented by the input image. t ) can be changed (or adjusted). For example, the motion information generation module (240) can apply acceleration (a) to an object (401) included in an input image representing winter. t ) can be changed (or adjusted) to a relatively low value. For example, the motion information generation module (240) can apply acceleration (a) to an object (401) included in an input image based on an input image of an object shaken by wind. t ) can be changed (or adjusted) so that it faces the direction of the wind appearing in the input image.

[0141] In one embodiment, the motion information generation module (240) determines the acceleration (a) to be applied to the object (401) based on the background and / or objects (or foreground) within the still image (310). t ) can be changed (or adjusted). In one embodiment, the motion information generation module (240) can determine shadow and / or lighting effects according to the movement of the object (401) according to the weather, time, and / or season depicted by the still image (310).

[0142] For example, the motion information generation module (240) applies acceleration (a) to the object (401) according to the weather, time, and / or season depicted in the still image (310). t ) can be changed (or adjusted). For example, if the weather in the still image (310) indicates winter, the motion information generation module (240) can change (or adjust) the acceleration (a) to be applied to the object (401). t) can be changed (or adjusted) to a relatively low level. For example, if the weather in the still image (310) indicates heavy rain, the motion information generation module (240) can apply acceleration (a) to the object (401) so that the object (401) moves relatively downward. t ) can be changed (or adjusted). For example, if the weather in the still image (310) indicates windy weather, the motion information generation module (240) can change (or adjust) the acceleration (a) to be applied to the object (401) so that the object (401) has a shaking movement. t ) can be changed (or adjusted). According to an embodiment, if other objects other than the object (401) in the still image (310) are bent in a specific direction (or are bent by an external force (e.g., wind), the motion information generation module (240) applies an acceleration (a) to the object (401) so that the object (401) has the same (or corresponding) shaking motion as the other objects. t ) can be changed (or adjusted). According to an embodiment, if there are shadows on objects other than the object (401) in the still image (310), the motion information generation module (240) can determine shadow and / or lighting effects according to the movement of the object (401) so that the object (401) has the same (or corresponding) shading as the other objects.

[0143] In one embodiment, the motion information generation module (240) can identify the position information of an object (401) at each of the identified time points through a motion model based on a position table, a motion model based on a position formula, and / or a motion model based on a physics formula. In one embodiment, the motion information generation module (240) can generate motion information (350) including information for transforming the position based on the positions of the object (401) at each of two adjacent time points. For example, the motion information generation module (240) can generate motion information (350) including a matrix representing rotation (e.g., a rotation matrix) and / or a matrix representing translation (e.g., a translation matrix) for transforming the position of the object (401) at the earlier of the two adjacent time points to the position of the object (401) at the later of the two adjacent time points.

[0144] According to an embodiment, the motion information generation module (240) may store motion information (350) in memory (130) separately from the video (360). In one embodiment, the motion information generation module (240) may reuse motion information (350) stored in memory (130) separately from the video (360). For example, reusing motion information (350) may include generating motion effects applied to objects included in another still image through motion information (350) when generating a video based on a still image different from the still image (310). In one embodiment, the motion information generation module (240) may store motion information (350) extracted from a still image used for video generation more than a reference number of times in memory (130) separately from the video (360). In one embodiment, the motion information generation module (240) may store motion information (350) of an object determined to be preferred by the user, extracted from a still image, in memory (130) separately from the video (360). In one embodiment, the electronic device (101) may automatically (or without a user's request) generate a video (360) for a still image (310) captured according to specified conditions. For example, the electronic device (101) may automatically (or without a user's request) generate a video (360) for a still image (310) captured when a scene or object for which the user has a history of generating a video (360) a certain number of times or more is captured, when a user-preferred object is captured, and / or when an object (or scene) for which the effect of generating motion can be prominent is captured. In one embodiment, the electronic device (101) may store the automatically generated video (360) together with the still image (310) in memory (130).

[0145] According to an embodiment, the motion information generation module (240) can store motion information (350) generated according to real-time input together with the video (360) in memory (130). By storing the generated motion information (350) together with the video (360) in memory (130), when playing the video (360), the user can see the object (401) moving according to the motion information (350) without separate real-time input. The motion information (350) stored in memory (130) together with the video (360) may be motion information (350) extracted from a still image used to generate the video more than a reference number of times, motion information (350) of an object determined to be preferred by the user, and / or motion information (350) having special graphic effects.

[0146] According to an embodiment, the motion information generation module (240) may generate motion information for each of the objects of interest by taking into account the motion information of other objects of interest. For example, when the motion information generation module (240) generates motion information to apply motion effects to a high-priority object (e.g., an object determined to be preferred by the user) (e.g., an object that is frequently photographed, a specific relationship (family)), it may generate motion information to stop (or suspend) the motion effects of a low-priority object. Accordingly, while the motion of a high-priority object among multiple objects is being played, the playback of the motion of other objects may be put on hold and / or stopped.

[0147] According to an embodiment, the motion information generation module (240) can generate motion information such that one of the objects covers another object according to a position in the depth direction. According to an embodiment, the motion information generation module (240) can generate motion information such that any two of the objects interfere (or collide) according to the movement of each of the objects. Motion information such as any two objects interfering (or colliding) can be generated based on acceleration according to the priority (or weight) of the two objects. For example, when the two objects interfere (or collide), motion information can be generated such that the object with higher priority (or weight) changes more slowly (e.g., has lower acceleration) and the object with lower priority (or weight) changes more quickly (e.g., has higher acceleration).

[0148] The motion generator can generate natural motion by reflecting the physical properties or priorities of each object when interference occurs during the simultaneous playback of multiple object movements. For example, motion can be generated by taking into account perspective and depth information within the image.

[0149] In one embodiment, the data input to the motion information generation module (240) may be transformation matrices of 3D mesh data. In one embodiment, the data output from the motion information generation module (240) may be 3D mesh data transformed according to the transformation matrices.

[0150] As described above, a method for the motion information generation module (240) to generate motion information (350) of the object (401) when the object (401) is a rigid body has been described. However, the embodiments are not limited thereto. Even when the object (401) is a soft body (e.g., clothing) or a fluid, the motion information generation module (240) can generate motion information (350) of the object (401).

[0151] In one embodiment, the motion information generation module (240) can generate motion information (350) of a soft body (e.g., clothing) or a fluid object (401) using an algorithm for conventional 3D graphics. For example, in the case of a fluid, the motion information generation module (240) can generate motion information (350) of the object (401) based on a specified fluid simulation algorithm (e.g., stable fluids, level set method, or MAC (marker and cell) Grid). For example, in the case of a soft body, the motion information generation module (240) can generate motion information (350) of the object (401) based on a specified soft body simulation algorithm (e.g., point-spring model, finite element method, or PBD (position-based dynamics).

[0152] In one embodiment, the rendering module (250) can receive 3D model information (340) from the 3D modeling module (220). In one embodiment, the rendering module (250) can receive motion information (350) from the motion information generation module (240).

[0153] In one embodiment, the rendering module (250) can generate a frame corresponding to each of the viewpoints based on 3D model information (340) and motion information (350).

[0154] In one embodiment, the rendering module (250) can generate a frame corresponding to each of the viewpoints through a rendering pipeline. In one embodiment, the rendering pipeline may include determining the position and shape of a 3D object within a frame at each of the viewpoints, determining the position of a 3D object at each of the viewpoints, projecting onto a frame at each of the viewpoints, and shading. In one embodiment, the rendering module (250) can perform the rendering pipeline using a designated rendering module (e.g., OpenGL, Vulkan).

[0155] In one embodiment, to determine the position and shape of a 3D object, the rendering module (250) can transform the position of each vertex included in the 3D object using motion information (350) for the object (401). For example, the rendering module (250) can determine the position and shape of the 3D object at each viewpoint by using motion information (350) for the object (401) to multiply the coordinate values ​​of each vertex included in the 3D object by matrices (e.g., rotation matrix and / or translation matrix). For example, the rendering module (250) can determine the position and shape of the 3D object in world coordinates with the camera as the origin at each viewpoint by using motion information (350) for the object (401) to multiply the coordinate values ​​of each vertex included in the 3D object by matrices (e.g., rotation matrix and / or translation matrix).

[0156] In one embodiment, for projection into a frame at each of the viewpoints, the rendering module (250) can generate a frame at each of the viewpoints by projecting an area containing a 3D object in world coordinates with the camera at each of the viewpoints as the origin into a 2D frame.

[0157] In one embodiment, for shading the frame at each of the viewpoints, the rendering module (250) can determine the color of each pixel included in the frame based on the color, texture, and / or lighting of the objects and / or background within the frame at each of the viewpoints.

[0158] In one embodiment, the rendering module (250) can determine the color of each pixel representing a shadow and / or lighting effect according to the movement of the object of interest (or, main object) based on the type of scene represented by the input image (e.g., type according to time, season, or weather), objects within the input image, or the relationship with the background, through shading of the frame at each of the viewpoints. For example, the shadow and / or lighting effect may represent the color of the light source under rainy conditions. For example, the shadow and / or lighting effect may represent the shadow of the object (401) on an object obscured by the object (401).

[0159] In one embodiment, the rendering module (250) can generate a video (360) containing frames at different viewpoints through a rendering pipeline. For example, the video (360) may include one or more frames depicting the movement of an object (401). For example, referring to FIG. 6a, the video (360) may include a frame (611) in which the object (401) is moved to the right. For example, referring to FIG. 6b, the video (360) may include a frame (615) in which the object (401) is moved to the left.

[0160] In one embodiment, the rendering module (250) may in-paint the parts (610, 620) that are obscured by the object (401) in the original image (or still image (310)) as the object (401) moves. In one embodiment, the in-painting may be a process (or image filling process) that generates a new image on the parts (610, 620) that are exposed as the object (401) moves. In one embodiment, the rendering module (250) may in-paint the parts (610, 620) that are obscured by the object (401) in the original image (or still image (310)) as the object (401) moves using a specified artificial intelligence model (e.g., a diffusion model, a transformer model, a generative adversarial network (GAN)). However, the embodiment is not limited thereto. For example, as the camera's coordinates in world coordinates and / or the camera's FoV change, parts not visible in the original image (or still image (310)) can be out-painted. In one embodiment, as the camera's FoV changes, the user can see the object moving in a two-dimensional direction and / or a three-dimensional direction.

[0161] FIG. 7a is a drawing showing an example of a frame included in a video generated by real-time input according to one embodiment. FIG. 7b is a drawing showing an example of a frame included in a video generated by real-time input according to one embodiment.

[0162] FIGS. 7a and FIGS. 7b can be explained with reference to FIGS. 1 to 6b.

[0163] A frame (711) according to FIG. 7a may be displayed through a display (260) of an electronic device (101). In one embodiment, the frame (711) may be an image that is interactive with a user (e.g., an image capable of reproducing at least some area (movement)) generated from an original image (or input image). For example, the frame (711) may be a frame in which a motion effect is applied to one or more objects (721, 723, 725, 727, 729) within the original image (or input image). For example, the frame (711) may be an image in which a motion effect is applied within the image to an object of interest, and the remaining parts (or remaining objects) unrelated to the motion effect of the object of interest do not move (or maintain their shape within the original image). For example, applying a motion effect to one or more objects (721, 723, 725, 727, 729) may indicate that the position and / or shape of one or more objects (721, 723, 725, 727, 729) differs from each other between the frame (711) and the original image (or input image). In one embodiment, the frame (711) may be a frame to which a motion effect is applied to one or more objects (721, 723, 725, 727, 729) according to the state (or movement) of the electronic device (101). In one embodiment, one or more objects (721, 723, 725, 727, 729) may be a plurality of objects recognized from the original image (or input image) by the object recognition module (201). In one embodiment, one or more objects (721, 723, 725, 727, 729) may be objects separated by the object separation module (210) among a plurality of objects recognized from the original image (or input image).In one embodiment, one or more objects (721, 723, 725, 727, 729) may be objects selected by the object selection module (215) among a plurality of objects recognized (or separated) from the original image (or input image).

[0164] In one embodiment, the electronic device (101) may receive user input for an object (725) while displaying a frame (711) through a display (260). In one embodiment, the user input may be a touch input (e.g., drag input) for the object (725). However, the embodiment is not limited thereto. For example, the user input for the object (725) may include voice input from the user (e.g., “Move a specific balloon downwards”). For example, the user input for the object (725) may include text (or a prompt) (e.g., “Move a specific balloon downwards”). However, the embodiment is not limited thereto. For example, the type of user input for generating movement information for the object (725) may be specified by the user during the video generation process or automatically specified by an artificial intelligence model. In one embodiment, the type of user input can be classified according to the input device for receiving user input, the intensity of the input, and the direction of the input.

[0165] In one embodiment, the electronic device (101) can generate motion information for each of one or more objects (721, 723, 725, 727, 729) by using a motion model corresponding to motion attributes according to object classification information for each of one or more objects (721, 723, 725, 727, 729) based on user input.

[0166] In one embodiment, the electronic device (101) can generate motion information including, based on user input, an acceleration that causes an object (725) to move downward and an acceleration that causes one or more objects (721, 723, 727, 729) to move to the left as the object (725) moves downward.

[0167] In one embodiment, the electronic device (101) can render a frame (715) to be displayed after a frame (711) by using 3D model information representing a 3D object of each of one or more objects (721, 723, 725, 727, 729) and motion information of each of one or more objects (721, 723, 725, 727, 729). In one embodiment, the interval between the time when the frame (711) is displayed and the time when the frame (715) is displayed may be based on the type of each of one or more objects (721, 723, 725, 727, 729) and a frame rate suitable for the motion model to be applied to each of one or more objects (721, 723, 725, 727, 729).

[0168] In one embodiment, the electronic device (101) may display a rendered frame (715) through a display (260). Referring to FIG. 7b, in the frame (715), the object (725) may be positioned relatively lower compared to the frame (711), and the objects (721, 723, 727, 729) may be positioned relatively left. In one embodiment, to show the movement effect of each of the objects (721, 723, 725, 727, 729), one or more intermediate frames may be displayed through the display (260) between the frame (715) and the frame (711), in which the positions of the objects (721, 723, 725, 727, 729) sequentially move from their positions within the frame (711) to their positions within the frame (715).

[0169] Afterward, the electronic device (101) may display through the display (260) one or more frames showing a motion effect in which objects (721, 723, 725, 727, 729) return to their original positions (e.g., positions in frame (711)) as user input to the object (725) is interrupted.

[0170] FIG. 8a is a drawing showing an example of a frame included in a video generated by real-time input according to one embodiment. FIG. 8b is a drawing showing an example of a frame included in a video generated by real-time input according to one embodiment.

[0171] FIGS. 8a and FIGS. 8b can be explained with reference to FIGS. 1 to 7b.

[0172] A frame (811) according to FIG. 8a may be displayed through a display (260) of an electronic device (101). In one embodiment, the frame (811) may be an image that is interactive with a user (e.g., an image that can reproduce at least some area (movement)) generated from an original image (or, input image). For example, the frame (811) may be a frame in which a motion effect is applied to one or more objects (721, 723, 725, 727, 729) within the original image (or, input image). For example, the application of a motion effect to one or more objects (721, 723, 725, 727, 729) may indicate that the position and / or shape of one or more objects (721, 723, 725, 727, 729) differs between the frame (811) and the original image (or, input image). In one embodiment, the frame (811) may be a frame to which a motion effect is applied to one or more objects (721, 723, 725, 727, 729) according to the state (or movement) of the electronic device (101).

[0173] In one embodiment, the electronic device (101) can identify a change in the state of the electronic device (101) as a real-time input while displaying the frame (811) through the display (260). In one embodiment, the change in the state of the electronic device (101) may include movement (or change in inertia) of the electronic device (101) (or housing of the electronic device (101)). In one embodiment, the movement (or change in inertia) of the electronic device (101) (or housing of the electronic device (101)) may include rotation (e.g., clockwise or counterclockwise rotation) of the electronic device (101) (or housing of the electronic device (101)). In one embodiment, the movement (or change in inertia) of the electronic device (101) (or housing of the electronic device (101)) may include movement of the electronic device (101) (or housing of the electronic device (101)).

[0174] In one embodiment, the electronic device (101) can generate movement information for each of one or more objects (721, 723, 725, 727, 729) by using a movement model corresponding to a movement attribute according to object classification information for each of one or more objects (721, 723, 725, 727, 729) based on a change in the state of the electronic device (101).

[0175] In one embodiment, the electronic device (101) can generate movement information in which the direction of gravitational acceleration applied to one or more objects (721, 723, 725, 727, 729) is changed as the electronic device (101) (or the housing of the electronic device (101)) is rotated counterclockwise based on a change in the state of the electronic device (101).

[0176] In one embodiment, the electronic device (101) can render a frame (815) to be displayed after a frame (811) by using 3D model information representing a 3D object of each of one or more objects (721, 723, 725, 727, 729) and motion information of each of one or more objects (721, 723, 725, 727, 729).

[0177] In one embodiment, the electronic device (101) can display a rendered frame (815) through a display (260). Referring to FIG. 8b, while the electronic device (101) (or the housing of the electronic device (101)) is rotated counterclockwise, the objects (721, 723, 725, 727, 729) in the frame (815) may be positioned to face the right corner of the electronic device (101) compared to the frame (811). To show the movement effect of the objects (721, 723, 725, 727, 729), between frame (815) and frame (811), one or more intermediate frames may be displayed through the display (260) in which the positions of the objects (721, 723, 725, 727, 729) move sequentially from their positions within frame (811) to their positions within frame (815).

[0178] FIG. 9a is a drawing showing an example of a frame included in a video displayed in a gallery application according to one embodiment.

[0179] FIG. 9a can be explained with reference to FIGS. 1 to 8b.

[0180] In one embodiment, the electronic device (101) can display the screen of a gallery application through the display (260). In one embodiment, referring to FIG. 9a, the screen of the gallery application may include a selected image (910).

[0181] In one embodiment, the electronic device (101) may receive user input for creating a video based on a selected image (910). In one embodiment, the electronic device (101) may receive user input for creating a video while displaying the selected image (910) on the screen of a gallery application. For example, the electronic device (101) may receive user input for selecting an object (915) for creating a video based on the selected image (910).

[0182] In one embodiment, the electronic device (101) can generate a video with motion effects applied to one or more objects based at least partially on receiving user input selecting an object (915). In one embodiment, the electronic device (101) can display the generated video instead of an image (910) on the screen of a gallery application.

[0183] In FIG. 9a, a video suitable for a gallery application is exemplified, but this is merely an example. In one embodiment, the electronic device (101) may determine the resolution, size, and / or format (e.g., emoji) of the video based on the size of the screen on which the image (910) or the video generated from the image (910) is to be displayed, and / or the type of application. For example, the resolution, size, and / or format (e.g., emoji) of the video may be determined based on the user's intention to generate the video, which is predicted according to the context of the electronic device (101) (e.g., the application currently in use, the usage pattern of the electronic device (101), the content currently displayed on the screen). For example, when a user generates a video from an image (910) while conversing with another user through a messenger application (or for file sharing), the electronic device (101) may generate a video having a resolution, size, and format (e.g., motion emoji) suitable for sharing with another user.

[0184] FIG. 9b is a drawing showing an example of a frame within specified content of a gallery application according to one embodiment.

[0185] FIG. 9b can be explained with reference to FIGS. 1 to 8b.

[0186] In one embodiment, referring to FIG. 9b, the electronic device (101) may display designated content (920) through a display (260). In one embodiment, the designated content (920) may be content (e.g., a story) that displays one or more images in chronological order. In one embodiment, the designated content (920) may include a video generated based on an image (e.g., 910 in FIG. 9a) to be played for a designated time.

[0187] In one embodiment, the electronic device (101) may play audio corresponding to the movement of an object while playing designated content (920). For example, the electronic device (101) may include an object (931) for controlling the playback of audio and an object (935) for controlling the volume of audio. In one embodiment, the audio corresponding to the movement of an object may be audio corresponding to the movement information of an object among a plurality of audios classified according to movement information. For example, if the object is a ball and the object bounces off a floor surface, the audio corresponding to the movement information of the object may include a sound effect of the object hitting the floor surface. For example, if the object is a cloud and the object moves in a designated direction, the audio corresponding to the movement information of the object may include a sound effect having a number of bits corresponding to the movement speed of the object. For example, the audio may be audio generated (through an artificial intelligence model) based on an image (e.g., 910 in FIG. 9a). For example, the audio may be audio generated (substantially) simultaneously with the generation of specified content (920) based on an image (e.g., 910 in FIG. 9a). For example, the audio may include a melody that corresponds to the scene or theme of the specified content (920). For example, the audio may include sound effects corresponding to the movement of an object within the specified content (920). According to an embodiment, the audio may include voice. For example, the audio may include voice generated (via an artificial intelligence model) based on a voice signal stored in the electronic device (101).

[0188] In one embodiment, a multimedia thumbnail may be combined with a video in the specified content (920). In one embodiment, an audio and / or multimedia thumbnail generated in relation to the movement of an object may be stored in memory (130) as metadata of the specified content (920).

[0189] FIG. 10a is a drawing showing an example of a lock screen displayed in a locked state of an electronic device according to one embodiment. FIG. 10b is a drawing showing an example of user input for unlocking the electronic device according to one embodiment.

[0190] Referring to FIG. 10a, the electronic device (101) may display a lock screen (1001) in a locked state. In one embodiment, the lock screen (1001) may be an image containing objects (721, 723, 725, 727, 729) that can interact with the user, generated from a still image. According to an embodiment, the lock screen (1001) may include a visual object (1021) indicating the location of a fingerprint sensor for identifying a fingerprint for unlocking the electronic device (101), and a visual object (1025) indicating that the electronic device (101) is in a locked state.

[0191] In one embodiment, the electronic device (101) may generate a lock screen (1001) to be displayed on the display (260) in a locked state based on an original image (or, input image). In one embodiment, when the electronic device (101) generates the lock screen (1001) from the original image (or, input image), it may set a method for unlocking the lock state. For example, the method for unlocking the lock state may include unlocking through a fingerprint obtained through a fingerprint sensor by placing a finger on a visual object (1021) indicating the location of the fingerprint sensor. For example, the method for unlocking the lock state may include unlocking through user input that causes movement of one or more objects (721, 723, 725, 727, 729) within the lock screen (1001).

[0192] In one embodiment, the electronic device (101) may receive a pattern of one or more user inputs that cause movement of one or more objects (721, 723, 725, 727, 729) within the lock screen (1001) while setting a method for unlocking the lock state. In one embodiment, the electronic device (101) may acquire the acquired pattern of user inputs as a pattern for unlocking the electronic device (101) while setting a method for unlocking the lock state.

[0193] In one embodiment, referring to FIG. 10b, the electronic device (101) can receive user input (1011, 1013, 1015) for objects (721, 723, 725, 727, 729) while displaying a lock screen (1001) in a locked state. For example, the electronic device (101) can identify a pattern of user input (1011, 1013, 1015) for objects (721, 723, 725, 727, 729).

[0194] For example, the electronic device (101) can unlock the electronic device (101) based on the pattern of user input (1011, 1013, 1015) corresponding to a pattern for unlocking the electronic device (101). For example, the electronic device (101) can keep the electronic device (101) locked based on the pattern of user input (1011, 1013, 1015) being different from a pattern for unlocking the electronic device (101).

[0195] For example, a pattern in which user input (1011) for moving object (721) to the left, user input (1013) for moving object (725) to the right, and user input (1015) for moving object (729) downward are sequentially input may be a pattern for unlocking the electronic device (101).

[0196] In this case, if user input (1011) for moving object (721) to the left, user input (1013) for moving object (725) to the right, and user input (1015) for moving object (729) downward are entered sequentially, the electronic device (101) can unlock the electronic device (101). If user input (1011) for moving object (721) to the left, user input (1013) for moving object (725) to the right, and user input (1015) for moving object (729) downward are entered in a different order, or if user input for other objects (e.g., 723 727) is entered, the electronic device (101) can maintain the lock of the electronic device (101).

[0197] According to an embodiment, a visual object (1021) indicating the location of a fingerprint sensor for identifying a fingerprint for unlocking the electronic device (101) instead of a pattern for unlocking the electronic device (101) may be displayed on the lock screen (1001). In one embodiment, as the electronic device (101) is set to be unlockable through a pattern for unlocking the electronic device (101) for the objects (721, 723, 725, 727, 729), the display of the visual object (1021) may be stopped (or, refrained from).

[0198] In one embodiment, when unlocking is configured via fingerprint, the electronic device (101) can acquire fingerprint data representing a fingerprint by detecting the user's finger located on a visual object (1021) through a fingerprint sensor. In one embodiment, the electronic device (101) can unlock the electronic device (101) if the fingerprint data represents the user's fingerprint. In one embodiment, the electronic device (101) can maintain the locked state of the electronic device (101) if the fingerprint data does not represent the user's fingerprint.

[0199] In one embodiment, the electronic device (101) may apply a designated motion effect to objects (721, 723, 725, 727, 729) when the fingerprint data represents the user's fingerprint. For example, the designated motion effect may be a preset motion effect to indicate that the fingerprint data represents the user's fingerprint. In one embodiment, the electronic device (101) may apply another designated motion effect to objects (721, 723, 725, 727, 729) when the fingerprint data does not represent the user's fingerprint. For example, the designated other motion effect may be a preset other motion effect to indicate that the fingerprint data does not represent the user's fingerprint.

[0200] FIG. 11a is a drawing showing an example of a frame displayed according to a notification of an electronic device according to one embodiment. FIG. 11b is a drawing showing an example of a frame displayed according to a notification of an electronic device according to one embodiment.

[0201] Referring to FIGS. 11a and 11b, the electronic device (101) may display different movements of objects (721, 723, 725, 727, 729) depending on the type of notifications (1110, 1120) while displaying a lock screen (1001) in a locked state. However, the embodiments are not limited thereto. For example, the electronic device (101) may display different movements of objects as it identifies an event (e.g., an event to display a notification) while displaying a screen containing objects that can interact with the user on a background screen.

[0202] In one embodiment, the motion information generation module (240) can identify a notification (e.g., a phone call notification from a phone application, a message reception notification from a message application). In one embodiment, the motion information generation module (240) can determine the acceleration (a) to be applied to the object (401) according to the notification.t ) can be changed (or adjusted). For example, the motion information generation module (240) can change (or adjust) the acceleration (a) to be applied to the object (401) depending on the type of notification (or the application that generated the notification), and / or the sender associated with the notification. t ) can be changed (or adjusted). For example, acceleration (a) to be applied to the object (401) according to the intensity set according to the type of notification. t ) can be changed (or adjusted). For example, the intensity set for a phone call notification may be stronger than the intensity set for a message notification. For example, the acceleration (a) applied to the object (401) according to the intensity set according to the caller associated with the notification t ) may be changed (or adjusted). For example, the intensity set for notifications for callers stored in contacts may be stronger than the intensity set for notifications for callers not stored in contacts. For example, the intensity set for notifications for designated callers (e.g., family, friends) among callers stored in contacts may be stronger than the intensity set for notifications for undesignated callers (e.g., colleagues) among callers stored in contacts.

[0203] According to an embodiment, notifications may include notifications based on phone calls and / or messages received from other electronic devices, as well as notifications generated within the electronic device (101). For example, when the battery level drops below a specified threshold, the electronic device (101) may play the movement of objects. For example, according to a schedule saved by the user (e.g., a specified time for medication, a wake-up call, or a departure time), the electronic device (101) may play the movement of objects.

[0204] Additionally, the electronic device (101) can play the movement of objects in response to being connected to an external electronic device (e.g., access point, base station, wearable device, IoT device) directly or indirectly. For example, when connected to a designated external electronic device, the electronic device (101) can play the designated movement of at least one object. For example, the electronic device (101) can play the movement of objects as it identifies that the distance between the electronic device (101) and an object outside the electronic device (101) (e.g., person, animal, other electronic device) has changed. For example, as it identifies that the distance to an external object is a designated distance, the electronic device (101) can play the designated movement of at least one object.

[0205] FIG. 12 is a flowchart illustrating the operation of an electronic device determining acceleration based on real-time input according to one embodiment.

[0206] FIG. 12 can be explained with reference to FIGS. 1 to FIGS. 11b.

[0207] Referring to FIG. 12, in operation 1210, the electronic device (101) can determine whether a touch input is present. For example, the electronic device (101) can determine whether a touch input is present on an object to which a motion effect within the interactive image can be applied while an interactive image (e.g., an image capable of reproducing at least some area (movement)) is displayed on the display (260).

[0208] In operation 1210, based on the presence of a touch input, the electronic device (101) can perform operation 1215. In operation 1210, based on the absence of a touch input, the electronic device (101) can perform operation 1220.

[0209] In operation 1215, the electronic device (101) can generate motion information based on touch input. For example, the electronic device (101) can generate motion information using a motion model corresponding to the motion attributes of the object toward which the touch input is directed. In one embodiment, the electronic device (101) can detect the touch input through the display (260) (or the touch sensor of the display (260)). In one embodiment, the electronic device (101) can identify acceleration based on the direction of the touch input. For example, the electronic device (101) can obtain the direction of the touch input and / or acceleration based on the amount of change in the position (or coordinates) of the press input obtained through the display (260) (or the touch sensor of the display (260)). In one embodiment, the electronic device (101) may determine that the touch input is a touch input that affects the movement of an object when the location of the touch input overlaps with at least a part of the object and / or comes within a specified distance from the object. In one embodiment, the electronic device (101) may generate movement information based on the touch input that affects the movement of the object.

[0210] In operation 1220, the electronic device (101) can determine whether there is movement of the electronic device (101). For example, the electronic device (101) can determine whether there is movement of the electronic device (101) while an interactive image (e.g., an image capable of reproducing at least some area (movement)) is displayed on the display (260).

[0211] In operation 1220, based on the existence of movement of the electronic device (101), the electronic device (101) can perform operation 1225. In operation 1220, based on the absence of movement of the electronic device (101), the electronic device (101) can perform operation 1230.

[0212] In operation 1225, the electronic device (101) can generate motion information based on the movement of the electronic device (101). For example, the electronic device (101) can generate motion information using a motion model corresponding to the motion attributes of an object to which a motion effect in an interactive image can be applied. In one embodiment, the electronic device (101) can detect the movement and / or rotation of the electronic device (101) using a sensor module (176) (e.g., a gyroscope and / or accelerometer). In one embodiment, the electronic device (101) can determine the motion effect that affects the object based on the information indicating the movement and / or rotation.

[0213] In operation 1230, the electronic device (101) can determine whether an audio signal is present. For example, the electronic device (101) can determine whether an audio signal is being output through the sound output module (155) while an interactive image (e.g., an image capable of reproducing at least some area (movement)) is displayed on the display (260).

[0214] In operation 1230, based on the presence of an audio signal, the electronic device (101) can perform operation 1235. In operation 1230, based on the absence of an audio signal, the electronic device (101) can perform operation 1240.

[0215] In operation 1235, the electronic device (101) can generate motion information based on an audio signal. For example, the electronic device (101) can generate motion information using a motion model corresponding to the motion attributes of an object to which a motion effect can be applied within an interactive image. For example, the electronic device (101) can acquire an audio signal using an input module (150) (e.g., a microphone) inside the electronic device (101). For example, the electronic device (101) can determine the direction and / or intensity of the audio signal based on the audio signal acquired using at least one microphone, and determine a motion effect to be applied to the object (e.g., a motion effect due to wind) using the determined direction and / or intensity of the audio signal. In one embodiment, the electronic device (101) can generate motion information using the motion effect to be applied to the object (e.g., a motion effect due to wind) determined based on the audio signal.

[0216] In operation 1240, the electronic device (101) can determine whether a notification event exists. For example, the electronic device (101) can determine whether a notification event exists while an interactive image (e.g., an image capable of reproducing at least some area (movement)) is displayed on the display (260).

[0217] In operation 1240, based on the presence of a notification event, the electronic device (101) can perform operation 1245. In operation 1240, based on the absence of a notification event, the electronic device (101) can perform operation 1250.

[0218] In operation 1245, the electronic device (101) can generate motion information based on a notification event. For example, the electronic device (101) can generate motion information using a motion model corresponding to the motion attributes of an object to which a motion effect can be applied within an interactive image. For example, the electronic device (101) can determine the motion effect to be applied to the object based on the type of notification event (or the application that generated the notification), the content of the notification event, and / or the sender associated with the notification event. For example, the electronic device (101) can determine the intensity of the motion effect to be applied to the object based on the type of notification event. For example, the electronic device (101) can determine the intensity of the motion effect to be applied to the object based on the sender associated with the notification event. For example, the intensity set for a notification event for a sender stored in contacts may be stronger than the intensity set for a notification event for a sender not stored in contacts. For example, the intensity set for notification events for a designated caller (e.g., family, friend) among the callers stored in the contacts can be stronger than the intensity set for notification events for an undesignated caller (e.g., colleague) among the callers stored in the contacts.

[0219] In operation 1250, the electronic device (101) can determine whether there is a change in the state of the electronic device (101). For example, the electronic device (101) can determine whether there is a change in the state of the electronic device (101) while an interactive image (e.g., an image capable of reproducing at least some area (movement)) is displayed on the display (260).

[0220] In operation 1250, based on whether there is a change in the state of the electronic device (101), the electronic device (101) may perform operation 1255. In operation 1250, based on whether there is no change in the state of the electronic device (101), the electronic device (101) may terminate the operation according to FIG. 12.

[0221] In operation 1255, the electronic device (101) can generate motion information based on a change in the state of the electronic device (101). For example, the electronic device (101) can generate motion information by using a motion model corresponding to the motion attributes of an object to which a motion effect can be applied within an interactive image. In one embodiment, the electronic device (101) can use the electronic device (101) to identify weather information and / or a change in weather information as a change in the state of the electronic device (101). Accordingly, the electronic device (101) can provide a motion effect according to a change in weather information. For example, if the current weather is winter, the electronic device (101) can determine the acceleration to be applied to the motion information to be relatively low. For example, if the current weather is windy, the electronic device (101) can determine the acceleration to be applied to the motion information according to the wind strength.

[0222] FIG. 13 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0223] FIG. 13 can be explained with reference to FIGS. 1 to 11b.

[0224] Referring to FIG. 13, in operation 1310, the electronic device (101) can obtain two-dimensional information about at least one object from a still image. For example, the two-dimensional information may represent the location (or center location) (or boundary surface) of each of one or more objects within the still image, and / or the region of each of one or more objects within the still image.

[0225] In operation 1320, the electronic device (101) can identify at least one type of object related to the three-dimensional movement effect of at least one object using two-dimensional information. In one embodiment, at least one type of object can define and / or indicate movement attributes related to the object. In one embodiment, at least one type of object can represent one or more types of movement (or one or more movement classifications) among a plurality of types of movement (or a plurality of movement classifications).

[0226] In one embodiment, the electronic device (101) can identify, using two-dimensional information, a pattern of physical motion associated with at least one object (e.g., a pattern of motion described with reference to Equations 1 to 9) and a deformation of the external shape corresponding to the pattern of physical motion (e.g., rotational motion, and / or translational motion). In one embodiment, the electronic device (101) can identify the type of at least one object based on the pattern of physical motion and the deformation of the external shape.

[0227] In operation 1330, the electronic device (101) can generate a video containing three-dimensional motion effects of at least one object based on a motion model corresponding to the type of at least one object. In one embodiment, the motion model may include a motion model based on a position table, a motion model based on a position formula, and a motion model based on a physics formula. The electronic device (101) can generate motion information based on a motion model corresponding to the type of at least one object. The electronic device (101) can generate a video containing motion effects of at least one object within a frame according to the motion information. In one embodiment, the motion effects may include movement, rotation, and shape change of at least one object.

[0228] FIG. 14 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0229] FIG. 14 can be explained with reference to FIGS. 1 to 11b.

[0230] Referring to FIG. 14, in operation 1410, the electronic device (101) can identify an object in a still image containing one or more objects. In one embodiment, the electronic device (101) can recognize and / or separate one or more objects within the still image. In one embodiment, the electronic device (101) can identify an object of interest among the identified one or more objects. In one embodiment, the object of interest may be an object selected by user input or an object satisfying specified conditions.

[0231] In operation 1420, the electronic device (101) can identify a motion model corresponding to a motion attribute to be applied to a 3D object corresponding to an identified object.

[0232] In one embodiment, the electronic device (101) may define (or identify) motion attributes related to an object of interest. For example, the motion attributes of the object of interest may represent one or more types of motion (or one or more classifications of motion) among a plurality of types of motion (or a plurality of classifications of motion). For example, the types of motion may be defined and / or indicated by one or more motions (e.g., circular motion, and reciprocating motion), or motions based on laws of physics (e.g., elastic motion, sliding motion, uniform motion, free fall, accelerated motion).

[0233] In one embodiment, the electronic device (101) can identify a motion model corresponding to the motion attributes of an object of interest. In one embodiment, the motion model may include a motion model based on a position table, a motion model based on a position formula, and a motion model based on a physical formula.

[0234] In operation 1430, the electronic device (101) can generate frames in which the position of the 3D object is updated according to the motion information of the 3D object from the motion model.

[0235] In one embodiment, the electronic device (101) can identify location information of an object of interest at each of the points in time identified through a motion model based on a location table, a motion model based on a location formula, and / or a motion model based on a physical formula. In one embodiment, the electronic device (101) can generate motion information including information for transforming the location based on the locations of the object of interest at each of two adjacent points in time.

[0236] In one embodiment, the electronic device (101) can generate frames in which the position of a 3D object is updated at each of the viewpoints based on 3D model information and motion information corresponding to an object of interest. In one embodiment, the electronic device (101) can generate frames corresponding to each of the viewpoints through a rendering pipeline. In one embodiment, the rendering pipeline may include determining the position and shape of a 3D object within a frame at each of the viewpoints, determining the position of a 3D object at each of the viewpoints, projecting onto a frame at each of the viewpoints, and shading.

[0237] FIG. 15 is a drawing showing an example of a frame displayed on a wearable device according to one embodiment.

[0238] In one embodiment, referring to FIG. 15, the electronic device (101) may be a wearable device that can be worn by a user (1500). For example, the electronic device (101) may display a FoV (1505) in a three-dimensional virtual space (1501).

[0239] In one embodiment, the electronic device (101) can generate an output image to be displayed in the FoV (1505) based on an input image. In one embodiment, the output image displayed in the FoV (1505) may include one or more objects (1511, 1513, 1515, 1517, 1519).

[0240] In one embodiment, the electronic device (101) can identify real-time inputs related to one or more objects (1511, 1513, 1515, 1517, 1519) included in an output image displayed in the FoV (1505).

[0241] In one embodiment, real-time input may be identified by a sensor module (176) (e.g., a motion sensor, or an infrared sensor) and / or a camera module (180). In one embodiment, real-time input may include the user (1500) moving while wearing the electronic device (101). In one embodiment, real-time input may include the user (1500) moving the user's (1500) head while wearing the electronic device (101). In one embodiment, real-time input may include the user (1500) moving a gaze determined by the user's (1500) two eyes while wearing the electronic device (101). In one embodiment, real-time input may include a gesture taken by the user (1500) while wearing the electronic device (101). In one embodiment, the real-time input in the bar type electronic device (101) exemplified with reference to FIGS. 1 to 11b is a two-dimensional input toward the display (260), whereas the real-time input in the wearable type electronic device (101) exemplified with reference to FIG. 15 may be a three-dimensional input by gesture, motion, and / or gaze.

[0242] In one embodiment, the electronic device (101) can apply movement effects of one or more objects (1511, 1513, 1515, 1517, 1519) in a three-dimensional direction according to real-time input in a three-dimensional direction. In one embodiment, the electronic device (101) can generate movement information including acceleration to move an object of high importance (or high priority) among one or more objects (1511, 1513, 1515, 1517, 1519) toward the user (1500) and to move an object of low importance (or low priority) away from the user (1500) according to real-time input in a three-dimensional direction. However, the embodiment is not limited thereto. For example, the electronic device (101) may generate motion information including acceleration to move an object representing a high-priority notification toward the user (1500) and move an object representing a low-priority notification away from the user (1500) when a notification is received. For example, the electronic device (101) may determine an object controlled by user input and / or close to user input as a high-priority object. Alternatively, the electronic device (101) may determine, based on the user's gaze, that the closer an object is to the user's gaze, the higher its importance.

[0243] FIG. 16 is a drawing illustrating various types of electronic devices according to one embodiment.

[0244] In FIGS. 1 to 11b, the electronic device (101) is depicted as a bar-type device, but the electronic device (101) to which the operation of the present disclosure applies is not limited thereto. For example, the electronic device may be devices having various form factors (e.g., a foldable type device including two housing parts (101-3), a foldable type device including three or more housing parts (101-4), or a sliderable (or rollable) type device (101-5)).

[0245] For example, a foldable type device (101-3) may include a pair of housing parts (e.g., a first housing part (1621) and a second housing part (1622)) and a flexible display (1624) that are directly or indirectly connected to each other so as to be rotatable with respect to a folding axis (1623). The flexible display (1624) may include a first display area (1625) accommodated in the first housing part (1621) and a second display area (1626) accommodated in the second housing part (1622). Between the first housing part (1621) and the second housing part (1622), a hinge assembly may be disposed to rotatably connect the first housing part (1621) and the second housing part (1622). Between the first housing part (1621) and the second housing part (1622), a hinge housing may be disposed to accommodate at least a portion of the hinge assembly.

[0246] For example, a multi-foldable type device (101-4) may include a first housing part (1631) and a second housing part (1632) that are directly or indirectly connected to each other rotatable with respect to a first folding axis (1634), and a third housing part (1633) that is rotatablely connected to the second housing part (1632) with respect to a second folding axis (1635). The multi-foldable type device (101-4) may include a flexible display (1636). The flexible display (1636) may include a first display area (1637) accommodated in the first housing part (1631), a second display area (1638) accommodated in the second housing part (1632), and a third display area (1639) accommodated in the third housing part (1633). A multi-foldable type device (101-4) may support an in-folding method or an out-folding method. An in-folding method may correspond to a method in which corresponding display areas are folded in a direction facing each other. A hinge assembly providing an in-folding method may include a hinge housing part. An out-folding method may correspond to a method in which corresponding display areas are folded in a direction not facing each other. A hinge assembly providing an out-folding method may not include a hinge housing part. For example, the first folding axis (1634) may correspond to the folding axis of the in-folding method, and the second folding axis (1635) may correspond to the folding axis of the out-folding method. The first housing part (1631) and the second housing part (1632) may be folded in an out-folding method, and the second housing part (1632) and the third housing part (1633) may be folded in an in-folding method. For example, a multi-foldable type device (101-4) may be formed to include three folding axes and four housing parts.

[0247] In one embodiment, in the electronic devices (101-3, 101-4, 101-5), motion information of an object in an image displayed on a display (e.g., 1624, 1625, 1626, 1636, 1637, 1638, 1639) can be determined in response to physical movement of a housing (e.g., 1621, 1622, 1631, 1632, 1633) and / or a display (e.g., 1624, 1625, 1626, 1636, 1637, 1638, 1639).

[0248] In one embodiment, when an image including a movable object is displayed on a background screen (or, lock screen, or AoD (always on display) screen), the electronic devices (101-3, 101-4, 101-5) respond to the physical movement of the housing (e.g., 1621, 1622, 1631, 1632, 1633) and / or the display (e.g., 1624, 1625, 1626, 1636, 1637, 1638, 1639) in response to the physical movement of the housing (e.g., 1621, 1622, 1631, 1632, 1633) and / or the display (e.g., 1624, 1625, 1626, 1636, 1637, 1638, 1639) (or, You can play the movement of the object moving in the opposite direction.

[0249] In one embodiment, in various types of electronic devices (101-3, 101-4, 101-5), an object to which a motion effect is applied may be selected according to the specifications of the electronic devices (101-3, 101-4, 101-5) (or electronic components mounted in the electronic devices (101-3, 101-4, 101-5)) (e.g., the type of input / output device that can be provided in the electronic devices (101-3, 101-4, 101-5), the specifications of the display). For example, in various types of electronic devices (101-3, 101-4, 101-5), for the same input image, a motion effect may be applied differently to different objects depending on the type of input detectable by the sensor module (176) (e.g., touch input, gyroscope input).

[0250] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure pertains.

[0251] As described above, the electronic device (101) may include at least one processor (120) including a processing circuit; and a memory (130) that stores instructions and includes one or more storage media. When the instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to obtain two-dimensional information about at least one object from a still image. When the instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may use the two-dimensional information to identify the type of the at least one object related to the three-dimensional motion effect of the at least one object. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the at least one object to generate a video containing the three-dimensional motion effect of the at least one object based on a motion model corresponding to the type of the at least one object.

[0252] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to identify a pattern of physical movement associated with the at least one object and a deformation of the external shape corresponding to the pattern of physical movement using the two-dimensional information. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to identify the type of the at least one object based on the pattern of physical movement and the deformation of the external shape.

[0253] The above instructions, when executed individually or collectively by the at least one processor (120), may cause the electronic device (101) to receive a specified user input for the at least one object. The above instructions, when executed individually or collectively by the at least one processor (120), may cause the electronic device (101) to determine the three-dimensional motion effect based on the user input and the motion model. The above instructions, when executed individually or collectively by the at least one processor (120), may cause the electronic device (101) to generate the video containing the three-dimensional motion effect determined according to the user input and the motion model.

[0254] When the above instructions are executed individually or collectively by the at least one processor (120), they may cause the electronic device (101) to detect state information of the electronic device (101). When the above instructions are executed individually or collectively by the at least one processor (120), they may cause the electronic device (101) to determine the three-dimensional motion effect based on the detected state information and the motion model. When the above instructions are executed individually or collectively by the at least one processor (120), they may cause the electronic device (101) to generate the video containing the three-dimensional motion effect determined according to the detected state information and the motion model.

[0255] When the above instructions are executed individually or collectively by the at least one processor (120), they may cause the electronic device (101) to detect the movement of the housing of the electronic device (101) moving in a specified direction and with a specified intensity. When the above instructions are executed individually or collectively by the at least one processor (120), they may cause the electronic device (101) to detect the movement of the housing of the electronic device (101) as state information of the electronic device (101).

[0256] When the above instructions are executed individually or collectively by the at least one processor (120), they may cause the electronic device (101) to identify a designated notification. When the above instructions are executed individually or collectively by the at least one processor (120), they may cause the electronic device (101) to detect the designated notification as state information of the electronic device (101).

[0257] When the above instructions are executed individually or collectively by the at least one processor (120), they may cause the electronic device (101) to identify that the electronic device (101) is changing between a locked state and an unlocked state. When the above instructions are executed individually or collectively by the at least one processor (120), they may cause the electronic device (101) to detect the change between the locked state and the unlocked state as state information of the electronic device (101).

[0258] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause at least some of the positional deformation and viewpoint deformation of the at least one object to be generated using other images. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the at least one object to be three-dimensionally modeled based on the positional deformation and viewpoint deformation of the at least one object.

[0259] The above at least one object includes a first object and a second object, and the video includes a first motion effect corresponding to the first object and a second motion effect corresponding to the second object, and the first motion effect and the second motion effect can be selected based on priorities related to the first object and the second object.

[0260] At least in part while the three-dimensional motion effect is provided by playing the video, the video may include a portion inpainted from the background area of ​​the still image and a portion outpainted from the at least one object.

[0261] As described above, the electronic device (101) may include at least one processor (120) comprising a display (260) and a processing circuit; and a memory (130) that stores instructions and includes one or more storage media. When the instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to identify an object in a still image comprising one or more objects. When the instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to identify a motion model corresponding to a motion attribute to be applied to a three-dimensional object corresponding to the identified object. The motion attribute may include one or more types of motion among a plurality of types of motion. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the display to show a first frame containing the 3D object corresponding to the object at a first position. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the position of the 3D object to be updated according to the motion information of the 3D object from the motion model. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the display to show a second frame following the first frame, containing the 3D object at a second position instead of the first position according to the motion information.

[0262] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to determine the type of the identified object among a plurality of types. The plurality of types may represent different combinations of the plurality of types of movement. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to identify predefined movement information for the type of the identified object. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the position of the 3D object from the movement model according to the predefined movement information.

[0263] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to receive an input. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to determine the motion information corresponding to the input. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to update the position of the 3D object from the motion model according to the motion information corresponding to the input.

[0264] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to receive a notification of an ongoing call of a phone application as the input. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to determine the motion information as first motion information based on the fact that the caller of the ongoing call corresponds to a first caller. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to determine the motion information as second motion information based on the fact that the caller of the ongoing call corresponds to a second caller other than the first caller. The displacement between the second position and the first position according to the first movement information may be greater than the displacement between the second position and the first position according to the second movement information.

[0265] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to receive user input toward the 3D object as the input. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to determine the movement information corresponding to the intensity and / or direction of the user input. The displacement between the second position and the first position may correspond to the intensity of the user input, and the direction from the first position to the second position may correspond to the direction of the user input.

[0266] The above multiple types of movement may include circular motion, reciprocating motion, rotational motion, or translational movement.

[0267] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause to identify an empty area within the second frame that is defined and / or indicated as the 3D object moves from the first position to the second position. The empty area may be an area within the second frame that is not overlapped by the second area occupied by the 3D object at the second position among the first areas occupied by the 3D object at the first position. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause to display the second frame in which the empty area is drawn after the first frame by in-painting the empty area.

[0268] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the screen of the gallery application to be displayed through the display. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to receive user input requesting the creation of an image in which the position of the 3D object is updated according to the motion information based on the still image among a plurality of still images within the screen. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to create the image in which the position of the 3D object is updated according to the motion information upon receiving the user input. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the position of the 3D object to be updated according to the motion information and to display a plurality of frames through the display in response to another user input requesting the display of the generated image. The plurality of frames may include the first frame and the second frame.

[0269] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to identify a still image containing a designated object among a plurality of still images acquired within a designated place and / or a designated period. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to identify the designated object within the still image as the object. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may be caused to generate an image in which the position of the 3D object is updated according to the motion information. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) causes the position of the 3D object to be updated according to the motion information and to display a plurality of frames through the display in response to a user input requesting the display of the generated image, and the plurality of frames may include the first frame and the second frame.

[0270] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to display a plurality of frames through the display in which the position of the 3D object is updated according to the movement information while the electronic device (101) is in a locked state. The plurality of frames may include the first frame and the second frame. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to receive user input toward the 3D object. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) displays the plurality of frames through the display in which the position of the 3D object is updated according to the user input: identifying whether the user input corresponds to an input pattern for unlocking the lock state, and determining to unlock the lock state based on identifying that the user input corresponds to the input pattern, and maintaining the lock state based on identifying that the user input does not correspond to the input pattern, and determining to unlock the lock state, thereby causing the home screen to be displayed instead of the plurality of frames.

[0271] When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to determine the type of the identified object among a plurality of types through an artificial intelligence model. Each of the plurality of types represents a combination of the plurality of types of movement, and the artificial intelligence model may be trained to output a type corresponding to the input among the plurality of types. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to identify the movement model corresponding to the determined type of object.

[0272] Each "processor" herein may include a processing circuit and / or a plurality of processors. For example, the term "processor" as used herein, including in the claims, may include various processing circuits including at least one processor, and at least one of the at least one processor may be configured to perform the various functions described herein collectively and / or individually in a distributed manner. When "processor," "at least one processor," and "one or more processors" are described as being configured to perform a plurality of functions as used herein, these terms may include, for example and not limited thereto, situations in which one processor performs part of the mentioned functions and other processor(s) perform other parts of the mentioned functions, and situations in which a single processor can perform all the mentioned functions. Additionally, at least one processor may include a combination of processors that perform the various mentioned / disclosed functions, for example, in a distributed manner. At least one processor may execute program instructions for achieving or performing the various functions.

[0273] A method performed by an electronic device (101) as described above may include the operation of the electronic device (101) acquiring two-dimensional information about at least one object from a still image. The method may include the operation of identifying the type of the at least one object related to the three-dimensional movement effect of the at least one object using the two-dimensional information. The method may include the operation of generating a video containing the three-dimensional movement effect of the at least one object based on a movement model corresponding to the type of the at least one object. The term "based on" as used herein includes at least "based on".

[0274] A method performed by an electronic device (101) as described above may include an operation of identifying an object in a still image containing one or more objects. The method may include an operation of identifying a motion model corresponding to a motion attribute to be applied to a 3D (three-dimensional) object corresponding to the identified object. The motion attribute may include one or more types of motion among a plurality of types of motion. The method may include an operation of displaying a first frame containing the 3D object corresponding to the object at a first position through the display. The method may include an operation of updating the position of the 3D object according to motion information of the 3D object from the motion model. The method may include an operation of displaying a second frame containing the 3D object at a second position instead of the first position according to the motion information through the display after the first frame.

[0275] A non-transitory computer-readable storage medium as described above may store a program containing instructions. When the instructions are executed individually or collectively by at least one processor (120), the electronic device (101) may cause the electronic device (101) to obtain two-dimensional information about at least one object from a still image. When the instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may use the two-dimensional information to identify the type of the at least one object related to the three-dimensional motion effect of the at least one object. When the instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to generate a video containing the three-dimensional motion effect of the at least one object based on a motion model corresponding to the type of the at least one object.

[0276] A non-transient computer-readable storage medium as described above may store a program containing instructions. When the instructions are executed individually or collectively by at least one processor (120), the electronic device (101) may cause the electronic device (101) to identify an object in a still image containing one or more objects. When the instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to identify a motion model corresponding to motion attributes to be applied to a three-dimensional object corresponding to the identified object. The motion attributes may include one or more types of motion among a plurality of motion types. When the instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the electronic device (101) to display a first frame containing the three-dimensional object corresponding to the object at a first position through the display. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the position of the 3D object to be updated according to the motion information of the 3D object from the motion model. When the above instructions are executed individually or collectively by the at least one processor (120), the electronic device (101) may cause the second frame containing the 3D object at a second position instead of a first position according to the motion information to be displayed through the display after the first frame.

[0277] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs.

[0278] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0279] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., first) component is referred to as "coupled" or "connected" to another (e.g., second) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through at least third component(s).

[0280] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0281] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0282] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., CD-ROM (compact disc read only memory)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0283] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101), At least one processor (120) including a processing circuit; and The electronic device (101) includes a memory (130) that stores instructions and includes one or more storage media, and when the instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Acquire two-dimensional information about at least one object from a still image; Based on the above two-dimensional information, identify the type of the at least one object related to the three-dimensional movement effect of the at least one object; Based on a motion model corresponding to the type of at least one object, causing to generate a video including the three-dimensional motion effect of the at least one object, Electronic device.

2. In Claim 1, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Based on the above two-dimensional information, a pattern of physical motion related to at least one object and a deformation of the external shape corresponding to the pattern of physical motion are identified, and Causing to identify the type of at least one object based on the pattern of physical movement and the deformation of the external shape, Electronic device.

3. In Claim 1, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Receiving specified user input for at least one of the above objects, and Based on the above user input and the above motion model, the above three-dimensional motion effect is determined, and Causing to generate the video containing the three-dimensional motion effect determined based on the user input and the motion model, Electronic device.

4. In Claim 1, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Detecting state information of the above electronic device (101), Based on the detected state information and the motion model, the three-dimensional motion effect is determined, and Causing to generate the video containing the three-dimensional motion effect determined according to the detected state information and the motion model, Electronic device.

5. In Claim 4, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Detecting the movement of the housing of the electronic device (101) moving in a specified direction and with a specified intensity, and Causing the movement of the housing of the electronic device (101) to be detected as at least part of the state information of the electronic device (101), Electronic device.

6. In Claim 4, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Identify designated notifications, Causing to detect the above-mentioned notification as at least part of the state information of the electronic device (101), Electronic device.

7. In Claim 4, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Identifying that the above electronic device (101) changes between a locked state and an unlocked state, Causing to detect a change between the lock state and the unlock state as at least a part of the state information of the electronic device (101), Electronic device.

8. In Claim 1, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), At least some of the deformation according to the position and deformation according to the viewpoint of the above-mentioned at least one object are generated based on another image, and Causing to model the at least one object in three dimensions based on deformation according to the position and deformation according to the time point of the at least one object, Electronic device.

9. In Claim 1, The above at least one object includes a first object and a second object, and The above video includes a first motion effect corresponding to the first object and a second motion effect corresponding to the second object, and The first motion effect and the second motion effect are selected based on the priority related to the first object and the second object, Electronic device.

10. In Claim 1, At least in part while the three-dimensional motion effect is provided by the playback of the video, the video includes a portion inpainted from the background area of ​​the still image and a portion outpainted from the at least one object. Electronic device.

11. In Claim 1, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Recognizing multiple objects from the above still image, and A UI (user interface) configured to request the selection of one or more of the aforementioned plurality of objects is displayed through the display (260), and Receiving user input to select at least one object among the plurality of objects, and Upon receiving the user input, causing to obtain the two-dimensional information about the at least one object represented by the user input from the still image, Electronic device.

12. In the electronic device (101), display, At least one processor (120) including a processing circuit; and The electronic device (101) includes a memory (130) that stores instructions and includes one or more storage media, and when the instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Identify objects in a still image containing one or more objects, and Identify a motion model corresponding to a motion attribute to be applied to a 3D (three-dimensional) object corresponding to the above-identified object, wherein the motion attribute includes one or more motion types among a plurality of motion types, and A first frame containing the 3D object corresponding to the above object at a first position is displayed through the display, and The position of the 3D object is updated according to the motion information of the 3D object from the motion model, and Through the above display, configured to cause the second frame, which includes the 3D object at a second position instead of the first position according to the motion information, to be displayed after the first frame, Electronic device.

13. In Claim 12, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Determine the type of the identified object among a plurality of types, and the plurality of types represent different combinations of the plurality of movement types, Identifying predefined movement information for the above type of the identified object, and Configured to cause the position of the 3D object to be updated from the motion model according to the above-defined predefined motion information, Electronic device.

14. In Claim 12, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Receive input, Determining the movement information corresponding to the above input, and Configured to cause the position of the 3D object to be updated from the motion model according to the motion information corresponding to the above input, Electronic device.

15. In Claim 14, When the above instructions are executed individually and / or collectively by the at least one processor (120), the electronic device (101), Receive a notification of an ongoing call from a phone application as the above input, and Based on the fact that the caller of the above ongoing call corresponds to the first caller, the motion information is determined as the first motion information, and Based on the fact that the caller of the above ongoing call corresponds to a second caller other than the first caller, the motion information is configured to be determined as second motion information, and The displacement between the second position and the first position according to the first movement information is greater than the displacement between the second position and the first position according to the second movement information. Electronic device.

Images